Apparatus and method for storage of encoded mesh frame in base-mesh bitstream

Abstract

Various embodiments describe apparatuses, methods, and computer program products. An example apparatus includes: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: providing one or more signaling elements in the base mesh parameters for indicating existence of the header component in the base mesh parameters and the payload component in the base mesh frame; and constructing a base mesh bitstream, to obtain a constructed base mesh bitstream, with the one or more signaling elements, the header component in the base mesh parameters and the payload component in the base mesh frame.

Description

APPARATUS AND METHOD FOR STORAGE OF ENCODED MESH FRAME IN BASE- MESH BITSTREAM TECHNICAL FIELD

[0001] The examples and non-limiting embodiments relate generally to volumetric video coding and, more particularly to, storage of encoded mesh frame in base-mesh bitstream. BACKGROUND

[0002] It is known to perform data encoding and decoding of video and images. SUMMARY

[0003] Example 1: An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: providing one or more signaling elements in the base mesh parameters for indicating existence of the header component in the base mesh parameters and the payload component in the base mesh frame; and constructing a base mesh bitstream, to obtain a constructed base mesh bitstream, with the one or more signaling elements, the header component in the base mesh parameters and the payload component in the base mesh frame.

[0004] Example 2. The apparatus of example 1, wherein the apparatus is further caused to perform: obtaining one or more encoded mesh frames; analyzing an encoded mesh frame of the one or more encoded mesh frames to determine a header component and a payload component of the encoded mesh frame; and packing the payload component of the encoded mesh frame into a base mesh frame; packing the header component of the encoded mesh frame into base mesh parameters.

[0005] Example 3: The apparatus of any of the examples 1 or 2, wherein the apparatus is further caused to perform: providing the constructed base mesh bitstream to a decoder; and providing the one or more signaling elements, in or along the constructed base mesh bitstream to the decoder.

[0006] Example 4: The apparatus of any of examples 1 to 3, wherein: the one or more encoded mesh frames are encoded as motion picture experts group (MPEG) EdgeBreaker static mesh; or the one or more encoded mesh frames are represented as Draco header, metadata, connectivity, and attributes.

[0007] Example 5. The apparatus of any of the examples 1 to 3, wherein the base mesh frame and the base mesh parameters are identified based on base mesh coding layer unit types.

[0008] Example 6: The apparatus of any of the previous examples, wherein the base mesh parameters comprise a syntax element for identifying the base mesh codec, and wherein the syntax element for identifying the base mesh codec is represented by one of the following: a codec identifier of an intra mesh codec; four character code (4CC) provided by a combination of a codec identifier of an intra mesh codec and a supplemental enhancement information message information; or information providing the used base mesh profile tier and level indices.

[0009] Example 7: The apparatus of any of the previous examples, wherein the header component of the encoded mesh frame is stored in one of: a base mesh sequence parameter set, the base mesh frame parameters, or the supplemental enhancement information message.

[0010] Example 8: The apparatus of any of the previous examples, wherein when the header component of the encoded mesh frame is not same for a sentence and changes during a series of frames, the header component of the encoded mesh frame is updated in a base mesh frame parameter set and is guarded by an update static mesh codec header, wherein the update static mesh codec header is signaled to indicate that a first header component is provided in a current frame parameter set and overrides a second header component at the base mesh sequence parameter set level.

[0011] Example 9: The apparatus of any of the previous examples, wherein the payload component of encoded mesh frame is stored in a base mesh intra submesh data unit syntax structure of submesh data unit.

[0012] Example 10: An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: extracting from the one or more base mesh frames the payload component of the encoded mesh frame; extracting from the base mesh parameter, the header component of the encoded mesh frame; and re-multiplexing the header component of the encoded mesh frame and the payload component of the encoded mesh frame into one encoded mesh frame.

[0013] Example 11. The apparatus of example 10, wherein the apparatus is further caused to perform: receiving a base mesh bitstream; demultiplexing the base mesh bitstream comprising one or more base mesh frames; receiving one or more signaling elements; examining the one or more signaling elements to determine whether the one or more base mesh frames comprise a payload component of an encoded mesh frame; and examining the one or more signaling elements to determine whether the base mesh parameters comprise header component of the encoded meshframe.

[0014] Example 12: The apparatus of any of the examples 10 or 11, wherein the apparatus is further caused to perform: identifying the header component of the encoded mesh frame based on the signaling information in the base mesh parameters, wherein the signaling information is comprised at a base mesh sequence parameter set, a base mesh frame parameter set, or a supplemental enhancement information level; identifying the payload component of the encoded mesh frame based on the signaling information in the base mesh parameters; and identifying a relation between the base mesh frame and the base mesh parameters is based on: the signaling information in case of the base mesh sequence parameter set and the base mesh frame parameter set; or base mesh bitstream structure constrains.

[0015] Example 13: The apparatus of any of the examples 10 to 12, wherein the header component of encoded mesh frame and payload component of encoded mesh frame that have a relation are re- multiplexed to create the one encoded mesh frame.

[0016] Example 14: A method comprising: providing one or more signaling elements in the base mesh parameters for indicating existence of the header component in the base mesh parameters and the payload component in the base mesh frame; and constructing a base mesh bitstream, to obtain a constructed base mesh bitstream, with the one or more signaling elements, the header component in the base mesh parameters and the payload component in the base mesh frame.

[0017] Example 15. The method of example 14 further comprising: obtaining one or more encoded mesh frames; analyzing an encoded mesh frame of the one or more encoded mesh frames to determine a header component and a payload component of the encoded mesh frame; and packing the payload component of the encoded mesh frame into a base mesh frame; packing the header component of the encoded mesh frame into base mesh parameters.

[0018] Example 16: The method of any of the examples 14 or 15 further comprising: providing the constructed base mesh bitstream to a decoder; and providing the one or more signaling elements, in or along the constructed base mesh bitstream to the decoder.

[0019] Example 17: The method of any of examples 14 to 16, wherein: the one or more encoded mesh frames are encoded as motion picture experts group (MPEG) EdgeBreaker static mesh; or the one or more encoded mesh frames are represented as Draco header, metadata, connectivity, and attributes.

[0020] Example 18: The method of any of examples 14 to 16, wherein: the base mesh frames is identified based on base mesh coding layer units type.

[0021] Example 19: The method of any of the examples 14 to 18, wherein the base mesh parameters comprise a syntax element for identifying the base mesh codec, and wherein the syntax element for identifying the base mesh codec is represented by one of the following: a codec identifier of an intra mesh codec; four character code (4CC) provided by a combination of a codec identifier of an intra mesh codec and a supplemental enhancement information message information; or information providing the used base mesh profile tier and level indices.

[0022] Example 20: The method of any of the examples 14 to 19, wherein the header component of the encoded mesh frame is stored in one of: a base mesh sequence parameter set, the base mesh frame parameters, or the supplemental enhancement information message.

[0023] Example 21: The method of any of the examples 14 to 20, wherein when the header component of the encoded mesh frame is not same for a sentence and changes during a series of frames, the header component of the encoded mesh frame is updated in a base mesh frame parameter set and is guarded by an update static mesh codec header, wherein the update static mesh codec header is signaled to indicate that a first header component is provided in a current frame parameter set and overrides a second header component at the base mesh sequence parameter set level.

[0024] Example 22: The method of any of the examples 14 to 21, wherein the payload component of encoded mesh frame is stored in a base mesh intra submesh data unit syntax structure of submesh data unit.

[0025] Example 23: A method comprising: extracting from the one or more base mesh frames the payload component of the encoded mesh frame; extracting from the base mesh parameter the header component of the encoded mesh frame; and re-multiplexing the header component of the encoded mesh frame and the payload component of the encoded mesh frame into one encoded mesh frame.

[0026] Example 24. The method of example 23 further comprising: receiving a base mesh bitstream; demultiplexing the base mesh bitstream comprising one or more base mesh frames; receiving one or more signaling elements; examining the one or more signaling elements to determine whether the one or more base mesh frames comprise a payload component of an encoded mesh frame; and examining the one or more signaling elements to determine whether the base mesh parameters comprise header component of the encoded mesh frame.

[0027] Example 25: The method of any of the examples 23 or 24 further comprising: identifying the header component of the encoded mesh frame based on the signaling information in the base mesh parameters, wherein the signaling information is comprised at a base mesh sequence parameter set, a base mesh frame parameter set, or a supplemental enhancement information level; identifying the payload component of the encoded mesh frame based on the signaling information in the base mesh parameters; and identifying a relation between the base mesh frame and the base mesh parameters is based on: the signaling information in case of the base mesh sequence parameter set and the base mesh frame parameter set; or base mesh bitstream structure constrains.

[0028] Example 26: The method of any of the examples 23 to 25, wherein the header component of encoded mesh frame and payload component of encoded mesh frame that have a relation are re- multiplexed to create the one encoded mesh frame.

[0029] Example 27: An apparatus comprising: means for providing one or more signaling elements in the base mesh parameters for indicating existence of the header component in the base mesh parameters and the payload component in the base mesh frame; and means for constructing a base mesh bitstream, to obtain a constructed base mesh bitstream, with the one or more signaling elements, the header component in the base mesh parameters and the payload component in the base mesh frame.

[0030] Example 28: The apparatus of example 27, wherein the apparatus further comprises means for performing the methods as described in any of the examples 12 to 22.

[0031] Example 29: An apparatus comprising: means for extracting from the one or more base mesh frames the payload component of the encoded mesh frame; means for extracting from the base mesh parameter the header component of the encoded mesh frame; and means for re-multiplexing the header component of the encoded base frame and the payload component of the encoded mesh frame into one encoded mesh frame.

[0032] Example 30: The apparatus of example 29, wherein the apparatus further comprises means for performing the methods as described in examples 24 to 26.

[0033] Example 31: A computer readable medium comprising program instructions which, when executed by an apparatus, cause the apparatus to perform at least the following: providing one or more signaling elements in the base mesh parameters for indicating existence of the headercomponent in the base mesh parameters and the payload component in the base mesh frame; and constructing a base mesh bitstream, to obtain a constructed base mesh bitstream, with the one or more signaling elements, the header component in the base mesh parameters and the payload component in the base mesh frame.

[0034] Example 32: The computer readable medium of example 31, wherein the computer readable medium comprises a non-transitory computer readable medium.

[0035] Example 33: The computer readable medium of any of examples 31 or 32, wherein the computer readable medium causes the apparatus to further perform the methods as described in any of the examples 15 to 22.

[0036] Example 34: A computer readable medium comprising program instructions which, when executed by an apparatus, cause the apparatus to perform at least the following: extracting from the one or more base mesh frames the payload component of the encoded mesh frame; extracting from the base mesh parameter the header component of the encoded mesh frame; and re-multiplexing the header component of the encoded mesh frame and the payload component of the encoded mesh frame into one encoded mesh frame.

[0037] Example 35: The computer readable medium of example 34, wherein the computer readable medium comprises a non-transitory computer readable medium.

[0038] Example 36: The computer readable medium of any of examples 34 or 35, wherein the computer readable medium causes the apparatus to further perform the methods as described in any of the examples 24 to 26. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The foregoing embodiments and other features are explained in the following description, taken in connection with the accompanying drawings, wherein:

[0040] FIG.1 shows schematically an electronic device employing embodiments of the examples described herein.

[0041] FIG.2 shows schematically a user equipment suitable for employing embodiments of the examples described herein.

[0042] FIG. 3 illustrates a general structure of a compressed bitstream, where three different alternative structures for the connectivity data have been separated.

[0043] FIG. 4 illustrates extracting encoded header and payload components of encoded mesh frame from base-mesh bitstream and re-multiplexing encoded mesh frames.

[0044] FIG.5 is an example apparatus configured to implement the examples described herein.

[0045] FIG.6 shows a representation of an example of non-volatile memory media used to store instructions that implement the examples described herein.

[0046] FIG.7 is an example method, based on the examples described herein.

[0047] FIG.8 is another example method, based on the examples described herein.

[0048] FIG.9 is yet another example method, based on the examples described herein.

[0049] FIG.10 is still another example method, based on the examples described herein. DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

[0050] Described herein is a method and apparatus for implementing storage of encoded mesh frame in a base-mesh bitstream.

[0051] The following describes in detail a suitable apparatus and possible mechanisms for a mesh encoding process or storage of encoded mesh frame in a base-mesh bitstream according to embodiments. In this regard reference is first made to FIG. 1 and FIG. 2, where FIG. 1 shows an example block diagram of an apparatus 50. The apparatus may be an Internet of Things (IoT) apparatus configured to perform various functions, such as for example, gathering information by one or more sensors, receiving or transmitting information, analyzing information gathered or received by the apparatus, or the like. The apparatus may comprise a video coding system, which may incorporate a codec. FIG. 2 shows a layout of an apparatus according to an example embodiment. The elements of FIG.1 and FIG.2 are explained next.

[0052] The electronic device 50 may for example be a mobile terminal or user equipment of a wireless communication system, a sensor device, a tag, or other lower power device. However, it may be appreciated that embodiments of the examples described herein may be implemented withinany electronic device or apparatus which may process data by neural networks.

[0053] The apparatus 50 may comprise a housing 30 for incorporating and protecting the device. The apparatus 50 further may comprise a display 32 in the form of a liquid crystal display. In other embodiments of the examples described herein the display may be any suitable display technology suitable to display an image or video. The apparatus 50 may further comprise a keypad 34. In other embodiments of the examples described herein any suitable data or user interface mechanism may be employed. For example the user interface may be implemented as a virtual keyboard or data entry system as part of a touch-sensitive display.

[0054] The apparatus may comprise a microphone 36 or any suitable audio input which may be a digital or analog signal input. The apparatus 50 may further comprise an audio output device which in embodiments of the examples described herein may be any one of: an earpiece 38, speaker, or an analog audio or digital audio output connection. The apparatus 50 may also comprise a battery (or in other embodiments of the examples described herein the device may be powered by any suitable mobile energy device such as solar cell, fuel cell or clockwork generator). The apparatus may further comprise a camera 42 capable of recording or capturing images and / or video. The apparatus 50 may further comprise an infrared port for short range line of sight communication to other devices. In other embodiments the apparatus 50 may further comprise any suitable short range communication solution such as for example a Bluetooth wireless connection or a USB / firewire wired connection.

[0055] The apparatus 50 may comprise a controller 56, processor or processor circuitry for controlling the apparatus 50. The controller 56 may be connected to memory 58 which in embodiments of the examples described herein may store both data in the form of image and audio data and / or may also store instructions for implementation on the controller 56. The controller 56 may further be connected to codec circuitry 54 suitable for carrying out coding and / or decoding of audio and / or video data or assisting in coding and / or decoding carried out by the controller.

[0056] The apparatus 50 may further comprise a card reader 48 and a smart card 46, for example a UICC and UICC reader for providing user information and being suitable for providing authentication information for authentication and authorization of the user at a network.

[0057] The apparatus 50 may comprise radio interface circuitry 52 connected to the controller and suitable for generating wireless communication signals for example for communication with a cellular communications network, a wireless communications system or a wireless local area network. The apparatus 50 may further comprise an antenna 44 connected to the radio interfacecircuitry 52 for transmitting radio frequency signals generated at the radio interface circuitry 52 to other apparatus(es) and / or for receiving radio frequency signals from other apparatus(es).

[0058] The apparatus 50 may comprise a camera capable of recording or detecting individual frames which are then passed to the codec 54 or the controller for processing. The apparatus may receive the video image data for processing from another device prior to transmission and / or storage. The apparatus 50 may also receive either wirelessly or by a wired connection the image for coding / decoding. The structural elements of apparatus 50 described above represent examples of means for performing a corresponding function.

[0059] Volumetric video

[0060] There are many ways to capture and represent a volumetric frame. The format used to capture and represent it depends on the processing to be performed on the volumetric frame, and the target application using volumetric frame. Some example representations are listed below.

[0061] A volumetric frame may be represented as a point cloud. A point cloud is a set of unstructured points in 3D space, where each point is characterized by its position in a 3D coordinate system (e.g., Euclidean), and some corresponding attributes (e.g., color information provided as red, green, blue, alpha (RGBA) value, or normal vectors).

[0062] A volumetric frame may be represented as images, with or without depth, captured from multiple viewpoints in 3D space. In other words, it may be represented by one or more view frames (where a view is a projection of a volumetric scene on to a plane (the camera plane) using a real or virtual camera with known / computed extrinsics and intrinsics). Each view may be represented by a number of components (e.g., geometry, color, transparency, and occupancy picture), which may be part of the geometry picture or represented separately.

[0063] A volumetric frame may be represented as a mesh. A mesh is a collection of points, called vertices, and connectivity information between vertices, called edges. Vertices along with edges form faces. The combination of vertices, edges and faces may uniquely approximate shapes of objects.

[0064] Depending on the capture, a volumetric frame may provide viewers the ability to navigate a scene with six degrees of freedom, e.g., both translational and rotational movement of their viewing pose (which includes yaw, pitch, and role). The data to be coded for a volumetric frame may also besignificant, as a volumetric frame may include many objects, and the positioning and movement of these objects in the scene may result in many dis-occluded regions. Furthermore, the interaction of light and materials in objects and surfaces in a volumetric frame may generate complex light fields that may produce texture variations for even a slight change of pose.

[0065] A sequence of volumetric frames is a volumetric video. Due to large amount of information, storage and transmission of a volumetric video requires compression. A way to compress a volumetric frame may be to project the 3D geometry and related attributes into a collection of 2D images along with additional associated metadata. The projected 2D images may then be coded using 2D video and image coding technologies, for example, ISO / IEC 14496-10 (H.264 / AVC) and ISO / IEC 23008-2 (H.265 / HEVC). The metadata may be coded with technologies specified in specification such as ISO / IEC 23090-5. The coded images and the associated metadata may be stored or transmitted to a client that may decode and render the 3D volumetric frame.

[0066] Visual Volumetric Video-base Coding (V3C) – ISO / IEC 23090-5

[0067] ISO / IEC 23090-5 specifies the syntax, semantics, and process for coding volumetric video. The specified syntax is designed to be generic so that it may be reused for a variety of applications. Point clouds, immersive video with depth, and mesh representations may all use ISO / IEC 23090-5 standard with extensions that deal with the specific nature of the final representation. The purpose of the specification is to define how to decode and interpret the associated data (for example atlas data in ISO / IEC 23090-5) which tells a renderer how to interpret 2D frames to reconstruct a volumetric frame.

[0068] Two applications of V3C (ISO / IEC 23090-5) have been defined, V-PCC (ISO / IEC 23090- 5) and MIV (ISO / IEC 23090-12). MIV and V-PCC use number of V3C syntax elements with a slightly modified semantics. An example on how the generic syntax element may be differently interpreted by the application is pdu_projection_id.

[0069] In case of V-PCC the syntax element, pdu_projection_id specifies the index of the projection plane for the patch. There may be 6 or 18 projection planes in V-PCC, and they are implicit, e.g., pre-determined.

[0070] In case of MIV pdu_projection_id corresponds to a view ID, e.g., identifies which view the patch originated from. View IDs and their related information is explicitly provided in MIV view parameters list and may be tailored for each content.

[0071] MPEG 3DG (ISO SC29 WG7) group has started work on a third application of V3C – the mesh compression. It is also envisaged that mesh coding may re-use V3C syntax as much as possible and may also slightly modify the semantics.

[0072] To differentiate between applications of V3C bitstream, that allow a client to properly interpret the decoded data, V3C uses the ptl_profile_toolset_idc parameter.

[0073] V3C – V3C bitstream

[0074] V3C bitstream is a sequence of bits that forms the representation of coded volumetric frames and the associated data making one or more coded V3C sequences (CVS). Where CVS is a sequence of bits identified and separated by appropriate delimiters, and is required to start with a VPS, includes a V3C unit, and includes one or more V3C units with atlas sub-bitstream or video sub-bitstream. Video sub-bitstreams and atlas sub-bitstreams may be referred to as V3C sub- bitstreams. Which V3C sub-bitstream includes a V3C unit and how to interpret it is identified by a V3C unit header in conjunction with VPS information.

[0075] V3C bitstream may be stored according to Annex C of ISO / IEC 23090-5 which specifies syntax and semantics of a sample stream format to be used by applications that deliver some or all of the V3C unit stream as an ordered stream of bytes or bits within which the locations of V3C unit boundaries need to be identifiable from patterns in the data.

[0076] Video-based Point Cloud Compression (V-PCC) – ISO / IEC 23090-5

[0077] The generic mechanism of V3C may be used by applications targeting volumetric content. One of such application is video-based point cloud compression (ISO / IEC 20390-5). V-PCC enables enables volumetric video coding for application in which a scene is represented by point cloud. V- PCC uses the patch data unit concept from V3C and for each patch assign one of 6 (18) pre-defined orthogonal camera views for reprojection.

[0078] MPEG Immersive Video (MIV) – ISO / IEC 23090-12

[0079] Another application of V3C is MPEG immersive video (ISO / IEC 23090-12). MIV enables volumetric video coding for applications in which a scene is recorded with multiple red, green, blue, and optionally depth (RGB(D)) cameras with overlapping fields of view (FoVs). One example setupis a linear array of cameras pointing towards a scene. This multi-scopic view of the scene allows a 3D reconstruction and therefore 6DoF / 3DoF+ consumption.

[0080] MIV uses the patch data unit concept from V3C and extends it by allow using application specific camera views for reprojection. In contrast to V-PCC, which uses pre-defined 6 or 18 orthogonal camera views for reprojection. Additionally, MIV introduces additional occupancy packing modes and other improvements to V3C base syntax. One such example is support for multiple atlases, for example when there is too much information to pack everything in a single video frame. It also adds support for common atlas data, which includes information that is shared between all atlases. This is particularly useful for storing camera details of the input camera models, which are frequently shared between different atlases.

[0081] Video-based dynamic mesh coding (V-DMC) – ISO / IEC 23090-29

[0082] V-DMC (ISO / IEC 23090-29) is another application form of V3C that aims on integration of mesh compression into the V3C family of standards. The standard is under development and at WD stage (MDS22775_WG07_N00611).

[0083] The retained technology after the CfP result analysis is based on multiresolution mesh analysis and coding. This approach includes: – generating a base-mesh that is a simplified (low resolution) mesh approximation of the original mesh, called base-mesh (this is done for all frames of the dynamic mesh sequence); – performing several mesh subdivision iterative steps (e.g., each triangle is converted into four triangles by connecting the triangle edge midpoints on the generated base mesh, generating other approximation meshes; – defining displacement vectors, also named error vectors, for each vertex of each mesh approximation; – for each subdivision level by adding the displacement vectors to the subdivided mesh vertices generates the best approximation of the original mesh at that resolution, given the base-mesh and prior subdivision levels; – the displacement vectors may undergo a lazy wavelet transform prior to compression; and – the attribute map of the original mesh is transferred to the deformed mesh at the highest resolution (e.g., subdivision level) such that texture coordinates are obtained for the deformed mesh and a new attribute map is generated.

[0084] The V-DMC encoder generates compressed bitstreams, which later on are packed in V3C units and create V3C bitstream by concatenating one or more of the following V3C units: – a sub-bitstream (e.g., a base-mesh bistream) with the encoded base-mesh using a mesh codec; – a sub-bitstream with the displacement vectors; packed in an image and encoded using a video codec or arithmetic encoded as defined in Annex J of WD ISO / IEC 23090-29, MDS22775_WG07_N00611; – a sub-bitstream with the attribute map encoded using a video codec; or – a sub-bitstream (atlas) that includes all metadata required to decode and reconstruct the mesh sequence based on the aforementioned sub-bitstreams. The signaling of the metadata is based on the V3C syntax and includes necessary extensions that are specific to meshes.

[0085] Base-mesh bitstream (ISO / IEC 23090-29)

[0086] An elementary unit for the output of a base-mesh encoder (Annex H of ISO / IEC 23090- 29) is a NAL unit.

[0087] A NAL unit may be defined as a syntax structure including an indication of the type of data to follow and bytes including that data in the form of a raw byte sequence payload (RBSP) interspersed as necessary with emulation prevention bytes. An The RBSP may be defined as a syntax structure including an integer number of bytes that is encapsulated in a NAL unit. An RBSP is either empty or has the form of a string of data bits including syntax elements followed by an RBSP stop bit and followed by zero or more subsequent bits equal to 0.

[0088] NAL units may be categorized into base-mesh coding layer (BMCL) NAL units and non- BMCL NAL units. BMCL NAL units may be coded sub-mesh NAL units. A non-BMCL NAL unit may be for example one of the following types: a base-mesh sequence parameter set, a base-mesh frame parameter set, a supplemental enhancement information (SEI) NAL unit, an access unit delimiter, an end of sequence NAL unit, an end of bitstream NAL unit, or a filler data NAL unit. Parameter sets may be needed for the reconstruction of decoded bas-mesh, whereas many of the other non-BMCL NAL units are not necessary for the reconstruction of decoded sample values.

[0089] V-DMC specifications may include a set of constraints for associating data units (e.g. NAL units) into coded base-mesh access units.

[0090] MPEG EdgeBreaker Static Mesh (ISO / IEC 23090-29:Annex I)

[0091] The MPEG EdgeBreaker static mesh codec specifies signaling and decoding processes for intra mesh coding based on the well-known EdgeBreaker algorithm, which is extended with several algorithmic enhancements allowing to handle a wider variety of mesh connectivity types and better geometry and UV texture coordinate predictions. It achieves comparable compression performances as the Draco compression that is introduced later in this document.

[0092] ISO / IEC 23090-29:Annex I specifies a mesh coding header, a mesh position coding payload and mesh attributes coding payload.

[0093] General Mesh coding syntax mesh_coding(^)^{ Descriptor mesh_coding_header(^) mesh_position_coding_payload(^) mesh_attributes_coding_payload(^) }

[0094] The mesh coding header syntax is detailed in the following table and includes the mesh codec type (e.g., the mesh connectivity traversal chosen, for the moment only the forward traversal is implemented), position encoding parameters (that are detailed later in this document), position dequantization information, the number of mesh attributes and for each of them its number of components (2 for texture coordinates, 3 for normal and colors, 1 for material type and a generic type with n components), and finally attribute coding and dequantization parameters that are explained in detail in following paragraphs.

[0095] Mesh coding header syntaxmesh_coding_header(^)^{Descriptormesh_codec_type u(2) mesh_position_encoding_parameters(^) mesh_position_dequantize_flag u(1)if^(mesh_position_dequantize_flag^) mesh_position_dequantize_parameters(^) mesh_attribute_count u(5) for(^i=0;^i<mesh_attribute_count;^i++^){ mesh_attribute_type[^i^] u(3) if(^mesh_attribute_type[^i^]^==^MESH_ATTR_TEXCOORD^) NumComponents[^i^]^=^2 else if(^mesh_attribute_type[^i^]^==^MESH_ATTR_NORMAL^) NumComponents[^i^]^=^3 else if(^mesh_attribute_type[^i^]^==^MESH_ATTR_COLOR^) NumComponents[^i^]^=^3 else if(^mesh_attribute_type[^i^]^==^MATERIAL_ID^) NumComponents[^i^]^=^1 else if(^mesh_attribute_type[^i^]^==^GENERIC^)^{ mesh_attribute_num_components_minus1[^i^] u(2) NumComponents[^i^]^=^mesh_attribute_num_components_minus1[^i^] +1 } mesh_attribute_encoding_parameters^(^i^) mesh_attribute_dequantize_flag[^i^] u(1) if^(mesh_attribute_dequantize_flag[^i^]^) mesh_attribute_dequantize_parameters^(^i^) } length_alignment(^) }

[0096] The position coding parameters are specified as follows, and include information related to the position bitdepth, the EdgeBreaker CLERS symbols coding method chosen, the position prediction methods used, the position residuals encoding method chosen and finally the position duplicate removal method.

[0097] Mesh position encoding parameters syntax mesh_position_encoding_parameters(^)^{ Descriptor mesh_position_bit_depth_minus1 u(4)mesh_clers_symbols_encoding_method ue(v) mesh_position_prediction_method ue(v) mesh_position_prediction_max_parallelograms_minus1 u(4) mesh_position_residuals_encoding_method ue(v) mesh_position_deduplicate_method ue(v) }

[0098] The mesh attributes encoding parameters are provided per attribute (signaled by its index) in the following table and signal the attribute bitdepth, whether it is a per face attribute and when a separate index table is provided for these attributes (several attributes may be mapped to the same face and in that case it is suitable to provide a separate index table with a mapping between attribute value index and face index), the mesh attribute prediction method used and the mesh attribute residuals encoding method chosen.

[0099] Mesh attributes encoding parameters syntax mesh_attribute_encoding_parameters(^index^)^{ Descriptor mesh_attribute_bit_depth_minus1[^index^] u(4) mesh_attribute_per_face_flag[^index^] u(1) if(^!mesh_attribute_per_face_flag[^index^]^)^{ mesh_attribute_separate_index_flag[^index^] u(1) if(^!mesh_attribute_separate_index_flag[^index^]^) mesh_attribute_reference_index_plus1[^index^] ue(v) } mesh_attribute_prediction_method[^index^] ue(v) mesh_attribute_residuals_encoding_method[^index^] ue(v) }

[0100] Draco compression

[0101] Draco is a technology for compressing and decompressing 3D geometric meshes and point clouds. It is intended to improve the storage and transmission of 3D graphics. It supports compressing points, connectivity information, texture coordinates, color information, normals, and any other generic attributes associated with geometry. Draco may compress the 3D mesh either sequentially or using edgebreaker algorithm. The input to Draco encoder may be any 3D model andthe encoder compresses it into a Draco bitstream. The compressed bitstream may be decoded back into the original 3D model, or if needed transcoded into a different format.

[0102] The compressed bitstream is divided into four parts: Draco header, metadata, connectivity and attributes.

[0103] FIG.3 illustrates a general structure of a compressed bitstream 300, where three different alternative structures for the connectivity data have been separated. A Draco header 302 includes high-level information about the bitstream 300 and identifies the bitstream as Draco bitstream 300. The fields of the Draco header 302 are described below: ParseHeader() { draco_string major_version minor_version encoder_type encoder_method flags }

[0104] The metadata 304 part allows to associate per attribute metadata or file level metadata with the Draco bitstream 300. It could be, for example, used to describe attribute names 306. The metadata 304 includes of following information and enables recursively adding more levels of sub-metadata. The metadata 304 is represented by key-value pairs. An example syntax is provided below: void DecodeMetadata() { ParseMetadataCount(); for (i = 0; i < num_att_metadata; ++i) { ParseAttributeMetadataId(i); DecodeMetadataElement(att_metadata[i]); } DecodeMetadataElement(file_metadata); } void ParseMetadataCount() { num_att_metadata} void ParseAttributeMetadataId(index) { att_metadata_id[index] } void ParseMetadataElement(metadata) { metadata.num_entries for (i = 0; i < metadata.num_entries; ++i) { sz = metadata.key_size[i] metadata.key[i] sz = metadata.value_size[i] metadata.value[i] } metadata.num_sub_metadata } void ParseSubMetadataKey(metadata, index) { sz = metadata.sub_metadata_key_size[index] metadata.sub_metadata_key[index] } void DecodeMetadataElement(metadata) { ParseMetadataElement(metadata); for (i = 0; i < metadata.num_sub_metadata; ++i) { ParseSubMetadataKey(metadata, i); DecodeMetadataElement(metadata.sub_metadata[i]); } }

[0105] The structure of the connectivity 308 part depends on the encoder method. It includes of either sequential or edgebreaker information. The type of connectivity 308 information is defined in the Draco header (encoder_method). The sequential connectivity header 310 includes the following fields: void ParseSequentialConnectivityData() { num_faces num_pointsconnectivity_method }

[0106] The connectivity header 310 includes information such as the number of faces and points in the connectivity bitstream as well as the connectivity method which identifies if the sequential connectivity bitstream includes of compressed or uncompressed indices 312. The rest of the connectivity data 312 includes the connectivity bitstream.

[0107] Alternatively, the compressed Draco bitstream 300 may include of edgebreaker encoded connectivity data instead of the sequential connectivity data. In this case the connectivity header 310 includes information as defined by following ParseEdgebreakerConnectivityData structure: void ParseEdgebreakerConnectivityData() { edgebreaker_traversal_type num_encoded_vertices num_faces num_attribute_data num_encoded_symbols num_encoded_split_symbols }

[0108] The connectivity header 310 provides information such as the traversal type, which indicates the type of the edgebreaker connectivity bitstream. In current version this may be either standard edgebreaker (0) or valence edgebreaker (2). Additionally, it includes information such as the number of encoded vertices and attributes. It also provides information on the number of encoder symbols and split symbols, which are required to decode different parts of the edgebreaker encoded connectivity bitstream. In addition to the connectivity header 310, the connectivity data includes the connectivity bitstream 314.

[0109] The structure of the connectivity bitstream 314 depends on the traversal type and may include encoded split data 316, encoded edgebreaker symbol data 318, encoded start face configuration data 320 and the attribute connectivity data 322. It could additionally include valence header 324 and context data 326 in case edgebreaker valence traversal type is used.

[0110] The attribute data 328 includes two sections. The first part is the attribute header 330, which indicates how many attributes need to be decoded as well as what components each attributeincludes. An example syntax is provided below: void ParseAttributeDecodersData() { num_attributes_decoders if (encoder_method == MESH_EDGEBREAKER_ENCODING) { for (i = 0; i < num_attributes_decoders; ++i) { att_dec_data_id[i] att_dec_decoder_type[i] att_dec_traversal_method[i] } } for (i = 0; i < num_attributes_decoders; ++i) { att_dec_num_attributes[i] for (j = 0; j < att_dec_num_attributes[i]; ++j) { att_dec_att_type[i][j] att_dec_data_type[i][j] att_dec_num_components[i][j] att_dec_normalized[i][j] att_dec_unique_id[i][j] } for (j = 0; j < att_dec_num_attributes[i]; ++j) { seq_att_dec_decoder_type[i][j] } } }

[0111] The second part of the attribute data is includes compressed attributes 332a to 332d, such as positions, texture coordinates, normals, and the like. Each attribute type 334 section is comprised of one or more unique components.

[0112] The compressed bitstream is described in detail in specification available from [https: / / google.github.io / draco / spec / (last accessed on October 4, 2023)].

[0113] In the current text of the ISO / IEC 23090-29, Annex H specifies high level syntax (HLS) for base-mesh sub-bitstream. The HLS provides a base-mesh sequence parameter set (BMSPS), base-mesh frame parameter set (BMFPS), and base mesh coding layer NAL units. The HLS isdefined independently of the static mesh codec (so called Intra-coded (I) base-mesh frames) and of motion vector coding (so called Predicted (P) base-mesh frames). What codecs are used for the I frame and P frames in base-mesh sub-bitstream are identified by bmsps_intra_mesh_codec_id and bmsps_inter_mesh_codec_id, respectively.

[0114] The codec ids or no other signaling impact BMSPS and BMFPS syntax element and they are the same for any codec used.

[0115] The I base-mesh frame codec may include header information, for example, MPEG EdgeBreaker Static Mesh defined in Annex I of ISO / IEC WD 23090-29 has mesh_coding_header(), which may include header information that may be the same for every I base-mesh frame, such as the number or type of different attributes.

[0116] That means that each I base-mesh frame may replicate this information. It may be beneficial to avoid replication and send that information in BMSPS or BMFPS.

[0117] An encoder receives compressed base mesh frame (e.g. using the codec specified in Annex I of ISO / IEC WD 23090-29), extracts the static parts of the compressed frame representation (e.g., mesh_coding_header( ) as specified in Annex I of ISO / IEC WD 23090-29) and inserts it into base mesh parameter set along with additional signalling information allowing a decoder to re-insert the static information to the compressed frame representation before forwarding the compressed mesh frame to the mesh frame decoder.

[0118] Encoder

[0119] An example apparatus includes: means for obtaining one or more encoded mesh frame; means for analyzing the encoded mesh frame to determine header and payload components of encoded mesh frame; means for packing payload component of encoded mesh frame into one base mesh frame; means for packing header component of encoded mesh frame into a base mesh parameters; means for providing one or more signaling elements in base mesh parameters, indicating existence of the header component in the base mesh parameters and the payload component in the base mesh frame; means for constructing a base-mesh bitstream with the one or more signaling elements and the header component in the base mesh parameter set and the payload component in the base mesh frame.

[0120] The example apparatus may further include means for providing the constructed base-mesh bitstream to a decoder; and means for providing the one or more signaling elements, in or along the constructed bitstream to the decoder.

[0121] Decoder

[0122] Another example apparatus includes: means for receiving a base-mesh bitstream; means for demultiplexing base-mesh bitstream including base-mesh frames; means for receiving one or more signaling elements; means for examining the one or more signaling elements to determine whether the base-mesh frames include payload component of encoded mesh frame; means for examining the one or more signaling elements to determine when the base-mesh parameters includes header component of the encoded base mesh frame; means for extracting from the base-mesh frame the payload component of the encoded base mesh frame; means for extracting from the base-mesh parameter the header component of the encoded base mesh frame; and means for re-multiplexing the header component of encoded base mesh frame and the payload component of encoded base mesh frame into one encoded base mesh frame.

[0123] In an embodiment, the encoded mesh frame may be encoded as MPEG EdgeBreaker Static Mesh defined in ISO / IEC 23090-29:Annex I.

[0124] In an embodiment, the encoded mesh frame may be represented by three syntax structures comprising header information, encoded position payload, and encoded attributes payload as ISO / IEC 23090-29 I.8.3.1. mesh_coding( ) { Descriptor mesh_coding_header( ) mesh_position_coding_payload( ) mesh_attributes_coding_payload( ) }

[0125] In an embodiment, the encoded mesh frame may be represented as Draco header, metadata, connectivity, and attributes, as described in FIG.3.

[0126] In an embodiment, the base mesh frame may be identified by base-mesh coding layer units, e.g., bmesh_nal_unit_type from range 0 to 29 in ISO / IEC 23090-29:Table H-1.

[0127] In an embodiment, base mesh parameters may be identified by non-base-mesh coding layerunits, e.g., bmesh_nal_unit_type equal to 30 or 31, or from range 36 to 63 in ISO / IEC 23090- 29:Table H-1.

[0128] In an embodiment, a syntax_element_identifying_bm_codec in base mesh parameters may be represented by: • bmsps_intra_mesh_codec_id; • 4CC code provided by combination of bmsps_intra_mesh_codec_id and SEI message information; or • bmptl_profile_codec_group_idc signalled in bmesh_profile_tier_level( )

[0129] In an embodiment, a header component of the encoded mesh frame is stored in base mesh sequence parameter set. bmesh_sequence_parameter_set_rbsp( ) { Descriptor bmsps_sequence_parameter_set_id u(4) bmesh_profile_tier_level( ) bmsps_intra_mesh_codec_id u(8) bmsps_inter_mesh_codec_id u(8) bmsps_inter_mesh_motion_group_size_minus1 u(8) bmsps_inter_mesh_max_num_neighbours_minus1 u(8) bmsps_geometry_3d_bit_depth_minus1 u(5) bmsps_facegroup_segmentation_method ue(v) bmsps_mesh_attribute_count u(7) for( i = 0; i < bmsps_mesh_attribute_count; i++ ) { bmsps_mesh_attribute_type_id[ i ] u(4) bmsps_attribute_bit_depth_minus1[ i ] u(5) bmsps_attribute_msb_align_flag[ i ] u(1) } bmsps_log2_max_mesh_frame_order_cnt_lsb_minus4 ue(v) bmsps_max_dec_mesh_frame_buffering_minus1 ue(v) bmsps_long_term_ref_mesh_frames_flag u(1) bmsps_num_ref_mesh_frame_lists_in_bmsps ue(v) for( i = 0; i < bmsps_num_ref_mesh_frame_lists_in_bmsps; i++ ) bmesh_ref_list_struct( i ) if ( syntax_element_identifying_bm_codec == mesh_codec_type_x ) {mesh_codec_type_x_header() } bmsps_extension_present_flag u(1) if( bmsps_extension_present_flag ) { bmsps_extension_count u(8) } if( bmsps_extension_count ){ bmsps_extensions_length_minus1 ue(v) for( i = 0; i < bmsps_extension_count; i++ ) { bmsps_extension_type[ i ] u(8) bmsps_extension_length[ i ] u(16) bmsps_extension( bmsps_extension_type[ i ], bmsps_extension_length[ i ] ) } } rbsp_trailing_bits( ) }

[0130] In an embodiment, the syntax_element_identifying_bm_codec identifies mesh codec defined in ISO / IEC 23090-29 Annex I, in that case mesh_codec_type_x_header() syntax structure may comprise mesh_coding_header( ) (ISO / IEC 23090-29:I.8.3.2).

[0131] In an embodiment, the syntax_element_identifying_bm_codec identifies mesh codec as in Draco available from [https: / / google.github.io / draco / spec / (last accessed on October 4, 2023)], in that case mesh_codec_type_x_header() syntax structure may comprise draco header and metadata as described in FIG.3.

[0132] In an embodiment, in case header component of the encoded mesh frames is not the same for the whole sequence but may change during a series of frames, an header component of encoded mesh header may be updated in BMFPS and it may be guarded by update_static_mesh_codec_header. update_static_mesh_codec_header is signaled to indicate that the header is provided in the current frame parameter set and it overrides the one provided on BMSPS level.

[0133] In an embodiment, the header component of encoded mesh frame is stored in SEI message.

[0134] In an embodiment payload component of encoded mesh frame is stored in sdu_intra_sub_mesh_unit () syntax structure and / or sismu_intra_unit(). An example syntax for sdu_intra_sub_mesh_unit and sismu_intra_unit() is as follows: sdu_intra_sub_mesh_unit( subMeshID, vertexCount ) { Descriptor sismu_intra_unit( subMeshID ) length_alignment( ) } sismu_intra_unit( subMeshID ) { Descriptor if ( syntax_element_identifying_bm_codec == mesh_codec_type_x ) { mesh_codec_type_x_payload() } }

[0135] In anembodiment, the syntax_element_idetifying_bm_codec identifies mesh codec defined in ISO / IEC 23090-29 Annex I, in that case mesh_codec_type_x_payload() syntax structure may comprise mesh_position_coding_payload( ), mesh_attributes_coding_payload( ) (ISO / IEC 23090-29:I.8.3.7, I.8.3.9).

[0136] In an embodiment, the syntax_element_idetifying_bm_codec identifies mesh codec as in Draco available from [https: / / google.github.io / draco / spec / (last accessed on October 4, 2023)], in that case mesh_codec_type_x_payload() syntax structure may comprise Connectivity and Attributes, as described in FIG.3.

[0137] FIG.4 illustrates extracting encoded mesh frame header (410) from base mesh parameter (400) of base-mesh bitstream (440) and extracting encoded mesh payload components (421...42N) from base mesh frames (401..40N) of base-mesh bitstream (440) and re-multiplexing encoded mesh frame header (410) and encoded mesh payload components (421...42N) to create encoded mesh frames (431..43N).

[0138] In an embodiment, header component of encoded mesh frame is identified based on the signaling information in base mesh parameters (it may be on BMSPS, BMFPS or SEI level), payload component of encoded mesh frame is identified based on the signaling information in base mesh parameters. In this embodiment, the relation between base mesh frame and base mesh parameters isidentified based on signalling information (e.g., based on the relation of bmsps_sequence_parameter_set_id, bfps_mesh_frame_parameter_set_id, bfps_mesh_sequence_parameter_set_id, bfps_mesh_sequence_parameter_set_id syntax elements in ISO / IEC 23090-29) in case of BMSPS and BMFPS, or a base mesh bitstream structure constrains (e.g., SEI message being valid for given base mesh access unit).

[0139] The header component of encoded mesh frame and payload component of encoded mesh frame that have relation are re-multiplex to create one encoded mesh frame.

[0140] FIG.5 is an example apparatus 500, which may be implemented in hardware, configured to implement the examples described herein. The apparatus 500 comprises at least one processor 502 (e.g., an FPGA and / or CPU), one or more memories 504 including computer program code 505, the computer program code 505 having instructions to carry out the methods described herein, wherein the at least one memory 504 and the computer program code 505 are configured to, with the at least one processor 502, cause the apparatus 500 to implement circuitry, a process, component, module, or function (implemented with control module 506) to implement the examples described herein, including storage of encoded mesh frame in base mesh bitstream. Optionally included encoder 530 of the control module 506 performs encoding, and optionally included decoder 540 implements decoding. The memory 504 may be a non-transitory memory, a transitory memory, a volatile memory (e.g. RAM), or a non-volatile memory (e.g., ROM).

[0141] The apparatus 500 includes a display and / or I / O interface 508, which includes user interface (UI) circuitry and elements, that may be used to display features or a status of the methods described herein (e.g., as one of the methods is being performed or at a subsequent time), or to receive input from a user such as with using a keypad, camera, touchscreen, touch area, microphone, biometric recognition, one or more sensors, and the like. The apparatus 500 includes one or more communication, e.g., network (N / W) interfaces (I / F(s)) 510. The communication I / F(s) 510 may be wired and / or wireless and communicate over the Internet / other network(s) via any communication technique including via one or more links 524. The communication I / F(s) 510 may comprise one or more transmitters or one or more receivers.

[0142] The transceiver 516 comprises one or more transmitters 518 and one or more receivers 520. The transceiver 516 and / or communication I / F(s) 510 may comprise standard well-known components such as an amplifier, filter, frequency-converter, (de)modulator, and encoder / decoder circuitries and one or more antennas, such as antennas 514 used for communication over wireless link 522.

[0143] The control module 506 of the apparatus 500 comprises one of or both parts 506-1 and / or 506-2, which may be implemented in a number of ways. The control module 506 may be implemented in hardware as control module 506-1, such as being implemented as part of the one or more processors 502. The control module 506-1 may be implemented also as an integrated circuit or through other hardware such as a programmable gate array. In another example, the control module 506 may be implemented as control module 506-2, which is implemented as computer program code (having corresponding instructions) 505 and is executed by the one or more processors 502. For instance, the one or more memories 504 store instructions that, when executed by the one or more processors 502, cause the apparatus 500 to perform one or more of the operations as described herein. Furthermore, the one or more processors 502, one or more memories 504, and example algorithms (e.g., as flowcharts and / or signaling diagrams), encoded as instructions, programs, or code, are means for causing performance of the operations described herein.

[0144] The apparatus 500 to implement the functionality of control 506 may correspond to any of the apparatuses depicted herein. Alternatively, apparatus 500 and its elements may not correspond to any of the other apparatuses depicted herein, as apparatus 500 may be part of a self- organizing / optimizing network (SON) node or other node, such as a node in a cloud.

[0145] The apparatus 500 may also be distributed throughout the network (e.g., internet) including within and between apparatus 500 and any network element (such as a base station).

[0146] Interface 512 enables data communication and signaling between the various items of apparatus 500, as shown in FIG.5. For example, the interface 512 may be one or more buses such as address, data, or control buses, and may include any interconnection mechanism, such as a series of lines on a motherboard or integrated circuit, fiber optics or other optical communication equipment, and the like. Computer program code (e.g., instructions) 505, including control 506 may comprise object-oriented software configured to pass data or messages between objects within computer program code 505. The apparatus 500 need not comprise each of the features mentioned, or may comprise other features as well. The various components of apparatus 500 may at least partially reside in a common housing 528, or a subset of the various components of apparatus 500 may at least partially be located in different housings, which different housings may include housing 528.

[0147] FIG. 6 shows a schematic representation of non-volatile memory media 600a (e.g. computer / compact disc (CD) or digital versatile disc (DVD)) and 600b (e.g. universal serial bus(USB) memory stick) and 600c (e.g. cloud storage for downloading instructions and / or parameters 602 or receiving emailed instructions and / or parameters 602) storing instructions and / or parameters 602 which when executed by a processor allows the processor to perform one or more of the operations of the methods described herein.

[0148] FIG. 7 is an example method 700, based on the example embodiments described herein. At 702, the method 700 includes obtaining one or more encoded mesh frames. At 704, the method 700 includes analyzing an encoded mesh frame of the one or more encoded mesh frames to determine a header component and a payload component of the encoded mesh frame. At 706, the method 700 includes packing the payload component of the encoded mesh frame into a base mesh frame. At 708, the method 700 includes packing the header component of the encoded mesh frame into base mesh parameters. At 710, the method 700 includes providing one or more signaling elements in the base mesh parameters for indicating existence of the header component in the base mesh parameters and the payload component in the base mesh frame. At 712, the method 700 includes and constructing a base mesh bitstream, to obtain a constructed base mesh bitstream, with the one or more signaling elements, the header component in the base mesh parameters and the payload component in the base mesh frame.

[0149] The method 700 may be performed with an encoding apparatus, such as the apparatus 50 or the apparatus 500 with the encoder 530.

[0150] FIG. 8 is another example method 800, based on the example embodiments described herein. At 802, the method 800 includes receiving a base mesh bitstream. At 804, the method 800 includes demultiplexing the base mesh bitstream comprising one or more base mesh frames. At 806, the method 800 includes receiving one or more signaling elements. At 808, the method 800 includes examining the one or more signaling elements to determine whether the one or more base mesh frames comprise a payload component of an encoded mesh frame. At 810, the method 800 includes examining the one or more signaling elements to determine whether the base mesh parameters comprise header component of the encoded mesh frame. At 812, the method 800 includes extracting from the one or more base mesh frames the payload component of the encoded mesh frame. At 814, the method 800 includes extracting from the base mesh parameter the header component of the encoded mesh frame. At 816, the method 800 includes re-multiplexing the header component of the encoded mesh frame and the payload component of the encoded mesh frame into one encoded mesh frame.

[0151] The method 800 may be performed with a decoding apparatus, such as the apparatus 50 orthe apparatus 500 with the decoder 540.

[0152] FIG.9 is yet another example method 900, based on the example embodiments described herein. At 902, the method 900 includes providing one or more signaling elements in the base mesh parameters for indicating existence of the header component in the base mesh parameters and the payload component in the base mesh frame. At 904, the method 900 includes constructing a base mesh bitstream, to obtain a constructed base mesh bitstream, with the one or more signaling elements, the header component in the base mesh parameters and the payload component in the base mesh frame.

[0153] In an embodiment, the method may further include obtaining one or more encoded mesh frames; analyzing an encoded mesh frame of the one or more encoded mesh frames to determine a header component and a payload component of the encoded mesh frame; packing the payload component of the encoded mesh frame into a base mesh frame; and packing the header component of the encoded mesh frame into base mesh parameters.

[0154] The method 900 may be performed with an encoding apparatus, such as the apparatus 50 or the apparatus 500 with the encoder 530.

[0155] FIG. 10 is still another example method 1000, based on the example embodiments described herein. At 1002, the method 1000 includes extracting from the one or more base mesh frames the payload component of the encoded mesh frame. At 1004, the method 1000 includes extracting from the base mesh parameter, the header component of the encoded mesh frame. At 1006, the method 1000 includes re-multiplexing the header component of the encoded mesh frame and the payload component of the encoded mesh frame into one encoded mesh frame.

[0156] In an embodiment, the method may further include receiving a base mesh bitstream; demultiplexing the base mesh bitstream comprising one or more base mesh frames; receiving one or more signaling elements; examining the one or more signaling elements to determine whether the one or more base mesh frames comprise a payload component of an encoded mesh frame; and examining the one or more signaling elements to determine whether the base mesh parameters comprise header component of the encoded mesh frame.

[0157] The method 1000 may be performed with a decoding apparatus, such as the apparatus 50 or the apparatus 500 with the decoder 540.

[0158] In the above, some example embodiments have been described with the help of syntax ofthe bitstream. It needs to be understood, however, that the corresponding structure and / or computer program may reside at the encoder for generating the bitstream and / or at the decoder for decoding the bitstream.

[0159] In the above, where example embodiments have been described with reference to an encoder, it needs to be understood that the resulting bitstream and the decoder have corresponding elements in them. Likewise, where example embodiments have been described with reference to a decoder, it needs to be understood that the encoder has structure and / or computer program for generating the bitstream to be decoded by the decoder.

[0160] References to a ‘computer’, ‘processor’, etc. should be understood to encompass not only computers having different architectures such as single / multi-processor architectures and sequential / parallel architectures but also specialized circuits such as field-programmable gate arrays (FPGAs), application specific circuits (ASICs), signal processing devices and other processing circuitry. References to computer program, instructions, code etc. should be understood to encompass software for a programmable processor or firmware such as, for example, the programmable content of a hardware device such as instructions for a processor, or configuration settings for a fixed- function device, gate array or programmable logic device, etc.

[0161] As used herein, the term ‘circuitry’, ‘circuit’ and variants may refer to any of the following: (a) hardware circuit implementations, such as implementations in analog and / or digital circuitry, and (b) combinations of circuits and software (and / or firmware), such as (as applicable): (i) a combination of processor(s) or (ii) portions of processor(s) / software including digital signal processor(s), software, and one or more memories that work together to cause an apparatus to perform various functions, and (c) circuits, such as a microprocessor(s) or a portion of a microprocessor(s), that require software or firmware for operation, even when the software or firmware is not physically present. As a further example, as used herein, the term ‘circuitry’ would also cover an implementation of merely a processor (or multiple processors) or a portion of a processor and its (or their) accompanying software and / or firmware. The term ‘circuitry’ would also cover, for example and when applicable to the particular element, a baseband integrated circuit or applications processor integrated circuit for a mobile phone or a similar integrated circuit in a server, a cellular network device, or another network device. Circuitry or circuit may also be used to mean a function or a process used to execute a method.

[0162] It should be understood that the foregoing description is only illustrative. Various alternatives and modifications may be devised by those skilled in the art. For example, featuresrecited in the various dependent claims could be combined with each other in any suitable combination(s). In addition, features from different embodiments described above could be selectively combined into a new embodiment. Accordingly, the description is intended to embrace all such alternatives, modifications and variances which fall within the scope of the appended claims.

[0163] The following acronyms and abbreviations that may be found in the specification and / or the drawing figures are defined as follows (the abbreviations may be appended with each other or with other characters using e.g. a hyphen or dash (-), and may be case insensitive): 2D two-dimensional 3D three-dimensional ASPS atlas sequence parameter set AFPS atlas frame parameter set ATL atlas tile layer AVC advanced video coding BMSPS base-mesh sequence parameter set BMFPS base-mesh frame parameter set BMTL base-mesh tile layer HEVC high efficiency video coding HLS high level syntax MIV mpeg immersive video SEI supplemental enhancement information V3C visual volumetric video-based coding V-DMC video-based dynamic mesh coding V-PCC video-based point cloud compression

Claims

CLAIMS What is claimed is:

1. An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: providing one or more signaling elements in the base mesh parameters for indicating existence of the header component in the base mesh parameters and the payload component in the base mesh frame; and constructing a base mesh bitstream, to obtain a constructed base mesh bitstream, with the one or more signaling elements, the header component in the base mesh parameters and the payload component in the base mesh frame.

2. The apparatus of claim 1, wherein the apparatus is further caused to perform: obtaining one or more encoded mesh frames; analyzing an encoded mesh frame of the one or more encoded mesh frames to determine a header component and a payload component of the encoded mesh frame; packing the payload component of the encoded mesh frame into a base mesh frame; and packing the header component of the encoded mesh frame into base mesh parameters.

3. The apparatus of any of the claims 1 or 2, wherein the apparatus is further caused to perform: providing the constructed base mesh bitstream to a decoder; and providing the one or more signaling elements, in or along the constructed base mesh bitstream to the decoder.

4. The apparatus of any of the previous claims, wherein: the one or more encoded mesh frames are encoded as motion picture experts group (MPEG) EdgeBreaker static mesh; or the one or more encoded mesh frames are represented as Draco header, metadata, connectivity, and attributes.

5. The apparatus of any of claims 1 to 3, wherein the base mesh frame and the base mesh parameters are identified based on base mesh coding layer unit types.

6. The apparatus of any of the previous claims, wherein the base mesh parameters comprise a syntax element for identifying a base mesh codec, and wherein the syntax element for identifying the base mesh codec is represented by one of the following: a codec identifier of an intra mesh codec; four character code (4CC) provided by a combination of a codec identifier of an intra mesh codec and a supplemental enhancement information message information; or information providing the used base mesh profile tier and level identifiers.

7. The apparatus of any of the previous claims, wherein the header component of the encoded mesh frame is stored in one of: a base mesh sequence parameter set, the base mesh frame parameters, or the supplemental enhancement information message.

8. The apparatus of any of the previous claims, wherein when the header component of the encoded mesh frame is not same for a sentence and changes during a series of frames, the header component of the encoded mesh frame is updated in a base mesh frame parameter set and is guarded by an update static mesh codec header, wherein the update static mesh codec header is signaled to indicate that a first header component is provided in a current frame parameter set and overrides a second header component at the base mesh sequence parameter set level.

9. The apparatus of any of the previous claims, wherein the payload component of encoded mesh frame is stored in a basemesh intra submesh data unit syntax structure of submesh data unit.

10. An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: extracting from the one or more base mesh frames the payload component of the encoded mesh frame; extracting from the base mesh parameter, the header component of the encoded mesh frame; and re-multiplexing the header component of the encoded mesh frame and the payload component of the encoded mesh frame into one encoded mesh frame.

11. The apparatus of claim 10, wherein the apparatus is further caused to perform: receiving a base mesh bitstream;demultiplexing the base mesh bitstream comprising one or more base mesh frames; receiving one or more signaling elements; examining the one or more signaling elements to determine whether the one or more base mesh frames comprise a payload component of an encoded mesh frame; and examining the one or more signaling elements to determine whether the base mesh parameters comprise header component of the encoded mesh frame.

12. The apparatus of any of the claims 10 or 11, wherein the apparatus is further caused to perform: identifying the header component of the encoded mesh frame based on the signaling information in the base mesh parameters, wherein the signaling information is comprised at a base mesh sequence parameter set, a base mesh frame parameter set, or a supplemental enhancement information level; identifying the payload component of the encoded mesh frame based on the signaling information in the base mesh parameters; and identifying a relation between a base mesh frame and the base mesh parameters is based on: the signaling information in case of the base mesh sequence parameter set and the base mesh frame parameter set; or base mesh bitstream structure constrains.

13. The apparatus of any of the claims 10 to 12, wherein the header component of encoded mesh frame and payload component of encoded mesh frame that have a relation are re- multiplexed to create the one encoded mesh frame.

14. A method comprising: providing one or more signaling elements in the base mesh parameters for indicating existence of the header component in the base mesh parameters and the payload component in the base mesh frame; and constructing a base mesh bitstream, to obtain a constructed base mesh bitstream, with the one or more signaling elements, the header component in the base mesh parameters and the payload component in the base mesh frame.

15. The method of claim 14 further comprising: obtaining one or more encoded mesh frames; analyzing an encoded mesh frame of the one or more encoded mesh frames to determinea header component and a payload component of the encoded mesh frame; packing the payload component of the encoded mesh frame into a base mesh frame; and packing the header component of the encoded mesh frame into base mesh parameters.

16. The method of any of the claims 14 or 15further comprising: providing the constructed base mesh bitstream to a decoder; and providing the one or more signaling elements, in or along the constructed base mesh bitstream to the decoder.

17. The method of any of the claims 14 to 16, wherein: the one or more encoded mesh frames are encoded as motion picture experts group (MPEG) EdgeBreaker static mesh; or the one or more encoded mesh frames are represented as Draco header, metadata, connectivity, and attributes.

18. The method of any of the claims 14 to 16, wherein the base mesh frame and the base mesh parameters are identified based on base mesh coding layer unit types.

19. The method of any of the claims 14 to 18, wherein the base mesh parameters comprise a syntax element for identifying a base mesh codec, and wherein the syntax element for identifying the base mesh codec is represented by one of the following: a codec identifier of an intra mesh codec; four character code (4CC) provided by a combination of a codec identifier of an intra mesh codec and a supplemental enhancement information message information; or information providing the used base mesh profile tier and level identifier.

20. The method of any of the claims 14 to 19, wherein the header component of the encoded mesh frame is stored in one of: a base mesh sequence parameter set, the base mesh frame parameters, or the supplemental enhancement information message.

21. The method of any of the claims 14 to 20, wherein when the header component of the encoded mesh frame is not same for a sentence and changes during a series of frames, the header component of the encoded mesh frame is updated in a base mesh frame parameter set and is guarded by an update static mesh codec header, wherein the update static mesh codec header is signaled to indicate that a first header component is provided in a current frame parameter setand overrides a second header component at the base mesh sequence parameter set level.

22. The method of any of the claims 14 to 21, wherein the payload component of encoded mesh frame is stored in a base mesh intra submesh data unit syntax structure of submesh data unit.

23. A method comprising: extracting from the one or more base mesh frames the payload component of an encoded mesh frame; extracting from the base mesh parameter, the header component of the encoded mesh frame; and re-multiplexing the header component of the encoded mesh frame and the payload component of the encoded mesh frame into one encoded mesh frame.

24. The method of claim 23 further comprising: receiving a base mesh bitstream; demultiplexing the base mesh bitstream comprising one or more base mesh frames; receiving one or more signaling elements; examining the one or more signaling elements to determine whether the one or more base mesh frames comprise a payload component of an encoded mesh frame; and examining the one or more signaling elements to determine whether the base mesh parameters comprise header component of the encoded mesh frame.

25. The method of any of the claims 23 or 24 further comprising: identifying the header component of the encoded mesh frame based on the signaling information in the base mesh parameters, wherein the signaling information is comprised at a base mesh sequence parameter set, a base mesh frame parameter set, or a supplemental enhancement information level; identifying the payload component of the encoded mesh frame based on the signaling information in the base mesh parameters; and identifying a relation between a base mesh frame and the base mesh parameters is based on: the signaling information in case of the base mesh sequence parameter set and the base mesh frame parameter set; or base mesh bitstream structure constrains.

26. The method of any of the claims 23 to 25, wherein the header component of encoded mesh frame and payload component of encoded mesh frame that have a relation are re- multiplexed to create the one encoded mesh frame.

27. An apparatus comprising: means for providing one or more signaling elements in the base mesh parameters for indicating existence of the header component in the base mesh parameters and the payload component in the base mesh frame; and means for constructing a base mesh bitstream, to obtain a constructed base mesh bitstream, with the one or more signaling elements, the header component in the base mesh parameters and the payload component in the base mesh frame.

28. The apparatus of claim 27, wherein the apparatus further comprises means for performing the methods as claimed in any of the claims 15 to 22.

29. An apparatus comprising: means for extracting from the one or more base mesh frames the payload component of the encoded mesh frame; means for extracting from the base mesh parameter, the header component of the encoded mesh frame; and means for re-multiplexing the header component of the encoded mesh frame and the payload component of the encoded mesh frame into one encoded mesh frame.

30. The apparatus of claim 29, wherein the apparatus further comprises means for performing the methods as claimed in claims 24 to 26.

31. A computer readable medium comprising program instructions which, when executed by an apparatus, cause the apparatus to perform at least the following: providing one or more signaling elements in the base mesh parameters for indicating existence of the header component in the base mesh parameters and the payload component in the base mesh frame; and constructing a base mesh bitstream, to obtain a constructed base mesh bitstream, with the one or more signaling elements, the header component in the base mesh parameters and the payload component in the base mesh frame.

32. The computer readable medium of claim 31, wherein the computer readable medium comprises a non-transitory computer readable medium.

33. The computer readable medium of any of claims 31 or 32, wherein the computer readable medium causes the apparatus to further perform the methods as claimed in any of the claims 15 to 22.

34. A computer readable medium comprising program instructions which, when executed by an apparatus, cause the apparatus to perform at least the following: extracting from the one or more base mesh frames the payload component of the encoded mesh frame; extracting from the base mesh parameter, the header component of the encoded mesh frame; and re-multiplexing the header component of the encoded mesh frame and the payload component of the encoded mesh frame into one encoded mesh frame.

35. The computer readable medium of claim 34, wherein the computer readable medium comprises a non-transitory computer readable medium.

36. The computer readable medium of any of claims 34 or 35, wherein the computer readable medium causes the apparatus to further perform the methods as claimed in any of the claims 24 to 26.

Images