Mesh data transmission device, mesh data transmission method, mesh data reception device, and mesh data reception method

Abstract

In order to achieve the above-described purpose and other advantages, a mesh data decoding method according to embodiments may comprise the steps of: receiving a file including mesh data; decapsulating the file; and decoding the mesh data. A mesh data encoding method according to embodiments may comprise the steps of: encoding mesh data; encapsulating the mesh data into a file; and transmitting the file.

Description

Mesh data transmission device, mesh data transmission method, mesh data reception device, and mesh data reception method

[0001] The embodiments provide a method for providing Point Cloud content to provide users with various services such as Virtual Reality (VR), Augmented Reality (AR), Mixed Reality (MR), and autonomous driving services.

[0002] A point cloud is a collection of points in 3D space. The sheer number of points in 3D space makes it difficult to generate point cloud data.

[0003] There is a problem that a lot of processing power is required to transmit and receive point cloud data.

[0004] The technical problem according to the embodiments is to provide a point cloud data transmission device, transmission method, point cloud data reception device and reception method for efficiently transmitting and receiving point clouds in order to solve the problems described above.

[0005] The technical problem according to the embodiments is to provide a point cloud data transmission device, transmission method, point cloud data reception device, and reception method for resolving latency and encoding / decoding complexity.

[0006] However, the scope of the embodiments is not limited to the aforementioned technical tasks, and the scope of the embodiments may be expanded to other technical tasks that can be inferred by a person skilled in the art based on the entire contents of this document.

[0007] To achieve the above-described purpose and other advantages, a mesh data decoding method according to embodiments may include a step of receiving a file including mesh data; a step of decapsulating the file; and a step of decrypting the mesh data. A mesh data encoding method according to embodiments may include a step of encoding mesh data; a step of encapsulating mesh data into a file; and a step of transmitting the file.

[0008] The point cloud data transmission method, transmission device, point cloud data reception method, and reception device according to the embodiments can provide a high-quality point cloud service.

[0009] The point cloud data transmission method, transmission device, point cloud data reception method, and reception device according to the embodiments can achieve various video codec methods.

[0010] The point cloud data transmission method, transmission device, point cloud data reception method, and reception device according to the embodiments can provide general-purpose point cloud content such as autonomous driving services.

[0011] The drawings are included to further understand the embodiments, and the drawings illustrate the embodiments together with the description related to the embodiments. For a better understanding of the various embodiments described below, reference should be made to the following description of the embodiments in conjunction with the following drawings, in which like reference numerals correspond to corresponding parts throughout the drawings.

[0012] Figure 1 illustrates a system for providing dynamic mesh content according to embodiments.

[0013] Figure 2 illustrates a V-MESH compression method according to embodiments.

[0014] Figure 3 illustrates pre-processing of V-MESH compression according to embodiments.

[0015] Figure 4 illustrates a mid-edge subdivision method according to embodiments.

[0016] Figure 5 shows a displacement generation process according to embodiments.

[0017] FIG. 6 illustrates an intra-frame encoding process of a V-MESH compression method according to embodiments.

[0018] Figure 7 illustrates an inter-frame encoding process of a V-MESH compression method according to embodiments.

[0019] Figure 8 shows a lifting conversion process for displacement according to embodiments.

[0020] Figure 9 illustrates a process of packing transformation coefficients according to embodiments into a 2D image.

[0021] Fig. 10 shows an attribute transfer process of a V-MESH compression method according to embodiments.

[0022] Fig. 11 illustrates an intra-frame decoding process of a V-MESH compression method according to embodiments.

[0023] Fig. 12 shows a V-MES and Fig. 13 shows a point cloud data transmission device according to embodiments.

[0024] Fig. 13 shows a point cloud data transmission device according to embodiments.

[0025] Fig. 14 shows a point cloud data receiving device according to embodiments.

[0026] Figure 15 illustrates a video-based dynamic mesh coding bitstream structure according to embodiments.

[0027] Figure 16 illustrates a V3C unit header according to embodiments.

[0028] Figure 17 shows a V3C unit type according to embodiments.

[0029] Figure 18 illustrates a V3C unit payload according to embodiments.

[0030] Figure 19 shows a base mesh sequence parameter set according to embodiments.

[0031] Figure 20 shows the base mesh profile tier levels according to embodiments.

[0032] Figure 21 shows base mesh profile tool set constraint information according to embodiments.

[0033] Figure 22 shows a base mesh frame parameter set according to embodiments.

[0034] Figures 23a, 23b, and 23c illustrate basemesh NAL unit types according to embodiments.

[0035] Figure 24 shows a displacement sequence parameter set according to embodiments.

[0036] Figure 25 shows displacement profile tier levels according to embodiments.

[0037] Figure 26 shows displacement profile toolset constraint information according to embodiments.

[0038] Figure 27 shows a displacement frame parameter set according to embodiments.

[0039] Figure 28 shows a displacement reference list structure according to embodiments.

[0040] Fig. 29 shows a displacement header according to embodiments.

[0041] Figure 30 illustrates a displacement intra data unit according to embodiments.

[0042] Figures 31a, 31b, and 31c illustrate displacement NAL unit types according to embodiments.

[0043] Figure 32 shows a mesh system structure according to embodiments.

[0044] Figure 33 shows a track reference according to embodiments.

[0045] Figure 34 shows a track reference according to embodiments.

[0046] Fig. 35 shows a decoder of a receiving device according to embodiments.

[0047] Figure 36 shows a mesh data encoding method according to embodiments.

[0048] Figure 37 shows a mesh data decryption method according to embodiments.

[0049] Preferred embodiments of the embodiments are described in detail, examples of which are illustrated in the accompanying drawings. The following detailed description, with reference to the accompanying drawings, is intended to illustrate preferred embodiments of the embodiments, rather than merely show embodiments that can be implemented according to the embodiments. The following detailed description includes details to provide a thorough understanding of the embodiments. However, it will be apparent to those skilled in the art that the embodiments may be practiced without these details.

[0050] While most of the terms used in the examples are commonly used in the field, some terms were arbitrarily selected by the applicant, and their meanings are described in detail in the following descriptions as needed. Therefore, the examples should be understood based on the intended meaning of the terms, not simply their names or meanings.

[0051] Figure 1 illustrates a system for providing dynamic mesh content according to embodiments.

[0052] The system of FIG. 1 includes a point cloud data transmission device (100) and a point cloud data reception device (110) according to embodiments. The point cloud data transmission device may include a dynamic mesh video acquisition unit (101), a dynamic mesh video encoder (102), a file / segment encapsulator (103), and a transmitter (104). The point cloud data reception device (110) may include a reception unit (111), a file / segment decapsulator (112), a dynamic mesh video decoder (113), and a renderer (114). Each component of FIG. 1 may correspond to hardware, software, a processor, and / or a combination thereof. Hereinafter, the point cloud data transmission device according to embodiments may be interpreted as a term referring to the transmission device (100) or a dynamic mesh video encoder (hereinafter, referred to as an encoder) (102). The point cloud data receiving device according to the embodiments may be interpreted as a term referring to a receiving device (110) or a dynamic mesh video decoder (hereinafter, decoder) (113).

[0053] The system of Fig. 1 can perform video-based dynamic mesh compression and decompression.

[0054] Advances in 3D capture, modeling, and rendering have enabled users to consume diverse forms of 3D content, such as AR, XR, metaverse, and holograms, across multiple platforms and devices. 3D content increasingly represents objects with greater precision and realism, enabling users to enjoy immersive experiences. To achieve this, the creation and use of 3D models requires a significant amount of data. Among various types of 3D content, 3D meshes are widely used for efficient data utilization and realistic object representation. Embodiments include a series of processing steps in a system that utilizes such mesh content.

[0055] First, the method of compressing dynamic mesh data starts with the V-PCC (Video-based point cloud compression) standard technology. Point cloud data is data that contains color information at the vertex coordinates (X, Y, Z). Mesh data refers to data in which connectivity information between vertices is added to this vertex information. When creating content, it can be created in the form of mesh data from the beginning. By adding connectivity information to point cloud data, it can be converted into mesh data and used.

[0056] Currently, the MPEG standards body defines two types of dynamic mesh data: Category 1: Mesh data with texture maps as color information. Category 2: Mesh data with vertex colors as color information.

[0057] Mesh coding standards for Category 1 data are currently under development, and work on Category 2 data standards is also planned for the future. The overall process for providing mesh content services may include acquisition, encoding, transmission, decoding, rendering, and / or feedback, as shown in Figure 1.

[0058] To provide mesh content services, 3D data acquired through multiple cameras or specialized cameras can be processed into mesh data types through a series of processes and then converted into video. The generated mesh video is then transmitted through a series of processes, and the receiving end can then reprocess the received data into mesh video and render it. This allows mesh video to be presented to users, who can then interact with the mesh content according to their intended intent.

[0059] A mesh compression system may include a transmitting device and a receiving device. The transmitting device can encode mesh video to output a bitstream, which can be delivered to the receiving device via digital storage media or a network in the form of a file or streaming segment. The digital storage media may include various storage media, such as USB, SD, CD, DVD, Blu-ray, HDD, or SSD.

[0060] The transmitting device may roughly include a mesh video acquisition unit, a mesh video encoder, and a transmitting unit. The receiving device may roughly include a receiving unit, a mesh video decoder, and a renderer. The encoder may be referred to as a mesh video / video / picture / frame encoding device, and the decoder may be referred to as a mesh video / video / picture / frame decoding device. The transmitter may be included in the mesh video encoder. The receiver may be included in the mesh video decoder. The renderer may include a display unit, and the renderer and / or the display unit may be configured as separate devices or external components. The transmitting device and the receiving device may further include separate internal or external modules / units / components for a feedback process.

[0061] Mesh data represents the surface of an object as a number of polygons. Each polygon is defined by its vertices in 3D space and connection information that describes how those vertices are connected. It can also contain vertex properties such as vertex color and normal. Mapping information that allows the surface of the mesh to be mapped to a 2D planar area can also be included as a mesh property. The mapping is typically described as a set of parametric coordinates, called UV coordinates or texture coordinates, associated with the mesh vertices. Meshes contain 2D attribute maps, which can be used to store high-resolution attribute information such as textures, normals, and displacement.

[0062] The mesh video acquisition unit may include processing 3D object data acquired through a camera, etc. into a mesh data type with the properties described above through a series of processes and generating a video composed of such mesh data. The mesh video may have properties of the mesh, such as vertices, polygons, connection information between vertices, colors, normals, etc., that may change over time. A mesh video with properties and connection information that change over time can be expressed as a dynamic mesh video.

[0063] A mesh video encoder can encode an input mesh video into one or more video streams. A single video can include multiple frames, and a single frame can correspond to a still image / picture. In this document, a mesh video can include a mesh image / frame / picture, and the mesh video can be used interchangeably with the mesh image / frame / picture. A mesh video encoder can perform a Video-based Dynamic Mesh (V-Mesh) Compression procedure. A mesh video encoder can perform a series of procedures such as prediction, transformation, quantization, and entropy coding for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0064] The encapsulation processing unit (file / segment encapsulation module) can encapsulate encoded mesh video data and / or mesh video-related metadata in the form of a file, etc. Here, the mesh video-related metadata may be received from the metadata processing unit, etc. The metadata processing unit may be included in the mesh video encoder, or may be configured as a separate component / module. The encapsulation processing unit may encapsulate the corresponding data in a file format such as ISOBMFF, or process it in the form of other DASH segments, etc. The encapsulation processing unit may include mesh video-related metadata in the file format according to an embodiment. The mesh video metadata may be included in boxes at various levels in the ISOBMFF file format, for example, or may be included as data in a separate track within the file. According to an embodiment, the encapsulation processing unit may encapsulate mesh video-related metadata itself in a file.

[0065] The transmission processing unit can process encapsulated mesh video data for transmission according to the file format. The transmission processing unit can be included in the transmission unit, or can be configured as a separate component / module. The transmission processing unit can process mesh video data according to any transmission protocol. The processing for transmission can include processing for transmission through a broadcast network or processing for transmission through broadband. According to an embodiment, the transmission processing unit can receive not only mesh video data but also mesh video-related metadata from the metadata processing unit and process the same for transmission.

[0066] The transmission unit can transmit encoded video / image information or data output in the form of a bitstream to the receiving unit of a receiving device via a digital storage medium or network in the form of a file or streaming. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmission unit can include an element for generating a media file via a predetermined file format and can include an element for transmission via a broadcasting / communication network. The receiving unit can extract the bitstream and transmit it to a decoding device.

[0067] The receiver can receive mesh video data transmitted by a mesh video transmission device. Depending on the transmission channel, the receiver can receive mesh video data via a broadcast network, via broadband, or via digital storage media.

[0068] The receiving processing unit can perform processing on the received mesh video data according to a transmission protocol. The receiving processing unit can be included in the receiving unit, or can be configured as a separate component / module. In response to the processing performed for transmission on the transmitting side, the receiving processing unit can perform the reverse process of the aforementioned transmitting processing unit. The receiving processing unit can transfer the acquired mesh video data to the decapsulation processing unit, and transfer the acquired mesh video-related metadata to a metadata parser. The mesh video-related metadata acquired by the receiving processing unit can be in the form of a signaling table.

[0069] A decapsulation processing unit (file / segment decapsulation module) can decapsulate mesh video data in file format received from a receiving processing unit. The decapsulation processing unit can decapsulate files according to ISOBMFF, etc., to obtain a mesh video bitstream or mesh video-related metadata (metadata bitstream). The obtained mesh video bitstream can be transmitted to a mesh video decoder, and the obtained mesh video-related metadata (metadata bitstream) can be transmitted to a metadata processing unit. The mesh video bitstream may include metadata (metadata bitstream). The metadata processing unit may be included in the mesh video decoder, or may be configured as a separate component / module. The mesh video-related metadata obtained by the decapsulation processing unit may be in the form of a box or track within a file format. If necessary, the decapsulation processing unit may receive metadata required for decapsulation from the metadata processing unit. Mesh video related metadata can be passed to a Mesh video decoder for use in the Mesh video decoding process, or passed to a renderer for use in the Mesh video rendering process.

[0070] A mesh video decoder can receive a bitstream and perform operations corresponding to those of a mesh video encoder to decode video / images. The decoded mesh video can be displayed via a display unit. Users can view all or part of the rendered result via a VR / AR display or a general display.

[0071] The feedback process may include a process of transmitting various feedback information that may be acquired during the rendering / display process to the transmitter or to the decoder on the receiver. Interactivity may be provided in mesh video consumption through the feedback process. Depending on the embodiment, head orientation information, viewport information indicating the area that the user is currently viewing, etc. may be transmitted during the feedback process. Depending on the embodiment, the user may interact with things implemented in the VR / AR / MR / autonomous driving environment, in which case information related to the interaction may be transmitted to the transmitter or the service provider during the feedback process. Depending on the embodiment, the feedback process may not be performed.

[0072] Head orientation information can refer to information about the user's head position, angle, and movement. Based on this information, information about the area the user is currently viewing within the mesh video, i.e. viewport information, can be calculated.

[0073] Viewport information can be information about the area the user is currently viewing in the mesh video. This can be used to perform gaze analysis to determine how the user consumes the mesh video, which area of ​​the mesh video they are gazing at, and for how long. Gaze analysis can be performed on the receiving side and transmitted to the transmitting side through a feedback channel. Devices such as VR / AR / MR displays can extract the viewport area based on the user's head position / orientation, the vertical or horizontal FOV supported by the device, etc.

[0074] Depending on the embodiment, the aforementioned feedback information may not only be transmitted to the transmitter but may also be consumed by the receiver. That is, the aforementioned feedback information may be utilized to perform decoding, rendering, and other processes on the receiver. For example, head orientation information and / or viewport information may be utilized to preferentially decode and render only the mesh video for the area currently being viewed by the user.

[0075] This document relates to dynamic mesh video compression as described above. The method / embodiment disclosed in this document can be applied to the Video-based Dynamic Mesh Compression (V-Mesh) standard of the Moving Picture Experts Group (MPEG) or the next-generation video / image coding standard. Dynamic mesh video compression is a method for processing mesh connection information and properties that change over time, and it can perform lossy and lossless compression for various applications such as real-time communication, storage, free-view video, and AR / VR.

[0076] The dynamic mesh video compression method described below is based on MPEG's V-Mesh method.

[0077] In this document, picture / frame can generally mean a unit representing one video of a specific time period.

[0078] A pixel or pel can refer to the smallest unit that constitutes a picture (or image). Additionally, the term "sample" can be used as a counterpart to a pixel. A sample can generally represent a pixel or a pixel value, and can represent only the pixel / pixel value of the luma component, only the pixel / pixel value of the chroma component, or only the pixel / pixel value of the depth component.

[0079] A unit may represent a basic unit of image processing. A unit may include at least one of a specific region of a picture and information related to the region. In some cases, the term "unit" may be used interchangeably with terms such as "block" or "area." In general, an MxN block may include a set (or array) of samples (or sample array) or transform coefficients consisting of M columns and N rows.

[0080] The encoding process of Fig. 1 is as follows.

[0081] Video-based dynamic mesh compression (V-Mesh) compression methods can provide a method for compressing dynamic mesh video data based on 2D video codecs such as HEVC and VVC. The V-Mesh compression process receives the following data as input and performs compression.

[0082] Input mesh: Contains the 3D coordinates (geometry) of the vertices that make up the mesh, normal information for each vertex, mapping information that maps the mesh surface to a 2D plane, and connection information between the vertices that make up the surface. The mesh surface can be expressed as triangles or more polygons, and connection information between the vertices that make up each surface is stored according to a set shape. The input mesh can be saved in the OBJ file format.

[0083] Attribute map: (Hereinafter, texture map is also used in the same meaning): Contains information about the properties (color, normal, displacement, etc.) of the mesh, and stores data in the form of a mapping of the surface of the mesh onto a 2D image. Mapping which part (surface or vertex) of the mesh each data of this attribute map corresponds to is based on the mapping information contained in the input mesh. Since the attribute map has data for each frame of the mesh video, it can also be expressed as an attribute map video (or attribute for short). The attribute map in the V-Mesh compression method mainly contains the color information of the mesh, and is saved in an image file format (PNG, BMP, etc.).

[0084] Material Library File: Contains information about the material properties used in a mesh, particularly information that links the input mesh to its corresponding attribute map. It is saved in the Wavefront Material Template Library (MTL) file format.

[0085] In the V-Mesh compression method, the following data and information can be generated through the compression process.

[0086] Base mesh: The input mesh is simplified (decimated) through a preprocessing process, and the objects of the input mesh are expressed using the minimum number of vertices determined by the user's standards.

[0087] Displacement: This is displacement information used to express the input mesh as similarly as possible to the base mesh, and is expressed in the form of 3D coordinates.

[0088] Atlas information: This is the metadata required to reconstruct a mesh using base mesh, displacement, and attribute map information. It can be created and utilized as sub-mesh units (such as patches) that make up the mesh.

[0089] Referring to FIGS. 2 to 7, a method for encoding mesh position information (vertex) is described, and referring to FIGS. 6-10, etc., a method for encoding attribute information (attribute map) by restoring mesh position information is described.

[0090] Figure 2 illustrates a V-MESH compression method according to embodiments.

[0091] Fig. 2 illustrates the encoding process of Fig. 1, and the encoding process may include a pre-processing process and an encoding process. The encoder of Fig. 1 may include a pre-processor (200) and an encoder (201) as in Fig. 2. The transmitting device of Fig. 1 may be broadly referred to as an encoder, and the dynamic mesh video encoder of Fig. 1 may be referred to as an encoder. The V-Mesh compression method may include a pre-processing (200) and an encoding (201) process as in Fig. 2. The pre-processor of Fig. 2 may be located in front of the encoder of Fig. 2. The pre-processor and the encoder of Fig. 2 may be referred to as a single encoder.

[0092] The preprocessor can receive a static dynamic mesh and / or an attribute map. The preprocessor can generate a base mesh and / or displacement through preprocessing. The preprocessor can receive feedback information from the encoder and generate the base mesh and / or displacement based on the feedback information.

[0093] The encoder can receive a base mesh, displacement mesh, static dynamic mesh, and / or attribute map. The encoder can encode mesh-related data to generate a compressed bitstream.

[0094] Figure 3 illustrates pre-processing of V-MESH compression according to embodiments.

[0095] Figure 3 shows the configuration and operation of the pre-processor of Figure 2.

[0096] Fig. 3 shows a process of performing preprocessing on an input mesh. The preprocessing process (200) can be broadly divided into four steps: 1) Group of Frame (GoF) generation, 2) Mesh Decimation, 3) UV parameterization, and 4) Fitting subdivision surface (300). The preprocessor (200) can receive an input mesh, generate a displacement and / or base mesh, and transmit the generated displacement and / or base mesh to the encoder (201). The preprocessor (200) can transmit GoF information related to GoF generation to the encoder (201).

[0097] Below, each step of Figure 3 is described.

[0098] GoF Generation: This is the process of generating a reference structure for mesh data. If the number of vertices, number of texture coordinates, vertex connection information, and texture coordinate connection information of the mesh of the previous frame and the current mesh are all the same, the previous frame can be set as the reference frame. In other words, if only the vertex coordinate values ​​are different between the current input mesh and the reference input mesh, inter-frame encoding can be performed. Otherwise, the frame performs intra-frame encoding.

[0099] Mesh Decimation: This process simplifies the input mesh to create a simplified mesh, or base mesh. Vertices to be removed from the original mesh are selected based on user-defined criteria, and the selected vertices and the triangles connected to them can be removed.

[0100] In the process of performing mesh simplification (Mesh decimation), the input mesh (voxelized), target triangle ratio (TTR), and minimum triangle component (CCCount) information are passed as input, and the simplified mesh (decimated mesh) can be obtained as output. In this process, connected triangle components smaller than the set minimum triangle component (CCCount) can be removed.

[0101] UV parameterization: This is the process of mapping a 3D surface of a decimated mesh into a texture domain. Parameterization can be performed using the UVAtlas tool. This process generates mapping information, which indicates where each vertex of the decimated mesh can be mapped to on a 2D image. This mapping information is expressed and stored as texture coordinates, and through this process, the final base mesh is created.

[0102] Fitting subdivision surface: This is the process of performing subdivision on a simplified mesh. The subdivision method can be a user-defined method, such as the mid-edge method. The fitting process ensures that the input mesh and the subdivision mesh are similar to each other.

[0103] Figure 4 illustrates a mid-edge subdivision method according to embodiments.

[0104] Figure 4 illustrates the mid-edge method of the fitting subdivision surface described in Figure 3. Referring to Figure 4, an original mesh containing four vertices is subdivided to create a sub-mesh. A sub-mesh can be created by creating a new vertex midway between the edges between the vertices.

[0105] When a fitted subdivided mesh (hereinafter referred to as a fitted subdivided mesh) is generated, displacement is calculated using this result and a pre-compressed and decrypted base mesh (hereinafter referred to as a reconstructed base mesh). That is, the reconstructed base mesh is subdivided in the same way as the fitting subdivision surface. The difference in position of each vertex between this result and the fitted subdivided mesh is the displacement for each vertex. Since displacement represents the position difference in three-dimensional space, it is also expressed as a value in the (x, y, z) space of a Cartesian coordinate system. Depending on the user input parameters, the (x, y, z) coordinate values ​​can be converted to (normal, tangential, bi-tangential) coordinate values ​​of the local coordinate system.

[0106] Figure 5 shows a displacement generation process according to embodiments.

[0107] FIG. 5 illustrates in detail the displacement calculation method of the fitting subdivision surface (300) as described in FIG. 4.

[0108] An encoder and / or pre-processor according to embodiments may include 1) a subdivision unit, 2) a local coordinate system calculation unit, and 3) a displacement calculation unit. The subdivision unit may receive a reconstructed base mesh and generate a subdivided reconstructed base mesh. The local coordinate system calculation unit may receive a fitted subdivision mesh and a subdivided reconstructed base mesh, and transform a coordinate system of the mesh into a local coordinate system. The local coordinate system calculation operation may be optional. The displacement calculation unit may calculate a positional difference between the fitted subdivision mesh and the subdivided reconstructed base mesh. For example, a positional difference value between vertices of two input meshes may be generated. The vertex positional difference value becomes a displacement.

[0109] The method and device for transmitting point cloud data according to the embodiments can encode the point cloud as follows. The point cloud data (which may be referred to as a point cloud for short) according to the embodiments can refer to data including vertex coordinates and color information. The term "point cloud" includes mesh data, and in this document, point cloud and mesh data can be used interchangeably.

[0110] The V-Mesh compression (reconstruction) method according to the embodiments may include intra frame encoding (Fig. 6) and inter frame encoding (Fig. 7).

[0111] Based on the results of the GoF generation described above, intra-frame encoding or inter-frame encoding is performed. In the case of intra-encoding, the data to be compressed may be a base mesh, displacement, attribute map, etc. In the case of inter-encoding, the data to be compressed may be a displacement, attribute map, and a motion field between a reference base mesh and the current base mesh.

[0112] FIG. 6 illustrates an intra-frame encoding process of a V-MESH compression method according to embodiments.

[0113] The encoding process of Fig. 6 details the encoding of Fig. 1. That is, it shows the configuration of an encoder when the encoding of Fig. 1 is an intra-frame method. The encoder of Fig. 6 may include a preprocessor (200) and / or an encoder (201).

[0114] The preprocessor can receive an input mesh and perform the preprocessing described above. The preprocessing can generate a base mesh and / or a fitted subdivided mesh. The quantizer can quantize the base mesh and / or the fitted subdivided mesh. The static mesh encoder can encode the static mesh. The static mesh encoder can generate a bitstream including the encoded base mesh. The static mesh decoder can decode the encoded static mesh. The inverse quantizer can inversely quantize the quantized static mesh. The displacement calculation unit can receive the reconstructed static mesh and generate displacement, which is a position difference, based on the fitted subdivided mesh. The forward linear lifting unit can receive the displacement and generate lifting coefficients. The quantizer can quantize the lifting coefficients. The image packing unit can pack an image based on the quantized lifting coefficients. A video encoder can encode a packed image. A video decoder decodes the encoded video. An image unpacker can unpack a packed image. A dequantizer can inversely quantize an image. An inverse linear lifting unit applies inverse lifting to the image to generate a reconstructed displacement. A mesh restoration unit restores a warped mesh using the reconstructed displacement and the reconstructed base mesh. An attribute transfer unit receives an input mesh and / or an input attribute map, and generates an attribute map based on the reconstructed warped mesh. A push-pull padding unit can pad data in the attribute map based on a push-pull method. A color space transformation unit can transform the space of a color component, which is an attribute. A video encoder can encode an attribute. A multiplexer can generate a bitstream by multiplexing a compressed base mesh, compressed displacement, and compressed attributes.

[0115] Figure 7 illustrates an inter-frame encoding process of a V-MESH compression method according to embodiments.

[0116] The encoding process of Fig. 7 details the encoding of Fig. 1. That is, it shows the configuration of an encoder when the encoding of Fig. 1 is an inter-frame method. The encoder of Fig. 7 may include a preprocessor (200) and / or an encoder (201).

[0117] Among the encoding operations of Fig. 7, the corresponding configuration of the encoding operation of Fig. 6 refers to the description of Fig. 7. For the inter-frame-based encoding of Fig. 7, the motion encoder can encode motion based on the restored quantized reference base mesh. The base mesh restoration unit can restore the base mesh based on the restored quantized reference base mesh.

[0118] The encoder of Fig. 6 generates a bitstream by compressing the base mesh, displacement, and attributes within the frame, and the encoder of Fig. 7 generates a bitstream by compressing the motion, displacement, and attributes between the current frame and the reference frame.

[0119] The encoding method according to the embodiments includes base mesh encoding (intra encoding). When performing intra frame encoding on the current input mesh frame, the base mesh generated in the preprocessing process can be encoded using a static mesh compression technique after going through a quantization process. In the V-Mesh compression method, for example, Draco technology is applied, and the vertex position information, mapping information (texture coordinates), vertex connection information, etc. of the base mesh are compressed.

[0120] The encoding method according to the embodiments may include motion field encoding (inter encoding). Inter frame encoding may be performed when a one-to-one correspondence of vertices is established between a reference mesh and a current input mesh, and only the position information of the vertices is different. When performing inter frame encoding, instead of compressing the base mesh, the difference between the vertices of the reference base mesh and the current base mesh, i.e., the motion field, may be calculated and this information may be encoded. The reference base mesh is the result of quantizing the already decoded base mesh data and is determined according to the reference frame index determined in the GoF generation. The motion field may be encoded as a value. Alternatively, the predicted motion field may be calculated by averaging the motion fields of the restored vertices among the vertices connected to the current vertex, and this predicted motion field The residual motion field, which is the difference between the value and the motion field value of the current vertex, can be encoded. This value can be encoded using entropy coding. The process of encoding the displacement and attribute map, excluding the motion field encoding process of inter-frame encoding, is the same as the structure of the intra-frame encoding method except for the base mesh encoding.

[0121] Figure 8 shows a lifting conversion process for displacement according to embodiments.

[0122] Figure 9 illustrates a process of packing transformation coefficients according to embodiments into a 2D image.

[0123] Figures 8-9 show the process of converting the displacement of the encoding process of Figures 6-7 and the process of packing the conversion coefficients, respectively.

[0124] The encoding method according to the embodiments includes displacement encoding.

[0125] After encoding the base mesh through base mesh encoding and / or motion field encoding, a reconstructed base mesh is generated through restoration and dequantization, and the displacement between the result of performing subdivision on the reconstructed base mesh and the fitted subdivided mesh generated through the fitting subdivision surface can be calculated. For effective encoding, a data transform process such as wavelet transform can be applied to the displacement information.

[0126] Figure 8 shows the process of transforming displacement information using lifting transform in V-Mesh. The transform coefficients generated through the transform process are quantized and then packed into a 2D image as shown in Figure 9. The transform coefficients are organized into one block for each 256 (=16×16) units, and each block can be packed in z-scan order. The horizontal number of blocks is fixed to 16, but the vertical number of blocks can be determined according to the number of vertices of the subdivided base mesh. The transform coefficients can be packed by sorting them with Morton code within a block. The packed images generate a displacement video for each GoF unit, and this displacement video can be encoded using an existing video compression codec.

[0127] Referring to FIG. 8, the base mesh (original) may include vertices and edges for LoD0. The first subdivision mesh generated by dividing the base mesh includes vertices generated by further dividing the edges of the base mesh. The first subdivision mesh includes vertices for LoD0 and vertices for LoD1. LoD1 includes the subdivided vertices and the vertices (LoD0) of the base mesh. The first subdivision mesh may be generated by dividing the second subdivision mesh. The second subdivision mesh includes LoD2. LoD2 includes the base mesh vertices (LoD0), LoD1 including the vertices additionally generated from LoD0, and the vertices additionally divided from LoD1. LoD is a level indicating the degree of detail (Level of Detail), and as the level index increases, the distance between vertices becomes closer and the level of detail increases. LoD N includes the vertices included in the previous LoDN-1 as they are. When a vertex is further divided through subdivision, considering the previous vertices v1, v2 and the subdivided vertex v, the mesh can be encoded based on a prediction and / or update method. Instead of still encoding information about the current LoD N, a residual value between the previous LoD N-1 can be generated, and the mesh can be encoded using the residual value to reduce the size of the bitstream. The prediction process means the operation of predicting the current vertex v using the previous vertices v1, v2. Since adjacent subdivision meshes have similar data, efficient encoding can be achieved by utilizing this property. The current vertex position information is predicted as the residual for the previous vertex position information, and the previous vertex position information is updated using the residual.

[0128] Referring to Figure 9, the vertices have coefficients generated through the lifting transformation. The coefficients of the vertices related to the lifting transformation can be encoded by packing them into an image.

[0129] Fig. 10 shows an attribute transfer process of a V-MESH compression method according to embodiments.

[0130] Figure 10 shows the detailed operation of attribute transfer of encoding such as Figures 6-7.

[0131] Encoding according to embodiments includes attribute map encoding.

[0132] Information about the input mesh is compressed through base mesh encoding, motion field encoding, and displacement encoding. In the encoding process, the compressed input mesh is restored through base mesh decoding (intra frame), motion field decoding (inter frame), and displacement video decoding processes, and the restored result, the reconstructed deformed mesh (hereinafter referred to as Recon. deformed mesh), is used to compress the input attribute map as shown in FIGS. 6 and 7. The reconstructed deformed mesh (Recon. deformed mesh) has vertex position information, texture coordinates, and corresponding connection information, but does not have color information corresponding to the texture coordinates. Therefore, as shown in Fig. 10, in the V-Mesh compression method, a new attribute map having color information corresponding to the texture coordinates of the reconstructed deformed mesh is created through the attribute transfer process.

[0133] Attribute transfer first checks whether each point P(u, v) in the 2D texture domain belongs to a texture triangle of the reconstructed deformed mesh, and if it is in the texture triangle T, calculates the barycentric coordinate (α, β γ) of P(u, v) according to the triangle T. Then, using the 3D vertex position and (α, β γ) of triangle T, calculate the 3D coordinate M(x, y, z) of P(u, v). Find the vertex coordinate M'(x', y', z') that corresponds to the position most similar to the calculated M(x, y, z) in the input mesh domain and the triangle T' that contains this point. And in this triangle T', the center of mass coordinates (α', β', γ') of M'(x', y', z') are calculated. Using the texture coordinates corresponding to the three vertices of triangle T' and (α', β', γ'), the texture coordinates (u', v') are calculated, and the color information corresponding to these coordinates is found in the input attribute map. The color information found in this way is immediately assigned to the pixel location (u, v) of the new attribute map. If P(u, v) does not belong to any triangle, the pixel at that location in the new attribute map can be filled with a color value using a padding algorithm such as the push-pull algorithm.

[0134] The new attribute map generated through attribute transfer is grouped into GoF units to form an attribute map video, which is then compressed using a video codec.

[0135] Referring to Figure 10, the reference relationship between the input mesh, input attribute map, restored mesh, and generated attribute map can be seen.

[0136] The decoding process of Fig. 1 can perform the reverse process of the corresponding encoding process of Fig. 1. The specific decoding process is as follows.

[0137] Fig. 11 illustrates an intra-frame decoding process of a V-MESH compression method according to embodiments.

[0138] Fig. 11 shows the configuration and operation of the decoder of the receiving device of Fig. 1.

[0139] Figure 11 illustrates the intra decoding process of V-Mesh technology. First, the input bitstream can be separated into a mesh sub-stream, a displacement sub-stream, an attribute map sub-stream, and a sub-stream containing mesh patch information, such as V3C / V-PCC.

[0140] The mesh sub-stream is decoded by the decoder of the static mesh codec used in encoding, such as Google Draco, and as a result, the connection information, vertex geometry information, vertex texture coordinates, etc. of the base mesh can be restored. The displacement sub-stream is decoded into a displacement video by the decoder of the video compression codec used in encoding, and goes through the image unpacking, inverse quantization, and inverse transform processes to restore displacement information for each vertex. Inverse quantization is applied to the restored base mesh, and this result is combined with the restored displacement information to generate the final decoded mesh.

[0141] The attribute map sub-stream is decoded through the decoder of the video compression codec used in encoding, and then restored to the final attribute map through processes such as color format conversion.

[0142] The restored decoded mesh and decoded attribute map can be utilized by the receiver as final mesh data that can be utilized by the user.

[0143] Referring to FIG. 11, the bitstream includes patch information, a mesh substream, a displacement substream, and an attribute map substream. The term "substream" is interpreted as referring to a portion of the bitstream included in the bitstream. The bitstream includes patch information (data), mesh information (data), displacement information (data), and attribute map information (data).

[0144] The decoder performs the following decoding operations within the frame. The static mesh decoder decodes the mesh to generate a reconstructed quantized base mesh, and the inverse quantizer applies the quantization parameters of the quantizer inversely to generate the reconstructed base mesh. The video decoder decodes the displacement, the unpacker unpacks the decoded video image, and the inverse quantizer inversely quantizes the quantized image. The linear lifting inverse transform applies a lifting transform in the reverse process of the encoder to generate the reconstructed displacement. The mesh restoration unit generates a warped mesh based on the base mesh and the displacement. The video decoder decodes the attribute map, and the color transformation unit transforms the color format and / or space to generate the decoded attribute map.

[0145] Figure 12 shows the inter-frame decoding process of the V-MESH compression method.

[0146] Fig. 12 shows the configuration and operation of the decoder of the receiving device of Fig. 1.

[0147] Figure 12 illustrates the inter-decoding process of V-Mesh technology. First, the input bitstream can be separated into a motion sub-stream, a displacement sub-stream, an attribute sub-stream, and a sub-stream containing mesh patch information, such as V3C / V-PCC.

[0148] The motion sub-stream is decoded through entropy decoding and inverse prediction processes, and the reconstructed motion information is combined with the already reconstructed reference base mesh to generate a reconstructed quantized base mesh for the current frame. The result of applying inverse quantization to this is combined with the displacement information reconstructed in the same way as the intra decoding described above to generate the final decoded mesh. The attribute map sub-stream is decoded in the same way as the intra decoding. The reconstructed decoded mesh and the decoded attribute map can be utilized by the receiver as the final mesh data that can be utilized by the user.

[0149] Referring to Fig. 12, the bitstream includes motion, displacement, and attribute maps. Since inter-frame decoding is performed, a process of decoding inter-frame motion information is further included. The motion is decoded, and a restored quantized base mesh for the motion is generated based on the reference base mesh, thereby generating a restored base mesh. For a description of the operations in Fig. 12, which are identical to those in Fig. 11, refer to the description in Fig. 11.

[0150] Fig. 13 shows a point cloud data transmission device according to embodiments.

[0151] Fig. 13 corresponds to the transmitting device (100), the dynamic mesh video encoder (102), the Fig. 2 encoder (preprocessor and encoder), and / or the transmitting encoding device corresponding thereto of Fig. 1. Each component of Fig. 13 corresponds to hardware, software, a processor, and / or a combination thereof.

[0152] The operation process of a transmitter for compressing and transmitting dynamic mesh data using V-Mesh compression technology may be as shown in Fig. 13.

[0153] The mesh preprocessor receives the original mesh as input and generates a simplified mesh (decimated mesh). Simplification can be performed based on the target number of vertices or the target number of polygons that constitute the mesh. Parameterization can be performed on the simplified mesh to generate texture coordinates and texture connection information per vertex. Additionally, quantization of floating-point mesh information into fixed-point information can be performed. This result can be encoded as a base mesh through a static mesh encoding unit. The mesh preprocessor can perform mesh subdivision on the base mesh to generate additional vertices. Depending on the subdivision method, vertex connection information, texture coordinates, and texture coordinate connection information including the added vertices can be generated. The subdivided mesh can be fitted by adjusting the vertex positions to resemble the original mesh, thereby generating a fitted subdivided mesh.

[0154] When performing intra-encoding on the corresponding mesh frame, the base mesh generated through the mesh preprocessing unit can be compressed through the static mesh encoding unit. In this case, encoding can be performed on the connection information, vertex geometry information, vertex texture information, normal information, etc. of the base mesh. The base mesh bitstream generated through encoding is transmitted to the multiplexing unit.

[0155] When performing inter encoding for the corresponding mesh frame, a motion vector encoding unit is performed, which can calculate a motion vector between the base mesh and the reference reconstruction base mesh as input and encode the value. The motion vector encoding unit can perform prediction based on connection information using a previously encoded / decoded motion vector as a predictor, and encode a residual motion vector obtained by subtracting the predicted motion vector from the current motion vector. The motion vector bitstream generated through encoding is transmitted to the multiplexing unit.

[0156] The encoded base mesh and motion vectors can be used to generate a restored base mesh through the base mesh restoration unit.

[0157] The displacement vector calculator can perform mesh refinement on the restored base mesh. The displacement vector can be calculated as the difference in vertex positions between the refined restored base mesh and the fitted subdivision mesh generated in the preprocessing unit. As a result, the number of displacement vectors can be calculated equal to the number of vertices in the refined mesh. The displacement vector calculation unit can convert the displacement vector calculated in the 3D Cartesian coordinate system into a local coordinate system based on the normal vector of each vertex.

[0158] A displacement vector video generator can transform displacement vectors for effective encoding. The transformation can be performed by a lifting transformation, a wavelet transformation, etc., depending on the embodiment. In addition, quantization can be performed on the transformed displacement vector values, i.e., the transform coefficients. Different quantization parameters can be applied to each axis of the transform coefficients, and the quantization parameters can be derived according to the promise of the encoder / decoder. The transformed and quantized displacement vector information can be packed into a 2D image. A displacement vector video can be generated by bundling the packed 2D images for each frame, and the displacement vector video can be generated for each GoF (Group of Frame) unit of the input mesh.

[0159] A displacement vector video encoder can encode the generated displacement vector video using a video compression codec. The generated displacement vector video bitstream is transmitted to a multiplexer.

[0160] The displacement vector restored through the displacement vector restorer and the base mesh restored through the base mesh restorer and refined are restored through the mesh restorer, and the restored mesh has restored vertices, connection information between vertices, texture coordinates, and connection information between texture coordinates.

[0161] The texture map of the original mesh can be regenerated as a texture map for the restored mesh through the texture map video generation unit. The color information per vertex of the texture map of the original mesh can be assigned to the texture coordinates of the restored mesh. The regenerated texture maps for each frame can be bundled into GoF units to generate a texture map video.

[0162] The generated texture map video can be encoded using a video compression codec through a texture map video encoding unit. The texture map video bitstream generated through encoding is transmitted to a multiplexing unit.

[0163] The generated motion vector bitstream, base mesh bitstream, displacement vector bitstream, and texture map bitstream may be multiplexed into a single bitstream and transmitted to a receiver via a transmitter. Alternatively, the generated motion vector bitstream, base mesh bitstream, displacement vector bitstream, and texture map bitstream may be generated as a file with one or more track data or encapsulated into segments and transmitted to a receiver via a transmitter.

[0164] Referring to FIG. 13, a transmitting device (encoder) can encode a mesh in an intra-frame or inter-frame manner. A transmitting device according to intra-encoding can generate a base mesh, a displacement vector (displacement), and a texture map (attribute map). A transmitting device according to inter-encoding can generate a motion vector (motion), a base mesh, a displacement vector (displacement), and a texture map (attribute map). A texture map obtained from a data input unit is generated and encoded based on a restored mesh. Displacement is generated and encoded through the difference in vertex positions between the base mesh and the segmented mesh. The base mesh is generated by preprocessing, simplifying, and encoding the original mesh. Motion is generated as a motion vector for the mesh of the current frame based on the reference base mesh of the previous frame.

[0165] Fig. 14 shows a point cloud data receiving device according to embodiments.

[0166] Fig. 14 corresponds to the receiving device (110), the dynamic mesh video decoder (113), the decoder of Figs. 11-12, and / or the receiving decoding device corresponding thereto of Fig. 1. Each component of Fig. 14 corresponds to hardware, software, a processor, and / or a combination thereof. The receiving (decoding) operation of Fig. 14 may follow the reverse process of the corresponding process of the transmitting (encoding) operation of Fig. 13.

[0167] The bitstream of the received Mesh is demultiplexed into a compressed motion vector bitstream or base mesh bitstream, displacement vector bitstream, and texture map bitstream after file / segment decapsulation.

[0168] If the current mesh has inter-frame encoding applied according to the frame header information, the motion vector decoding unit can perform decoding on the motion vector bitstream. The final motion vector can be reconstructed by adding the previously decoded motion vector to the residual motion vector decoded from the bitstream using the previously decoded motion vector as a predictor.

[0169] If the current mesh has been encoded within the screen according to the frame header information, the base mesh bitstream can restore the connection information, vertex geometry information, texture coordinates, normal information, etc. of the base mesh through the static mesh encoding unit.

[0170] In the base mesh restoration unit, if the current mesh has inter-frame encoding applied, the current base mesh can be restored by adding the decoded motion vector to the reference base mesh and then performing inverse quantization. If the current mesh has intra-frame encoding applied, the static mesh decoding unit can perform inverse quantization on the decoded mesh to generate a restored base mesh.

[0171] The displacement vector bitstream can be decoded as a video bitstream using a video codec in a displacement vector video decoding unit.

[0172] In the displacement vector restoration unit, displacement vector transformation coefficients are extracted from the decoded displacement vector video, and the displacement vector is restored through the inverse quantization and inverse transformation processes. If the restored displacement vector is a value in the local coordinate system, a process of inverse transformation to the Cartesian coordinate system can be performed.

[0173] The mesh restoration unit can generate additional vertices by performing subdivision on the restored base mesh. Subdivision can generate vertex connection information, texture coordinates, and texture coordinate connection information, including the added vertices. The subdivided restored base mesh can be combined with the restored displacement vector to generate the final restored mesh.

[0174] The texture map bitstream can be decoded as a video bitstream using a video codec in a texture map video decoding unit. The restored texture map contains color information for each vertex contained in the restored mesh, and the color value of each vertex can be obtained from the texture map using the texture coordinates of the corresponding vertex.

[0175] The restored mesh and texture map are displayed to the user through a rendering process using a mesh data renderer, etc.

[0176] Referring to FIG. 14, a receiving device (decoder) can decode a mesh in an intra-frame or inter-frame manner. A receiving device according to intra-decoding can receive a base mesh, a displacement vector (displacement), and a texture map (attribute map), and decode the restored mesh and the restored texture map to render the mesh data. A receiving device according to inter-decoding can receive a motion vector (motion), a base mesh, a displacement vector (displacement), and a texture map (attribute map), and decode the restored mesh and the restored texture map to render the mesh data.

[0177] The point cloud data transmission device and method according to the embodiments can encode mesh data and transmit a bitstream including the encoded mesh data. The point cloud data reception device and method according to the embodiments can receive a bitstream including mesh data and decode the mesh data. The point cloud data transmission and reception method / device according to the embodiments may be abbreviated as the method / device according to the embodiments. The point cloud data transmission and reception method / device according to the embodiments may also be referred to as the mesh data transmission and reception method / device according to the embodiments.

[0178] The mesh data encoding method / device according to the embodiments may refer to the transmitting device (100) of FIG. 1, the obtaining unit (101), the encoder (102), the encapsulator (103), the transmitter (104), the preprocessor (200) of FIG. 2, the encoder (201), the encoder of FIG. 6-7, the encoder of FIG. 13, the bitstream generation of FIG. 15 to FIG. 31, the transmitting device of FIG. 32, the encoding method of FIG. 36, etc.

[0179] The mesh data decoding method / device according to the embodiments may refer to the receiving device (110) of FIG. 1, the receiver (111), the decapsulator (112), the decoder (113), the renderer (114), the decoder of FIG. 11-12, the decoder of FIG. 14, the bitstream parsing of FIG. 15 to FIG. 31, the receiving device of FIG. 32, the receiving device of FIG. 35, the decoding method of FIG. 37, etc.

[0180] Figure 15 illustrates a video-based dynamic mesh coding bitstream structure according to embodiments.

[0181] The method and apparatus according to the embodiments can include and perform a method of storing and transmitting a multiple track file of a dynamic mesh coding bitstream (Carriage of Dynamic Mesh Coding Bitstream in Multiple Track).

[0182] Embodiments may include a method of storing and signaling a bitstream as multiple tracks within a file by video-based dynamic mesh coding.

[0183] Embodiments include a transmitter or receiver for providing a mesh content service that efficiently stores and signalizes a video-based dynamic mesh coding (V-DMC) or mesh bitstream within multiple tracks within a file.

[0184] Embodiments include a transmitter or receiver for providing mesh content services that handles file storage techniques to enable efficient access to V-DMC bitstreams stored within files. The transmitter according to the embodiments may be referred to as an encoder, and the receiver may be referred to as a decoder.

[0185] Although the embodiments describe video-based dynamic mesh coding, the present disclosure may be applied to bitstreams using other mesh codings constructed in the same manner or in a similar bitstream format as video-based dynamic mesh coding.

[0186] Mesh content encoded with V-DMC can be encapsulated into multi-tracks based on ISOBMFF files to provide services such as streaming. In this regard, embodiments may include information on the type and entry point signaling of multiple tracks, syntax and semantics information of sample entries within multiple tracks, syntax and semantics information of sample formats within multiple tracks, and track reference signaling information between multiple tracks.

[0187] Figure 15 shows an example of a bitstream structure including encoded mesh data.

[0188] Referring to Fig. 15a, the sequence header of the bitstream may include decoder configuration information. The encoded base mesh of the bitstream is generated by encoding the input mesh file. The encoding method may include Edgebreaker or Draco. The encoded displacement (which may be referred to as displacement information or geometry, etc.) of the bitstream refers to the difference information between vertices between the reconstructed subdivision meshes after fitting and subdividing the base mesh. It is generated by packing the difference information between vertices into a 2D image and encoding the packed image based on a video method. The video method may include HEVC, etc. Alternatively, the displacement may be generated by encoding the packed image based on an arithmetic coding method. The encoded attributes (or textures) of the bitstream are generated by generating and encoding new 2D attributes (color information, PNG images) that match the coordinates of the attributes (textures, 2D) of the reconstructed subdivision mesh using the input attribute map (e.g., PNG images). The encoding method may include HEVC, etc.

[0189] The bitstream according to the embodiments may be a bitstream resulting from video-based dynamic mesh encoding. As described above, the bitstream may include encoded base mesh data, displacement data, texture, or attribute data, respectively.

[0190] A bitstream according to embodiments may be generated as illustrated in FIG. 15b. The bitstream may include a sample stream header and a sample stream unit. The unit may be a DMC unit. The header and the unit may be referred to as a sample stream DMC header, a sample stream DMC unit, etc., respectively.

[0191] The sample stream DMC unit according to the embodiments includes the following information:

[0192] VPS (V-DMC Parameter Set): May include decoder configuration information related to mesh encoding / decoding and parameter set information such as sequence number, etc.

[0193] AD (Atlas Data): May contain information related to 2D mapping or texture mapping for 3D objects.

[0194] BMD (Base Mesh Data): Encoded base mesh information for mesh coding / decoding.

[0195] DD (Displacement Data): Displacement information encoded using arithmetic coding. However, in the case of lossless encoding, displacement data may not be included.

[0196] GVD (Displacements Video Data): Displacement information encoded with a video codec.

[0197] AVD (Attribute Video Data): Attribute or texture information encoded with a video codec.

[0198] A bitstream according to embodiments may include a V3C unit.

[0199] The syntax of the V3C unit is as follows:

[0200] v3c_unit( numBytesInV3CUnit ) {Descriptorv3c_unit_header( )v3c_unit_payload( numBytesInV3CUnit - 4)}

[0201] Figure 16 illustrates a V3C unit header according to embodiments.

[0202] Figure 17 shows a V3C unit type according to embodiments.

[0203] A V3C unit may include a V3C parameter set, atlas data, accumulator video data, geometry video data, attribute video data, packed video data, common atlas data, etc. As shown in Fig. 17, each value of the type of the V3C unit may represent each V3C unit. Although not shown in Fig. 17, the V3C unit may further include BMD (Base Mesh Data: compressed base mesh information for mesh encoding / decoding), DD (Displacements Data: displacement information compressed by arithmetic coding), etc.

[0204] The V3C parameter set represents V3C level parameters. Atlas data represents atlas information. Accuracy video data represents accuracy information. Geometry video data represents geometry information. Attribute video data represents attribute information. Packed video data represents packing information. Common atlas data represents common information for the atlas. For example, it can be atlas information defined in the CVS described in ISO / IEC 23090-12.

[0205] As shown in FIG. 16, the V3C unit header may include a V3C unit header type (vuh_unit_type) indicating the type of the unit header, a V3C parameter set ID (vuh_v3c_parameter_set_id) for the V3C unit header identifying a parameter set for the V3C unit, an atlas ID (vuh_atlas_id) for the V3C unit header identifying an atlas for the V3C unit, an attribute index (vuh_attribute_index) for the V3C unit header indicating an attribute for the V3C unit, an attribute partition index (vuh_attribute_partition_index) indicating a partition of an attribute for the V3C unit, a map index (vuh_map_index) for the V3C unit, an auxiliary video flag (vuh_auxiliary_video_flag) indicating whether an auxiliary video for the V3C unit exists, etc.

[0206] The semantics of each element in Figures 16-17 are as follows.

[0207] vuh_unit_type indicates the V3C unit type.

[0208] vuh_v3c_parameter_set_id represents the vps_v3c_parameter_set_id value for the active V3C VPS.

[0209] vuh_atlas_id represents the ID of the atlas corresponding to the current V3C unit.

[0210] vuh_attribute_index indicates the index of the attribute data included in the attribute video data unit.

[0211] vuh_attribute_partition_index indicates the index of the attribute dimension group included in the attribute video data unit.

[0212] If vuh_map_index is present, it indicates the map index of the current geometry or property stream.

[0213] A vuh_auxiliary_video_flag equal to 1 indicates that the associated geometry or attribute video data unit is a sub-bitstream dedicated to RAW and / or EOM code points. A vuh_auxiliary_video_flag equal to 0 indicates that the associated geometry or attribute video data unit may contain RAW and / or EOM code points.

[0214] Figure 18 illustrates a V3C unit payload according to embodiments.

[0215] A V3C unit payload can contain a V3C parameter set (v3c_parameter_set), an atlas sub-bitstream (atlas_sub_bitstream), a basemesh sub-bitstream (bmesh_sub_bitstream), a video sub-bitstream, etc.

[0216] The semantics of each element contained in the V3C unit payload are as follows.

[0217] v3c_parameter_set may be parameter set information that includes information about dynamic mesh bits, such as profiles, sub-bitstreams for each type, etc., such as ISO / IEC 23090-29.

[0218] atlas_sub_bitstream can be a sub-bitstream containing atlas data, such as ISO / IEC 23090-29.

[0219] bmesh_sub_bitstream can be a sub-bitstream containing basemesh data, such as ISO / IEC 23090-29.

[0220] The format of a V-DMC bitstream can be defined by the syntax and semantics of the sample stream DMC header and the sample stream DMC unit as follows.

[0221] The sample stream data syntax and semantics can be in the sample stream format defined in ISO / IEC 23090-5, i.e., the V3C codec standard, which is referenced in the V-DMC standard. Furthermore, it can be flexibly implemented based on a bitstream format newly defined in the V-DMC or V3C codec.

[0222] The syntax of the sample stream DMC header is as follows:

[0223] sample_stream_dmc_header() {Descriptorsdmh_unit_size_precision_bytes_minus1u(3)sdmh_reserved_zero_5bitsu(5)}

[0224] Sample Stream Header Precision (sdmh_unit_size_precision_bytes_minus1): This value plus 1 represents the precision (in bytes) of sdmu_dmc_unit_size elements in every sample stream DMC unit. The range for sdmh_unit_size_precision_bytes_minus1 is 0 to 7.

[0225] The syntax of a sample stream DMC unit is as follows. Each sample stream DMC unit contains one type of DMC unit among VPS, AD, BMD, DD, GVD, and AVD. The contents of each sample stream DMC unit are associated with the same access unit as the DMC unit contained in the sample stream DMC unit.

[0226] sample_stream_dmc_unit() {Descriptorsdmu_dmc_unit_sizeu(v)dmc_unit(sdmu_dmc_unit_size )}

[0227] The sample stream unit size (sdmu_dmc_unit_size) represents the size (in bytes) of the subsequent dmc_unit. The number of bits used to represent sdmu_dmc_unit_size is equal to (sdmh_unit_size_precision_bytes_minus1 + 1) * 8.

[0228] A DMC unit (dmc_unit) can contain a unit header and a unit payload. The unit header can contain type information about the unit payload. The unit payload can contain data of that type.

[0229] The DMC unit (dmc_unit) data syntax and semantics can be in the format of the V3C unit (v3c_unit) used in the V3C codec specification, i.e., ISO / IEC 23090-5, which is referenced in the V-DMC standard. It can also be flexibly implemented based on a bitstream format newly defined in the V-DMC or V3C codec.

[0230] If the sample stream of the bitstream is a NAL unit, the syntax and semantics of the sample stream NAL unit are as follows.

[0231] The following is the sample stream NAL unit syntax and semantics, as per ISO / IEC23090-5.

[0232] Sample stream NAL header syntax

[0233] sample_stream_nal_header() {Descriptorssnh_unit_size_precision_bytes_minus1u(3)ssnh_reserved_zero_5bitsu(5)}

[0234] Sample stream NAL unit syntax

[0235] sample_stream_nal_unit() {Descriptorssnu_nal_unit_sizeu(v)nal_unit (ssnu_nal_unit_size)}

[0236] The sample stream NAL header can always be at the beginning of the NAL stream.

[0237] ssnh_unit_size_precision_bytes_minus1 + 1 represents the precision (in bytes) of ssnu_nal_unit_size elements in every sample stream NAL unit.

[0238] The order of the sample stream NAL units of the sample stream may follow the decoding order of the NAL units contained in the sample stream NAL units.

[0239] nal_unit may be replaced with basemesh data that can be output to a sample of a basemesh track according to embodiments, i.e., bmesh_nal_unit, and / or displacement data encoded with arithmetic coding, i.e., displ_nal_unit.

[0240] ssnu_nal_unit_size indicates the size (in bytes) of the subsequent NAL_unit.

[0241] The structure of the Basemesh sub-bitstream according to the embodiments included in the bitstream is as follows.

[0242] A NAL sample stream format can be constructed from a NAL unit stream format by arranging the NAL units in decoding order and prepending a title to each NAL unit that specifies the precise size (in bytes) of the NAL unit. A sample stream header is included at the beginning of the sample stream bitstream that specifies the precision (in bytes) of the signaled NAL unit size. The NAL unit stream format can be extracted from the sample stream format by iterating over the sample stream format and reading the size information to appropriately extract each NAL unit.

[0243] The syntax of the basemesh sub-bitstream is as follows:

[0244] General NAL unit syntax

[0245] bmesh_nal_unit( NumBytesInNalUnit ) {Descriptorbmesh_nal_unit_header( )NumBytesInRbsp = 0for( i = 2; i < NumBytesInNalUnit; i++ )rbsp_byte[ NumBytesInRbsp++ ]b(8)}

[0246] NAL unit header syntax

[0247] bmesh_nal_unit_header() {Descriptorbmesh_nal_forbidden_zero_bitf(1)bmesh_nal_unit_typeu(6)bmesh_nal_layer_idu(6)bmesh_nal_temporal_id_plus1u(3)}

[0248] Figure 19 shows a base mesh sequence parameter set according to embodiments.

[0249] Figure 20 shows the base mesh profile tier levels according to embodiments.

[0250] Figure 21 shows base mesh profile tool set constraint information according to embodiments.

[0251] Figure 22 shows a base mesh frame parameter set according to embodiments.

[0252] The semantics of the elements included in the aforementioned basemesh sub-bitstream are as follows.

[0253] Semantics of Basemesh NAL units:

[0254] General NAL Unit Semantics:

[0255] NumBytesInNalUnit represents the size of a NAL unit in bytes. This value is required for NAL unit decoding. Enabling inference of NumBytesInNalUnit requires some form of boundary setting for NAL unit boundaries.

[0256] rbsp_byte[ i ] is the ith byte of the RBSP. The RBSP represents a sequence of bytes ordered as follows:

[0257] An RBSP contains a data bit string (SODB) that:

[0258] If the SODB is empty (i.e., has a length of 0 bits), the RBSP is also empty.

[0259] Otherwise, the RBSP contains the following SODBs:

[0260] 1) The first byte of the RBSP contains the first (most significant, leftmost) 8 bits of the SODB. The next byte of the RBSP contains the next 8 bits of the SODB, and so on until there are fewer than 8 bits of the SODB left.

[0261] 2) The rbsp_trailing_bits( ) syntax structure exists after SODB as follows.

[0262] i) The first (most significant, leftmost) bit of the last RBSP byte contains the remaining bits of the SODB (if any).

[0263] ii) The next bit consists of a single bit equal to 1 (i.e. rbsp_stop_one_bit).

[0264] iii) If rbsp_stop_one_bit is not the last bit of a byte that is aligned, byte alignment is achieved by the presence of one or more bits equal to 0 (i.e., instances of rbsp_alignment_zero_bit).

[0265] Syntax structures with these RBSP properties are indicated in the syntax table using the "_rbsp" suffix. These structures are conveyed within a NAL unit as the contents of the rbsp_byte[ i ] data byte.

[0266] Figures 23a, 23b, and 23c illustrate basemesh NAL unit types according to embodiments.

[0267] Semantics of NAL unit headers:

[0268] A NAL unit type similar to that of an atlas is defined for the base mesh, enabling similar functionality for random access and segmentation of the mesh. Unlike an atlas, which is segmented into tiles, the embodiments define the concept of a sub-mesh and define specific final units corresponding to the coded mesh data. Additionally, NAL units that can contain metadata, such as SEI messages, are defined.

[0269] Semantics of BaseMesh raw byte sequence payload, trailing bits, and byte alignment:

[0270] Basemesh Sequence Parameter Set RBSP Semantics:

[0271] General Basemesh Sequence Parameter Set RBSP Semantics:

[0272] The parameter set ID (bmsps_sequence_parameter_set_id) represents an identifier for the basemesh sequence parameter set so that it can be referenced from other syntax elements.

[0273] The intra-mesh codec ID (bmsps_intra_mesh_codec_id) indicates the identifier of the codec used to compress the static mesh. This codec may be identified by a profile defined in ISO / IEC 23090-29, a component codec mapping SEI message, or by means external to this document. It may also be associated with a specific mesh or motion mesh codec via a profile specified in that specification, or it may be explicitly indicated in an SEI message, as is done in the V3C specification for video sub-bitstreams.

[0274] The inter-mesh codec ID (bmsps_inter_mesh_codec_id) indicates the identifier of the codec used to compress the motion data. This codec may be identified by a profile defined in ISO / IEC 23090-29, a component codec mapping SEI message, or by means external to this document. It may also be associated with a specific mesh or motion mesh codec through a profile specified in that specification, or it may be explicitly indicated in an SEI message, as is done in the V3C specification for video sub-bitstreams.

[0275] Inter-mesh motion group size (bmsps_inter_mesh_motion_group_size_minus1): Adding 1 to this value indicates the size of the vertex grouping in motion vector coding.

[0276] Inter-mesh maximum number of neighbors (bmsps_inter_mesh_max_num_neighbours_minus1): Adding 1 to this value indicates the maximum number of vertex neighbors to use for calculating the motion vector predictor.

[0277] Geometry bit length (bmsps_geometry_3d_bit_length_minus1): Adding 1 to the value indicates the bit depth of the geometry coordinates of the reconstructed mesh.

[0278] The facegroup segmentation method (bmsps_facegroup_segmentation_method) indicates the identifier of the method for deriving facegroup IDs from a mesh.

[0279] Mesh attribute count (bmsps_mesh_attribute_count) indicates the number of attributes associated with the mesh.

[0280] The mesh attribute type ID (bmsps_mesh_attribute_type_id[i]) indicates the attribute type of the attribute with index i for the mesh. The following describes the list of supported attributes and their relationship to bmsps_mesh_attribute_type_id[ i ].

[0281] bmsps_mesh_attribute_type_id[ i ]IdentifierAttribute type0ATTR_TEXTURETexture1ATTR_MATERIAL_IDMaterial ID2ATTR_TRANSPARENCYTransparency3ATTR_REFLECTANCEReflectance4ATTR_NORMAL5ATTR_FACEGROUP_ID6..14ATTR_RESERVED15ATTR_UNSPECIFIED

[0282] Attribute bit length (bmsps _attribute_bit_length_minus1[i]): Adding 1 to this value indicates the bit depth of the attribute with index i for the mesh.

[0283] Mesh Frame Order Maximum Count (bmsps_log2_max_mesh_frame_order_cnt_lsb_minus4): Adding 4 to this value gives the values ​​of the Log2MaxMeshFrmOrderCntLsb and MaxMeshFrmOrderCntLsb variables used in the decoding process for the number of atlas frame orders.

[0284] Log2MaxMeshFrmOrderCntLsb = bmsps_log2_max_mesh_frame_order_cnt_lsb_minus4 + 4

[0285] MaxMeshFrmOrderCntLsb = 2Log2MaxMeshFrmOrderCntLsb

[0286] Mesh Frame Buffering Maximum (bmsps_max_dec_mesh_frame_buffering_minus1): Adding 1 to this value indicates the maximum required size of a decoded atlas frame buffer for CAS in atlas frame storage buffer units.

[0287] If the long-term reference mesh frame flag (bmsps_long_term_ref_mesh_frames_flag) is 0, it indicates that long-term reference atlas frames are not used for inter prediction of coded atlas frames in CAS. If bmsps_long_term_ref_mesh_frames_flag is 1, it indicates that long-term reference atlas frames can be used for inter prediction of one or more coded atlas frames in CAS.

[0288] The number of reference mesh frame lists (bmsps_num_ref_mesh_frame_lists_in_bmsps) represents the number of bmesh_ref_list_struct(rlsIdx) syntax structures included in the atlas sequence parameter set.

[0289] NOTE: The decoder allocates memory for a total number of bmesh_ref_list_struct(rlsIdx) syntax structures equal to (bmsps_num_ref_mesh_frame_lists_in_bmsps + 1), since there can be one bmesh_ref_list_struct(rlsIdx) syntax structure that directly signals the atlas tile header of the current atlas tile.

[0290] If bmsps_extension_present_flag is 1, it indicates that bmsps_extension_count_minus1 is present in the basemesh sequence parameter set.

[0291] bmsps_extension_count indicates the number of BMSPS extensions present in the v3c_parameter_set() syntax structure.

[0292] If bmsps_extensions_length_minus1 is present, it represents the cumulative length (in bytes) BmspsExtensionsLength for all extensions that follow this syntax element. BmspsExtensionsLength is calculated as follows:

[0293] if( bmsps_extension_count == 0 )

[0294] BmspsExtensionsLength = 0

[0295] else

[0296] BmspsExtensionsLength = bmsps_extensions_length_minus1 + 1

[0297] If bmsps_extension_count is not 0, BmspsExtensionsLength is equal to 3 * bmsps_extension_count plus the sum of all bmsps_extension_length[ i ].

[0298] bmsps_extension_length[ i ] represents the number of bytes used to represent the payload size of the syntax structure of the extension associated with index i. If bmsps_extension_length[ i ] is 0, there is no extension payload for the extension with index i. Otherwise, the extension with index i has a payload size in bits in the range 8 * ( bmsps_extension_length[ i ] - 1 ) + 1 to 8 * bmsps_extension_length[ i ].

[0299] Basemesh profile, tier, and level semantics:

[0300] Tier flag (bmptl_tier_flag) indicates the tier context for bmptl_level_idc interpretation, as per ISO / IEC 23090-29.

[0301] Profile Codec Group (bmptl_profile_codec_group_idc) represents a codec group profile component that CVS complies with, such as ISO / IEC 23090-29.

[0302] bmptl_profile_toolset_idc represents a tool set combination profile component that CVS complies with, such as ISO / IEC 23090-29.

[0303] Profile Reconstruction (bmptl_profile_reconstruction_idc) indicates the reconstruction profile component that CVS is recommended to adhere to, such as ISO / IEC 23090-29. A decoder may choose to use a different reconstruction profile than the one indicated in the bitstream.

[0304] Level (bmptl_level_idc) indicates the level to which CVS complies, such as ISO / IEC 23090-29.

[0305] The number of sub-profiles (bmptl_num_sub_profiles) represents the number of bmptl_sub_profile_idc[ i ] syntax elements.

[0306] If the extended subprofile flag (bmptl_extended_sub_profile_flag) is 1, it indicates that the bmptl_sub_profile_idc[ i ] syntax element, if present, should be represented using 64 bits. If bmptl_extended_sub_profile_flag is 0, it indicates that the bmptl_sub_profile_idc[ i ] syntax element, if present, should be represented using 32 bits.

[0307] A subprofile (bmptl_sub_profile_idc[ i ]) represents the ith interoperability metadata registered as specified in Rec. The number of bits used to represent bmptl_sub_profile_idc[ i ] is equal to (bmptl_extended_sub_profile_flag == 0 ? 32 : 64).

[0308] If the toolset constraints presence flag (bmptl_toolset_constraints_present_flag) is 1, it indicates that the additional structure profile toolset constraints information (profile_toolset_constraints_information()) is present in the bitstream. If bmptl_toolset_constraints_present_flag is 0, it indicates that the profile_toolset_constraints_information() structure is not present.

[0309] BaseMesh Profile Toolset Constraint Information Semantics:

[0310] If the mesh frame flag (bmptc_one_mesh_frame_only_flag) is present, it has a semantic system specified as ISO / IEC 23090-29, where the profile indicated by bmptl_profile_toolset_idc is the profile specified in ISO / IEC 23090-29.

[0311] A bmptc_intra_frames_only_flag of 1 indicates that the bitstream contains only sdu_intra_sub_mesh_unit().

[0312] BaseMesh Frame Parameter Set RBSP Semantics:

[0313] General BaseMesh Frame Parameter Set RBSP Semantics:

[0314] The mesh sequence parameter set ID (bfps_mesh_sequence_parameter_set_id) indicates the bmsps_sequence_parameter_set_id value for the active basemesh sequence parameter set.

[0315] The mesh parameter set ID (bfps_mesh_parameter_set_id) identifies the basemesh frame parameters set for reference in other syntax elements.

[0316] If the Output Flag Present Flag (bfps_output_flag_present_flag) is 1, it indicates that the smh_output_flag syntax element is present in the associated submesh header. If bfps_output_flag_present_flag is 0, it indicates that the smh_output_flag syntax element is not present in the associated submesh header.

[0317] Number of reference indices (bfps_num_ref_idx_default_active_minus1): Adding 1 to this value gives the inferred value of the variable NumRefIdxActive for tiles where smh_num_ref_idx_active_override_flag is 0.

[0318] bfps_additional_lt_mfoc_lsb_len indicates the value of the MaxLtMeshFrmOrderCntLsb variable used in the decoding process for the reference atlas frame list.

[0319] MaxLtMeshFrmOrderCntLsb =2 * (Log2MaxMeshFrmOrderCntLsb+ bfps_additional_lt_mfoc_lsb_len)

[0320] Figure 24 shows a displacement sequence parameter set according to embodiments.

[0321] A bitstream according to embodiments may include an arithmetic code displacement sub-bitstream.

[0322] A NAL sample stream format can be constructed from a NAL unit stream format by arranging the NAL units in decoding order and prepending a title to each NAL unit that specifies the precise size (in bytes) of the NAL unit. A sample stream header is included at the beginning of the sample stream bitstream that specifies the precision (in bytes) of the signaled NAL unit size. The NAL unit stream format can be extracted from the sample stream format by iterating over the sample stream format and reading the size information to appropriately extract each NAL unit.

[0323] With respect to the syntax of the arithmetic code displacement sub-bitstream, the NAL unit syntax is as follows.

[0324] General NAL unit syntax:

[0325] displ_nal_unit( NumBytesInNalUnit ) {Descriptordispl_nal_unit_header( )NumBytesInRbsp = 0for( i = 2; i < NumBytesInNalUnit; i++ )rbsp_byte[ NumBytesInRbsp++ ]b(8)}

[0326] NAL unit header syntax

[0327] displ_nal_unit_header() {Descriptordispl_nal_forbidden_zero_bitf(1)displ_nal_unit_typeu(6)displ_nal_layer_idu(6)displ_nal_temporal_id_plus1u(3)}

[0328] Describes the displacement sequence parameter set RBSP syntax, with respect to raw byte sequence payload, trailing bits, and byte alignment syntax.

[0329] Figure 24 is the general displacement sequence parameter set RBSP syntax.

[0330] Figure 25 shows displacement profile tier levels according to embodiments.

[0331] Figure 26 shows displacement profile toolset constraint information according to embodiments.

[0332] Below, the displacement frame parameter set RBSP syntax is described.

[0333] Figure 27 shows a displacement frame parameter set according to embodiments.

[0334] Figure 28 shows a displacement reference list structure according to embodiments.

[0335] Fig. 29 shows a displacement header according to embodiments.

[0336] The displacement layer RBSP syntax is as follows:

[0337] displ_layer_rbsp() {Descriptordispl_header()displ_data_unit(displID)rbsp_trailing_bits()}

[0338] The syntax of the displacement data unit is as follows:

[0339] displ_data_unit( displID ) {Descriptorif( dh_type == I_DISPLACEMENT ) {displ_intra_unit( displID )}else if( dh_type == P_DISPLACEMENT ) {displ_inter_unit( displID )}}

[0340] Figure 30 illustrates a displacement intra data unit according to embodiments.

[0341] Describes the displacement inter data unit syntax.

[0342] The arithmetic decoding engine according to the embodiments is a context-separated binary arithmetic decoder that performs binary renormalization and produces binary output. The displacement residual is derived from the arithmetic decoding.

[0343] displ_inter_unit( dispID ) {Descriptor / * Can be the same as specified in 4.3.1.3.7 * / }

[0344] Displacement Inter-Data Unit Semantics:

[0345] NAL unit semantics:

[0346] General NAL unit semantics:

[0347] NumBytesInNalUnit indicates the size of a NAL unit in bytes. This value is required for NAL unit decoding. Some form of boundary setting is required to enable inference of NumBytesInNalUnit.

[0348] Note: The displacement coding layer (DCL) is directed to efficiently represent the contents of displacement data. The NAL is directed to specify the format of that data and to provide header information in a manner suitable for transmission over various communication channels or storage media. All data is contained in NAL units, each unit containing an integer number of bytes. NAL units represent a common format for use in both packet-oriented and bitstream systems. The format of NAL units for both packet-oriented transport and sample streams is the same, except that each NAL unit may be preceded by an additional element specifying the size of the NAL unit, as specified in the sample stream format in Appendix TBD.

[0349] rbsp_byte[ i ] is the ith byte of the RBSP. An RBSP is indicated by a sequence of bytes ordered as follows:

[0350] An RBSP contains the following SODB (Data Bit String):

[0351] If the SODB is empty (i.e., has a length of 0 bits), the RBSP is also empty.

[0352] Otherwise, the RBSP contains the following SODBs:

[0353] 3) The first byte of the RBSP contains the first (most significant, leftmost) 8 bits of the SODB. The next byte of the RBSP contains the next 8 bits of the SODB, and so on until there are fewer than 8 bits of the SODB left.

[0354] 4) The rbsp_trailing_bits( ) syntax structure exists after SODB as follows:

[0355] iv) The first (most significant, leftmost) bit of the last RBSP byte contains the remaining bits of the SODB (if any).

[0356] v) The next bit consists of a single bit equal to 1 (i.e. rbsp_stop_one_bit).

[0357] vi) If rbsp_stop_one_bit is not the last bit of a byte that is aligned, byte alignment is achieved by the presence of one or more bits equal to 0 (i.e., instances of rbsp_alignment_zero_bit).

[0358] Syntax structures with these RBSP properties are indicated in the syntax table using the suffix "_rbsp". These structures are carried within the NAL unit as the contents of the rbsp_byte[ i ] data byte.

[0359] Note: Once the RBSP boundary is known, the decoder can extract the SODB from the RBSP by concatenating the bits of the RBSP bytes and discarding the last (least significant, rightmost) bit, rbsp_stop_one_bit, which is set to 1 and discarding the bits following it (those with less significance and farther to the right). This is equivalent to 0. The data required for the decoding process is contained in the SODB portion of the RBSP.

[0360] NAL unit header semantics:

[0361] As with atlases, a similar NAL unit type is defined for displacement, enabling similar random access capabilities and defining a specific final unit corresponding to the coded displacement data. NAL units that can contain metadata, such as SEI messages, are also defined.

[0362] In particular, the supported displacement NAL unit types are indicated as in the following Figures 31a, 31b, and 31c.

[0363] Figures 31a, 31b, and 31c illustrate displacement NAL unit types according to embodiments.

[0364] Below, the order of NAL units and displacement frames, and their association with coded displacement frames, access units, and coded displacement sequences are described.

[0365] Raw byte sequence payload, trailing bits, and byte alignment semantics:

[0366] Displacement Order Parameter Set RBSP Semantics:

[0367] General displacement sequence parameter set RBSP semantics:

[0368] The sequence parameter set ID (dsps_sequence_parameter_set_id) represents an identifier for the displacement sequence parameter set so that it can be referenced from other syntax elements.

[0369] The codec ID (dsps_codec_id) indicates the identifier of the codec used to compress the displacement. This codec may be identified through a profile defined in ISO / IEC 23090-29, a component codec mapping SEI message, or means external to this document. It may also be associated with a specific displacement codec through a profile specified in that specification, or it may be explicitly indicated in an SEI message, as is done in the V3C specification for video sub-bitstreams.

[0370] Range (dsps_range_log2_minus2): Adding 2 to this value gives the range of the geometric displacement coordinates of the displacement.

[0371] The single dimension flag (dsps_single_dimension_flag) indicates the number of dimensions for the displacement associated with the displacement. If dsps_single_dimension_flag is 0, it indicates that three components are used for the displacement. If dsps_single_dimension_flag is 1, it indicates that only the normal component is used for the displacement.

[0372] dsps_msb_align_flag indicates how decoded displacement samples are converted to samples of displacement range bit depth.

[0373] dsps_log2_max_displ_frame_order_cnt_lsb_minus4 + 4 specifies the values ​​of the Log2MaxDisplFrmOrderCntLsb and MaxDisplFrmOrderCntLsb variables used in the decoding process for the displacement frame order number.

[0374] Log2MaxDisplFrmOrderCntLsb = dsps_log2_max_displ_frame_order_cnt_lsb_minus4 + 4

[0375] MaxDisplFrmOrderCntLsb = 2Log2MaxDisplFrmOrderCntLsb

[0376] dsps_max_dec_displ_frame_buffering_minus1 + 1 represents the maximum required size of the decoded displacement frame buffer for CDS in displacement frame storage buffer units.

[0377] A dsps_long_term_ref_displ_frames_flag of 0 indicates that the long-term reference disparity is not used for inter prediction of coded disparity frames in the CDS. A dsps_long_term_ref_displ_frames_flag of 1 indicates that the long-term reference disparity frame may be used for inter prediction of one or more coded disparity frames in the CDS.

[0378] dsps_num_ref_displ_frame_lists_in_dsps indicates the number of displ_ref_list_struct( rlsIdx ) syntax structures contained in the displacement sequence parameter set.

[0379] NOTE: The decoder may have one displ_ref_list_struct( rlsIdx ) syntax structure that directly signals the displacement header of the current displacement frame, so it allocates memory for a total number of displ_ref_list_struct( rlsIdx ) syntax structures equal to (dsps_num_ref_displ_frame_lists_in_dsps + 1).

[0380] If dsps_extension_present_flag is 1, it indicates that dsps_extension_count_minus1 and dsps_extension_length_minus1 are present in the displacement sequence parameter set.

[0381] dsps_extension_count_minus1 + 1 represents the number of extensions in the current displacement sequence parameter set.

[0382] dsps_extension_length_minus1 + 1 represents the length of the dsps_extension_data_byte element following this syntax element.

[0383] Displacement profile, tier, and level semantics:

[0384] The tier flag (dptl_tier_flag) indicates the tier context for the interpretation of dptl_level_idc as specified in ISO / IEC 23090-29.

[0385] The Profile Codec Group ID (dptl_profile_codec_group_id) indicates the Codec Group Profile component to which the CDS conforms as specified in ISO / IEC 23090-29.

[0386] dptl_profile_toolset_idc represents the toolset combination profile component that CDS complies with as specified in ISO / IEC 23090-29.

[0387] Level (dptl_level_idc) indicates the level of the CDS specified in ISO / IEC 23090-29.

[0388] The number of sub-profiles (dptl_num_sub_profiles) represents the number of dptl_sub_profile_idc[ i ] syntax elements.

[0389] If dptl_extended_sub_profile_flag is 1, it indicates that the dptl_sub_profile_idc[ i ] syntax element, if present, should be represented using 64 bits. If dptl_extended_sub_profile_flag is 0, it indicates that the dptl_sub_profile_idc[ i ] syntax element, if present, should be represented using 32 bits.

[0390] dptl_sub_profile_idc[ i ] represents the ith interoperability metadata registered as specified in Rec. The number of bits used to represent dptl_sub_profile_idc[ i ] is equal to (dptl_extended_sub_profile_flag == 0 ? 32 : 64).

[0391] If dptl_toolset_constraints_present_flag is 1, it indicates that the additional structure dptl_profile_toolset_constraints_information() is present in the bitstream. If dptl_toolset_constraints_present_flag is 0, it indicates that the dptl_profile_toolset_constraints_information() structure is not present.

[0392] Displacement Profile Toolset Constraint Information Semantics:

[0393] If dptc_one_displacemnt_frame_only_flag is present, it has the meaning specified in ISO / IEC 23090-29.

[0394] Displacement Frame Parameter Set RBSP Semantics:

[0395] General displacement frame parameter set RBSP semantics:

[0396] The displacement sequence parameter set ID (dfps_displ_sequence_parameter_set_id) indicates the dsps_sequence_parameter_set_id value for the active displacement sequence parameter set.

[0397] The displacement parameter set ID (dfps_displ_parameter_set_id) identifies the displacement frame parameters set for reference in other syntax elements.

[0398] A dfps_output_flag_present_flag equal to 1 indicates that the displ_output_flag syntax element is present in the associated displacement header. A dfps_output_flag_present_flag equal to 0 indicates that the displ_output_flag syntax element is not present in the associated displacement header.

[0399] dfps_num_ref_idx_default_active_minus1 + 1 represents the inferred value of the variable NumRefIdxActive for tiles where displ_num_ref_idx_active_override_flag is 0.

[0400] dfps_additional_lt_dfoc_lsb_len indicates the value of the MaxLtDisplFrmOrderCntLsb variable used in the decoding process for the reference atlas frame list.

[0401] MaxLtDisplFrmOrderCntLsb = 2 *(Log2MaxDisplFrmOrderCntLsb+ dfps_additional_lt_dfoc_lsb_len)

[0402] If dfps_extension_present_flag is 1, it indicates that the syntax element dfps_extension_8bits is present in the displacement frame parameter set. If dfps_extension_present_flag is 0, it indicates that the syntax element dfps_extension_8bits is not present.

[0403] displ_no_output_of_prior_displ_frames_flag affects the output of previously decoded displacement frames in the DDB after decoding displacement frames in a CDS AU that is not the first AU in the bitstream as specified in ISO / IEC 23090-29.

[0404] It is a bitstream conformance requirement that the value of no_output_of_prior_displ_frames_flag must be the same for all displacement frames in an AU.

[0405] The no_output_of_prior_displ_frames_flag value in the displacement header is the output_of_prior_displ_frames_flag value in the AU.

[0406] displ_frame_parameter_set_id indicates the dfps_displ_frame_parameter_set_id value for the active displacement frame parameters set for the current displacement frame.

[0407] dislp_type indicates the coding type of the current displacement frame, as shown in the table below.

[0408] smh_typeName of smh_type0P_DISPLACEMENT1I_DISPLACEMENT2...RESERVED

[0409] displ_output_flag affects the decoded displacement output and removal process as specified in ISO / IEC 23090-29.

[0410] displ_frm_order_cnt_lsb indicates the number of displacement frame orders modulo MaxDisplFrmOrderCntLsb for the current displacement frame.

[0411] If ref_displ_frame_list_dsps_flag is 1, it indicates that the reference displacement frame list of the current displacement frame is derived based on one of the displ_ref_list_struct( rlsIdx ) syntax structures of the active DSPS. If ref_displ_frame_list_dsps_flag = 0, it indicates that the reference displacement frame list of the current displacement frame is derived based on the displ_ref_list_struct( rlsIdx ) syntax structure contained directly in the displacement frame header of the current displacement frame.

[0412] ref_displ_frame_list_idx represents an index into the list of displ_ref_list_struct( rlsIdx ) syntax structures contained in the active DSPS of the displ_ref_list_struct( rlsIdx ) syntax structure, which is used to derive the reference displ_frame list for the current displ_frame. The syntax element ref_displ_frame_list_idx is expressed in Ceil( Log2( dsps_num_ref_displ_frame_lists_in_dsps ) ) bits.

[0413] The variable RlsIdx of the current atlas tile is derived as follows:

[0414] RlsIdx = ref_displ_frame_list_dsps_flag ? ref_displ_frame_list_idx : dsps_num_ref_displ_frame_lists_in_dsps

[0415] If added_dfoc_lsb_present_flag[ j ] is equal to 1, it indicates that added_dfoc_lsb_val[ j ] exists in the current displacement frame. If external_dfoc_lsb_present_flag[ j ] is equal to 0, it indicates that added_dfoc_lsb_val[ j ] does not exist.

[0416] The additional_dfoc_lsb_val[ j ] represents the FullFrmOrderCntLsbLt[ RlsIdx ][ j ] value for the current atlas tile as follows:

[0417] FullDisplFrmOrderCntLsbLt[ RlsIdx ][ j ] = add_dfoc_lsb_val[ j ] * MaxDisplFrmOrderCntLsb +dfoc_lsb_lt[ RlsIdx ][ j ]

[0418] The syntax element added_dfoc_lsb_val[ j ] is represented by dfps_additional_lt_dfoc_lsb_len bits.

[0419] If num_ref_idx_active_override_flag is 1, it indicates that syntax element num_ref_idx_active_minus1 exists for the current displacement frame. If num_ref_idx_active_override_flag is 0, it indicates that syntax element num_ref_idx_active_minus1 does not exist.

[0420] num_ref_idx_active_minus1 is used to derive the variable NumRefIdxActive specified in Equation 5 for the current displacement frame.

[0421] If the current displacement frame is a P_DISPLACEMENT displacement frame, num_ref_idx_active_override_flag is inferred to be 1, and if num_ref_idx_active_minus1 does not exist, num_ref_idx_active_minus1 is inferred to be equal to 0.

[0422] The NumRefIdxActive variable is derived as follows:

[0423] if( displ_type == P_DISPLACEMENT ) {

[0424] if( num_ref_idx_active_override_flag == 1 )

[0425] NumRefIdxActive = num_ref_idx_active_minus1 + 1

[0426] else {

[0427] if( num_ref_entries[ RlsIdx ] >= dfps_num_ref_idx_default_active_minus1 + 1 )

[0428] NumRefIdxActive = dfps_num_ref_idx_default_active_minus1 + 1

[0429] else

[0430] NumRefIdxActive = num_ref_entries[RlsIdx]

[0431] }

[0432] }

[0433] else

[0434] NumRefIdxActive = 0

[0435] NumRefIdxActive minus 1 represents the maximum number of displacement reference frame indices that can be used to decode the current displacement frame.

[0436] Displacement Reference List Structure Semantics:

[0437] drl_num_ref_entries[ rlsIdx ] represents the number of entries in the displ_ref_list_struct( rlsIdx ) syntax structure, where rlsIdx is the index into the displacement frame reference list.

[0438] drl_st_ref_displ_frame_flag[ rlsIdx ][ i ] = 1 indicates that the ith entry in the displ_ref_list_struct( rlsIdx ) syntax structure is a short-term reference displacement frame entry. st_ref_displ_frame_flag[ rlsIdx ][ i ] = 0 indicates that the ith entry in the displ_ref_list_struct( rlsIdx ) syntax structure is a long-term reference displacement frame entry.

[0439] The NumLtrDisplFrmEntries[ rlsIdx ] variable is derived as follows:

[0440] NumLtrDisplFrmEntries[ rlsIdx ] = 0

[0441] for( i = 0; i < drl_num_ref_entries[ rlsIdx ]; i++ )

[0442] if( !drl_st_ref_displ_frame_flag[ rlsIdx ][ i ] )

[0443] NumLtrDisplFrmEntries[ rlsIdx ]++

[0444] drl_abs_delta_dfoc_st[ rlsIdx ][ i ], if the i-th item is the first short-term reference displacement frame entry in the displ_ref_list_struct( rlsIdx ) syntax structure, indicates the absolute difference between the displacement frame order count values ​​of the current displacement frame referenced by the i-th item or if the i-th item is a short-term reference displacement frame entry but is not the first short-term reference displacement frame entry in the displ_ref_list_struct( rlsIdx ) syntax structure. indicates the absolute difference between the displacement frame order count values ​​of the displacement frames referenced by the i-th item and the previous short-term reference displacement frame entry in the displ_ref_list_struct( rlsIdx ) syntax structure.

[0445] drl_straf_entry_sign_flag[ rlsIdx ][ i ] If it is equal to 1, it indicates that the i-th entry of the syntax structure displ_ref_list_struct( rlsIdx ) has a value greater than or equal to 0. If drl_straf_entry_sign_flag[ rlsIdx ][ i ] is equal to 0, the i-th entry of the syntax structure displ_ref_list_struct( rlsIdx ) has a value less than 0.

[0446] The DeltaDfocSt[ rlsIdx ][ i ] list is derived as follows:

[0447] for( i = 0; i < drl_num_ref_entries[ rlsIdx ]; i++ )

[0448] if( drl_st_ref_displ_frame_flag[ rlsIdx ][ i ] )

[0449] DeltaDfocSt[ rlsIdx ][ i ] =

[0450] ( 2 * drl_straf_entry_sign_flag[ rlsIdx ][ i ] - 1 ) * drl_abs_delta_dfoc_st[ rlsIdx ][ i ]

[0451] else

[0452] DeltaDfocSt[rlsIdx][i] = 0

[0453] drl_dfoc_lsb_lt[ rlsIdx ][ i ] represents the Modulo MaxDisplFrmOrderCntLsb displacement frame order count value of the displacement frame referenced by the i-th entry of the displ_ref_list_struct( rlsIdx ) syntax structure. The length of the drl_dfoc_lsb_lt[ rlsIdx ][ i ] syntax element is Log2MaxDisplFrmOrderCntLsb bits.

[0454] Displacement header semantics:

[0455] dh_no_output_of_prior_displ_frames_flag affects the output of previously decoded displacement frames in the DDB after decoding displacement frames in a CDS AU that is not the first AU in the bitstream as specified in ISO / IEC 23090-29.

[0456] It is a bitstream conformance requirement that the value of no_output_of_prior_displ_frames_flag must be the same for all displacement frames in an AU.

[0457] The no_output_of_prior_displ_frames_flag value in the displacement header is the output_of_prior_displ_frames_flag value in the AU.

[0458] dh_frame_parameter_set_id indicates the dfps_displ_frame_parameter_set_id value for the active displacement frame parameter set in the current displacement frame.

[0459] dh_type indicates the coding type of the current displacement frame according to the table below.

[0460] smh_typeName of smh_type0P_DISPLACEMENT1I_DISPLACEMENT2...RESERVED

[0461] dh_frm_order_cnt_lsb represents the displacement frame order number modulo MaxDisplFrmOrderCntLsb for the current displacement frame. The length of the dh_frm_order_cnt_lsb syntax element is equal to Log2MaxDisplFrmOrderCntLsb bits.

[0462] If dh_ref_displ_frame_list_dsps_flag is 1, it indicates that the reference displacement frame list of the current displacement frame is derived based on one of the displ_ref_list_struct(rlsIdx) syntax structures of the active DSPS. If dh_ref_displ_frame_list_dsps_flag = 0, it indicates that the reference displacement frame list of the current displacement frame is derived based on the displ_ref_list_struct( rlsIdx ) syntax structure contained directly in the displacement frame header of the current displacement frame.

[0463] dh_ref_displ_frame_list_idx represents an index into the list of displ_ref_list_struct( rlsIdx ) syntax structures contained in the active DSPS of the displ_ref_list_struct( rlsIdx ) syntax structure, which is used to derive the reference displ_frame list for the current displ_frame. The syntax element dh_ref_displ_frame_list_idx is expressed in Ceil( Log2( dsps_num_ref_displ_frame_lists_in_dsps ) ) bits.

[0464] The variable RlsIdx of the current atlas tile is derived as follows:

[0465] RlsIdx = dh_ref_displ_frame_list_dsps_flag ? ref_displ_frame_list_idx : dsps_num_ref_displ_frame_lists_in_dsps

[0466] If dh_additional_dfoc_lsb_present_flag[ j ] is 1, it indicates that dh_additional_dfoc_lsb_val[ j ] is present in the current displacement frame. If dh_additional_dfoc_lsb_present_flag[ j ] is 0, it indicates that dh_additional_dfoc_lsb_val[ j ] is not present.

[0467] dh_additional_dfoc_lsb_val[ j ] represents the FullFrmOrderCntLsbLt[ RlsIdx ][ j ] value for the current atlas tile as follows:

[0468] FullDisplFrmOrderCntLsbLt[ RlsIdx ][ j ] = dh_additional_dfoc_lsb_val[ j ] * MaxDisplFrmOrderCntLsb +dfoc_lsb_lt[ RlsIdx ][ j ]

[0469] The syntax element dh_additional_dfoc_lsb_val[ j ] is represented by the dfps_additional_lt_dfoc_lsb_len bits.

[0470] If dh_num_ref_idx_active_override_flag is 1, it indicates that syntax element num_ref_idx_active_minus1 exists for the current displacement frame. If dh_num_ref_idx_active_override_flag is 0, it indicates that syntax element num_ref_idx_active_minus1 does not exist.

[0471] dh_num_ref_idx_active_minus1 is used to derive the variable NumRefIdxActive for the current displacement frame.

[0472] If the current displacement frame is a P_DISPLACEMENT displacement frame, dh_num_ref_idx_active_override_flag is 1, and dh_num_ref_idx_active_minus1 does not exist, then dh_num_ref_idx_active_minus1 is inferred to be equal to 0.

[0473] The NumRefIdxActive variable is derived as follows:

[0474] if( dh_type == P_DISPLACEMENT ) {

[0475] if( dh_num_ref_idx_active_override_flag == 1 )

[0476] NumRefIdxActive = dh_num_ref_idx_active_minus1 + 1

[0477] else {

[0478] if( num_ref_entries[ RlsIdx ] >= dfps_num_ref_idx_default_active_minus1 + 1 )

[0479] NumRefIdxActive = dfps_num_ref_idx_default_active_minus1 + 1

[0480] else

[0481] NumRefIdxActive = num_ref_entries[RlsIdx]

[0482] }

[0483] }

[0484] else

[0485] NumRefIdxActive = 0

[0486] NumRefIdxActive minus 1 represents the maximum number of displacement reference frame indices that can be used to decode the current displacement frame.

[0487] If we add 6 to dh_log2_subblock_size_minus6, the value of the subblockSize variable will be as follows:

[0488] subblockSize = 1 << ( log2_subblock_size_minus6 + 6 )

[0489] Displacement Data Unit Semantics:

[0490] displ_intra_unit( displID ) contains a displacement unit stream, an aligned stream of bytes or bits that identifies the location of unit boundaries in a pattern of data. The format of this displacement unit stream is identified by the 4CC code defined by dptl_profile_codec_group_idc or by the component codec mapping SEI message.

[0491] displ_inter_unit( displID ) contains a displacement unit stream, an aligned stream of bytes or bits that identifies the location of unit boundaries in a pattern of data. The format of this displacement unit stream is identified by the 4CC code defined by dptl_profile_codec_group_idc or by the component codec mapping SEI message.

[0492] Displacement Intra Data Unit Semantics:

[0493] The arithmetic decoding engine is a context-sensitive binary arithmetic decoder that performs binary renormalization and produces binary output.

[0494] The displacement values ​​are derived from arithmetic decoding.

[0495] diu_lod_count[ displID ] indicates the number of granularity levels used for the displacement signaled in the data unit associated with displId displID.

[0496] diu_vertex_count_lod[ displID ] [ i ] represents the displacement count for the i-th level of the wavelet transform for the data unit associated with displID.

[0497] diu_last_sig_coeff[ k ] represents the index of the last non-zero displacement coefficient level in the k-th component.

[0498] diu_coded_block_flag[ k ][ b ] indicates whether the block with index b has a non-zero displacement coefficient level in the kth component (if 1) or not (if 0).

[0499] diu_coded_subblock_flag[k][b][s] indicates whether the subblock with index s in the block with index b has a non-zero displacement coefficient level in the kth component (if 1) or not (if 0).

[0500] diu_coeff_abs_level_gt0[ k ][ b ][ s ][ v ] indicates whether the kth component of the displacement coefficient level associated with the vertex with index v in the subblock with index s in the block with index b has an absolute value higher than

[0501] diu_coeff_abs_level_gt1[ k ][ b ][ s ][ v ] indicates whether the kth component of the displacement coefficient level associated with the vertex with index v in the subblock with index s in the block with index b has an absolute value higher than

[0502] diu_coeff_sign[ k ][ b ][ s ][ v ] indicates whether the kth component of the displacement coefficient level associated with the vertex with index v in the subblock with index s in the block with index b has a positive sign.

[0503] diu_coeff_abs_level_rem[ k ][ b ][ s ][ v ] represents the absolute value of the kth component of the displacement coefficient level associated with the vertex with index v in the block with index b - 2.

[0504] Displacement Inter-Data Unit Semantics:

[0505] The arithmetic decoding engine is a context-separated binary arithmetic decoder that performs binary renormalization and produces binary output. The displacement residual is derived from the arithmetic decoding. This can be the same as described above.

[0506] Figure 32 shows a mesh system structure according to embodiments.

[0507] Embodiments include a method of storing and transmitting a dynamic mesh coding bitstream in a file (Carriage of Dynamic Mesh Coding Bitstream in Single Track).

[0508] Embodiments include methods for storing and / or signaling a Video-based Dynamic Mesh Coding (VDMC) bitstream as a single track within a file.

[0509] Embodiments include a transmission method / device and / or a reception method / device for providing a mesh content service that efficiently stores a V-DMC (video-based dynamic mesh coding) or mesh bitstream within a single track within a file and provides signaling therefor.

[0510] Embodiments include a transmission method / device and / or a reception method / device for providing mesh content services that handle file storage techniques to enable efficient access to stored V-DMC bitstreams.

[0511] Although the embodiments describe the embodiments using V-DMC (video-based dynamic mesh coding), the embodiments can be applied not only to V-DMC but also to bitstreams using other mesh codings that are structured in the same manner or in a similar bitstream format.

[0512] Embodiments can encapsulate mesh content encoded with V-DMC (Video-based Dynamic Mesh Coding) into a single track based on an ISOBMFF file to provide services such as streaming.

[0513] The embodiments generate and transmit information contained in a sample entry within a single track of a file.

[0514] The embodiments generate and transmit decoder configuration information as signaling information included in a sample entry within a single track of a file.

[0515] The embodiments generate and transmit information contained in a sample format within a single track of a file.

[0516] The decoder receives a single track of the file, parses the information within the single track, and efficiently decodes the mesh data.

[0517] Figure 32 illustrates the structure of a mesh system. Scenes and / or objects captured in the real world using multiple cameras, sensors, and / or virtual cameras can be encoded into a V-DMC bitstream based on mesh data preprocessing and / or mesh data encoding. This mesh bitstream can be generated into a file format that can be efficiently stored and transmitted through file encapsulation.

[0518] The dynamic mesh player can decompress the files acquired and received through this process into a mesh bitstream through file decapsulation. The decompressed mesh bitstream can be provided to a display device and displayed based on mesh data decoding and / or mesh data processing / rendering.

[0519] Examples include mesh encoding, file / segment encapsulation, file / segment decapsulation, mesh decoding, etc.

[0520] Common data structures included in the Sample Stream DMC header and / or Sample Stream DMC unit are as follows:

[0521] DMC decoder configuration record:

[0522] Decoder configuration information contains decoder configuration information for mesh-based point cloud content. This decoder configuration information includes a version field. Decoder configuration information can define version 1. Incompatible changes to the decoder configuration information are indicated by a change in the version number. Compatible extensions to this record extend this record and do not change the configuration version code.

[0523] The syntax of the DMC decoder configuration information is as follows.

[0524] aligned(8) class DMCDecoderConfigurationRecord {

[0525] unsigned int(8) configurationVersion = 1;

[0526] unsigned int(8) num_of_setup_units;

[0527] for (i=0; I < num_of_setup_units; i++) {

[0528] unsigned int(1) array_completeness;

[0529] unsigned int(7) setup_unit_type; / VPS, SPS, FPS, GPS, APS, etc.

[0530] SetupUnit setup_unit;}

[0531] / additional fields

[0532] }

[0533] Setup unit (setup_unit): Contains a set of V3C parameters. The setup unit contains NAL units according to their associated nal unit type (nal_unit_type). If present, the setup_unit contains SEI messages of type NAL_PREFIX_ESEI, NAL_PREFIX_NSEI, NAL_SUFFIX_ESEI, or NAL_SUFFIX_NSEI, i.e. SEI messages that provide information about the entire stream.

[0534] The setup_unit array contains a set of atlas sub-bitstream parameters and atlas sub-bitstream SEI messages that are constants for the CVS referenced by the sample entry for which the decoder configuration record exists.

[0535] The semantics of the DMC decoder configuration information are as follows.

[0536] The ConfigurationVersion field is a version field. Incompatible changes to a record are indicated by a change in the version number.

[0537] The number of setup units (num_of_setup_units) indicates the number of DMC configuration units in the decoder configuration record.

[0538] An array completeness of 1 indicates that all constituent units of the indicated type are present in the following array and not in the stream. A value of 0 indicates that additional constituent units of the indicated type may be present in the stream. The default and allowed values ​​are constrained by the sample entry name.

[0539] The setup unit type (setup_unit_type) indicates the DMC-related parameter set type. This parameter set type may contain information based on the parameter set type defined in the V-DMC specification.

[0540] A setup unit (SetupUnit) includes a VPS (V-DMC Parameter Set), a SPS (Sequence Parameter Set), a FPS (Frame Parameter Set), a DPS (n Parameter Set), a GPS (Geometry Parameter Set), a BPS (Basemesh Parameter Set), and / or an APS (Attribute Parameter Set), and the parameter sets may be information based on parameter sets defined in the V-DMC standard.

[0541] DMC Decoder Configuration Box:

[0542] The DMC Decoder Configuration box contains a DMCDecoderConfigurationRecord.

[0543] The syntax of the DMC decoder configuration box is as follows:

[0544] class DMCConfigurationBox extends FullBox('dmcC', version = 0, 0) {

[0545] DMCDecoderConfigurationRecord();

[0546] }

[0547] The semantics of the DMC decoder configuration box are the same as the DMCDecoderConfigurationRecord described above.

[0548] DMC Component Information Record:

[0549] A DMC component information record represents DMC component information, including the type of DMC component (e.g., geometry or attributes).

[0550] Syntax:

[0551] aligned(8) class DMCComponentInfoRecord(){

[0552] unsigned int(8) component_type;

[0553] if(component_type == 4){ / Attribute component

[0554] unsigned int(8) attr_index;

[0555] utf8string attr_name;

[0556] }

[0557] / additional fields

[0558] }

[0559] Semantics:

[0560] The component_type field identifies the type of DMC component, as specified in the table below. In this version of the document, the value of this field is 1, 2, or 4.

[0561] component_type valueDescription1Displacement component2Geometry component3Reserved4Attribute component5...31Reserved

[0562] attr_index can specify the type, i.e. kind, of the attribute, and can be mapped to the bmsps_mesh_attribute_type_id value in the basemesh sequence parameter set.

[0563] attr_name represents a human-readable name for the DMC attribute component type.

[0564] DMC Component Information Box:

[0565] If this box is present in a sample entry for a track, it indicates the type of DMC component that the track carries. If this box is present in a sample entry for a DMC Attribute track, it also provides the attribute name and optional attribute type information.

[0566] Syntax:

[0567] aligned(8) class DMCComponentInfoBox extends FullBox('dcin', 0, flags){

[0568] DMCComponentInfoRecord();

[0569] }

[0570] Semantics:

[0571] DMCComponentInfoRecord refers to the previously described content.

[0572] The file encapsulator of the transmitter and encoder according to the embodiments can encapsulate the encoded bitstream described above into a file based on multiple tracks as follows.

[0573] A file decapsulator of a receiving device and decoder according to embodiments can receive a file including the encoded bitstream described above, decapsulate point cloud data, i.e., mesh data, from multiple tracks within the file, and decode the file.

[0574] Multiple Track Encapsulation:

[0575] When encapsulating a V-DMC bitstream into one or more tracks, the track type can be configured according to the data type contained in the bitstream. The tracks within the file can include at least one of a basemesh track, an atlas track, a geometry track, a displacement track, or an attribute track. The geometry track can correspond to cases where displacement data is recorded using a video codec. The displacement track can correspond to cases where arithmetic coding is used. In addition, when displacement data is processed using arithmetic coding, the basemesh data, atlas data, and displacement data can be configured as a single track for encapsulation.

[0576] Basemesh track sample entries:

[0577] Sample item types: 'bmc1', 'bmcg'

[0578] Container: SampleDescriptionBox

[0579] Required: 'bmc1' or 'bmcg' sample entry is required

[0580] Quantity: One or more

[0581] Basemesh data of a V-DMC bit stream can be encapsulated into a basemesh track. A sample entry of a basemesh track can contain DMCConfigurationBox information. If the sample entry type is 'bmc1', all parameter sets related to the basemesh data can be inserted into the setup_unit of the DMCConfigurationBox. If the sample entry type is 'bmcg', all parameter sets related to the basemesh data can be inserted into the setup_unit of the DMCConfigurationBox and / or inserted into the sample of the basemesh track. A receiver can recognize and operate a track with a sample entry type of 'bmc1' or 'bmcg' as an entry point.

[0582] Syntax:

[0583] aligned(8) class DMCBaseMeshSampleEntry() extends VolumetricVisualSampleEntry (type) {

[0584] / type is 'bmc1' or 'bmcg'

[0585] DMCConfigurationBox config; / as defined in section 4.5.2

[0586] }

[0587] Semantics:

[0588] config can be the decoder configuration box described above.

[0589] If the basemesh track is an entry point, the config information may include a V-DMC or V3C parameter set (VPS) and / or a basemesh bitstream and associated parameter sets such as SPS, FPS, etc.

[0590]

[0591] Basemesh Track Sample Format:

[0592] Each sample in a basemesh track corresponds to a single coded basemesh access unit as defined in ISO / IEC 23090-29.

[0593] Syntax:

[0594] aligned(8) class DMCBaseMeshSample {

[0595] / sample_size size of sample from SampleSizeBox

[0596] for (int i = 0; i < sample_size; i++) {

[0597] sample_stream_nal_unit ss_nal_unit; / as defined in section 4.1.2

[0598] i += ss_nal_unit.ssnu_nal_unit_size;

[0599] }

[0600] }

[0601] Semantics:

[0602] ss_nal_unit contains a single NAL unit (bmesh_nal_unit) in the NAL unit sample stream format as defined in ISO / IEC 23090-29.

[0603] ssnu_nal_unit_size represents the size (in bytes) of the sample stream NAL unit.

[0604] Displacement Track Sample Entry

[0605] Sample item types: 'dpc1', 'dpcg'

[0606] Container: SampleDescriptionBox

[0607] Required: 'dpc1' or 'dpcg' sample entry

[0608] Quantity: One or more

[0609] If the displacement data of the DMC bitstream is encoded with arithmetic coding, it can be encapsulated into a displacement track (displacement track). The Sample Entry of the Displacement track can contain DMCConfigurationBox information. If the Sample Entry Type is 'dpc1', all parameter sets can be inserted into the setup_unit of the DMCConfigurationBox. If the Sample Entry Type is 'dpcg', all parameter sets can be inserted into the setup_unit of the DMCConfigurationBox and / or inserted into the samples of the displacement track.

[0610] Syntax:

[0611] aligned(8) class DMCDisplSampleEntry() extends VolumetricVisualSampleEntry (type) {

[0612] / type is 'dpc1' or 'dpcg'

[0613] DMCConfigurationBox config; / as defined in section 4.5.2

[0614] }

[0615] Semantics:

[0616] config can be the decoder configuration box described above.

[0617] Displacement Track Sample Format

[0618] Syntax:

[0619] aligned(8) class DMCDisplSample {

[0620] / sample_size size of sample from SampleSizeBox

[0621] for (int i = 0; i < sample_size; i++) {

[0622] sample_stream_nal_unit ss_nal_unit; / as defined in section 4.1.2

[0623] i += ss_nal_unit.ssnu_nal_unit_size;

[0624] }

[0625] }

[0626] Semantics:

[0627] ss_nal_unit contains a single NAL unit (displ_nal_unit) in the NAL unit sample stream format as defined in ISO / IEC 23090-29.

[0628] ssnu_nal_unit_size represents the size (in bytes) of the sample stream NAL unit.

[0629] Atlas Track

[0630] The V-DMC specification, ISO / IEC 23090-29, is currently under technical development and integration within MPEG for utilizing atlas data that constitutes dynamic meta bitstreams. If atlas data is used for a similar purpose as in the V3C specification, ISO / IEC 23090-5, file encapsulation of the atlas data can follow the V3C delivery specification, ISO / IEC 23090-10. The syntax and semantics of the atlas sample entry and sample format can be followed. However, the sample entry type can be newly defined within V-DMC, such as 'dmc1' if the parameter set does not change within the stream and is the same, or 'dmcg' if the parameter set changes within the stream. The receiver can recognize and operate a track with a sample entry type of 'dmc1' or 'dmcg' as an entry point. When an atlas track is an entry point, config information that can be inserted into a sample entry may include a V-DMC or V3C parameter set (VPS) and / or parameter sets such as SPS, FPS, etc. related to the atlas bitstream.

[0631] DMC Video Component Track

[0632] For the displacement data that constitutes the dynamic mesh bitstream, the geometry data and attribute data encoded with the video codec can follow the file encapsulation method for 2D video of ISOBMFF referenced in the V3C standard, i.e. ISO / IEC 23090-10. The receiver can determine information about each data type included in the corresponding track through the V3CUnitHeaderBox information included in the SchemeInformationBox.

[0633] The syntax and semantics of V3CunitHeaderBox can be referenced together with the V3C specification, i.e. ISO / IEC 23090-5.

[0634] The V3C video component track carries 2D video encoding data for a V3C video component. Storage of the V3C video component track leverages existing features of the ISO base media file format and its derivatives. For example, ISO / IEC 14496-15 defines a mechanism for transmitting ISO / IEC 14496-10 and ISO / IEC 23008-2 code V3Cs.

[0635] The V3C video component track is marked as a restricted video in the file and uses the generic restricted sample item 'resv' with additional requirements.

[0636] SchemeTypeBox exists in RestrictedSchemeInfoBox and Scheme_type is set to 'vvvc'.

[0637] SchemeInformationBox is contained in RestrictedSchemeInfoBox and contains V3CUnitHeaderBox.

[0638] In the track header, the track_in_movie flag is set to 0 to indicate that this track should not be displayed alone.

[0639] Additionally, when not using the existing V3C specification method, the receiver can identify the data component type included in the track and information about each data through DMCComponentInfoBox signaling according to the embodiments.

[0640] Figure 33 shows a track reference according to embodiments.

[0641] Regarding track references, if the basemesh track is an entry track:

[0642] Embodiments can designate a basemesh track as an entry point where a file parser can begin parsing among tracks composed of each track type. References between tracks can utilize the TrackReferenceBox of a track box defined in the ISOBMFF specification (ISO / IEC 14496-12) within each track.

[0643] The possible reference relationships between the tracks are as follows.

[0644] Track Reference Method 1:

[0645] Figure 33(a): An example where the base mesh track references the atlas track, and the atlas track references the geometry track and the attribute track.

[0646] To connect different types of tracks, use the track reference tool from ISO / IEC 14496-12.

[0647] A TrackReferenceTypeBox is added to the TrackReferenceBox within the TrackBox of a basemesh track. The TrackReferenceTypeBox contains an array of track_IDs that specify the tracks referenced by the DMC track. To associate a basemesh track with an atlas track, the reference type (reference_type) of the TrackReferenceTypeBox in the basemesh track identifies the associated atlas track, and the atlas track identifies the associated DMC track. The 4CCs for these track reference types are as follows:

[0648] 'bmct': Referenced atlas track.

[0649] 'atcg': Referenced geometry track.

[0650] 'atca': Referenced attribute track.

[0651] Track Reference Method 2:

[0652] Figure 33(b): An example where a base mesh track references an atlas track, a geometry track, and an attribute track.

[0653] To associate a base mesh track with each track, the reference_type of the TrackReferenceTypeBox in the base mesh track identifies the associated DMC track. The 4CCs for these track reference types are as follows:

[0654] 'bmct': Referenced atlas track.

[0655] 'bmcg': Referenced geometry track.

[0656] 'bmca': Referenced property track.

[0657] Figure 34 shows a track reference according to embodiments.

[0658] If the Atlas track is an entry track:

[0659] Embodiments can designate an atlas track as the entry point where a file parser can begin parsing among tracks composed of each track type. References between tracks can utilize the TrackReferenceBox of the TrackBox defined in the ISOBMFF specification (ISO / IEC 14496-12) within each track.

[0660] The reference relationship between possible tracks according to this is as follows.

[0661] To connect different types of tracks, use the track reference tool from ISO / IEC 14496-12.

[0662] A TrackReferenceTypeBox is added to the TrackReferenceBox within the TrackBox of a basemesh track. The TrackReferenceTypeBox contains an array of track_IDs that specify the tracks referenced by the V-DMC track. To associate a basemesh track with an atlas track, the reference_type of the TrackReferenceTypeBox in the basemesh track identifies the associated atlas track, and the atlas track identifies the associated V-DMC track. The 4CCs for these track reference types are as follows:

[0663] 'atcb': Referenced basemesh track.

[0664] 'atcg': Referenced geometry track.

[0665] 'atca': Referenced attribute track.

[0666] Fig. 35 shows a decoder of a receiving device according to embodiments.

[0667] File Receiver: The receiver can receive dynamic mesh content in file form and parse one or more tracks contained within the file.

[0668] File Parser: The receiver's file parser can parse the sample entries of one or more tracks within a dynamic mesh content file to identify base mesh tracks, i.e. entry tracks, whose sample entry types are 'bmc1' or 'bmcg'. It can also parse the TrackReferenceTypeBox of a base mesh track sample entry to identify the referenced, i.e. related, tracks.

[0669] Additionally, the file parser can parse the DMCDecoderConfiguration information contained in the sample entry of each track to find out information such as codec information through the parameter sets contained in each track. Afterwards, the file parser can parse the type-specific data contained in the samples of each track.

[0670]

[0671] Bitstream Packager: The bitstream packager of the receiver can collect data by type, parsed by the aforementioned file parser, in units of frames and / or specific sections, and configure it in the form of a dynamic mesh bitstream that can be decodable.

[0672] V-DMC Decoder: Can decode dynamic mesh bitstreams constructed via bitstream packager.

[0673] Figure 36 shows a mesh data encoding method according to embodiments.

[0674] The mesh data encoding method according to the embodiments may include a step (S3600) of encoding mesh data.

[0675] The mesh data encoding method according to the embodiments may further include a step (S3610) of encapsulating mesh data into a file.

[0676] The mesh data encoding method according to the embodiments may further include a step of transmitting a file (S3620).

[0677] A bitstream containing mesh data and parameter information about the mesh data within the file may be encapsulated into multiple tracks of the file.

[0678] The encoding method of Fig. 36 can be performed by an encoder such as Fig. 1.

[0679] The encoder includes an encoder for encoding mesh data; a file encapsulator for encapsulating the mesh data into a file; and a transmission unit for transmitting the file; and a bitstream including mesh data and parameter information about the mesh data can be encapsulated into multiple tracks of the file.

[0680] Figure 37 shows a mesh data decryption method according to embodiments.

[0681] A method for decrypting mesh data according to embodiments may include a step (S3700) of receiving a file containing mesh data.

[0682] The mesh data decryption method according to the embodiments may further include a step of decapsulating a file (S3710).

[0683] The mesh data decryption method according to the embodiments may further include a step (S3720) of decrypting mesh data.

[0684] The file contains a bitstream containing mesh data and parameter information about the mesh data.

[0685] Referring to FIG. 15, with respect to the format of the bitstream, the bitstream includes a sample stream header and one or more sample stream units,

[0686] The sample stream header contains information about one or more sample stream units,

[0687] One or more sample stream units may include at least one of atlas data about mesh data, information about a base mesh, displacement information encoded based on arithmetic coding, displacement information encoded based on a video codec, and attribute data encoded based on a video codec.

[0688] Referring to FIGS. 16 to 18, with respect to data units within a bitstream, the bitstream may include units including at least one of basemesh data and displacement data. A first unit of the bitstream may include basemesh data, and a second unit of the bitstream may include displacement data. The first unit and the second unit refer to data units of the bitstream, and may be units according to the V3C or V-DMC method.

[0689] Referring to FIGS. 19 to 31, the bitstream includes a basemesh sub-bitstream, and the basemesh sub-bitstream may include at least one of a basemesh sequence parameter set or a basemesh frame parameter set.

[0690] The bitstream further includes a displacement sub-bitstream, and the displacement sub-bitstream may include at least one of a displacement sequence parameter set, a displacement frame parameter set, a displacement reference list structure, or a displacement layer.

[0691] In relation to the file format according to the embodiments, the file may include at least one of a basemesh track including basemesh data of the bitstream, an atlas track including atlas data of the bitstream, a displacement track including displacement data of the bitstream, an atlas track including atlas data of the bitstream, or a dynamic mesh component track including at least one of geometry data or attribute data.

[0692] With respect to a basemesh track of a file according to embodiments, a sample entry of the basemesh track may include dynamic mesh configuration information including a set of parameters regarding mesh data, and a sample of the basemesh track may include a sample stream unit for an encoded basemesh.

[0693] In relation to a displacement track of a file according to embodiments, a sample entry of the displacement track may include dynamic mesh configuration information including a parameter set regarding mesh data, and a sample of the displacement track may include a sample stream unit regarding displacement data of the mesh data. Here, the parameter set includes parameter information regarding displacement data encoded with arithmetic coding.

[0694] With respect to an atlas track of a file according to embodiments, a sample entry of the atlas track may include a parameter set regarding atlas data, and a dynamic mesh component track may include data type information indicating at least one of geometry data or attribute data encoded based on a video codec.

[0695] The decoding method of Fig. 37 can follow the reverse process of the encoding method of Fig. 36. The decoding method of Fig. 37 can be performed by a decoder such as that of Fig. 1.

[0696] A decoder according to embodiments includes a receiving unit for receiving a file including mesh data; a file decapsulator for decapsulating the file; and a decoder for decoding the mesh data; wherein the file may include a bitstream including mesh data and parameter information about the mesh data.

[0697] The embodiments provide the following effects: A transmitter or receiver for providing mesh content services can construct a V-DMC bit stream and store a file as described above.

[0698] A V-DMC bitstream can be effectively multiplexed. It enables metadata for data processing and rendering in a V-DMC bitstream to be transmitted within a file. A Video-based Dynamic Mesh Compression processing device, transmitter, receiver, mesh player, encoder or decoder provides the above-described effects. In other words, the above-described data representation method provides the effect of efficiently accessing a V-DMC bitstream. A transmitter or receiver according to embodiments of the present document can efficiently store and transmit a file of a V-DMC bitstream through a storage technique and signaling of a V-DMC bitstream as multiple tracks within a file.

[0699] The embodiments have been described in terms of methods and / or devices, and the descriptions of methods and devices may be applied complementarily.

[0700] For the convenience of explanation, each drawing has been described separately, but it is also possible to design a new embodiment by combining the embodiments described in each drawing. In addition, designing a computer-readable recording medium having a program recorded thereon for executing the previously described embodiments, as needed by a person skilled in the art, also falls within the scope of the embodiments. The devices and methods according to the embodiments are not limited to the configurations and methods of the embodiments described above, but the embodiments may be configured by selectively combining all or part of the embodiments so that various modifications can be made. Although preferred embodiments of the embodiments have been illustrated and described, the embodiments are not limited to the specific embodiments described above, and various modifications can be made by a person skilled in the art to which the present invention pertains without departing from the gist of the embodiments claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the embodiments.

[0701] The various components of the devices of the embodiments may be implemented by hardware, software, firmware, or a combination thereof. The various components of the embodiments may be implemented by a single chip, for example, a single hardware circuit. According to embodiments, the components according to the embodiments may be implemented by separate chips. According to embodiments, at least one of the components of the devices of the embodiments may be configured with one or more processors capable of executing one or more programs, and the one or more programs may perform, or include instructions for performing, one or more of the operations / methods according to the embodiments. The executable instructions for performing the methods / operations of the devices of the embodiments may be stored in non-transitory CRMs or other computer program products configured to be executed by one or more processors, or may be stored in temporary CRMs or other computer program products configured to be executed by one or more processors. In addition, the memory according to the embodiments may be used as a concept including not only volatile memory (e.g., RAM, etc.), but also non-volatile memory, flash memory, PROM, etc. Additionally, it may include implementations in the form of carrier waves, such as transmissions via the Internet. Furthermore, processor-readable recording media may be distributed across network-connected computer systems, allowing processor-readable code to be stored and executed in a distributed manner.

[0702] In this document, “ / ” and “,” are interpreted as “and / or”. For example, “A / B” is interpreted as “A and / or B”, and “A, B” is interpreted as “A and / or B”. Additionally, “A / B / C” means “at least one of A, B, and / or C”. Also, “A, B, C” means “at least one of A, B, and / or C”. Additionally, “or” in this document is interpreted as “and / or”. For example, “A or B” can mean 1) “A” only, 2) “B” only, or 3) “A and B”. In other words, “or” in this document can mean “additionally or alternatively”.

[0703] Terms such as first, second, etc. may be used to describe various components of the embodiments. However, the various components according to the embodiments should not be limited in their interpretation by the above terms. These terms are merely used to distinguish one component from another. For example, a first user input signal may be referred to as a second user input signal. Similarly, a second user input signal may be referred to as a first user input signal. The use of these terms should be interpreted as not departing from the scope of the various embodiments. Although a first user input signal and a second user input signal are both user input signals, they do not mean the same user input signals unless the context clearly indicates otherwise.

[0704] The terminology used to describe the embodiments is for the purpose of describing particular embodiments and is not intended to be limiting of the embodiments. As used in the description of the embodiments and in the claims, the singular is intended to include the plural unless the context clearly dictates otherwise. The expressions “and / or” are used to mean all possible combinations of terms. The expression “includes” describes the presence of features, numbers, steps, elements, and / or components, but does not mean that additional features, numbers, steps, elements, and / or components are not included. Conditional expressions such as “if” or “when” used to describe the embodiments are not intended to be limited to only optional cases. When a specific condition is satisfied, a related action is performed in response to a specific condition, or a related definition is intended to be interpreted.

[0705] Additionally, the operations according to the embodiments described in this document may be performed by a transceiver device including a memory and / or a processor according to the embodiments. The memory may store programs for processing / controlling the operations according to the embodiments, and the processor may control various operations described in this document. The processor may be referred to as a controller, etc. The operations according to the embodiments may be performed by firmware, software, and / or a combination thereof, and the firmware, software, and / or a combination thereof may be stored in the processor or in the memory.

[0706] Meanwhile, the operations according to the embodiments described above may be performed by a transmitting device and / or a receiving device according to the embodiments. The transmitting / receiving device may include a transmitting / receiving unit for transmitting and receiving media data, a memory for storing instructions (program code, algorithm, flowchart, and / or data) for a process according to the embodiments, and a processor for controlling the operations of the transmitting / receiving device.

[0707] The processor may be referred to as a controller or the like, and may correspond to, for example, hardware, software, and / or a combination thereof. The operations according to the above-described embodiments may be performed by the processor. Furthermore, the processor may be implemented as an encoder / decoder or the like for the operations of the above-described embodiments.

[0708] As described above, the relevant contents have been described in the best form for carrying out the embodiments.

[0709] As described above, the embodiments may be applied in whole or in part to a point cloud data transmission and reception device and system.

[0710] Those skilled in the art may make various changes or modifications to the embodiments within the scope of the embodiments.

[0711] Embodiments may include modifications / changes, which do not depart from the scope of the claims and their equivalents.

Claims

1. A step of receiving a file containing mesh data; a step of decapsulating the above file; and A step of decrypting the above mesh data; including; The file includes a bitstream including the mesh data and parameter information about the mesh data, How to decode mesh data.

2. In paragraph 1, The above bitstream includes a sample stream header and one or more sample stream units, The above sample stream header contains information about one or more sample stream units, The one or more sample stream units include at least one of atlas data about the mesh data, information about a base mesh, displacement information encoded based on arithmetic coding, displacement information encoded based on a video codec, and attribute data encoded based on a video codec. How to decode mesh data.

3. In paragraph 1, The above bitstream includes a unit including at least one of basemesh data or displacement data, The first unit of the above bitstream includes the basemesh data, The second unit of the bitstream includes the display period data, How to decode mesh data.

4. In paragraph 1, The above bitstream includes a basemesh sub-bitstream, The above basemesh sub-bitstream includes at least one of a basemesh sequence parameter set or a basemesh frame parameter set, The above bitstream further includes a displacement sub-bitstream, The above displacement sub-bitstream includes at least one of a displacement sequence parameter set, a displacement frame parameter set, a displacement reference list structure, or a displacement layer. How to decode mesh data.

5. In paragraph 1, The above file is A basemesh track containing basemesh data of the above bitstream, An atlas track containing atlas data of the above bitstream, A displacement track containing displacement data of the above bitstream, an atlas track containing atlas data of the above bitstream, or At least one dynamic mesh component track containing at least one of geometry data or attribute data, How to decode mesh data.

6. In paragraph 5, The sample entry of the above base mesh track contains dynamic mesh configuration information including a set of parameters regarding the above mesh data, The sample of the above basemesh track includes a sample stream unit for the encoded basemesh, How to decode mesh data.

7. In paragraph 5, The sample entry of the above displacement track contains dynamic mesh configuration information including a set of parameters regarding the above mesh data, The sample of the above displacement track includes a sample stream unit for displacement data of the above mesh data, How to decode mesh data.

8. In paragraph 5, A sample entry of the above atlas track contains a set of parameters regarding the above atlas data, The above dynamic mesh component track includes data type information representing at least one of encoded geometry data or attribute data based on a video codec. How to decode mesh data.

9. A receiving unit that receives a file containing mesh data; A file decapsulator for decapsulating the above file; and A decoder for decoding the above mesh data; The file includes a bitstream including the mesh data and parameter information about the mesh data, Mesh data decoding device.

10. Step of encoding mesh data; A step of encapsulating the above mesh data into a file; and A step of transmitting the above file; comprising: wherein a bitstream including the mesh data and parameter information about the mesh data is encapsulated into multiple tracks of the file. How to encode mesh data.

11. In paragraph 10, The above bitstream includes a sample stream header and one or more sample stream units, The above sample stream header contains information about one or more sample stream units, The one or more sample stream units include at least one of atlas data about the mesh data, information about a base mesh, displacement information encoded based on arithmetic coding, displacement information encoded based on a video codec, and attribute data encoded based on a video codec. How to encode mesh data.

12. In paragraph 10, The above bitstream includes a unit including at least one of basemesh data or displacement data, The first unit of the above bitstream includes the basemesh data, The second unit of the bitstream includes the display period data, How to encode mesh data.

13. In paragraph 10, The above bitstream includes a basemesh sub-bitstream, The above basemesh sub-bitstream includes at least one of a basemesh sequence parameter set or a basemesh frame parameter set, The above bitstream further includes a displacement sub-bitstream, The above displacement sub-bitstream includes at least one of a displacement sequence parameter set, a displacement frame parameter set, a displacement reference list structure, or a displacement layer. How to encode mesh data.

14. In paragraph 10, The above file is A basemesh track containing basemesh data of the above bitstream, An atlas track containing atlas data of the above bitstream, A displacement track containing displacement data of the above bitstream, an atlas track containing atlas data of the above bitstream, or At least one dynamic mesh component track containing at least one of geometry data or attribute data, How to encode mesh data.

15. Encoder that encodes mesh data; A file encapsulator that encapsulates the above mesh data into a file; and A transmission unit for transmitting the above file; including; wherein a bitstream including the mesh data and parameter information about the mesh data is encapsulated into multiple tracks of the file. Mesh data encoding device.

Images