Mesh data encoding device, mesh data encoding method, mesh data decoding device, and mesh data decoding method

Abstract

A decoding method according to embodiments may comprise the steps of: decoding a base mesh in a bitstream; decoding displacement in the bitstream; and decoding an attribute in the bitstream. An encoding method according to embodiments may comprise the steps of: encoding a base mesh of mesh data; encoding displacement of the mesh data; and encoding an attribute of the mesh data.

Description

Mesh data encoding device, mesh data encoding method, mesh data decoding device, and mesh data decoding method

[0001] The embodiments provide a method for providing Point Cloud content to provide users with various services such as Virtual Reality (VR), Augmented Reality (AR), Mixed Reality (MR), and autonomous driving services.

[0002] A point cloud is a collection of points in 3D space. The sheer number of points in 3D space makes it difficult to generate point cloud data.

[0003] Mesh data refers to a data format in which connectivity information between mesh vertices is added to point cloud data.

[0004] There is a problem that a lot of processing is required to transmit and receive dynamic mesh data.

[0005] The technical problem according to the embodiments is to provide a mesh data transmission device, a transmission method, a mesh data reception device, and a reception method for efficiently transmitting and receiving mesh data in order to solve the problems described above.

[0006] The technical problem according to the embodiments is to provide a mesh data transmission device, transmission method, mesh data reception device and reception method for resolving latency and encoding / decoding complexity.

[0007] However, the scope of the embodiments is not limited to the aforementioned technical tasks, and the scope of the embodiments may be expanded to other technical tasks that can be inferred by a person skilled in the art based on the entire contents of this document.

[0008] To achieve the above-described purpose and other advantages, a decoding method according to embodiments may include a step of decoding a base mesh in a bitstream; a step of decoding a displacement in the bitstream; and a step of decoding an attribute in the bitstream. An encoding method according to embodiments may include a step of encoding a base mesh of mesh data; a step of encoding a displacement of the mesh data; and a step of encoding an attribute of the mesh data.

[0009] The mesh data transmission method, transmission device, mesh data reception method, and reception device according to the embodiments can provide a quality mesh service.

[0010] The mesh data transmission method, transmission device, mesh data reception method, and reception device according to the embodiments can achieve various video codec methods.

[0011] The mesh transmission method, transmission device, mesh data reception method, and reception device according to the embodiments can provide general-purpose mesh content such as autonomous driving services.

[0012] The drawings are included to further understand the embodiments, and the drawings illustrate the embodiments together with the description related to the embodiments. For a better understanding of the various embodiments described below, reference should be made to the following description of the embodiments in conjunction with the following drawings, in which like reference numerals correspond to corresponding parts throughout the drawings.

[0013] Figure 1 illustrates a V-DMC based encoder and decoder according to embodiments.

[0014] Figure 2 illustrates a system for providing dynamic mesh content according to embodiments.

[0015] Figure 3 shows a V-MESH compression method according to embodiments.

[0016] Figure 4 illustrates pre-processing of V-MESH compression according to embodiments.

[0017] Figure 5 illustrates a mid-edge subdivision method according to embodiments.

[0018] Figure 6 shows a displacement generation process according to embodiments.

[0019] Figure 7 shows a V-DMC encoding process according to embodiments.

[0020] Figure 8 shows a lifting conversion process for displacement according to embodiments.

[0021] Figure 9 illustrates a process of packing transformation coefficients according to embodiments into a 2D image.

[0022] Fig. 10 shows an attribute transfer process of a V-MESH compression method according to embodiments.

[0023] Figure 11 illustrates a V-DMC decoding process according to embodiments.

[0024] Figure 12 illustrates a V-DMC encoding process according to embodiments.

[0025] Figure 13 illustrates a V-DMC decoding process according to embodiments.

[0026] Figure 14 shows a base mesh frame HRD buffer model according to embodiments.

[0027] Figure 15 shows the structure of basemesh NAL unit streams for HRD conformance checks according to embodiments.

[0028] FIG. 16 illustrates the basemesh buffering period SEI message syntax of the payload of SEI (Supplemental enhancement information) according to embodiments.

[0029] Figure 17 illustrates the Basemesh frame timing SEI message syntax of the payload of SEI according to embodiments.

[0030] Figure 18 illustrates the general basemesh sequence parameter set RBSP syntax in a bitstream according to embodiments.

[0031] Figure 19 illustrates the Basemesh VUI parameters syntax in a bitstream according to embodiments.

[0032] Figure 20 illustrates the Basemesh HRD parameters syntax in a bitstream according to embodiments.

[0033] Figure 21 shows an encoding method according to embodiments.

[0034] Figure 22 shows a decryption method according to embodiments.

[0035] Preferred embodiments of the embodiments are described in detail, examples of which are illustrated in the accompanying drawings. The following detailed description, with reference to the accompanying drawings, is intended to illustrate preferred embodiments of the embodiments, rather than merely show embodiments that can be implemented according to the embodiments. The following detailed description includes details to provide a thorough understanding of the embodiments. However, it will be apparent to those skilled in the art that the embodiments may be practiced without these details.

[0036] While most of the terms used in the examples are commonly used in the field, some terms were arbitrarily selected by the applicant, and their meanings are described in detail in the following descriptions as needed. Therefore, the examples should be understood based on the intended meaning of the terms, not simply their names or meanings.

[0037] Figure 1 illustrates a V-DMC based encoder and decoder according to embodiments.

[0038] The basic structure of the currently in-progress V-DMC (v-mesh) is shown in Figure 1. The encoder and decoder according to Figure 1 perform the encoding and decoding process of media representing dynamic meshes using V3C technology. The preprocessor converts the input dynamic mesh representation into several V3C components: a base mesh, a set of displacements, a 2D representation of attributes, and an atlas. The original mesh is simplified into a base mesh. The base mesh can be encoded using any mesh codec. Displacement vectors can be encoded into V3C geometric video components using any video codec, either indicated by a profile or using SEI messages. For example, depending on the profile, the displacement vectors (displacement data) can be encoded using arithmetic coding. The attribute data can include additional attributes. For example, texture or material information can be included as additional attributes and can be encoded using any video codec. The atlas data contains information on how to perform inverse reconstruction and is provided to the V3C decoding and / or rendering system. For example, atlas data may include how to perform subdivision of a base mesh, how to apply displacement vectors to subdivided mesh vertices, how to apply attributes to the reconstructed mesh, etc.

[0039] The encoder may comprise a memory and at least one processor connected to the memory. The at least one processor may be configured to perform operations such as a preprocessor, an atlas encoder, a basemesh encoder, a displacement vector encoder, a video encoder, and a multiplexer.

[0040] The atlas encoding unit encodes the atlas of mesh data to generate an atlas bitstream. The basemesh encoding unit encodes the basemesh of mesh data to generate a basemesh bitstream. The displacement vector encoding unit encodes the displacement vector of mesh data to generate a displacement vector bitstream. The video encoding unit encodes the properties (attributes) of mesh data to generate an attribute bitstream. The encoder generates parameter information (which may be referred to as signaling information, metadata, etc.) related to each encoding. The encoder may generate a bitstream including parameter information, atlas, basemesh, displacement vector, and / or attribute.

[0041] The decoder may be configured with a memory and at least one processor connected to the memory. The at least one processor may be configured to perform operations such as a demultiplexing unit, an atlas decoding unit, a basemesh decoding unit, a displacement vector decoding unit, and a video decoding unit.

[0042] The atlas decoding unit decodes the atlas within the bitstream. The basemesh decoding unit decodes the basemesh within the bitstream. The displacement vector decoding unit decodes the displacement vector within the bitstream. The video decoding unit decodes the attributes within the bitstream. The decoder can perform each decoding operation based on parameter information within the bitstream. The decoder can reconstruct dynamic mesh data based on the atlas, displacement vector, attributes, and base mesh.

[0043] Below, the operation of the V-DMC encoder and decoder of Fig. 1 is described in more detail.

[0044] Figure 2 illustrates a system for providing dynamic mesh content according to embodiments.

[0045] The system of FIG. 2 includes a point cloud data transmission device (100) and a point cloud data reception device (110) according to embodiments. The point cloud data transmission device may include a dynamic mesh video acquisition unit (101), a dynamic mesh video encoder (102), a file / segment encapsulator (103), and a transmitter (104). The point cloud data reception device (110) may include a reception unit (111), a file / segment decapsulator (112), a dynamic mesh video decoder (113), and a renderer (114). Each component of FIG. 1 may correspond to hardware, software, a processor, and / or a combination thereof. Hereinafter, the point cloud data transmission device according to embodiments may be interpreted as a term referring to the transmission device (100) or a dynamic mesh video encoder (hereinafter, referred to as an encoder) (102). The point cloud data receiving device according to the embodiments may be interpreted as a term referring to a receiving device (110) or a dynamic mesh video decoder (hereinafter, decoder) (113).

[0046] The system of FIG. 2 can perform video-based dynamic mesh compression and decompression.

[0047] Advances in 3D capture, modeling, and rendering have enabled users to consume diverse forms of 3D content, such as AR, XR, metaverse, and holograms, across multiple platforms and devices. 3D content increasingly represents objects with greater precision and realism, enabling users to enjoy immersive experiences. To achieve this, the creation and use of 3D models requires a significant amount of data. Among various types of 3D content, 3D meshes are widely used for efficient data utilization and realistic object representation. Embodiments include a series of processing steps in a system that utilizes such mesh content.

[0048] First, the method of compressing dynamic mesh data starts with the V-PCC (Video-based point cloud compression) standard technology. Point cloud data is data that contains color information at the vertex coordinates (X, Y, Z). Mesh data refers to data in which connectivity information between vertices is added to this vertex information. When creating content, it can be created in the form of mesh data from the beginning. By adding connectivity information to point cloud data, it can be converted into mesh data and used.

[0049] Currently, the MPEG standards body defines two types of dynamic mesh data: Category 1: Mesh data with texture maps as color information. Category 2: Mesh data with vertex colors as color information.

[0050] Mesh coding standards for Category 1 data are currently under development, and work on Category 2 data standards is also planned for the future. The overall process for providing mesh content services may include acquisition, encoding, transmission, decoding, rendering, and / or feedback, as shown in Figure 1.

[0051] To provide mesh content services, 3D data acquired through multiple cameras or specialized cameras can be processed into mesh data types through a series of processes and then converted into video. The generated mesh video is then transmitted through a series of processes, and the receiving end can then reprocess the received data into mesh video and render it. This allows mesh video to be presented to users, who can then interact with the mesh content according to their intended intent.

[0052] A mesh compression system may include a transmitting device and a receiving device. The transmitting device can encode mesh video to output a bitstream, which can be delivered to the receiving device via digital storage media or a network in the form of a file or streaming segment. The digital storage media may include various storage media, such as USB, SD, CD, DVD, Blu-ray, HDD, or SSD.

[0053] The transmitting device may roughly include a mesh video acquisition unit, a mesh video encoder, and a transmitting unit. The receiving device may roughly include a receiving unit, a mesh video decoder, and a renderer. The encoder may be referred to as a mesh video / video / picture / frame encoding device, and the decoder may be referred to as a mesh video / video / picture / frame decoding device. The transmitter may be included in the mesh video encoder. The receiver may be included in the mesh video decoder. The renderer may include a display unit, and the renderer and / or the display unit may be configured as separate devices or external components. The transmitting device and the receiving device may further include separate internal or external modules / units / components for a feedback process.

[0054] Mesh data represents the surface of an object as a number of polygons. Each polygon is defined by its vertices in 3D space and connection information that describes how those vertices are connected. It can also contain vertex properties such as vertex color and normal. Mapping information that allows the surface of the mesh to be mapped to a 2D planar area can also be included as a mesh property. The mapping is typically described as a set of parametric coordinates, called UV coordinates or texture coordinates, associated with the mesh vertices. Meshes contain 2D attribute maps, which can be used to store high-resolution attribute information such as textures, normals, and displacement.

[0055] The mesh video acquisition unit may include processing 3D object data acquired through a camera, etc. into a mesh data type with the properties described above through a series of processes and generating a video composed of such mesh data. The mesh video may have properties of the mesh, such as vertices, polygons, connection information between vertices, colors, normals, etc., that may change over time. A mesh video with properties and connection information that change over time can be expressed as a dynamic mesh video.

[0056] A mesh video encoder can encode an input mesh video into one or more video streams. A single video can include multiple frames, and a single frame can correspond to a still image / picture. In this document, a mesh video can include a mesh image / frame / picture, and the mesh video can be used interchangeably with the mesh image / frame / picture. A mesh video encoder can perform a Video-based Dynamic Mesh (V-Mesh) Compression procedure. A mesh video encoder can perform a series of procedures such as prediction, transformation, quantization, and entropy coding for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0057] The encapsulation processing unit (file / segment encapsulation module) can encapsulate encoded mesh video data and / or mesh video-related metadata in the form of a file, etc. Here, the mesh video-related metadata may be received from the metadata processing unit, etc. The metadata processing unit may be included in the mesh video encoder, or may be configured as a separate component / module. The encapsulation processing unit may encapsulate the corresponding data in a file format such as ISOBMFF, or process it in the form of other DASH segments, etc. The encapsulation processing unit may include mesh video-related metadata in the file format according to an embodiment. The mesh video metadata may be included in boxes at various levels in the ISOBMFF file format, for example, or may be included as data in a separate track within the file. According to an embodiment, the encapsulation processing unit may encapsulate mesh video-related metadata itself in a file.

[0058] The transmission processing unit can process encapsulated mesh video data for transmission according to the file format. The transmission processing unit can be included in the transmission unit, or can be configured as a separate component / module. The transmission processing unit can process mesh video data according to any transmission protocol. The processing for transmission can include processing for transmission through a broadcast network or processing for transmission through broadband. According to an embodiment, the transmission processing unit can receive not only mesh video data but also mesh video-related metadata from the metadata processing unit and process the same for transmission.

[0059] The transmission unit can transmit encoded video / image information or data output in the form of a bitstream to the receiving unit of a receiving device via a digital storage medium or network in the form of a file or streaming. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmission unit can include an element for generating a media file via a predetermined file format and can include an element for transmission via a broadcasting / communication network. The receiving unit can extract the bitstream and transmit it to a decoding device.

[0060] The receiver can receive mesh video data transmitted by a mesh video transmission device. Depending on the transmission channel, the receiver can receive mesh video data via a broadcast network, via broadband, or via digital storage media.

[0061] The receiving processing unit can perform processing on the received mesh video data according to a transmission protocol. The receiving processing unit can be included in the receiving unit, or can be configured as a separate component / module. In response to the processing performed for transmission on the transmitting side, the receiving processing unit can perform the reverse process of the aforementioned transmitting processing unit. The receiving processing unit can transfer the acquired mesh video data to the decapsulation processing unit, and transfer the acquired mesh video-related metadata to a metadata parser. The mesh video-related metadata acquired by the receiving processing unit can be in the form of a signaling table.

[0062] A decapsulation processing unit (file / segment decapsulation module) can decapsulate mesh video data in file format received from a receiving processing unit. The decapsulation processing unit can decapsulate files according to ISOBMFF, etc., to obtain a mesh video bitstream or mesh video-related metadata (metadata bitstream). The obtained mesh video bitstream can be transmitted to a mesh video decoder, and the obtained mesh video-related metadata (metadata bitstream) can be transmitted to a metadata processing unit. The mesh video bitstream may include metadata (metadata bitstream). The metadata processing unit may be included in the mesh video decoder, or may be configured as a separate component / module. The mesh video-related metadata obtained by the decapsulation processing unit may be in the form of a box or track within a file format. If necessary, the decapsulation processing unit may receive metadata required for decapsulation from the metadata processing unit. Mesh video related metadata can be passed to a Mesh video decoder for use in the Mesh video decoding process, or passed to a renderer for use in the Mesh video rendering process.

[0063] A mesh video decoder can receive a bitstream and perform operations corresponding to those of a mesh video encoder to decode video / images. The decoded mesh video can be displayed via a display unit. Users can view all or part of the rendered result via a VR / AR display or a general display.

[0064] The feedback process may include a process of transmitting various feedback information that may be acquired during the rendering / display process to the transmitter or to the decoder on the receiver. Interactivity may be provided in mesh video consumption through the feedback process. Depending on the embodiment, head orientation information, viewport information indicating the area that the user is currently viewing, etc. may be transmitted during the feedback process. Depending on the embodiment, the user may interact with things implemented in the VR / AR / MR / autonomous driving environment, in which case information related to the interaction may be transmitted to the transmitter or the service provider during the feedback process. Depending on the embodiment, the feedback process may not be performed.

[0065] Head orientation information can refer to information about the user's head position, angle, and movement. Based on this information, information about the area the user is currently viewing within the mesh video, i.e. viewport information, can be calculated.

[0066] Viewport information can be information about the area the user is currently viewing in the mesh video. This can be used to perform gaze analysis to determine how the user consumes the mesh video, which area of ​​the mesh video they are gazing at, and for how long. Gaze analysis can be performed on the receiving side and transmitted to the transmitting side through a feedback channel. Devices such as VR / AR / MR displays can extract the viewport area based on the user's head position / orientation, the vertical or horizontal FOV supported by the device, etc.

[0067] Depending on the embodiment, the aforementioned feedback information may not only be transmitted to the transmitter but may also be consumed by the receiver. That is, the aforementioned feedback information may be utilized to perform decoding, rendering, and other processes on the receiver. For example, head orientation information and / or viewport information may be utilized to preferentially decode and render only the mesh video for the area currently being viewed by the user.

[0068] This document relates to dynamic mesh video compression as described above. The method / embodiment disclosed in this document can be applied to the Video-based Dynamic Mesh Compression (V-Mesh) standard of the Moving Picture Experts Group (MPEG) or the next-generation video / image coding standard. Dynamic mesh video compression is a method for processing mesh connection information and properties that change over time, and it can perform lossy and lossless compression for various applications such as real-time communication, storage, free-view video, and AR / VR.

[0069] The dynamic mesh video compression method described below is based on MPEG's V-Mesh method.

[0070] In this document, picture / frame can generally mean a unit representing one video of a specific time period.

[0071] A pixel or pel can refer to the smallest unit that constitutes a picture (or image). Additionally, the term "sample" can be used as a counterpart to a pixel. A sample can generally represent a pixel or a pixel value, and can represent only the pixel / pixel value of the luma component, only the pixel / pixel value of the chroma component, or only the pixel / pixel value of the depth component.

[0072] A unit may represent a basic unit of image processing. A unit may include at least one of a specific region of a picture and information related to the region. In some cases, the term "unit" may be used interchangeably with terms such as "block" or "area." In general, an MxN block may include a set (or array) of samples (or sample array) or transform coefficients consisting of M columns and N rows.

[0073] The encoding process of Figure 2 is as follows.

[0074] Video-based dynamic mesh compression (V-Mesh) compression methods can provide a method for compressing dynamic mesh video data based on 2D video codecs such as HEVC and VVC. The V-Mesh compression process receives the following data as input and performs compression.

[0075] Input mesh: Contains the 3D coordinates (geometry) of the vertices that make up the mesh, normal information for each vertex, mapping information that maps the mesh surface to a 2D plane, and connection information between the vertices that make up the surface. The mesh surface can be expressed as triangles or more polygons, and connection information between the vertices that make up each surface is stored according to a set shape. The input mesh can be saved in the OBJ file format.

[0076] Attribute map: (Hereinafter, texture map is also used in the same meaning): Contains information about the properties (color, normal, displacement, etc.) of the mesh, and stores data in the form of a mapping of the surface of the mesh onto a 2D image. Mapping which part (surface or vertex) of the mesh each data of this attribute map corresponds to is based on the mapping information contained in the input mesh. Since the attribute map has data for each frame of the mesh video, it can also be expressed as an attribute map video (or attribute for short). The attribute map in the V-Mesh compression method mainly contains the color information of the mesh, and is saved in an image file format (PNG, BMP, etc.).

[0077] Material Library File: Contains information about the material properties used in a mesh, particularly information that links the input mesh to its corresponding attribute map. It is saved in the Wavefront Material Template Library (MTL) file format.

[0078] In the V-Mesh compression method, the following data and information can be generated through the compression process.

[0079] Base mesh: The input mesh is simplified (decimated) through a preprocessing process to express the objects of the input mesh using the minimum number of vertices determined by the user's standards.

[0080] Displacement: This is displacement information used to express the input mesh as similarly as possible to the base mesh, and is expressed in the form of 3D coordinates.

[0081] Atlas information: This is the metadata required to reconstruct a mesh using base mesh, displacement, and attribute map information. It can be created and utilized as sub-mesh units (such as patches) that make up the mesh.

[0082] Referring to FIGS. 3 to 7, a method for encoding mesh position information (vertex) is described, and referring to FIGS. 7-10, etc., a method for encoding attribute information (attribute map) by restoring mesh position information is described.

[0083] Figure 3 shows a V-MESH compression method according to embodiments.

[0084] Fig. 3 illustrates the encoding process of Fig. 2, and the encoding process may include a pre-processing process and an encoding process. The encoder of Fig. 2 may include a pre-processor (200) and an encoder (201) as in Fig. 3. The transmitting device of Fig. 2 may be broadly referred to as an encoder, and the dynamic mesh video encoder of Fig. 2 may be referred to as an encoder. The V-Mesh compression method may include a pre-processing (200) and an encoding (201) process as in Fig. 3. The pre-processor of Fig. 3 may be located in front of the encoder of Fig. 3. The pre-processor and the encoder of Fig. 3 may be referred to as a single encoder.

[0085] The preprocessor can receive a static dynamic mesh and / or an attribute map. The preprocessor can generate a base mesh and / or displacement through preprocessing. The preprocessor can receive feedback information from the encoder and generate the base mesh and / or displacement based on the feedback information.

[0086] The encoder can receive a base mesh, displacement mesh, static dynamic mesh, and / or attribute map. The encoder can encode mesh-related data to generate a compressed bitstream.

[0087] Figure 4 illustrates pre-processing of V-MESH compression according to embodiments.

[0088] Figure 4 shows the configuration and operation of the pre-processor of Figure 3.

[0089] Fig. 3 shows a process of performing preprocessing on an input mesh. The preprocessing process (200) can be broadly divided into four steps: 1) Group of Frame (GoF) generation, 2) Mesh Decimation, 3) UV parameterization, and 4) Fitting subdivision surface (300). The preprocessor (200) can receive an input mesh, generate a displacement and / or base mesh, and transmit the generated displacement and / or base mesh to the encoder (201). The preprocessor (200) can transmit GoF information related to GoF generation to the encoder (201).

[0090] Below, each step of Fig. 4 is described.

[0091] GoF Generation: This is the process of generating a reference structure for mesh data. If the number of vertices, number of texture coordinates, vertex connection information, and texture coordinate connection information of the mesh of the previous frame and the current mesh are all the same, the previous frame can be set as the reference frame. In other words, if only the vertex coordinate values ​​are different between the current input mesh and the reference input mesh, inter-frame encoding can be performed. Otherwise, the frame performs intra-frame encoding.

[0092] Mesh Decimation: This process simplifies the input mesh to create a simplified mesh, or base mesh. Vertices to be removed from the original mesh are selected based on user-defined criteria, and the selected vertices and the triangles connected to them can be removed.

[0093] In the process of performing mesh simplification (Mesh decimation), the input mesh (voxelized), target triangle ratio (TTR), and minimum triangle component (CCCount) information are passed as input, and the simplified mesh (decimated mesh) can be obtained as output. In this process, connected triangle components smaller than the set minimum triangle component (CCCount) can be removed.

[0094] UV parameterization: This is the process of mapping a 3D surface of a decimated mesh into a texture domain. Parameterization can be performed using the UVAtlas tool. This process generates mapping information, which indicates where each vertex of the decimated mesh can be mapped to on a 2D image. This mapping information is expressed and stored as texture coordinates, and through this process, the final base mesh is created.

[0095] OrthoAtlas technology generates texture coordinates using orthographic projection. OrthoAtlas technology sequentially generates patches and packs them. First, adjacent triangles are divided to generate Connected Components (CCs), and then the optimal CCs are merged using a cost function to generate patches. The cost function can measure the cost based on the degree of distortion that occurs when orthogonally projecting patches in each direction. By packing the patch that minimizes the cost function into the texture domain, the texture coordinates can be ultimately calculated. In the case of orthoAtlas technology, texture coordinates and texture connection information can be derived from the base mesh decoder without compressing them during the base mesh encoding process.

[0096] Fitting subdivision surface: This is the process of performing subdivision on a simplified mesh. The subdivision method can be a user-defined method, such as the mid-edge method. The fitting process ensures that the input mesh and the subdivision mesh are similar to each other.

[0097] This is the process of fitting a mesh that has undergone subdivision on a base mesh so that its surface resembles that of the input mesh. User-defined subdivision methods, such as the mid-edge method (Figure 5), the loop method, or the LS3 method, can be applied.

[0098] Figure 5 illustrates a mid-edge subdivision method according to embodiments.

[0099] Figure 5 illustrates the mid-edge method of the fitting subdivision surface described in Figure 4. Referring to Figure 5, an original mesh containing four vertices is subdivided to create a sub-mesh. A sub-mesh can be created by creating a new vertex midway between the edges between the vertices.

[0100] When a fitted subdivided mesh (hereinafter referred to as a fitted subdivided mesh) is generated, displacement is calculated using this result and a pre-compressed and decrypted base mesh (hereinafter referred to as a reconstructed base mesh). That is, the reconstructed base mesh is subdivided in the same way as the fitting subdivision surface. The difference in position of each vertex between this result and the fitted subdivided mesh is the displacement for each vertex. Since displacement represents the position difference in three-dimensional space, it is also expressed as a value in the (x, y, z) space of a Cartesian coordinate system. Depending on the user input parameters, the (x, y, z) coordinate values ​​can be converted to (normal, tangential, bi-tangential) coordinate values ​​of the local coordinate system.

[0101] Figure 6 shows a displacement generation process according to embodiments.

[0102] FIG. 6 illustrates in detail the displacement calculation method of the fitting subdivision surface (300) as described in FIG. 5.

[0103] An encoder and / or pre-processor according to embodiments may include 1) a subdivision unit, 2) a local coordinate system calculation unit, and 3) a displacement calculation unit. The subdivision unit may receive a reconstructed base mesh and generate a subdivided reconstructed base mesh. The local coordinate system calculation unit may receive a fitted subdivision mesh and a subdivided reconstructed base mesh, and transform a coordinate system of the mesh into a local coordinate system. The local coordinate system calculation operation may be optional. The displacement calculation unit may calculate a positional difference between the fitted subdivision mesh and the subdivided reconstructed base mesh. For example, a positional difference value between vertices of two input meshes may be generated. The vertex positional difference value becomes a displacement.

[0104] The method and device for transmitting point cloud data according to the embodiments can encode the point cloud as follows. The point cloud data (which may be referred to as a point cloud for short) according to the embodiments can refer to data including vertex coordinates and color information. The term "point cloud" includes mesh data, and in this document, point cloud and mesh data can be used interchangeably.

[0105] The V-Mesh compression (reconstruction) method according to the embodiments may include intra frame encoding (Fig. 6) and inter frame encoding (Fig. 7).

[0106] Based on the results of the GoF generation described above, intra-frame encoding or inter-frame encoding is performed. In the case of intra-encoding, the data to be compressed may be a base mesh, displacement, attribute map, etc. In the case of inter-encoding, the data to be compressed may be a displacement, attribute map, and a motion field between a reference base mesh and the current base mesh.

[0107] Figure 7 shows a V-DMC encoding process according to embodiments.

[0108] The encoding process of Fig. 7 details the encoding of Figs. 1 and 2.

[0109] The preprocessor can receive an input mesh and perform the preprocessing described above. The preprocessing can generate a base mesh and / or a fitted subdivided mesh. The quantizer can quantize the base mesh and / or the fitted subdivided mesh. The static mesh encoder can encode the static mesh. The static mesh encoder can generate a bitstream including the encoded base mesh. The motion encoder can encode a motion vector for the base mesh based on inter-frame motion estimation and motion compensation for inter-prediction. The atlas encoder can encode an atlas for the vertices of the base mesh. The encoded base mesh can be reconstructed and inversely quantized through a dequantizer. The displacement calculation unit can receive the reconstructed mesh and generate displacement, which is a position difference, based on the fitted subdivided mesh. A lifting transform unit can receive displacement and generate lifting coefficients. A quantizer can quantize the lifting coefficients. Depending on an encoding method, an image packing unit can pack an image based on the quantized lifting coefficients. A video encoder can encode the packed image. Depending on the encoding method, inter-prediction can be applied to the quantized lifting coefficients, and the predicted lifting coefficients can be encoded according to an arithmetic encoding method. A mesh restoration unit restores a warped mesh using the restored displacement and the restored base mesh. Displacement data is restored, and the warped mesh is restored based on the restored displacement data and the restored base mesh, and provided to an attribute transfer unit. The attribute transfer unit receives an input mesh and / or an input attribute map, and generates an attribute map based on the restored warped mesh. A push-pull padding unit can pad data in the attribute map based on a push-pull method. The color space conversion unit can convert the space of the color component, which is an attribute. The video encoder can encode the attribute.A multiplexer can generate a bitstream by multiplexing compressed base meshes, compressed displacements, and compressed attributes.

[0110] Base Mesh Encoding: The base mesh compression method can be divided into INTRA type, INTER type, and SKIP type depending on the base mesh type, and encoding can be performed in different ways for each. If the base mesh is INTRA type, it can be encoded using the static mesh encoding method. If the base mesh is INTER type, the motion field between the reference base mesh and the current base mesh can be encoded. If the current base mesh is SKIP type, the reference base mesh can be derived as the current base mesh.

[0111] After encoding in the encoder, the decoded base mesh can be subdivided into a subdivided mesh through a subdivision process. Subdivision algorithms such as mid-point subdivision and loop subdivision can be used.

[0112] Static Basemesh Encoding (Intra Basemesh Encoding): When performing Intra encoding on the current basemesh, the base mesh generated in the preprocessing process can be encoded using static mesh compression technology after going through a quantization process. Static mesh compression applies MPEG EdgeBreaker (MEB) technology, and the vertex position information, mapping information (texture coordinates), vertex connection information, and normals of the base mesh are compressed.

[0113] Technology for compressing connection information can be encoded using the edgebreaker algorithm. The edgebreaker algorithm sequentially traverses triangles according to rules, mapping symbols based on the characteristics of each triangle, and then encoding the corresponding symbols.

[0114] A technique for compressing vertex position information can encode the residual value, which is the difference between the current vertex and the predicted value, after obtaining the predicted value based on a prediction technique such as multiple parallelogram prediction.

[0115] A technique for compressing mapping information (texture coordinates) can encode the residual value, which is the difference between the current mapping information (texture coordinates) and the predicted value, after obtaining the predicted value based on a prediction technique such as stretch.

[0116] Techniques for compressing normals can encode residual values, which are the differences between the current normal and the predicted values, after obtaining predicted values ​​based on prediction techniques such as delta coding, multiple parallelogram prediction, and cross product-based prediction.

[0117] Motion Field Encoding (Inter Basemesh Encoding): Inter Basemesh encoding can be performed when a one-to-one correspondence exists between the reference mesh and the current input mesh, and only the vertex position information differs. When performing Inter encoding, instead of compressing the base mesh, the difference between the vertices of the reference base mesh and the current base mesh, i.e. the motion field, can be calculated and this information can be encoded. The reference base mesh is the result of quantizing the already decoded base mesh data and is determined by the reference frame index.

[0118] The motion field can be encoded as is, or the predicted motion field can be calculated by averaging the motion fields of the reconstructed vertices among the vertices connected to the current vertex, and the residual motion field, which is the difference between the predicted motion field value and the motion field value of the current vertex, can be encoded. This value can be encoded using entropy coding.

[0119] Displacement Encoding: After encoding the base mesh, it is restored and dequantized to generate a Recon. The base mesh is then subdivided to calculate the displacement between the results and the fitted subdivided mesh. For effective encoding, a data transform process such as the Wavelet transform can be applied to the displacement information, and Figure 7 shows the process of transforming the displacement information using the Lifting transform in V-Mesh. The transform coefficients generated through the transform process are quantized, and the quantized transform coefficients can be compressed using a video codec or through arithmetic encoding, depending on the compression method.

[0120] When compressed through a video codec, it is packed into a 2D image as shown in Figure 8. The transform coefficients are organized into one block for each N^2(N*N) unit, and each block can be packed in z-scan order. The horizontal number of blocks is fixed to N, but the vertical number of blocks can be determined according to the number of vertices of the subdivided base mesh. Within one block, the transform coefficients can be packed by sorting them with Morton code. The packed images generate a displacement video for each GoF unit, and this displacement video can be encoded using an existing video compression codec.

[0121] When compressed through arithmetic coding, inter-frame prediction can be performed on the quantized displacement vector transform coefficients. When inter-frame prediction is performed on the current quantized displacement vector transform coefficients, the residual value, which is the difference between the current displacement vector transform coefficients and the reference displacement vector transform coefficients, can be encoded, and information about the reference target can be encoded. Depending on the displacement vector type, if it is an INTRA type, the quantized displacement vector transform coefficients can be arithmetic-coded, and if it is an INTER type, the residual value can be arithmetic-coded. Arithmetic coding can be performed based on Context Adaptive Binary Arithmetic Coding (CABAC). The CABAC process can first binarize the displacement vector data and map it to a bin string. The bin string can be a binarized output of 0 and 1, and each 0 or 1 can be a bin. Each bin can be arithmetic-coded using context information selected from a context model, and a process of updating the probability can be performed.

[0122] Figure 8 shows a lifting conversion process for displacement according to embodiments.

[0123] Figure 9 illustrates a process of packing transformation coefficients according to embodiments into a 2D image.

[0124] Figures 8-9 show the process of converting the displacement of the encoding process of Figure 7 and the process of packing the conversion coefficients, respectively.

[0125] The encoding method according to the embodiments includes displacement encoding.

[0126] After encoding the base mesh through base mesh encoding and / or motion field encoding, a reconstructed base mesh is generated through restoration and dequantization, and the displacement between the result of performing subdivision on the reconstructed base mesh and the fitted subdivided mesh generated through the fitting subdivision surface can be calculated. For effective encoding, a data transform process such as wavelet transform can be applied to the displacement information.

[0127] Fig. 8 shows the process of transforming displacement information using lifting transform in V-Mesh. The transform coefficients generated through the transform process are quantized and then packed into a 2D image as shown in Fig. 9. The transform coefficients are organized into one block for every 256 (=16X16) units, and each block can be packed in z-scan order. The horizontal number of blocks is fixed to 16, but the vertical number of blocks can be determined according to the number of vertices of the subdivided base mesh. The transform coefficients can be packed by sorting them with Morton code within a block. The packed images generate a displacement video for each GoF unit, and this displacement video can be encoded using an existing video compression codec.

[0128] Referring to FIG. 8, the base mesh (original) may include vertices and edges for LoD0. The first subdivision mesh generated by dividing the base mesh includes vertices generated by further dividing the edges of the base mesh. The first subdivision mesh includes vertices for LoD0 and vertices for LoD1. LoD1 includes the subdivided vertices and the vertices (LoD0) of the base mesh. The first subdivision mesh may be generated by dividing the second subdivision mesh. The second subdivision mesh includes LoD2. LoD2 includes the base mesh vertices (LoD0), LoD1 including the vertices additionally generated from LoD0, and the vertices additionally divided from LoD1. LoD is a level indicating the degree of detail (Level of Detail), and as the level index increases, the distance between vertices becomes closer and the level of detail increases. LoD N includes the vertices included in the previous LoDN-1 as they are. When a vertex is further divided through subdivision, considering the previous vertices v1, v2 and the subdivided vertex v, the mesh can be encoded based on a prediction and / or update method. Instead of still encoding information about the current LoD N, a residual value between the previous LoD N-1 can be generated, and the mesh can be encoded using the residual value to reduce the size of the bitstream. The prediction process means the operation of predicting the current vertex v using the previous vertices v1, v2. Since adjacent subdivision meshes have similar data, efficient encoding can be achieved by utilizing this property. The current vertex position information is predicted as the residual for the previous vertex position information, and the previous vertex position information is updated using the residual.

[0129] Referring to Figure 9, the vertices have coefficients generated through the lifting transformation. The coefficients of the vertices related to the lifting transformation can be encoded by packing them into an image.

[0130] Fig. 10 shows an attribute transfer process of a V-MESH compression method according to embodiments.

[0131] Figure 10 shows the detailed operation of attribute transfer of the encoding of Figure 7.

[0132] Encoding according to embodiments includes attribute map encoding.

[0133] Information about the input mesh is compressed through base mesh encoding, motion field encoding, and displacement encoding. In the encoding process, the compressed input mesh is restored through base mesh decoding (intra frame), motion field decoding (inter frame), and displacement video decoding processes, and the restored result, the reconstructed deformed mesh (hereinafter referred to as Recon. deformed mesh), is used to compress the input attribute map as shown in FIGS. 6 and 7. The reconstructed deformed mesh (Recon. deformed mesh) has vertex position information, texture coordinates, and corresponding connection information, but does not have color information corresponding to the texture coordinates. Therefore, as shown in Fig. 10, in the V-Mesh compression method, a new attribute map having color information corresponding to the texture coordinates of the reconstructed deformed mesh is created through the attribute transfer process.

[0134] Attribute transfer first checks whether each point P(u, v) in the 2D texture domain belongs to a texture triangle of the reconstructed deformed mesh, and if it is in a texture triangle T, then the barycentric coordinate (α, α) of P(u, v) according to the triangle T is calculated. , γ) is calculated. And the 3D vertex positions of triangle T and (α, , γ) to compute the 3D coordinates M(x, y, z) of P(u, v). Find the vertex coordinates M'(x', y', z') and the triangle T' containing this point that corresponds to the most similar position to the computed M(x, y, z) in the input mesh domain. Then, in this triangle T', the center of mass coordinates (α', ', γ') are calculated. The texture coordinates corresponding to the three vertices of Triangle T' and (α', ', γ') is used to calculate the texture coordinates (u', v'), and the color information corresponding to these coordinates is found in the input attribute map. The color information found in this way is immediately assigned to the pixel location (u, v) of the new attribute map. If P(u, v) does not belong to any triangle, the pixel at that location in the new attribute map can be filled with a color value using a padding algorithm such as the push-pull algorithm.

[0135] The new attribute map generated through attribute transfer is grouped into GoF units to form an attribute map video, which is then compressed using a video codec.

[0136] Atlas Encoding: Atlas information may be transmitted during the aforementioned process. The Atlas consists of information required for mesh decoding and / or rendering, and may include information required for subdivision, displacement decoding, base mesh decoding, tile information, patch information, and other information. Atlas data may be encoded using methods such as Exp-Golomb coding.

[0137] Referring to Figure 10, the reference relationship between the input mesh, input attribute map, restored mesh, and generated attribute map can be seen.

[0138] The decoding process of Fig. 1 can perform the reverse process of the corresponding encoding process of Fig. 1. The specific decoding process is as follows.

[0139] Figure 11 illustrates a VV-DMC decoding process according to embodiments.

[0140] Fig. 11 shows the configuration and operation of a decoder of a receiving device such as Fig. 1.

[0141] The input bitstream can be separated into a basemesh sub-stream, a displacement sub-stream, an attribute map sub-stream, and an atlas sub-stream.

[0142] Atlas sub-streams can be decoded through Exp-Golomb coding, etc., and as a result, information necessary for decoding, such as tile information and patch information, can be obtained.

[0143] If the Basemesh sub-stream is of INTRA type according to the Basemesh type, it can be decoded through a static mesh decoder based on MEB (MPEG EdgeBreaker) technology, and as a result, the connection information, vertex geometry information, and vertex mapping information (texture coordinates) of the Basemesh can be restored.

[0144] When the texture parameterization method in the encoder is orthoAtlas, the decoder can derive mapping information (texture coordinates) and attribute information (texture) connection information using vertex coordinates. The process of deriving mapping information (texture coordinates) and connection information can generate mapping information (texture coordinates) and attribute information (texture) connection information by calculating the homography transform of each face and then projecting the vertices based on this.

[0145] If the Basemesh type is INTER, motion information can be decoded through entropy decoding and inverse prediction. The reconstructed motion information is combined with the reference Basemesh, which has already been reconstructed and stored in the buffer, to create a Reconstructed Quantized Basemesh for the current frame. The reconstructed Basemesh can then undergo an inverse quantization process.

[0146] If the displacement sub-stream is compressed through a video codec according to the compression method used in encoding, it is decoded into displacement video through the decoder of the video compression codec, and then the image unpacking process is performed.

[0147] When compressed through arithmetic coding, the displacement vector bitstream can decode the binarized syntax elements through arithmetic decoding, and a contextual probability model (CPM) can be adaptively determined according to each bin of the syntax elements, and the occurrence probability of the bin can be predicted through the CPM to perform arithmetic decoding. The binarized syntax elements can be decoded through inverse binarization. The quantized displacement vector transform coefficients can be derived from the decoded syntax elements. If the displacement information type is INTER, an inverse inter prediction process is performed using reference information for the quantized displacement coefficients (if inter prediction is performed).

[0148] The quantized displacement coefficient is restored as displacement information for each vertex through inverse quantization, inverse transform, and coordinate system transformation processes.

[0149] The restored base mesh and restored displacement information are combined to generate the final decoded mesh. The attribute map sub-stream is decoded through the decoder of the video compression codec used in encoding, and then restored to the final attribute map through processes such as color format conversion.

[0150] The restored Decoded mesh and Decoded attribute map can be utilized by the receiver as final mesh data that can be utilized by the user.

[0151] The atlas decoder decodes the atlas data within the bitstream.

[0152] The motion decoder derives the motion field of the base mesh of the current frame through motion estimation and compensation based on the base mesh in the reference frame, if the mesh data in the bitstream is encoded based on inter prediction. The spatial decoder decodes the base mesh, if the mesh data in the bitstream is encoded based on intra prediction. Depending on the encoding method, the displacement data is decoded by applying arithmetic encoding decoding or video decoding. The video decoder decodes the attribute data in the bitstream.

[0153] The decoding method of Fig. 11 can follow the reverse process of the encoding method according to the embodiments.

[0154] Figure 12 illustrates a V-DMC encoding process according to embodiments.

[0155] Fig. 12 illustrates the configuration and operation of the encoder of the transmitting device of Figs. 1 and 2. Each component of Fig. 12 corresponds to hardware, software, a processor, and / or a combination thereof.

[0156] Figure 12 shows the encoding process of V-Mesh technology.

[0157] The mesh preprocessing unit receives the original mesh as input and generates a simplified mesh (decimated mesh). Simplification can be performed based on the target number of vertices or target number of polygons that constitute the mesh. Parameterization can be performed on the simplified mesh to generate mapping information (texture coordinates) and attribute information (texture) connection information per vertex. Additionally, quantization of floating-point mesh information into fixed-point information can be performed. This result can be encoded as a base mesh through a static mesh encoding unit. The mesh preprocessing unit can perform mesh subdivision on the base mesh to generate additional vertices. Depending on the subdivision method, vertex connection information, texture coordinates, and texture coordinate connection information including the added vertices can be generated. The subdivided mesh can be fitted by adjusting the vertex positions to resemble the original mesh, thereby generating a fitted subdivided mesh.

[0158] The base mesh generated through the mesh preprocessing unit can perform intra encoding or inter encoding depending on the base mesh type. If the base mesh frame performs intra encoding, it can be compressed through the static mesh encoding unit. In this case, encoding can be performed on the connection information, vertex geometry information, vertex texture information, normal information, etc. of the base mesh. If the base mesh frame performs inter encoding, a motion vector encoding unit is performed, which can use the base mesh and the reference reconstruction base mesh as input to calculate the motion vector between the two meshes and encode the value. The motion vector encoding unit can perform connection information-based prediction using the previously encoded / decoded motion vector as a predictor, and can encode the residual motion vector obtained by subtracting the predicted motion vector from the current motion vector. The base mesh bitstream generated through the base mesh encoding unit is transmitted to the multiplexing unit.

[0159] The encoded base mesh bitstream can generate a restored base mesh through a base mesh restoration unit.

[0160] The displacement vector calculation unit can perform mesh refinement on the restored base mesh. The displacement vector can be calculated as the difference in vertex positions between the refined restored base mesh and the fitted subdivision mesh generated in the preprocessing unit. As a result, the displacement vector can be calculated as many times as the number of vertices of the refined mesh. The displacement vector calculation unit can convert the displacement vector calculated in the 3D Cartesian coordinate system into a local coordinate system based on the normal vector of each vertex.

[0161] The displacement vector processing unit can transform the displacement vector for effective encoding. The transform can be performed by a lifting transform, a wavelet transform, etc. depending on the embodiment. In addition, quantization can be performed on the transformed displacement vector value, i.e., the transform coefficient. Different quantization parameters can be applied to each axis of the transform coefficient, and the quantization parameters can be derived according to the agreement of the encoder / decoder. The quantized displacement vector transform coefficients calculated by the displacement vector processing unit can be encoded through a displacement vector video encoding unit or a displacement vector arithmetic encoding unit depending on the compression method.

[0162] The displacement vector video encoding unit can pack displacement vector information that has undergone transformation and quantization into a 2D image. The packed 2D images can be bundled for each frame to generate a displacement vector video, and the displacement vector video can be generated for each GoF (Group of Frame) unit of the input mesh. The generated displacement vector video can be encoded using a video compression codec. The generated displacement vector video bitstream is transmitted to the multiplexing unit.

[0163] The displacement vector arithmetic encoding unit can perform inter-screen prediction on the quantized displacement vector transform coefficients if the displacement vector type is INTER. The inter-screen prediction process may be a process of obtaining a residual value, which is the difference between the current transform coefficient and the reference transform coefficient. The displacement vector transform coefficient or the residual value can be encoded through an arithmetic encoding process.

[0164] The displacement vector restored through the displacement vector restoration unit and the base mesh restored through the base mesh restoration unit and refined are restored through the mesh restoration unit, and the restored mesh has restored vertices, connection information between vertices, texture coordinates, and connection information between texture coordinates.

[0165] The attribute information (texture map) of the original mesh can be regenerated as attribute information (texture map) for the restored mesh through the attribute information (texture map) video generation unit. The color information per vertex of the texture map of the original mesh can be assigned to the texture coordinates of the restored mesh. The regenerated texture maps for each frame can be bundled by GoF unit to generate a texture map video.

[0166] The generated texture map video can be encoded using a video compression codec through a texture map video encoding unit. The texture map video bitstream generated through encoding is transmitted to a multiplexing unit.

[0167] The atlas encoding unit can encode atlases, which are additional information required for mesh decoding and rendering. The generated atlas bitstream is transmitted to the multiplexing unit.

[0168] The generated base mesh bitstream, displacement vector bitstream, texture map bitstream, and atlas bitstream can be multiplexed into a single bitstream and transmitted to a receiver via a transmitter. Alternatively, the generated base mesh bitstream, displacement vector bitstream, texture map bitstream, and atlas bitstream can be generated into a file with one or more track data or encapsulated into segments and transmitted to a receiver (decoder) via a transmitter.

[0169] The data input unit can receive an original mesh and / or an original texture map ('attribute'). The mesh preprocessing unit can simplify the original mesh to generate a base mesh and fit it to generate a refined mesh. The motion vector encoding unit can generate a motion vector (motion field) by referring to a reconstructed base mesh in a previously processed reference frame when the mesh encoding method is inter-prediction, and can encode it based on a motion estimation and compensation method. The static mesh encoding unit can encode the base mesh in the frame when the mesh encoding method is intra-prediction. The displacement vector calculation unit can calculate a displacement vector for a vertex from the fitted refined mesh based on the reconstructed base mesh. The displacement vector processing unit can process the displacement vector into a form suitable for encoding. Depending on the encoding method for the displacement vector, the displacement vector can be encoded based on a video method or an arithmetic encoding method. The displacement vector can be reconstructed and provided to the mesh restoration unit together with the reconstructed base mesh. Based on the restored mesh, the attribute (texture map) video generation unit can generate a video for encoding the texture map using the original mesh and the texture map for the original mesh. The attribute is encoded based on the video method. The atlas is encoded by the atlas encoding unit.

[0170] Figure 13 illustrates a V-DMC decoding process according to embodiments.

[0171] Fig. 13 corresponds to the decoders of Figs. 1 to 3. Each component of Fig. 13 corresponds to hardware, software, a processor, and / or a combination thereof.

[0172] The bitstream of the received Mesh is demultiplexed into a compressed base mesh bitstream, a displacement vector bitstream, an attribute information (texture map) bitstream, and an atlas bitstream after file / segment decapsulation.

[0173] If the current mesh has inter-frame encoding applied according to the frame header information, the motion vector decoding unit can perform decoding on the base mesh bitstream. The final motion vector can be reconstructed by adding the previously decoded motion vector to the residual motion vector decoded from the bitstream using the previously decoded motion vector as a predictor. The current base mesh can be reconstructed by adding the decoded motion vector to the reference base mesh.

[0174] If the current mesh has been encoded within the screen according to the frame header information, the base mesh bitstream can be used to restore the connection information, vertex geometry information, texture coordinates, normal information, etc. of the base mesh through the static mesh decoder.

[0175] The base mesh restoration unit can perform inverse quantization on the decoded base mesh to generate a restored base mesh.

[0176] Depending on the encoding codec type, the displacement vector bitstream may be decoded using a video codec and then subjected to a depacking process. If encoded using arithmetic coding, arithmetic decoding may be performed using a displacement vector arithmetic decoding unit. If inter-screen prediction is performed, the current displacement vector transform coefficient may be generated by adding a residual value to the reference displacement vector transform coefficient through inter-screen prediction.

[0177] The displacement vector restoration unit restores the displacement vector by inverse quantization and inverse transformation of the decoded displacement vector transform coefficients. If the restored displacement vector is a value in the local coordinate system, a process of inverse transformation to the Cartesian coordinate system can be performed.

[0178] The mesh restoration unit can generate additional vertices by performing subdivision on the restored base mesh. Subdivision can generate vertex connection information, texture coordinates, and texture coordinate connection information, including the added vertices. The subdivided restored base mesh can be combined with the restored displacement vector to generate the final restored mesh.

[0179] The texture map bitstream can be decoded as a video bitstream using a video codec in a texture map video decoding unit. The restored texture map contains color information for each vertex contained in the restored mesh, and the color value of each vertex can be obtained from the texture map using the texture coordinates of the corresponding vertex.

[0180] The atlas bitstream can be decrypted by the atlas decryptor.

[0181] The restored mesh and texture map are displayed to the user through a rendering process using a mesh data renderer, etc.

[0182] The decoder receives an encoded bitstream and decodes the base mesh, displacement vectors, attributes, and atlas within the bitstream based on parameter information (which may be referred to as signaling information, metadata, etc.) contained within the bitstream. The decoding process may follow the reverse process of the encoding process. Based on the decoded atlas, a mesh is reconstructed from the reconstructed base mesh and the reconstructed displacement mesh. The mesh can be rendered based on the reconstructed mesh and the reconstructed attributes.

[0183] A point cloud data encoding device and method according to embodiments can encode mesh data and transmit a bitstream including the encoded mesh data. A point cloud data decoding device and method according to embodiments can receive a bitstream including mesh data and decode the mesh data. The point cloud data encoding / decoding method / device according to embodiments may be referred to as the method / device according to embodiments. The point cloud data encoding / decoding method / device according to embodiments may also be referred to as the mesh data encoding / decoding method / device according to embodiments. In addition, the term encoding / decoding method / device may be used in this document for short.

[0184] The encoding method and device according to the embodiments can include and perform the operation of the encoding process of FIGS. 1 to 10 and FIG. 12, the encoding support for the HRD (Hypothetical reference decoder) decoding process of FIGS. 14 to 17, the generation of syntax elements of FIGS. 18 to 22, and the encoding method of FIG. 23.

[0185] The decryption method and device according to the embodiments can include and perform the decryption process of FIGS. 1, 11, and 13, the HRD decoding process of FIGS. 14 to 17, the acquisition of syntax elements of FIGS. 18 to 22, and the decryption method of FIG. 24.

[0186] The method and device according to the embodiments may include and perform operations related to a basemesh hypothetical reference decoder (HRD) for V-DMC and acquisition of related syntax elements. The method and device according to the embodiments may include and perform operations of generating and transmitting HLS (High Level Syntax) for HRD operations, acquiring HLS, and decoding mesh data based on HLS.

[0187] The embodiments relate to V-DMC (V-mesh), a method for compressing 3D dynamic mesh data based on a 2D video codec.

[0188] Since the current BaseMesh design supports all final unit types defined in HEVC, the high-level operation of the BaseMesh codec can, in principle, be similar to that of HEVC. Therefore, the HRD design of the BaseMesh codec can be based on the HEVC HRD model. Furthermore, there are features that can further simplify the BaseMesh codec's HRD model.

[0189] HEVC has several syntax elements that provide information related to HRD operation.

[0190] The buffering period SEI message provides the initial removal timing of the coded picture buffer to initialize the HRD at the associated access unit location.

[0191] The picture timing SEI message provides CPB removal delay and DPB output delay information for the access unit associated with the SEI message.

[0192] HRD parameters (parameters) within the VUI provide HRD parameters used for the HRD operation of the layer set.

[0193] In V3C, the buffering period SEI message, the atlas frame timing SEI message, and the HRD parameters provide the information necessary to define the operation of the atlas HRD.

[0194] Corresponding syntax elements are required to provide the information necessary for the operation of the BaseMesh HRD.

[0195] Figure 14 shows a base mesh frame HRD buffer model according to embodiments.

[0196] Fig. 14 shows the HRD decoding configuration of the decoding method and device (Figs. 1, 11, and 13 decoding process, Figs. 15 to 17 HRD decoding process, Figs. 18 to 22 syntax element acquisition, and Fig. 24 decoding method) according to embodiments. The encoding method and device (Figs. 1 to 10, and 12 encoding process, Figs. 14 to 17 encoding support for HRD (Hypothetical reference decoder) decoding process, and Figs. 18 to 22 syntax element generation) according to embodiments can generate HLS (which may be referred to as parameters, signaling information, metadata, etc.) for the basemesh HRD decoding operation of Fig. 14 and transmit it by including it in a bitstream.

[0197] A high-level approach to basemesh HRD design according to embodiments is as follows:

[0198] 1) General

[0199] - In HEVC, intra-coded static mesh (also referred to as mesh, etc.) frames can be viewed as IRAP, and inter-coded motion frames can be viewed as non-IRAP. Since all basemesh (also referred to as basemesh, etc.) nal unit types can be mapped to the corresponding nal unit types, the high-level functionality of the basemesh codec in terms of buffer management can be the same as that of HEVC.

[0200] - The HRD buffer model includes a basemesh buffer (CBMB) corresponding to an input buffer that stores a coded compressed basemesh bitstream and a buffer (DBMB) corresponding to an output buffer that stores a decoded basemesh frame.

[0201] - The function of the submesh can be similar to the slice function of HEVC.

[0202] - The HRD model of HEVC is very similar to the high-level operation, including the decoding process and supported nal unit types, so it can be applied to the HRD model of the Basemesh codec.

[0203] - For basemesh, there is no equivalent feature such as output cropping or normative output windowing that exists in HEVC or V3C atlas.

[0204] A decoding method and device according to embodiments may receive an encoded bitstream from an encoding method and device according to embodiments, and decode mesh data within the bitstream. The method and device according to embodiments may utilize a method for decoding without buffer underflow or overflow by using a buffer of a decoder. A Hypothetical Stream Scheduler (HSS) may perform subsequent procedures based on a type of a basemesh sub-bitstream within a bitstream. The type of the basemesh sub-bitstream includes a first type and / or a second type. A Coded Basedmesh Buffer (CBMC) may output decoding units of intra-prediction and / or inter-prediction from a buffer based on intra-basemesh or inter-motion. The decoder decodes the basemesh instantaneously. A Decoded Basedmesh Buffer (DBMB) may output basemesh frames. DBMB can provide the decoder with a reference basemesh for inter-prediction.

[0205] Figure 15 shows the structure of basemesh NAL unit streams for HRD conformance checks according to embodiments.

[0206] BaseMesh Bitstream Conformance: Since there are no additional stream formats defined for the BaseMesh stream (e.g., byte stream format encapsulation), BaseMesh Bitstream Conformance is defined as in Figure 15.

[0207] Buffering Period SEI Message: The syntax elements of the Buffering Period SEI message provide important information about the timing and operation of CBMB. A separate Buffering Period SEI message may be defined that provides CBMB information for the Basemesh codec.

[0208] HRD Parameters: HRD parameters provide information used in HRD operations and are crucial for defining bitstream conformance. The basemesh decoding process is defined only when bmesh_nal_layer_id is 0, and there is no temporal layering, so HRD sublayer parameters (sub-layer parameters) for the basemesh are not required.

[0209] Skipped Basemesh Frames: The coded sub-meshes of skipped basemesh frames are unique to the basemesh codec, and this concept does not exist in HEVC. However, the behavior of skipped basemesh frames is exactly the same as other non-IRAP BMCL mesh data in terms of CBMB or DBMB management. Therefore, no special considerations or additional work are required when designing HRD for skipped basemesh frames.

[0210] Since the current CD of V-DMC does not support temporal hierarchy of basemesh bitstream, the HRD design and related syntax elements of buffering period SEI message and HRD parameters of VUI may not consider temporal hierarchy.

[0211] While submesh timing (or sub-access unit-level timing) can be useful for ultra-low-latency applications, it may be appropriate to define basemesh HRD behavior at the coded basemesh access unit level. Furthermore, the Atlas HRD design in the V3C specification does not consider such sub-access unit-level timing.

[0212] According to the embodiments, a NAL Unit (Network Abstraction Layer) is a basic unit that constitutes an HEVC bitstream.

[0213] According to the type 1 HRD and type 2 HRD according to the day unit, the HRD conformance check can be performed as shown in Fig. 15. The BMCL day unit and the filter data day unit are type 1 HRDs, and the non-BMCL day unit and the filter data day unit, which are not the filter data day unit, are type 2 HRDs, and the conformance check can be checked. The type 1 HRD can be used to check the timing of the day unit for the parameter information (metadata), and the type 2 HRD can be used to check the timing for the day unit of the mesh data to be decoded.

[0214] For reference, a BMCL raw unit is a raw unit that includes submesh data in the basemesh stream, and a raw unit corresponding to parameter information, etc. that does not include submesh data in the basemesh stream is referred to as a non-BMCL raw unit (or non-BMCL raw unit). Additionally, one BMCL raw unit includes one submesh data.

[0215] Referring to FIG. 15, a basemesh hypothetical reference decoder according to embodiments is described.

[0216] 1. General

[0217] Below, we describe a virtual reference decoder (HRD) and its use for basemesh bitstream and decoder conformance checking.

[0218] The method according to the embodiments performs HRD conformance checking on two types of bitstreams or subsets of bitstreams. The first type is a Type I bitstream, which is a BNAL unit stream that contains only BMCL NAL units and non-BMCL NAL units where bmesh_nal_unit_type is equal to BNAL_FD (filler data NAL units) for all coded basemesh access units of the bitstream. The second type is a Type II bitstream, which contains BMCL units and filler data NAL units for all coded basemesh access units of the basemesh bitstream, plus additional non-BMCL NAL units other than the filler data NAL units.

[0219] Figure 15 illustrates the types of basemesh bitstream conformance points that the basemesh HRD checks.

[0220] The syntax elements (or default values ​​for some syntax elements) of non-BMCL NAL units required for HRD are described below.

[0221] Two types of HRD parameter sets are used: BNAL HRD parameters and BMCL HRD parameters. HRD parameter sets are signaled via the bmhrd_parameters() syntax structure, which can be part of the BMSPS syntax structure.

[0222] To verify the conformance of the entire bitstream (the entire bitstream), two sets of bitstream conformance tests are required. The entire bitstream is denoted as entireBitstream. The first set of bitstream conformance tests is intended to test the conformance of the entire bitstream and its temporal subset, excluding non-BMCL NAL units. The second set of bitstream conformance tests is intended to test the conformance of the entire bitstream. In all these tests, only basemesh frames with a bmesh_nal_layer_id of 0 are decoded when the decoding process is invoked.

[0223] For each test, the following steps are applied in the order listed:

[0224] 1. The test target operation point, indicated by TargetOp, is selected by selecting the OpTid with the highest value of the target TemporalId. The OpTid value of the basemesh HRD is set to 0. The sub-bitstream BitstreamToDecode must satisfy both of the following conditions:

[0225] ― BitstreamToDecode contains one or more BMCL NAL units with bmesh_nal_layer_id equal to 0.

[0226] ― BitstreamToDecode contains one or more BMCL NAL units with TemporalId equal to OpTid.

[0227] 2. Set HighestTid to be the same as the OpTid of TargetOp.

[0228] 3. The bmhrd_parameters( ) syntax structure applicable to TargetOp is selected. Within the selected bmhrd_parameters( ) syntax structure, if BitstreamToDecode is a Type I bitstream, the sub_layer_hrd_parameters( HighestTid ) ​​syntax structure immediately following the condition "if( bmcl_hrd_parameters_present_flag )" is selected and the variable NalHrdModeFlag is set to 0. Otherwise (if BitstreamToDecode is a Type II bitstream), the sub_layer_hrd_parameters( HighestTid ) ​​syntax structure immediately following the condition "if( bmcl_hrd_parameters_present_flag )" (in which case the variable NalHrdModeFlag is set to 0) or the condition "if( bnal_hrd_parameters_present_flag )" (in which case the variable NalHrdModeFlag is set to 1) is selected. If BitstreamToDecode is a Type II bitstream and NalHrdModeFlag is 0, all non-BMCL NAL units except filler data NAL units are dropped from BitstreamToDecode and the remaining bitstream is assigned to BitstreamToDecode.

[0229] The coded basemesh access unit associated with the buffering period SEI message applicable to TargetOp (present in BitstreamToDecode or available through external means not specified herein) is selected as the basemesh HRD initialization point and is referred to as coded basemesh access unit 0.

[0230] For each coded basemesh access unit of BitstreamToDecode starting from coded basemesh access unit 0, a buffering duration SEI message (present in BitstreamToDecode or available via external means not specified herein) that applies to TargetOp associated with the coded basemesh access unit is selected, and a basemesh frame timing SEI message (present in BitstreamToDecode or available via external means not specified herein) that applies to TargetOp associated with the coded access unit is selected.

[0231] A SchedSelIdx value is selected. The selected SchedSelIdx must be in the range 0 to cbmb_cnt_minus1, where cbmb_cnt_minus1 is found in the bmhrd_parameters( ) syntax structure.

[0232] If the bmesh_nal_unit_type of the coded basemesh frame of coded basemesh access unit 0 is BNAL_CRA or BNAL_BLA_W_LP, and the irap_cbmb_params_present_flag of the selected buffering period SEI message is 1, one of the following applies to the initial CBMB removal delay and delay offset selection:

[0233] If NalHrdModeFlag is 1, the default initial CBMB removal delay and delay offset expressed in bnal_initial_cbmb_removal_delay[SchedSelIdx] and bnal_initial_cbmb_removal_offset[SchedSelIdx], respectively, of the selected buffering period SEI message are selected. Otherwise, the default initial CBMB removal delay and delay offset expressed in bmcl_initial_cbmb_removal_delay[SchedSelIdx] and bmcl_initial_cbmb_removal_offset[SchedSelIdx], respectively, of the selected buffering period SEI message are selected. The DefaultInitCbmbParamsFlag variable is set to 1.

[0234] If NalHrdModeFlag is 1, an alternate initial CBMB removal delay and delay offset are selected, represented by bnal_initial_alt_cbmb_removal_delay[SchedSelIdx] and bnal_initial_alt_cbmb_removal_offset[SchedSelIdx], respectively, in the selected buffering period SEI message. Otherwise, an alternate initial CBMB removal delay and delay offset are selected, represented by bmcl_initial_alt_cbmb_removal_delay[SchedSelIdx] and bmcl_initial_alt_cbmb_removal_offset[SchedSelIdx], respectively, in the selected buffering period SEI message. The DefaultInitCbmbParamsFlag variable is set to 0, the RASL coded basemesh access unit associated with coded basemesh access unit 0 is deleted from BitstreamToDecode, and the remaining bitstream is assigned to BitstreamToDecode.

[0235] Each conformance test is constructed by combining one option from each of the above steps. If a step has more than one option, only one option is selected for a particular conformance test. All possible combinations of all steps constitute the complete conformance test set.

[0236] The number of bitstream conformance tests to be performed for each operating point under test is n0 * n1 * (n2 * 2 + n3) * n4, where the values ​​of n0, n1, n2, n3, and n4 are specified as follows.

[0237] ― n0 is equal to 1 regardless of whether BitstreamToDecode is a Type I or Type II bitstream.

[0238] ― n1 is equal to cbmb_cnt_minus1 + 1.

[0239] ― n2 is the number of coded basemesh access units in BitstreamToDecode, each unit associated with a buffering period SEI message applicable to TargetOp, and both of the following conditions are true:

[0240] ― bmesh_nal_unit_type is equal to BNAL_CRA or BNAL_BLA_W_LP for BMCL NAL units.

[0241] ― irap_cbmb_params_present_flag of the SEI message applicable to TargetOp is 1.

[0242] ― n3 is the number of coded basemesh access units in BitstreamToDecode, each unit associated with a buffering period SEI message applicable to TargetOp and satisfying one or both of the following conditions:

[0243] - The bmesh_nal_unit_type of the BMCL NAL unit is not equal to BNAL_CRA or BNAL_BLA_W_LP.

[0244] - The irap_cbmb_params_present_flag of the SEI message applicable to the TargetOp is 0.

[0245] - n4 is 1.

[0246] If BitstreamToDecode is a Type II bitstream, the following applies:

[0247] ― If the sub_layer_hrd_parameters(HighestTid) syntax structure immediately following the condition "if(bmcl_hrd_parameters_present_flag)" is selected, the test is performed at the Type I conformance point shown in Fig. 15, and only BMCL and filler data NAL units are calculated for input bit rate and CBMB storage.

[0248] ― Otherwise (if the sub_layer_hrd_parameters(HighestTid) syntax structure immediately following the condition "if(bnal_hrd_parameters_present_flag)" is selected), the test is performed at the Type II conformance point shown in Fig. 15, where every byte of the Type II bitstream, which is the BNAL unit stream, is counted for the input bit rate and CBMB storage.

[0249] The BNAL HRD parameter set to the SchedSelIdx value for the Type II conformance point shown in Fig. 15 is sufficient to set the BMCL HRD conformance for the Type I conformance point shown in Fig. 15 for the same values ​​of InitCbmbRemovalDelay[SchedSelIdx], BitRate[SchedSelIdx] and CbmbSize[SchedSelIdx] for variable bit rate (VBR) (when cbr_flag[SchedSelIdx] is 0). This is because the data flow to the Type I conformance point is a subset of the data flow to the Type II conformance point, and for VBR, the CBMB can be empty and remain empty until the next basemesh frame starts to arrive.

[0250] All BMSPS and BMFPS and their buffering periods, basemesh frame timing SEI messages mentioned in the BMCL NAL units are delivered to the HRD in a timely manner, either via the bitstream (non-BMCL NAL units) or by other means not specified in this document.

[0251] The specification for the "presence" of a non-BMCL NAL unit containing a BMSPS, BMFPS, Buffering Period SEI message, or Basemesh Frame Timing SEI message is satisfied even when that BNAL unit (or portion thereof) is delivered to the Basemesh decoder (or HRD) by means other than those specified in this document. For the purpose of counting bits, only those bits that are actually present in the Basemesh sub-bitstream are counted.

[0252] For example, synchronization of non-BMCL NAL units conveyed via means other than those present in the bitstream with BNAL units present in the bitstream can be achieved by specifying two points in the bitstream. If the encoder had decided to convey non-BMCL NAL units in the bitstream, the non-BMCL NAL units would have been present in the bitstream between these two points.

[0253] If the content of these non-BMCL NAL units is delivered to the application by means other than those present in the bitstream, the representation of the non-BMCL NAL unit content need not use the same syntax as specified in this document.

[0254] If HRD information is included in the bitstream, compliance with the requirements of this section may be verified solely based on the information contained in the bitstream. If HRD information is not included in the bitstream, as is the case with all "standalone" Type I bitstreams, compliance may be verified only if the HRD data is provided through means other than those specified in this document.

[0255] Referring to Figure 14, HRD includes a coded basemesh frame buffer (CBMB), an immediate decoding process, a decoded basemesh frame buffer (DBMB), and a nominal format conversion.

[0256] For each bitstream conformance test, the CBMB size (number of bits) is CbmbSize[SchedSelIdx] specified in the lower-layer HRD parameter semantics, where SchedSelIdx and the basemesh HRD parameters are specified. The DBMB size (number of basemesh frame storage buffers) is bmsps_max_dec_bmfr_buffering_minus1[HighestTid] + 1.

[0257] To express the constraints of this appendix, the following are specified:

[0258] ― Each coded basemesh access unit is referred to as coded basemesh access unit n, where the number n represents a specific coded basemesh access unit. Coded basemesh access unit 0 is selected according to step 4 above. The value of n increases by 1 for each subsequent coded basemesh access unit in the decoding order.

[0259] ― Each decoding unit is referred to as decoding unit m, where the number m identifies a particular decoding unit. The first decoding unit in the decoding order from access unit 0 is referred to as decoding unit 0. The value of m increases by 1 for each subsequent decoding unit in the decoding order.

[0260] ― Basemesh frame n represents a coded basemesh frame or a decoded basemesh frame of coded basemesh access unit n.

[0261] HRD works like this:

[0262] ― HRD is initialized at decoding unit 0, and both CBMB and DBMB are set to empty (DBMB fullness is set to 0).

[0263] After initialization, HRD is not reinitialized by subsequent buffering period SEI messages.

[0264] ― Data related to decoding units flowing into CBMB according to a specified arrival schedule is delivered by HSS.

[0265] ― Data associated with each decoding unit is immediately removed and decoded by the instantaneous decoding process at the time of CBMB removal of the decoding unit.

[0266] ― Each decoded basemesh frame is placed in DBMB.

[0267] ― Decoded basemesh frames are removed from the DBMB when they are no longer needed for inter-prediction reference and output.

[0268] For each bitstream conformance test, the operation of CBMB, instantaneous decoder operation, DBMB operation, and nominal format conversion are described further below.

[0269] The number of enumerated transmission schedules and the associated bit rates and buffer sizes are specified in the HRD parameter syntax and semantics. The HRD is initialized according to the buffering period SEI message. The timing of removing a decoding unit from a CBMB and the timing of outputting a decoded basemesh frame from a DBMB are specified using information in the basemesh frame timing SEI message. All timing information related to a specific decoding unit must arrive before the CBMB removal time of the decoding unit.

[0270] Bitstream conformance requirements are described below, and HRD is used for conformance checks of the bitstream and conformance checks of the basemesh decoder.

[0271] Conformance is guaranteed under the assumption that all basemesh frame rates and clocks used to generate the bitstream exactly match the values ​​indicated in the bitstream; however, in actual systems, each of these values ​​may differ from the indicated or specified values.

[0272] All arithmetic operations are performed using real numbers, so rounding errors do not propagate. For example, the number of bits in a CBMB immediately before or after decoding unit removal does not necessarily have to be an integer.

[0273] The variable ClockTick is derived as follows and is called a clock tick.

[0274] ClockTick = vui_num_units_in_tick χ vui_time_scale

[0275] The operation of a coded basemesh buffer (CBMB) according to embodiments is described below.

[0276] The operation of the coded basemesh buffer (CBMB) is applied independently to each coded basemesh buffer (CBMB) parameter set and to the type I and type II conformance points shown in Fig. 15, and the CBMB parameter set is selected.

[0277] The method and device according to the embodiments can decode mesh data based on the timing of decoding unit arrival.

[0278] Timing of decoding unit arrival: The operation of the basemesh buffer (CBMB) is performed to derive the initial and final CBMB arrival times for coded basemesh access unit n by considering the decoding unit as a coded basemesh access unit.

[0279] The variables InitCbmbRemovalDelay[SchedSelIdx] and InitCbmbRemovalDelayOffset[SchedSelIdx] are derived as follows:

[0280] ― If one or more of the following conditions are true, NalHrdModeFlag is 1 or bmcl_initial_alt_cbmb_removal_delay[SchedSelIdx], and bmcl_initial_alt_cbmb_removal_offset[SchedSelIdx] is 0, respectively, when NalHrdModeFlag is 0, the buffering period SEI message syntax element is selected according to the embodiments.

[0281] ― Coded basemesh access unit 0 is a BLA access unit in which the bmesh_nal_unit_type of the coded basemesh frame is BNAL_BLA_W_RADL or BNAL_BLA_N_LP and the irap_cbmb_params_present_flag value of the buffering period SEI message is 1.

[0282] ― Coded BaseMesh Access Unit 0 is a BLA access unit for which the bmesh_nal_unit_type of the coded basemesh frame is BNAL_BLA_W_LP or is a BNAL_CRA access unit and the irap_cbmb_params_present_flag value in the buffering period SEI message is 1. If the buffering period SEI message is 1 and one or more of the following conditions are true:

[0283] ― If UseAltCbmbParamsFlag is 1 for coded basemesh access unit 0.

[0284] ― If DefaultInitCbmbbParamsFlag is 0

[0285] ― Otherwise, if NalHrdModeFlag is 1, InitCbmbRemovalDelay[SchedSelIdx] and InitCbmbRemovalDelayOffset[SchedSelIdx] are set equal to the values ​​of the buffering period SEI message syntax elements bnal_initial_cbmb_removal_delay[SchedSelIdx] and bnal_initial_cbmb_removal_offset[SchedSelIdx], respectively. Alternatively, bmcl_initial_cbmb_removal_delay[SchedSelIdx] and bmcl_initial_cbmb_removal_offset[SchedSelIdx] are applied when the buffering period SEI message syntax element is selected, respectively, when NalHrdModeFlag is 0.

[0286] The time when the first bit of decoding unit m begins to enter the CBMB is called the initial arrival time initArrivalTime[m].

[0287] The initial arrival time of decoding unit m is derived as follows.

[0288] ― If the decoding unit is decoding unit 0 (i.e., m is 0), initArrivalTime[0] is set to 0.

[0289] ― Otherwise (if the decoding unit is a decoding unit m with m > 0), the following applies.

[0290] ― If cbr_flag[SchedSelIdx] is 1, the initial arrival time of decoding unit m is equal to the final arrival time of decoding unit m-1 (derived below). That is,

[0291] initArrivalTime[m] = AuFinalArrivalTime[m-1] (H 2)

[0292] ― Otherwise (cbr_flag[SchedSelIdx] is 0), the initial arrival time of decoding unit m is derived as follows.

[0293] initArrivalTime[ m ] =

[0294] Max( AuFinalArrivalTime[ m - 1 ], initArrivalEarliestTime[ m ] )

[0295] Here, initArrivalEarliestTime[ m ] is derived as follows:

[0296] ― The variable tmpNominalRemovalTime is derived as follows.

[0297] tmpNominalRemovalTime = AuNominalRemovalTime[m]

[0298] Here, AuNominalRemovalTime[ m ] is the nominal CBMB removal time of the coded basemesh access unit m.

[0299] ― If decoding unit m is not the first decoding unit of the subsequent buffering period, initArrivalEarliestTime[ m ] is derived as follows.

[0300] initArrivalEarliestTime[ m ] = tmpNominalRemovalTime - ( InitCbmbRemovalDelay[ SchedSelIdx ]

[0301] + InitCbmbRemovalDelayOffset[ SchedSelIdx ] ) χ 90000

[0302] ― Otherwise (if decoding unit m is the first decoding unit of the subsequent buffering period), initArrivalEarliestTime[ m ] is derived as follows.

[0303] initArrivalEarliestTime[ m ] = tmpNominalRemovalTime -

[0304] (InitCbmbRemovalDelay[SchedSelIdx] χ 90000)

[0305] The final arrival time of decoding unit m is derived as follows.

[0306] AuFinalArrivalTime[ m ] =

[0307] initArrivalTime[m] + sizeInbits[m] χ BitRate[SchedSelIdx] (H 7)

[0308] Here, sizeInbits[ m ] is the size (in bits) of the decoding unit m, and calculates the bits of the BMCL NAL unit and the filler data NAL unit for a Type I conformance point, or all bits of the Type II bitstream for a Type II conformance point. Here, Type I and Type II conformance points are as shown in Fig. 15.

[0309] The values ​​of SchedSelIdx, BitRate[ SchedSelIdx ], and CbmbSize[ SchedSelIdx ] are restricted as follows:

[0310] ― If the content of the hrd_parameters( ) syntax structure selected for the coded basemesh access unit containing the decoding unit m is different from that of the previously coded basemesh access unit, the HSS selects the following value: Select SchedSelIdx1 of the SchedSelIdx values ​​provided in the hrd_parameters() syntax structure selected for the coded basemesh access unit containing the decoding unit m. This value generates BitRate[SchedSelIdx1] or CbmbSize[SchedSelIdx1] for the coded basemesh access unit containing the decoding unit m. The value of BitRate[SchedSelIdx1] or CbmbSize[SchedSelIdx1] may be different from the value of BitRate[SchedSelIdx0] or CbmbSize[SchedSelIdx0] for the SchedSelIdx0 value of SchedSelIdx used in the previously coded basemesh access unit.

[0311] ― Otherwise, HSS continues to operate using the previous values ​​of SchedSelIdx, BitRate[SchedSelIdx], and CbmbSize[SchedSelIdx].

[0312] If HSS chooses a BitRate[SchedSelIdx] or CbmbSize[SchedSelIdx] value that differs from the value of the previously coded basemesh access unit, the following applies:

[0313] ― The BitRate[SchedSelIdx] variable applies to the initial CBMB arrival time of the currently coded basemesh access unit.

[0314] ― The CbmbSize[SchedSelIdx] variable is applied as follows.

[0315] If the new value of CbmbSize[SchedSelIdx] is larger than the existing CBMB size, it is applied to the initial CBMB arrival time of the currently coded basemesh access unit.

[0316] ― Otherwise, the new value of CmbbSize[SchedSelIdx] is applied to the CBMB removal time of the currently coded basemesh access unit.

[0317] The method and device according to the embodiments can decode mesh data based on timing of decoding unit removal and decoding of decoding unit.

[0318] The variables InitCbmbRemovalDelay[SchedSelIdx], InitCbmbRemovalDelayOffset[SchedSelIdx], CbmbDelayOffset, and DbmbDelayOffset are derived as follows.

[0319] ― If one or more of the following conditions are true, CbmbDelayOffset is set equal to the value of the buffering period SEI message syntax element cbmb_delay_offset, DbmbDelayOffset is set equal to the value of the buffering period SEI message syntax element dbmb_delay_offset, and InitCbmbRemovalDelay[SchedSelIdx] and InitCbmbRemovalDelayOffset[SchedSelIdx] are set equal to the value of the buffering period SEI message syntax element bnal_initial_alt_cbmb_removal_delay[SchedSelIdx] and bnal_initial_alt_cbmb_removal_offset[SchedSelIdx] if NalHrdModeFlag is 1, or bmcl_initial_alt_cpb_removal_delay[SchedSelIdx] if NalHrdModeFlag is 0. bmcl_initial_alt_cpb_removal_offset[SchedSelIdx], where the buffering period SEI message containing the syntax element is selected according to embodiments.

[0320] ― Coded basemesh access unit 0 is a BLA access unit in which the bmesh_nal_unit_type of the coded basemesh frame is BNAL_BLA_W_RADL or BNAL_BLA_N_LP and the irap_cbmb_params_present_flag value of the buffering period SEI message is 1.

[0321] ― Coded BaseMesh Access Unit 0 is a BLA access unit whose bmesh_nal_unit_type of the coded basemesh frame is BNAL_BLA_W_LP or a CRA access unit, and the value of irap_cbmb_params_present_flag in the buffering period SEI message is 1 and one or more of the following conditions are true:

[0322] ― UseAltCbmbParamsFlag of coded basemesh access unit 0 is 1.

[0323] ― DefaultInitCbmbParamsFlag is 0.

[0324] ― Otherwise, InitCbmbRemovalDelay[SchedSelIdx] and InitCbmbRemovalDelayOffset[SchedSelIdx] are set equal to the value of the buffering period SEI message syntax element bnal_initial_cbmb_removal_delay[SchedSelIdx]. When NalHrdModeFlag is 1, bnal_initial_cbmb_removal_offset[SchedSelIdx] is set, and when NalHrdModeFlag is 0, bmcl_initial_cbmb_removal_delay[SchedSelIdx] and bmcl_initial_cpb_removal_offset[SchedSelIdx] are set, respectively. Here, a buffering period SEI message including the syntax element is selected according to embodiments, and both CbmbDelayOffset and DbmbDelayOffset are set to 0.

[0325] The nominal removal time of a basemesh access unit n coded in CBMB is given by:

[0326] ― If the coded basemesh access unit n is a coded basemesh access unit (an access unit that initializes HRD) with n equal to 0, the nominal removal time of the coded basemesh access unit in CBMB is as follows.

[0327] AuNominalRemovalTime

[0000] = InitCbmbRemovalDelay[SchedSelIdx] χ 90000 (H 8)

[0328] ― Otherwise, the following applies:

[0329] ― If the coded basemesh access unit n is the first access unit of a buffering period that does not initialize the HRD, the following applies:

[0330] The nominal removal time of a basemesh access unit n coded in CBMB is specified as follows:

[0331] if( !concatenationFlag ) {

[0332] baseTime = AuNominalRemovalTime[ firstPicInPrevBuffPeriod ]

[0333] tmpCbmbRemovalDelay = AuCbmbRemovalDelayVal

[0334] } else {

[0335] baseTime = AuNominalRemovalTime[ prevNonDiscardablePic ]

[0336] tmpCbmbRemovalDelay =

[0337] Max( ( auCbmbRemovalDelayDeltaMinus1 + 1 ), (H 9)

[0338] Ceil( ( InitCbmbRemovalDelay[ SchedSelIdx ] χ 90000 +

[0339] AuFinalArrivalTime[ n - 1 ] - AuNominalRemovalTime[ n - 1 ] ) / ClockTick ) )

[0340] }

[0341] AuNominalRemovalTime[ n ] = baseTime + ClockTick * ( tmpCbmbRemovalDelay - CbmbDelayOffset )

[0342] Here, AuNominalRemovalTime[ firstPicInPrevBuffPeriod ] is the nominal removal time of the first coded basemesh access unit in the previous buffering period, and AuNominalRemovalTime[ prevNonDiscardablePic ] is the nominal removal time of the previous coded basemesh access unit with TemporalId equal to 0 in decoding order and is not a RASL, RADL, or SLNR basemesh frame. AuCbmbRemovalDelayVal is a value of AuCbmbRemovalDelayVal derived according to au_cbmb_removal_delay_minus1 of a basemesh frame timing SEI message selected according to embodiments, and is associated with a coded basemesh access unit n, and concatenationFlag and auCbmbRemovalDelayDeltaMinus1 are values ​​of the syntax elements concatenation_flag and au_cbmb_removal_delay_delta_minus1 of a buffering period SEI message, respectively, and are associated with a coded basemesh access unit n.

[0343] After deriving the nominal CBMB removal time and before deriving the DBMB output time of the coded basemesh access unit n, the CbmbDelayOffset and DbmbDelayOffset values ​​are updated as follows.

[0344] ― If one or more of the following conditions are true, CbmbDelayOffset is set equal to the value of the buffering period SEI message syntax element cbmb_delay_offset, and DbmbDelayOffset is set equal to the value of the buffering period SEI message syntax element dbmb_delay_offset, where a buffering period SEI message containing the syntax element is selected.

[0345] ― The coded basemesh access unit n is a BLA access unit in which the bmesh_nal_unit_type of the coded basemesh frame is BNAL_BLA_W_RADL or BNAL_BLA_N_LP and the irap_cbmb_params_present_flag value of the buffering period SEI message is 1.

[0346] ― The coded basemesh access unit n is a BLA access unit in which the bmesh_nal_unit_type of the coded basemesh frame is BNAL_BLA_W_LP or a CRA access unit, the irap_cbmb_params_present_flag value of the buffering period SEI message is 1, and the UseAltCbmbParamsFlag for the coded basemesh access unit n is 1.

[0347] ― Otherwise, both CbmbDelayOffset and DbmbDelayOffset are set to 0.

[0348] ― If the coded basemesh access unit n is not the first coded basemesh access unit in the buffering period, the nominal removal time of the coded basemesh access unit n in the CBMB is specified as follows.

[0349] AuNominalRemovalTime[n] = AuNominalRemovalTime[firstBasemeshInCurrBuffPeriod] +

[0350] ClockTick * ( AuCbmbRemovalDelayVal - CbmbDelayOffset )

[0351] Here, AuNominalRemovalTime[ firstBasemeshInCurrBuffPeriod ] is the nominal removal time of the first coded basemesh access unit in the current buffering period, and AuCbmbRemovalDelayVal is derived according to au_cbmb_removal_delay_minus1 of the basemesh timing SEI message, which is associated with coded basemesh access unit n.

[0352] At the CBMB removal time of the coded basemesh access unit n, the coded basemesh access unit is immediately decoded.

[0353] The decoding method and device according to the embodiments can decode mesh data based on the operation of the decoded basemesh buffer (DBMB).

[0354] The behavior of a decoded basemesh buffer (DBMB) is applied independently to each set of DBMB parameters.

[0355] The decoded basemesh buffer contains a basemesh frame storage buffer. Each basemesh frame storage buffer may contain a decoded basemesh frame that is marked as "used for reference" or stored for future output. The sequential processing of the decoded basemesh buffer (DBMB) is described below.

[0356] The operation of the decoded basemesh buffer (DBMB) currently includes a step of removing basemesh frames from the DBMB before decoding the basemesh frame.

[0357] The removal of a basemesh frame from a DBMB before decoding the current basemesh frame (after parsing the submesh header of the first submesh of the current basemesh frame) occurs immediately at the time of CBMB removal of the first decoding unit of coded basemesh access unit n (including the current basemesh frame), and proceeds as follows:

[0358] ― After the reference mesh frame list creation process is called, the reference mesh frame display process is called.

[0359] ― If the current basemesh frame is an IRAP basemesh frame with NoOutputBeforeRecoveryFlag equal to 1 and is not basemesh frame 0, the following sequence of steps is applied.

[0360] 1. For the decoder under test, the NoOutputOfPriorMeshFramesFlag variable is derived as follows:

[0361] ― If the current base mesh frame is a CRA base mesh frame, NoOutputOfPriorMeshFramesFlag is set to 1 regardless of the bmsh_no_output_of_prior_mesh_frames_flag value.

[0362] ― Otherwise, if the value of bmsps_geometry_3d_bit_depth_minus1, bmsps_mesh_attribute_dimension_minus1, bmsps_attribute_bit_depth_minus1, or bmsps_max_dec_mesh_frame_buffering_minus1 derived from the active Basemesh SPS is different from the value of bmsps_geometry_3d_bit_depth_minus1, bmsps_mesh_attribute_dimension_minus1, bmsps_attribute_bit_depth_minus1, or bmsps_max_dec_mesh_frame_buffering_minus1 derived from the active Basemesh SPS for the previous basemesh frame, NoOutputOfPriorMeshFramesFlag may (but must not) be set to 1 by the basemesh decoder under test. bmsh_no_output_of_prior_mesh_frames_flag.

[0363] In these conditions, it is recommended to set NoOutputOfPriorMeshFramesFlag to bmsh_no_output_of_prior_mesh_frames_flag, but the basemesh decoder under test may set NoOutputOfPriorMeshFramesFlag to 1 in this case.

[0364] ― Otherwise, NoOutputOfPriorMeshFramesFlag is set to bmsh_no_output_of_prior_mesh_frames_flag.

[0365] 2. Apply the NoOutputOfPriorMeshFramesFlag value derived for the basemesh decoder under test to the HRD. If the NoOutputOfPriorMeshFramesFlag value is 1, all basemesh frame storage buffers in the DBMB are emptied, the output of the basemesh frames contained in the buffers is lost, and the DBMB fullness is set to 0.

[0366] ― If both of the following conditions are true for all basemesh frames k of DBMB, all basemesh frames k of DBMB are removed from DBMB.

[0367] ― Basemesh frame k is marked as “not used for reference”.

[0368] ― When MeshFrameOutputFlag of basemesh frame k is 0, or the corresponding DBMB output time is less than or equal to the CBMB removal time of the first decoding unit (indicated by decoding unit m) of the current basemesh frame n, i.e., when DbmbOutputTime[k] is less than or equal to AuCbmbRemovalTime[m].

[0369] ― For each basemesh frame removed from the DBMB, the DBMB fullness is reduced by 1.

[0370] The operation of the decoded basemesh buffer (DBMB) further includes a basemesh frame output step.

[0371] The basemesh frame output step occurs immediately at AuCbmbRemovalTime[n], which is the CBMB removal time of the coded basemesh access unit n. If MeshFrameOutputFlag of basemesh frame n is 1, the corresponding DBMB output time DbmbOutputTime[n] is derived as follows. Here, the variable firstMeshFrameInBufferingPeriodFlag is 1 if the coded basemesh access unit n is the first coded basemesh access unit in the buffering period, and 0 otherwise.

[0372] DbmbOutputTime[n] = AuCbmbRemovalTime[n] + ClockTick * meshFrameDbmbOutputDelay (H 11)

[0373] if( firstMeshFrameInBufferingPeriodFlag )

[0374] DbmbOutputTime[n] -= ClockTick * DbmbDelayOffset

[0375] Here, meshFrameDbmbOutputDelay is the mesh_frame_dbmb_output_delay value of the basemesh frame timing SEI message associated with the coded basemesh access unit n.

[0376] The output of the current base mesh frame is specified as follows:

[0377] ― If MeshFrameOutputFlag is 1 and DbmbOutputTime[n] is equal to AuCbmbRemovalTime[n], the current base mesh frame is output.

[0378] ― Otherwise, if MeshFrameOutputFlag is 0, the current base mesh frame is not output and is stored in DBMB according to the embodiments.

[0379] ― Otherwise (MeshFrameOutputFlag is 1 and DbmbOutputTime[ n ] is greater than AuCbmbRemovalTime[ n ]), the current basemesh frame is output later and stored in the DBMB, and output at time DbmbOutputTime[ n ], unless bmsh_no_output_of_prior_mesh_frames_flag is set to 1 at a time earlier than DbmbOutputTime[ n ], indicating that it will not be output via decoding or inference.

[0380] If basemesh frame n is the output basemesh frame and is not the last basemesh frame in the output bitstream, the value of the variable DbmbOutputInterval[n] is derived as follows.

[0381] DbmbOutputInterval[n] = DbmbOutputTime[nextMeshFrameInOutputOrder] - DbmbOutputTime[n]

[0382] Here, nextMeshFrameInOutputOrder is the base mesh frame that comes after base mesh frame n in the output order, and MeshFrameOutputFlag is 1.

[0383] The method and device according to the embodiments further include and perform the step of marking and storing a currently decoded basemesh frame.

[0384] The currently decoded basemesh frame is stored in the DBMB of the empty basemesh frame storage buffer, and the DBMB fullness is increased by 1. If bmsps_long_term_ref_mesh_frames_flag is 1, this basemesh frame is marked as "used for long-term references". After all submeshes of the current basemesh frame are decoded, this basemesh frame is marked as "used for short-term references".

[0385] If there is no more memory available to store the decoded basemesh frames than the level limit requires, the decoder will start storing the decoded portion of the current basemesh frame into the DBMB when the first submesh is decoded, and will continue to store more decoded basemesh data as the decoding process progresses.

[0386] Bitstream Conformance: A bitstream of coded data conforming to this document satisfies the requirements according to the embodiments.

[0387] The bitstream is structured according to syntax, semantics and constraints according to embodiments.

[0388] The first coded basemesh access unit of a bitstream is an IRAP coded basemesh frame, i.e., an IDR basemesh frame, a CRA basemesh frame, or a BLA basemesh frame.

[0389] The bitstream is tested in HRD according to embodiments.

[0390] For each current base mesh frame, the maxMeshFrmOrderCnt and minMeshFrmOrderCnt variables are set to the maximum and minimum values ​​of the MeshFrmOrderCntVal values ​​of the next base mesh frame, respectively.

[0391] - Current base mesh frame

[0392] ― The previous base mesh frame in decoding order with TemporalId equal to 0 and which is not a RASL, RADL, or SLNR base mesh frame.

[0393] ― Short-term reference base mesh frame in the list of reference mesh frames for the current base mesh frame.

[0394] ― All base mesh frames n where MeshFrameOutputFlag is 1, AuCbmbRemovalTime[n] is less than AuCbmbRemovalTime[currMeshFrm], and DbmbOutputTime[n] is greater than or equal to AuCbmbRemovalTime[currMeshFrm], where currMeshFrm is the current base mesh frame.

[0395] For each bitstream conformance test, all of the following conditions are met:

[0396] 1. For each coded basemesh access unit n, where n is greater than 0, associated with a buffering period SEI message, set the variable deltaTime90k[n] as follows:

[0397] deltaTime90k[n] = 90000 * ( AuNominalRemovalTime[n] - AuFinalArrivalTime[n - 1] )

[0398] The value of InitCbmbRemovalDelay[SchedSelIdx] is limited as follows:

[0399] ― If cbr_flag[SchedSelIdx] is 0, the following condition is true:

[0400] InitCbmbRemovalDelay[SchedSelIdx] <= Ceil(deltaTime90k[n] ) (H 14)

[0401] ― Otherwise (if cbr_flag[SchedSelIdx] is 1), the following condition is true:

[0402] Floor(deltaTime90k[n]) <= InitCpbRemovalDelay[SchedSelIdx] <= Ceil(deltaTime90k[n])

[0403] The exact number of bits in the CBMB at each basemesh frame removal point may vary depending on the buffering period SEI message selected to initialize the HRD. This should be taken into account by the encoder, as it must adhere to all specified constraints regardless of the buffering period SEI message selected to initialize the HRD, as the HRD can be initialized from any of the buffering period SEI messages.

[0404] 2. CBMB overflow is specified when the total number of bits in CBMB is greater than the CBMB size. CBMB never overflows.

[0405] 3. If low_delay_hrd_flag[ HighestTid ] is 0, CBMB does not underflow. CBMB underflow is specified as follows.

[0406] ― CBMB underflow is specified as the condition that the nominal CBMB removal time of coded basemesh access unit n AuNominalRemovalTime[ n ] is shorter than the final CBMB arrival time of coded basemesh access unit n AuFinalArrivalTime[ n ] by at least one value n .

[0407] 4. The nominal time to remove a basemesh frame from CBMB (starting from the second basemesh frame in decoding order) satisfies the constraints on AuNominalRemovalTime[n] and AuCbmbRemovalTime[n].

[0408] 5. For each current basemesh frame, after calling the process to remove the basemesh frame from the DBMB, the number of basemesh frames decoded from the DBMB (including all basemesh frames n marked as "used for reference", with MeshFrameOutputFlag equal to 1 and AuCbmbRemovalTime[n] less than AuCbmbRemovalTime[currMeshFrm]) is less than or equal to bmsps_max_dec_mesh_frame_buffering_minus1, where currMeshFrm is the current basemesh frame.

[0409] 6. All reference basemesh frames are present in the DBMB when needed for prediction. Each basemesh frame with MeshFrameOutputFlag set to 1 is present in the DBMB at DBMB output time, unless it is removed from the DBMB before output time.

[0410] 7. For each current basemesh frame that is not an IRAP basemesh frame with NoOutputBeforeRecoveryFlag equal to 1, the value of maxMeshFrmOrderCnt - minMeshFrmOrderCnt is less than MaxMeshFrmOrderCntLsb / 2.

[0411] 8. The value of DbmbOutputInterval[n] given by the formula (DbmbOutputInterval[n] = DbmbOutputTime[nextMeshFrameInOutputOrder] - DbmbOutputTime[n]) is the difference between the output time of a base mesh frame and the output time of the first base mesh frame that follows it in the output order and has MeshFrameOutputFlag equal to 1, and satisfies the constraints according to the embodiments for the profile, layer, and level specified in the bitstream using the decoding process according to the embodiments.

[0412] 9. For two basemesh frames m and n in the same CBMS, when DbmbOutputTime[m] is greater than DbmbOutputTime[n], MeshFrmOrderCntVal of basemesh frame m is greater than MeshFrmOrderCntVal of basemesh frame n.

[0413] In decoding order, all basemesh frames of a previous CBMS are output before all basemesh frames of a subsequent CBMS. Basemesh frames output within a specific CBMS are output in order of increasing MeshFrmOrderCntVal values.

[0414] Decoder Conformance: The decoder satisfies all requirements described below.

[0415] A decoder claiming conformance to a particular profile, tier, and level can successfully decode any bitstream that complies with the bitstream conformance requirements according to the embodiments in a manner according to the embodiments, provided that all BMSPS and BMFPS referenced in BMCL NAL units, and appropriate buffering period and basemesh frame timing SEI messages, are delivered to the decoder in a timely manner, either through the bitstream (non-BMCL NAL units) or through external means not specified in this document.

[0416] If a basemesh sub-bitstream contains a syntax element with a reserved value, and a decoder is specified to ignore the syntax element with the reserved value or the value of a NAL unit containing the syntax element, and the basemesh sub-bitstream conforms to this document, the conforming decoder decodes the basemesh sub-bitstream in the same manner as it decodes a conforming basemesh sub-bitstream, ignoring the syntax element or NAL unit containing the syntax element with the specified reserved value.

[0417] There are two types of conformance that a decoder can require: output timing conformance and output order conformance.

[0418] To verify the suitability of the decoder, a test bitstream that complies with the profile, tier, and level according to the embodiments is delivered to both the HRD and the basemesh decoder under test (DUT) by a virtual basemesh frame stream scheduler (HSS). Every decoded basemesh frame output by the HRD must also be output by the DUT, and each decoded basemesh frame output by the DUT must be a basemesh frame with MeshFrameOutputFlag equal to 1, and for each decoded basemesh frame output by the DUT, the information associated with all sub-meshes output is identical to the information associated with all sub-meshes generated by the specified decoding process.

[0419] For output timing decoder conformance, the HSS operates as described above. The transmission schedule is selected from only a subset of SchedSelIdx values, and the bit rate and CBMB size of these values ​​are constrained for the specified profile, layer, and level. Alternatively, the bit rate and CBMB size of the "interpolated" transmission schedule specified below are constrained. The same transmission schedule is used for both the HRD and the DUT.

[0420] When the HRD parameter and buffering period SEI message exist with cbmb_cnt_minus1[ HighestTid ] greater than 0, the basemesh frame decoder can decode the basemesh sub-bitstream delivered from the HSS using an "interpolated" delivery schedule specified as having peak bit rate r, CBMB size c(r), and initial CBMB removal delay (f(r) χ r) as follows:

[0421] α = ( r - BitRate[ SchedSelIdx - 1 ] ) / ( BitRate[ SchedSelIdx] - BitRate[ SchedSelIdx - 1 ] ), (H 16)

[0422] c(r) = α * CbmbSize[ SchedSelIdx ] + ( 1 - α ) * CbmbSize[ SchedSelIdx 1 ], (H 17)

[0423] f(r) = α * InitCbmbRemovalDelay[SchedSelIdx] * BitRate[SchedSelIdx] +

[0424] (1 - α) * InitCbmbRemovalDelay[SchedSelIdx 1] * BitRate[SchedSelIdx 1] (H 18)

[0425] For all SchedSelIdx > 0 and r such that BitRate[SchedSelIdx - 1] <= r <= BitRate[SchedSelIdx], r and c(r) are within the limits for the maximum bitrate and buffer size for the specified profile, layer, and level.

[0426] InitCbmbRemovalDelay[SchedSelIdx] may be different for each buffering cycle and is recalculated.

[0427] For output timing decoder suitability, the HRD described above is used, and the timing of the basemesh frame output (based on the propagation time of the first bit) is the same for both the HRD and the DUT up to the fixed delay time.

[0428] For output order decoder suitability, the following applies:

[0429] ― The HSS delivers the bitstream BitstreamToDecode to the DUT “on demand”. That is, the HSS delivers bits (in decoding order) only when the DUT needs more bits to process.

[0430] This means that in this test the coded basemesh buffer of the DUT can be as small as the size of the largest decoding unit.

[0431] ― The modified HRD described below is used, and the HSS passes the bitstream to the HRD according to one of the schedules specified in the bitstream BitstreamToDecode, with the bit rate and CBMB size limited. The basemesh frame output order is the same for both the HRD and the DUT.

[0432] ― The HRD CBMB size is given by CbmbSize[SchedSelIdx], where SchedSelIdx and HRD parameters are selected. The DBMB size is given by bmsps_max_dec_mesh_frame_buffering_minus1 + 1. The CBMB removal time for HRD is the last bit arrival time, and decoding is performed immediately. The DBMB operation of this HRD is described as follows.

[0433] Output order DBMB behavior:

[0434] The decoded basemesh buffer contains a basemesh frame storage buffer. Each basemesh frame storage buffer contains a decoded basemesh frame that is marked as "used for reference" or stored for future output. As described below, before decoding the current basemesh frame, a process is called to output and remove the basemesh frame from the DBMB, a process is called to display and store the currently decoded basemesh frame, and finally, an additional bumping process is called. The "bumping" process is called as described below.

[0435] The method and device according to the embodiments further include the step of outputting and removing basemesh frames from the DBMB.

[0436] The output and removal of the basemesh frame from the DBMB before decoding the current basemesh frame (after parsing the submesh header of the first submesh of the current basemesh frame) occurs immediately when the first decoding unit of the coded basemesh access unit containing the current basemesh frame is removed from the CBMB, and proceeds as follows:

[0437] ― Performs a decoding process for the base mesh reference frame list and reference classification for the decoded base mesh frame.

[0438] ― If the current basemesh frame is an IRAP coded basemesh frame with NoOutputOfPriorMeshFramesFlag equal to 1 and is not basemesh frame 0, the following steps are applied.

[0439] 1. For the decoder under test, the NoOutputOfPriorMeshFramesFlag variable is derived as follows:

[0440] ― If the current base mesh frame is a CRA base mesh frame, NoOutputOfPriorMeshFramesFlag is set to 1 regardless of the bmsh_no_output_of_prior_mesh_frames_flag value.

[0441] ― Otherwise, NoOutputOfPriorMeshFramesFlag may (but must not) be set to 1 by the decoder under test if the value of bmsps_geometry_3d_bit_depth_minus1, bmsps_mesh_attribute_dimension_minus1, bmsps_attribute_bit_depth_minus1, or bmsps_max_dec_mesh_frame_buffering_minus1 derived for the current basemesh frame is different from the value of bmsps_geometry_3d_bit_depth_minus1, bmsps_mesh_attribute_dimension_minus1, bmsps_attribute_bit_depth_minus1, or bmsps_max_dec_mesh_frame_buffering_minus1 derived from the basemesh SPS enabled for the previous basemesh frame. bmsh_no_output_of_prior_mesh_frames_flag.

[0442] NOTE: In these conditions, it is recommended to set NoOutputOfPriorMeshFramesFlag to bmsh_no_output_of_prior_mesh_frames_flag, but the decoder under test may set NoOutputOfPriorMeshFramesFlag to 1 in this case.

[0443] ― Otherwise, NoOutputOfPriorMeshFramesFlag is set to bms_no_output_of_prior_mesh_frames_flag.

[0444] 2. The NoOutputOfPriorMeshFramesFlag value derived for the decoder under test is applied to the HRD as follows.

[0445] ― If NoOutputOfPriorMeshFramesFlag is 1, all basemesh frame storage buffers in the DBMB are emptied without outputting any basemesh frames, and the DBMB fullness is set to 0.

[0446] ― Otherwise (if NoOutputOfPriorMeshFramesFlag is 0), all basemesh frame storage buffers containing basemesh frames marked as "Not needed for output" and "Not used for reference" are flushed without output, all non-empty basemesh frame storage buffers in the DBMB are flushed by repeatedly calling the "bumping" process described below, and the DBMB fullness is set to 0.

[0447] ― Otherwise (if the current basemesh frame is not an IRAP basemesh frame), if NoOutputBeforeRecoveryFlag is 1, all basemesh frame storage buffers containing basemesh frames marked as "Not needed for output" and "Not used for reference" are flushed (no output). For each basemesh frame storage buffer that is flushed, the DBMB fullness is decreased by 1. If one or more of the following conditions are true, the "fill" process described below is called repeatedly, and the DBMB fullness is decreased by 1 for each additional basemesh frame storage buffer that is flushed. This process continues until none of the following conditions are true:

[0448] ― The number of basemesh frames marked as “Required for output” in DBMB is greater than bmsps_max_num_reorder_frames.

[0449] ― bmsps_max_latency_increase_plus1 is not 0, and there is at least one basemesh frame marked as "needed for output" in the DBMB, and the associated variable MeshFrameLatencyCount of this frame is greater than or equal to MaxLatencyBaseMeshFrames.

[0450] ― The number of basemesh frames in DBMB is greater than or equal to bmsps_max_dec_mesh_frame_buffering_minus1 + 1.

[0451] The method and device according to the embodiments further include and perform an additional bumping step.

[0452] An additional bumping step occurs immediately when the last decoding unit of coded basemesh access unit n containing the current basemesh frame is removed from the CBMB.

[0453] If the current basemesh frame's MeshFrameOutputFlag is 1, it is marked as "needs output" in DBMB and the associated variable MeshFrameLatencyCount is set to MeshFrameLatencyCount + 1 for each basemesh frame that follows the current basemesh frame in the output order.

[0454] ― If the MeshFrameOutputFlag of the currently decoded base mesh frame is 1, it is marked as “output required” and the related variable MeshFrameLatencyCount is set to 0.

[0455] ― Otherwise (if MeshFrameOutputFlag of the currently decoded basemesh frame is 0), it is marked as “No output required”.

[0456] If one or more of the following conditions are true, the "bumping" process is called repeatedly until none of the following conditions are true:

[0457] ― The number of basemesh frames marked as “needs output” in DBMB is greater than bmsps_max_num_reorder_frames.

[0458] ― bmsps_max_latency_increase_plus1 is not 0 and there is one or more basemesh frames marked as "needs output" in the DBMB, and the associated variable MeshFrameLatencyCount for these frames is greater than or equal to MaxLatencyBaseMeshFrames.

[0459] The method according to the embodiments further comprises a “bumping” process.

[0460] The "bumping" process consists of the following steps:

[0461] 1. The base mesh frame to be output first is selected as the frame with the smallest MeshFrmOrderCntVal value among all base mesh frames marked as “Required for Output” in DBMB.

[0462] 2. The base mesh frame is output, and the base mesh frame is marked as “not required for output.”

[0463] 3. If the basemesh frame save buffer containing the output basemesh frame contains a basemesh frame that is marked as "unused for reference," the basemesh frame save buffer is emptied.

[0464] Note: For two basemesh frames meshFrmA and meshFrmB that belong to the same CBMS and are output by the "bumping process", if meshFrmA is output before meshFrmB, the MeshFrmOrderCntVal value of meshFrmA is smaller than the MeshFrmOrderCntVal value of meshFrmB.

[0465] The method and device according to the embodiments can generate and obtain a basemesh buffering period SEI message, a basemesh frame timing SEI message, and a basemesh HRD parameter. Since this information must be clearly associated with a coded basemesh access unit, this information is transmitted in a NAL unit rather than a BMCL.

[0466] FIG. 16 illustrates the basemesh buffering period SEI message syntax of the payload of SEI (Supplemental enhancement information) according to embodiments.

[0467] Figure 17 illustrates the Basemesh frame timing SEI message syntax of the payload of SEI according to embodiments.

[0468] The encoding method and device according to the embodiments (the encoding process of FIGS. 1 to 10, 12, encoding support for the HRD (Hypothetical reference decoder) decoding process of FIGS. 14 to 15, generation of syntax elements of FIGS. 16 to 17, encoding method of FIG. 23, etc.) can generate the syntax of FIGS. 16 to 17 and transmit it by including it in a bitstream.

[0469] The decoding method and device according to the embodiments (decoding process of FIGS. 1, 11, and 13, HRD decoding process of FIGS. 14 to 15, acquisition of syntax elements of FIGS. 16 to 17, decoding method of FIG. 24, etc.) can decode mesh data based on the syntax elements of FIGS. 16 to 17 in the bitstream.

[0470] The semantics of the syntax of Figures 16 and 17 are as follows:

[0471] Basemesh buffering period SEI message semantics:

[0472] The Basemesh Buffering Period SEI message provides initial CBMB removal delay and initial CBMB removal delay offset information for initializing the Basemesh HRD at the location of the associated coded Basemesh access unit in decoding order.

[0473] If the currently coded basemesh access unit is a CRA or BLA coded basemesh access unit, the skippedMeshFrmList is a list of skipped preceding basemesh frames consisting of RASL basemesh frames associated with the CRA or BLA coded basemesh access unit associated with the buffering period SEI message.

[0474] If a basemesh frame is not a RASL, RADL, or SLNR basemesh frame, then that basemesh frame is called a notDiscardableMeshFrm basemesh frame.

[0475] The following applies to the SEI message syntax and semantics for buffering periods:

[0476] ― The syntax elements initial_cbmb_removal_delay_length_minus1, au_cbmb_removal_delay_length_minus1, dbmb_output_delay_length_minus1, and the variables NalHrdBpPresentFlag and BmclHrdBpPresentFlag are found in or derived from syntax elements in the hrd_parameters() syntax structure applicable to at least one operation point to which the buffering period SEI message applies.

[0477] ― The CbmbSize[ i ], BitRate[ i ], and CbmbCnt variables are derived from syntax elements in the sub_layer_hrd_parameters( ) syntax structure applicable to one or more operation points to which the buffering period SEI message applies.

[0478] ― The bitstream (or part thereof) refers to a subset (or part thereof) of the bitstream associated with every operation point to which the buffering period SEI message applies.

[0479] The presence of a buffering period SEI message for an action point is specified as follows:

[0480] ― If NalHrdBpPresentFlag is 1 or BmclHrdBpPresentFlag is 1, the following applies to each coded basemesh access unit of the CBMS:

[0481] ― If the coded basemesh access unit is an IRAP coded basemesh access unit, the buffering period SEI message applicable to the corresponding operation point is associated with the coded basemesh access unit.

[0482] ― Otherwise, if the coded basemesh access unit contains notDiscardablePic, a buffering period SEI message applicable to that operation point may or may not be associated with the coded basemesh access unit.

[0483] ― Otherwise, the coded basemesh access unit is not associated with a buffering period SEI message applicable to that operation point.

[0484] ― Otherwise (both NalHrdBpPresentFlag and BmclHrdBpPresentFlag are 0), no coded basemesh access unit of the CBMS is associated with a buffering period SEI message applicable to that operation point.

[0485] For some applications, it may be desirable to have frequent buffering period SEI messages (e.g., random access or bitstream splicing in IRAP basemesh frames or non-IRAP basemesh frames).

[0486] bp_bmsps_seq_parameter_set_id indicates and is equal to the bmsps_sequence_parameter_set_id of the BMSPS that is activated for the coded basemesh frame associated with the buffering period SEI message. The value of bp_bmsps_seq_parameter_set_id is equal to the bmfps_sequence_parameter_set_id value of the BMFPS referenced by bmsh_basemesh_frame_parameter_set_id in the basemesh submesh header of the coded basemesh frame associated with the buffering period SEI message. The value of bp_bmsps_seq_parameter_set_id is in the range of 0 to 15.

[0487] If irap_cbmb_params_present_flag is 1, it indicates the presence of the nal_initial_alt_cbmb_removal_delay[ i ] and nal_initial_alt_cbmb_removal_offset[ i ] or bmcl_initial_alt_cbmb_removal_delay[ i ] and bmcl_initial_alt_cbmb_removal_offset[ i ] syntax elements. If not present, the value of irap_cbmb_params_present_flag is inferred to be 0. If the associated basemesh frame is neither a CRA basemesh frame nor a BLA basemesh frame, the value of irap_cbmb_params_present_flag is equal to 0.

[0488] cbmb_delay_offset indicates the offset to use to derive the nominal CBMB removal time of the coded basemesh access unit that follows the CRA or BLA coded basemesh access unit associated with the buffering period SEI message in decoding order, if there are no basemesh frames in the skippedMeshFrmList. The length of the syntax element is in bits given by au_cbmb_removal_delay_length_minus1 + 1. If this value is absent, the cbmb_delay_offset value is inferred to be 0.

[0489] dbmb_delay_offset represents the offset used to derive the DBMB output time for a CRA or BLA coded basemesh access unit associated with a buffering period SEI message when there are no basemesh frames in the skippedMeshFrmList. The length of the syntax element is in bits given by dbmb_output_delay_length_minus1 + 1. If this value is absent, the dbmb_delay_offset value is inferred to be 0.

[0490] If the current basemesh frame is not the first basemesh frame in the bitstream in decoding order, set prevNonDiscardableMeshFrm to the previous basemesh frame in decoding order that is not a RASL, RADL, or SLNR basemesh frame.

[0491] concatenation_flag indicates whether the nominal CBMB removal time of the current basemesh frame is determined based on the nominal CBMB removal time of the previous basemesh frame with a buffering period SEI message, or based on the nominal CBMB removal time of the prevNonDiscardableMeshFrm basemesh frame, when the current basemesh frame is not the first basemesh frame in the bitstream in decoding order.

[0492] au_cbmb_removal_delay_delta_minus1 + 1 represents the CBMB removal delay increment relative to the nominal CBMB removal time of basemesh frame prevNonDiscardableMeshFrm when the current basemesh frame is not the first basemesh frame in the bitstream in decoding order. The length of this syntax element is in bits given by au_cbmb_removal_delay_length_minus1 + 1.

[0493] If the current basemesh frame contains a buffering period SEI message, concatenation_flag is 0, and the current basemesh frame is not the first picture in the bitstream in decoding order, the following constraints apply for bitstream conformance:

[0494] ― If the basemesh frame prevNonDiscardableMeshFrm is not associated with a buffering period SEI message, au_cbmb_removal_delay_minus1 of the current basemesh frame is equal to au_cbmb_removal_delay_minus1 of prevNonDiscardableMeshFrm plus au_cbmb_removal_delay_delta_minus1 + 1.

[0495] ― Otherwise, au_cbmb_removal_delay_minus1 is equal to au_cbmb_removal_delay_delta_minus1.

[0496] If the current basemesh frame contains a buffering period SEI message and the concatenation_flag is 1, the au_cbmb_removal_delay_minus1 of the current basemesh frame is not used. The constraints specified above can, in some cases, allow splicing of bitstreams (using a properly designed reference structure) simply by changing the concatenation_flag value from 0 to 1 in the buffering period SEI message of the IRAP basemesh frame at the splice point. If the concatenation_flag is 0, the constraints specified above can be checked whether the decoder meets the constraints to detect the loss of the basemesh frame prevNonDiscardableMeshFrm.

[0497] nal_initial_cbmb_removal_delay[ i ] and nal_initial_alt_cbmb_removal_delay[ i ] represent the default and alternate initial CBMB removal delays, respectively, for the ith CBMB when the NAL HRD parameter is in use. The length of the syntax element is specified as initial_cbmb_removal_delay_length_minus1 + 1 and is in 90kHz clock units. The value of the syntax element must not be zero and is not greater than or equal to 90000 * ( CbmbSize[ i ] χ BitRate[ i ] ), which is the time corresponding to the CBMB size in 90kHz clock units.

[0498] nal_initial_cbmb_removal_offset[ i ] and nal_initial_alt_cbmb_removal_offset[ i ] represent the default and alternate initial CBMB removal offsets, respectively, for the i-th CBMB when the NAL HRD parameter is in use. The syntax element has a bit length specified as initial_cbmb_removal_delay_length_minus1 + 1 and is in 90kHz clock units.

[0499] In the entire CBMS, the sum of nal_initial_cbmb_removal_delay[ i ] and nal_initial_cbmb_removal_offset[ i ] must be constant for each value of i, and the sum of nal_initial_alt_cbmb_removal_delay[ i ] and nal_initial_alt_cbmb_removal_offset[ i ] must be constant for each value of i.

[0500] bmcl_initial_cbmb_removal_delay[ i ] and bmcl_initial_alt_cbmb_removal_delay[ i ] represent the default and alternate initial CBMB removal delays, respectively, for the i-th CBMB when the BMCL HRD parameter is in use. The length of the syntax element is specified as initial_cbmb_removal_delay_length_minus1 + 1 and is in 90kHz clock units. The value of the syntax element must be non-zero and not greater than or equal to 90000 * ( CbmbSize[ i ] χ BitRate[ i ] ), which is the time corresponding to the CBMB size in 90kHz clock units.

[0501] bmcl_initial_cbmb_removal_offset[ i ] and bmcl_initial_alt_cbmb_removal_offset[ i ] represent the default and alternate initial CBMB removal offsets, respectively, for the i-th CBMB when the BMCL HRD parameter is in use. The length of the syntax element is specified as initial_cbmb_removal_delay_length_minus1 + 1, in 90kHz clock units.

[0502] In the entire CBMS, the sum of bmcl_initial_cbmb_removal_delay[ i ] and bmcl_initial_cbmb_removal_offset[ i ] must be constant for each value of i, and the sum of bmcl_initial_alt_cbmb_removal_delay[ i ] and bmcl_initial_alt_cbmb_removal_offset[ i ] must be constant for each value of i.

[0503] An encoder is recommended not to include irap_cbmb_params_present_flag set to 1 in a buffering period SEI message associated with a CRA or BLA basemesh frame whose associated RASL basemesh frame follows at least one of its associated RADL basemesh frames in decoding order.

[0504] Basemesh Frame Timing SEI Message Semantics

[0505] The Basemesh Frame Timing SEI message provides CBMB removal delay and DBMB output delay information for the coded basemesh access unit associated with the SEI message.

[0506] The following applies to the BaseMesh Frame Timing SEI message syntax and semantics.

[0507] - The variable CbmbDbmbDelaysPresentFlag is derived from a syntax element found in the BaseMeshBufferingDuration SEI message and is applicable to at least one operation point to which the BaseMeshFrameTiming SEI message applies.

[0508] - The bitstream (or part thereof) refers to a subset (or part thereof) of the bitstream associated with every operation point to which the BaseMesh Frame Timing SEI message applies.

[0509] The syntax of a BaseMesh Frame Timing SEI message depends on the contents of the hrd_parameters() syntax structure applicable to the operation point to which the BaseMesh Frame Timing SEI message applies. These hrd_parameters() syntax structures are in the BMSPS that is enabled for the coded basemesh access unit associated with the BaseMesh Frame Timing SEI message. If the BaseMesh Frame Timing SEI message is associated with an IRAP coded basemesh access unit with NoOutputBeforeRecoveryFlag set to 1, activation of the BMSPS (and, for IRAP coded basemesh frames with NoOutputBeforeRecoveryFlag set to 1 that are not the first basemesh frame in the bitstream in decoding order, verification that the coded basemesh frame is IRAP with NoOutputBeforeRecoveryFlag set to 1) does not occur until the first coded submesh unit of the coded basemesh frame has been decoded, unless it is preceded by a BaseMesh Buffering Duration SEI message within the same coded basemesh access unit. Since the coded submesh units of a coded basemesh frame follow the basemesh frame timing SEI messages in NAL unit order, there may be cases where the decoder needs to store RBSPs containing basemesh frame timing SEI messages and then perform parsing of the basemesh frame timing SEI messages until it determines the active BMSPS of the coded basemesh.

[0510] au_cbmb_removal_delay_minus1 + 1 represents the number of clock ticks between the nominal CBMB removal time of a coded basemesh access unit associated with a basemesh frame timing SEI message and the previous coded basemesh access unit containing a buffering period SEI message in decoding order. This value is also used to compute the earliest time that coded basemesh access unit data arrives at the CBMB for HSS. The syntax element is a fixed-length code, with a length in bits of bp_au_cbmb_removal_delay_length_minus1 + 1.

[0511] The variable AuCbmbRemovalDelayMsb of the current base mesh frame is derived as follows.

[0512] - If the current basemesh frame is associated with a buffering period SEI message applicable to one or more operation points to which the basemesh frame timing SEI message applies, AuCbmbRemovalDelayMsb is set to 0.

[0513] - Otherwise, the following applies:

[0514] - Set prevAuCbmbRemovalDelayMinus1 and prevAuCbmbRemovalDelayMsb to be equal to au_cbmb_removal_delay_minus1 and AuCbmbRemovalDelayMsb of the previous basemesh frame that is not a RASL, RADL, or SLNR basemesh frame in decoding order, and ensure that they are within the same buffering period as the current basemesh frame.

[0515] - AuCbmbRemovalDelayMsb is derived as follows.

[0516] if( au_cbmb_removal_delay_minus1 <= prevAuCbmbRemovalDelayMinus1 )

[0517] AuCbmbRemovalDelayMsb = prevAuCbmbRemovalDelayMsb + 2au_cbmb_removal_delay_length_minus1 + 1 (F 1)

[0518] else

[0519] AuCbmbRemovalDelayMsb = prevAuCbmbRemovalDelayMsb

[0520] The variable AuCbmbRemovalDelayVal is derived as follows:

[0521] AuCbmbRemovalDelayVal = AuCbmbRemovalDelayMsb +

[0522] au_cbmb_removal_delay_minus1 + 1 (F 2)

[0523] The value of AuCbmbRemovalDelayVal is from 1 to 232 (inclusive).

[0524] mesh_dbmb_output_delay is used to calculate the DBMB output delay of a basemesh frame. It represents the number of clock ticks that must wait before a decoded basemesh frame is output from the DBMB after the coded basemesh access unit is removed from the CBMB.

[0525] If a basemesh frame is marked as "for short-term reference" or "for long-term reference", it is not removed from the DBMB at output time.

[0526] The length of the syntax element mesh_dbmb_output_delay is specified in bits as dbmb_output_delay_length_minus1 + 1. If bmsps_max_dec_mesh_frame_buffering_minus1 is 0, mesh_dbmb_output_delay will be 0.

[0527] The output time derived from mesh_dbmb_output_delay of any basemesh frame output from a decoder that respects output timing takes precedence over the output time derived from mesh_dbmb_output_delay of any basemesh frame in the subsequent CBMS in decoding order.

[0528] The base mesh output order set by the value of this syntax element is the same as the order set by the value of MeshFrmOrderCntVal.

[0529] For basemesh frames that are not output in the "bumping" process, the output time derived from mesh_dbmb_output_delay increases as the MeshFrmOrderCntVal value increases for all basemesh frames within the same CBMS, since they precede IRAP coded basemesh frames with NoOutputBeforeRecoveryFlag equal to 1 in the decoding order.

[0530] Figure 18 illustrates the general basemesh sequence parameter set RBSP syntax in a bitstream according to embodiments.

[0531] The general basemesh sequence parameters according to the embodiments include information (bm_vui_parameters_present_flag) indicating whether basemesh related VUI parameters (see FIG. 19) exist in the bitstream.

[0532] Figure 19 illustrates the Basemesh VUI parameters syntax in a bitstream according to embodiments.

[0533] The basemesh VUI parameter contains information (vui_bm_hrd_parameters_present_flag) indicating whether basemesh HRD parameters are present in the bitstream.

[0534] Figure 20 illustrates the Basemesh HRD parameters syntax in a bitstream according to embodiments.

[0535] Regarding Figures 18 to 20, the semantics of the basemesh VUI parameters are as follows:

[0536] If vui_timing_info_present_flag is 1, it indicates that vui_num_units_in_tick, vui_time_scale, vui_poc_proportional_to_timing_flag, and vui_hrd_parameters_present_flag are present in the vui_parameters( ) syntax structure. If vui_timing_info_present_flag is 0, it indicates that vui_num_units_in_tick, vui_time_scale, vui_poc_proportional_to_timing_flag, and vui_hrd_parameters_present_flag are not present in the vui_parameters( ) syntax structure.

[0537] vui_num_units_in_tick is the number of time units of a clock operating at a frequency of vui_time_scale Hz that correspond to one increment of the clock tick counter (called a clock tick). vui_num_units_in_tick is greater than 0. The clock tick in seconds is equal to the quotient of vui_num_units_in_tick divided by vui_time_scale. For example, if the frame rate of a BaseMesh signal is 25 Hz, vui_time_scale could be 27,000,000 and vui_num_units_in_tick could be 1,080,000, resulting in a clock tick of 0.04 seconds.

[0538] vui_time_scale is the number of time units that elapse in one second. For example, the vui_time_scale of a time coordinate system that measures time using a 27 MHz clock is 27,000,000. The value of vui_time_scale is greater than 0.

[0539] If vui_poc_proportional_to_timing_flag is 1, it indicates that the basemesh frame order count value for each basemesh frame of the CBMS other than the first basemesh frame of the CBMS in decoding order is proportional to the output time of that basemesh frame with respect to the output time of the first basemesh frame of the CBMS. If vui_poc_proportional_to_timing_flag is 0, it indicates that the basemesh frame order count value for each basemesh frame of the CBMS other than the first basemesh frame of the CBMS in decoding order may or may not be proportional to the output time of that basemesh frame with respect to the output time of the first basemesh frame of the CBMS.

[0540] vui_num_ticks_poc_diff_one_minus1 + 1 represents the number of clock ticks when the difference in basemesh frame order count values ​​is 1. The value of vui_num_ticks_poc_diff_one_minus1 ranges from 0 to 232 - 2.

[0541] If vui_bm_hrd_parameters_present_flag is 1, it indicates that the bm_hrd_parameters() syntax structure exists in the bm_vui_parameters() syntax structure. If vui_bm_hrd_parameters_present_flag is 0, it indicates that the bm_hrd_parameters() syntax structure does not exist in the bm_vui_parameters() syntax structure.

[0542] Baymesh HRD parameter semantics

[0543] The bm_hrd_parameters() syntax construct provides the Baymesh HRD parameters used in Baymesh HRD operations. For bitstream conformance to this version of this document, only one layer is supported, i.e., both nal_layer_id and nal_temporal_id are 0. When the bm_hrd_parameters() syntax construct is included in a BMSPS, the set of layers to which the bm_hrd_parameters() syntax construct applies is the set of layers whose associated layer identifier list contains all nal_layer_id values ​​in the CBMS.

[0544] To interpret the following semantics, a bitstream (or a portion thereof) refers to a subset of the bitstream (or a portion thereof) associated with the set of layers to which the bm_hrd_parameters() syntax construct applies.

[0545] If hrd_nal_parameters_present_flag is 1, it indicates that NAL HRD parameters (related to Type II bitstream conformance) are present in the bm_hrd_parameters( ) syntax structure. If hrd_nal_parameters_present_flag is 0, it indicates that NAL HRD parameters are not present in the bm_hrd_parameters( ) syntax structure.

[0546] If hrd_nal_parameters_present_flag is 0, conformance of the bitstream cannot be verified without providing the NAL HRD parameters and all buffering duration and Baymesh frame timing SEI messages through a method not specified in this document.

[0547] When hrd_bmcl_parameters_present_flag is 1, it indicates that BMCL HRD parameters (related to Type I conformance type) are present in the bm_hrd_parameters( ) syntax structure. When hrd_bmcl_parameters_present_flag is 0, it indicates that BMCL HRD parameters are not present in the bm_hrd_parameters( ) syntax structure.

[0548] If hrd_bmcl_parameters_present_flag is 0, conformance of the bitstream cannot be verified without providing the BMCL HRD parameters and all buffering duration and Baymesh frame timing SEI messages through a method not specified in this document.

[0549] hrd_bit_rate_scale, together with hrd_bit_rate_value_minus1[h][j], represents the maximum input bit rate of the jth CBMB. If hrd_bit_rate_scale is absent, its value is inferred to be 1.

[0550] hrd_cbmb_size_scale specifies the CBMB size of the j-th CBMB, along with hrd_cbmb_size_value_minus1[h][j], indicating that the CBMB operates at the coded baymesh access unit level. If hrd_cbmb_size_scale is absent, its value is inferred to be 1.

[0551] If hrd_fixed_basemesh_rate_general_flag is 1, it indicates that the temporal distance between the HRD output times of consecutive baymesh frames in the output order is constrained using additional syntax elements. If hrd_fixed_basemesh_rate_general_flag is 0, it indicates that this constraint is not applied. If hrd_fixed_basemesh_rate_general_flag is absent, it is inferred to be 0.

[0552] If hrd_fixed_basemesh_rate_within_cbms_flag is 1, it indicates that the temporal distance between the HRD output times of consecutive baymesh frames in the output order is constrained using additional syntax elements. If hrd_fixed_basemesh_rate_within_cbms_flag is 0, it indicates that this constraint is not applied. If hrd_fixed_basemesh_rate_within_cbms_flag is absent (when hrd_fixed_basemesh_rate_general_flag is 1), the value of hrd_fixed_basemesh_rate_within_cbms_flag is inferred to be 1.

[0553] The value of hrd_elemental_duration_in_tc_minus1 plus 1 represents the time interval (in clock ticks) between element units that specifies the HRD output time of consecutive Baymesh frames, according to the output order specified below. The value of hrd_elemental_duration_in_tc_minus1 ranges from 0 to 2047.

[0554] When hrd_fixed_basemesh_rate_general_flag is equal to 1 for a CBMS containing basemesh frame n, the value computed for DbmbOutputInterval[n] is equal to ClockTick * ( hrd_elemental_duration_in_tc_minus1 + 1 ), where ClockTick is a value specified according to embodiments (using the ClockTick value for the CBMS containing basemesh frame n), and one of the following conditions is true for the next basemesh in the output order nextMeshFrameInOutputOrder specified according to embodiments:

[0555] - basemesh frame nextMeshFrameInOutputOrder is in the same CBMS as basemesh frame n.

[0556] - Baymesh frame nextMeshFrameInOutputOrder is in another CBMS, hrd_fixed_basemesh_rate_general_flag is 1 in the CBMS containing the baymesh frame nextMeshFrameInOutputOrder, the value of ClockTick is the same in both CBMS, and the value of hrd_elemental_duration_in_tc_minus1 is the same in both CBMS.

[0557] When hrd_fixed_basemesh_rate_within_cbms_flag is equal to 1 for a CBMS containing basemesh frame n, the computed value for DbmbOutputInterval[n] is equal to ClockTick * ( hrd_elemental_duration_in_tc_minus1 + 1 ), where ClockTick is as specified in the formula according to the embodiments (using the ClockTick value for the CBMS containing basemesh frame n). This is the case when the next basemesh frame in the output order nextMeshFrameInOutputOrder specified for use in the formula according to the embodiments is in the same CBMS as basemesh frame n.

[0558] hrd_low_delay_flag indicates the HRD operating mode according to embodiments. If this flag is absent, the hrd_low_delay_flag value is assumed to be 0.

[0559] When hrd_low_delay_flag is 1, "large basemesh frames" that violate the nominal CBMB removal time due to the number of bits used in the coded basemesh access unit are permitted. These "large basemesh frames" are expected to occur intermittently, but are not required.

[0560] hrd_cbmb_cnt_minus1 + 1 indicates the number of alternative CBMB specifications in the CBMS bitstream. The value of hrd_cbmb_cnt_minus1 ranges from 0 to 31. If hrd_cbmb_cnt_minus1 is absent, the value is inferred to be 0.

[0561] The variable CbmbCnt is set equal to hrd_cbmb_cnt_minus1.

[0562] hrd_bit_rate_value_minus1[h][j] (together with hrd_bit_rate_scale) specifies the maximum input bit rate of the j-th CBMB when the CBMB operates at the access unit level, where h indicates the bitstream conformance type. hrd_bit_rate_value_minus1[h][j] ranges from 0 to 232-2 (232-*2). For j greater than 0, hrd_bit_rate_value_minus1[h][j] is greater than hrd_bit_rate_value_minus1[h][j-1].

[0563] The bit rate per second is:

[0564] BitRate[h][j] =

[0565] (hrd_bit_rate_value_minus1[h][j] + 1) * 2(6 + hrd_bit_rate_scale)

[0566] If the hrd_bit_rate_value_minus1[h][j] syntax element is absent, the BitRate[h][j] value is inferred to be equal to MaxBasemeshBR, where MaxBasemeshBR is specified according to embodiments.

[0567] hrd_cbmb_size_value_minus1[h][j] is used in conjunction with cbmb_size_scale to specify the size of the jth CBMB when CBMB operates at the access unit level, where h indicates the bitstream conformance type. hrd_cbmb_size_value_minus1[ h ][ j ] is in the range of 0 to 232 - 2. For j greater than 0 and a particular value of i, hrd_cbmb_size_value_minus1[ h ][ j ] is less than or equal to hrd_cbmb_size_value_minus1[ h ][ j - 1 ].

[0568] The CBMB size (in bits) is as follows:

[0569] CbmbSize[ h ][ j ] =

[0570] ( hrd_cbmb_size_value_minus1[ h ][ j ] + 1 ) * 2( 4 + hrd_cbmb_size_scale )

[0571] If the hrd_cbmb_size_value_minus1[ h ][ j ] syntax element is absent, the CbmbSize[ h ][ j ] value is inferred to be equal to MaxCBMBSize, where MaxCBMBSize is determined according to embodiments.

[0572] If hrd_cbr_flag[ h ][ j ] is 0, it indicates that the Virtual Stream Scheduler (HSS) operates in intermittent bit rate mode to decode this CBMS bitstream using the j-th CBMB specification, where h indicates the bitstream conformance type. If hrd_cbr_flag[ h ][ j ] is 1, it indicates that the HSS operates in constant bit rate (CBR) mode. If this value is absent, the value of hrd_cbr_flag[ h ][ j ] is assumed to be 0.

[0573] Figure 21 shows an encoding method according to embodiments.

[0574] The method according to the embodiments may include a step of encoding a base mesh of mesh data (S2100); a step of encoding a displacement of mesh data (S2110); and / or a step of encoding an attribute of mesh data (S2120).

[0575] The step of encoding the base mesh (S2100) may follow the description of the base mesh encoding operation of the transmitting device of FIG. 1, the base mesh encoding operation of the transmitting device of FIG. 2, the preprocessing of the base mesh of FIGS. 3 to 4, the encoding operation of FIG. 7, the encoder operation of FIG. 8, and the syntax information generation operation of FIGS. 16 to 20.

[0576] The step (S2110) of encoding displacement of mesh data may follow the description of the base mesh encoding operation of the transmitting device in FIG. 1, the base mesh encoding operation of the transmitting device in FIG. 2, the preprocessing of the base mesh in FIGS. 3 to 4, the encoding operation of FIG. 7, the encoder operation of FIG. 8, and the syntax information generation operation of FIGS. 16 to 20.

[0577] The step (S2120) of encoding attributes of mesh data may follow the description of the base mesh encoding operation of the transmitting device in FIG. 1, the base mesh encoding operation of the transmitting device in FIG. 2, the preprocessing of the base mesh in FIGS. 3 to 4, the encoding operation of FIG. 7, the encoder operation of FIG. 8, and the syntax information generation operation of FIGS. 16 to 20.

[0578] In relation to SEI, Basemesh buffering period SEI message, Basemesh frame timing SEI message, etc. according to embodiments, the method of FIG. 21 further includes a step of deriving SEI (Supplemental Enhancement Information); and the SEI may include a Basemesh buffering period SEI message and a Basemesh frame timing SEI message.

[0579] The bitstream may include offset information used to derive the removal timing of the buffer for the encoded basemesh, and offset information used to derive the output timing of the buffer for the decoded basemesh.

[0580] The bitstream may include delay information used to calculate the time to output a basemesh frame from the buffer for the decrypted basemesh.

[0581] The encoding method of FIG. 21 may be performed by an encoder device. The encoder device according to embodiments includes a memory; and at least one processor connected to the memory; wherein the at least one processor may be configured to: encode a base mesh of mesh data; encode a displacement of the mesh data; and encode an attribute of the mesh data.

[0582] The embodiments further include a computer-readable storage medium storing a bitstream generated by the method according to FIG. 21.

[0583] Embodiments may include a step of obtaining a bitstream for mesh data, the bitstream being generated based on a step of encoding a base mesh of the mesh data; a step of encoding a displacement of the mesh data; and a step of encoding an attribute of the mesh data; and a step of transmitting data including the bitstream.

[0584] Figure 22 shows a decryption method according to embodiments.

[0585] The method according to the embodiments may include a step of decoding a base mesh within a bitstream (S2200); a step of decoding a displacement within a bitstream (S2210); and / or a step of decoding an attribute within a bitstream (S2220).

[0586] The decryption method of Fig. 22 and the encoding method of Fig. 21 can follow the reverse process of each other.

[0587] The step of decoding the base mesh (S2200) may follow the description of the base mesh decoding operation of the decoder in Fig. 1, the receiving device in Fig. 2, the receiving device in Fig. 11, the receiving device in Fig. 13, etc., the HRD-related input buffer and / or output buffer control operation of Figs. 14 to 15, the base mesh-related syntax generation operation of Figs. 16 to 20, etc.

[0588] The step (S2210) of decoding displacement in a bitstream may follow the description of displacement decoding operations of the decoder in FIG. 1, the receiving device in FIG. 2, the receiving device in FIG. 11, the receiving device in FIG. 13, etc., the HRD-related input buffer and / or output buffer control operations of FIGS. 14 to 15, and the basemesh-related syntax generation operations of FIGS. 16 to 20.

[0589] The step (S2220) of decoding an attribute in a bitstream may follow the description of the attribute decoding operation of the decoder in FIG. 1, the receiving device in FIG. 2, the receiving device in FIG. 11, the receiving device in FIG. 13, etc., the HRD-related input buffer and / or output buffer control operation of FIGS. 14 to 15, the basemesh-related syntax generation operation of FIGS. 16 to 20, etc.

[0590] In relation to CBMB and / or SEI according to embodiments, the step of decoding the basemesh (S2200) may include: decoding the basemesh based on a buffer for decoding the basemesh and SEI (Supplemental Enhancement Information) in the bitstream.

[0591] With respect to the decoding unit arrival timing (AuFinalArrivalTime[m]) according to the embodiments, the decoding unit for the basemesh is a basemesh access unit, the arrival time of the buffer for the basemesh access unit is derived, the arrival time of the buffer is derived based on the size of the decoding unit, and the size of the decoding unit can be derived based on the type for suitability checking of the HRD (Hypothetical Reference Decoder).

[0592] With respect to the SEI delay offset syntax elements for the decoding unit removal timing (AuNominalRemovalTime) and / or the basemesh buffering period according to the embodiments, the removal time of the basemesh access unit from the buffer can be derived based on the delay offset information of the basemesh buffering period SEI in the bitstream.

[0593] With respect to the output delay (mesh_dbmb_output_delay i) of the basemesh frame output and the basemesh frame timing SEI message according to the embodiments, the decoded basemesh is stored in a buffer for the decoded basemesh, and the basemesh frame is output from the buffer based on the removal time of the basemesh access unit, and the basemesh frame is output from the buffer based on the output time, and the output time can be derived based on the output delay information of the basemesh frame timing SEI in the bitstream.

[0594] With respect to cbmb_delay_offset, dbmb_delay_offset, etc. of the basemesh buffering period SEI message according to FIG. 16 and the embodiments, the bitstream may include offset information used to derive the removal timing of the buffer for the encoded basemesh, and offset information used to derive the output timing of the buffer for the decoded basemesh.

[0595] With respect to the output delay (mesh_dbmb_output_delay) of the basemesh frame timing SEI message according to FIG. 17 and the embodiments, the bitstream may include delay information used to calculate the time for outputting the basemesh frame from the buffer for the decoded basemesh.

[0596] The decoding method of FIG. 22 may be performed by a decoder device. The decoder device according to embodiments includes a memory; and at least one processor connected to the memory; wherein the at least one processor may be configured to: decode a base mesh within a bitstream; decode a displacement within the bitstream; and decode an attribute within the bitstream.

[0597] The method and device according to the embodiments provide the following technical effects: The standardization process of V-DMC for dynamic mesh data and the basemesh hypothetical reference decoder (HRD) that can be commercialized can be efficiently utilized. Since reference information for decoding timing is transmitted from the decoder's perspective, there is an effect of supporting stable operation of the decoder. There is an effect of preventing buffer overflow and underflow. Stable decoding is performed (without data loss) even in an unstable network environment and an unstable data rate. A guide can be provided for output. The size required for the actual output can be effectively notified to the decoder. Cost-effective memory management is possible. There is an effect of guaranteeing decoding time and display output timing.

[0598] The embodiments have been described in terms of methods and / or devices, and the descriptions of methods and devices may be applied complementarily.

[0599] For the convenience of explanation, each drawing has been described separately, but it is also possible to design a new embodiment by combining the embodiments described in each drawing. In addition, designing a computer-readable recording medium having a program recorded thereon for executing the previously described embodiments, as needed by a person skilled in the art, also falls within the scope of the embodiments. The devices and methods according to the embodiments are not limited to the configurations and methods of the embodiments described above, but the embodiments may be configured by selectively combining all or part of the embodiments so that various modifications can be made. Although preferred embodiments of the embodiments have been illustrated and described, the embodiments are not limited to the specific embodiments described above, and various modifications can be made by a person skilled in the art to which the present invention pertains without departing from the gist of the embodiments claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the embodiments.

[0600] The various components of the devices of the embodiments may be implemented by hardware, software, firmware, or a combination thereof. The various components of the embodiments may be implemented by a single chip, for example, a single hardware circuit. According to embodiments, the components according to the embodiments may be implemented by separate chips. According to embodiments, at least one of the components of the devices of the embodiments may be configured with one or more processors capable of executing one or more programs, and the one or more programs may perform, or include instructions for performing, one or more of the operations / methods according to the embodiments. The executable instructions for performing the methods / operations of the devices of the embodiments may be stored in non-transitory CRMs or other computer program products configured to be executed by one or more processors, or may be stored in temporary CRMs or other computer program products configured to be executed by one or more processors. In addition, the memory according to the embodiments may be used as a concept including not only volatile memory (e.g., RAM, etc.), but also non-volatile memory, flash memory, PROM, etc. Additionally, it may be implemented in the form of a carrier wave, such as transmission via the Internet. Furthermore, the processor-readable recording medium may be distributed across network-connected computer systems, allowing the processor-readable code to be stored and executed in a distributed manner.

[0601] In this document, “ / ” and “,” are interpreted as “and / or”. For example, “A / B” is interpreted as “A and / or B”, and “A, B” is interpreted as “A and / or B”. Additionally, “A / B / C” means “at least one of A, B, and / or C”. Also, “A, B, C” means “at least one of A, B, and / or C”. Additionally, “or” in this document is interpreted as “and / or”. For example, “A or B” can mean 1) “A” only, 2) “B” only, or 3) “A and B”. In other words, “or” in this document can mean “additionally or alternatively”.

[0602] Terms such as "first," "second," etc. may be used to describe various components of the embodiments. However, the various components according to the embodiments should not be interpreted as limited by these terms. These terms are merely used to distinguish one component from another. For example, a first user input signal may be referred to as a "second user input signal." Similarly, a second user input signal may be referred to as a "first user input signal." The use of these terms should be interpreted as not departing from the scope of the various embodiments. Although "first user input signal" and "second user input signal" are both user input signals, they do not mean the same user input signals unless the context clearly indicates otherwise.

[0603] The terminology used to describe the embodiments is for the purpose of describing particular embodiments and is not intended to be limiting of the embodiments. As used in the description of the embodiments and in the claims, the singular is intended to include the plural unless the context clearly dictates otherwise. The expressions “and / or” are used to mean all possible combinations of terms. The expression “includes” describes the presence of features, numbers, steps, elements, and / or components, but does not mean that additional features, numbers, steps, elements, and / or components are not included. Conditional expressions such as “if” or “when” used to describe the embodiments are not intended to be limited to only optional cases. When a specific condition is satisfied, a related action is performed in response to a specific condition, or a related definition is intended to be interpreted.

[0604] Additionally, the operations according to the embodiments described in this document may be performed by a transceiver device including a memory and / or a processor according to the embodiments. The memory may store programs for processing / controlling the operations according to the embodiments, and the processor may control various operations described in this document. The processor may be referred to as a controller, etc. The operations according to the embodiments may be performed by firmware, software, and / or a combination thereof, and the firmware, software, and / or a combination thereof may be stored in the processor or in the memory.

[0605] Meanwhile, the operations according to the embodiments described above may be performed by a transmitting device and / or a receiving device according to the embodiments. The transmitting / receiving device may include a transmitting / receiving unit for transmitting and receiving media data, a memory for storing instructions (program code, algorithm, flowchart, and / or data) for a process according to the embodiments, and a processor for controlling the operations of the transmitting / receiving device.

[0606] The processor may be referred to as a controller or the like, and may correspond to, for example, hardware, software, and / or a combination thereof. The operations according to the above-described embodiments may be performed by the processor. Furthermore, the processor may be implemented as an encoder / decoder or the like for the operations of the above-described embodiments.

[0607] As described above, the relevant contents have been described in the best form for carrying out the embodiments.

[0608] As described above, the embodiments may be applied in whole or in part to a point cloud data transmission and reception device and system.

[0609] Those skilled in the art may make various changes or modifications to the embodiments within the scope of the embodiments.

[0610] Embodiments may include modifications / changes, which do not depart from the scope of the claims and their equivalents.

Claims

1. Step of decoding the base mesh in the bitstream; A step of decoding displacement within the bitstream; and A step of decoding an attribute within the bitstream; comprising: How to decrypt.

2. In paragraph 1, The steps for decoding the above base mesh are: Decoding the base mesh based on a buffer for decoding the base mesh and SEI (Supplemental Enhancement Information) in the bitstream, How to decrypt.

3. In paragraph 2, The decoding unit for the above base mesh is a base mesh access unit, The arrival time of the buffer for the above base mesh access unit is derived, The arrival time of the above buffer is derived based on the size of the above decoding unit, The size of the above decoding unit is derived based on the type for suitability testing of the HRD (Hypothetical Reference Decoder). How to decrypt.

4. In paragraph 2, The removal time of the base mesh access unit from the above buffer is Derived based on the delay offset information of the basemesh buffering cycle SEI in the above bitstream, How to decrypt.

5. In paragraph 4, The above decoded base mesh is stored in a buffer for the decoded base mesh, Based on the removal time of the above base mesh access unit, a base mesh frame is output from the buffer, The above base mesh frame is output from the buffer based on the output time, The above output time is derived based on the output delay information of the basemesh frame timing SEI in the bitstream. How to decrypt.

6. In paragraph 1, The above bitstream includes offset information used to derive the removal timing of the buffer for the encoded basemesh, and Contains offset information used to derive the output timing of the buffer for the decrypted basemesh. How to decrypt.

7. In paragraph 1, The above bitstream includes delay information used to calculate the time to output a basemesh frame from the buffer for the decrypted basemesh. How to decrypt.

8. Memory; and At least one processor connected to the memory; wherein the at least one processor comprises: Decode the basemesh within the bitstream; Decoding displacements within the above bitstream; and configured to decode an attribute within the above bitstream; Decryption device.

9. Step of encoding the base mesh of mesh data; a step of encoding the displacement of the above mesh data; and Encoding the attributes of the above mesh data; including; Encoding method.

10. In the 9th paragraph, the method A step of inducing SEI (Supplemantal Enhancement Information); further comprising, The above SEI includes a basemesh buffering cycle SEI message and a basemesh frame timing SEI message. Encoding method.

11. In paragraph 10, The above bitstream includes offset information used to derive the removal timing of the buffer for the encoded basemesh, and Contains offset information used to derive the output timing of the buffer for the decrypted basemesh. Encoding method.

12. In paragraph 10, The above bitstream includes delay information used to calculate the time to output a basemesh frame from the buffer for the decrypted basemesh. Encoding method.

13. Memory; and At least one processor connected to the memory; wherein the at least one processor comprises: Encode the base mesh of mesh data; Encoding the displacement of the above mesh data; and Encoding attributes of the above mesh data; configured to do so; Encoding device.

14. A computer-readable storage medium storing a bitstream generated by the method according to Article 13.

15. Step of obtaining bitstream for mesh data, The bitstream is generated based on the steps of encoding a base mesh of the mesh data; encoding a displacement of the mesh data; and encoding an attribute of the mesh data; and Comprising a step of transmitting data including the bitstream, method.

Images