Mesh compression of sub-mesh information with base mesh information signaled in first sub-bitstream and displacement information signaled in additional sub-bitstream

Abstract

The present disclosure relates to grid compression of sub-grid information having base grid information signaled in a first sub-bitstream and displacement information signaled in an additional sub-bitstream. The present disclosure relates to a system comprising an encoder configured to compress and encode data for a three-dimensional dynamic grid. The three-dimensional dynamic mesh uses a base mesh lattice bitstream and an atlas sub-bitstream (e.g., a shift and / or attribute sub-bitstream) to compress concurrent signaling. The atlas sub-bitstream includes displacement information to be applied at subdivision points of the base grid to reconstruct the three-dimensional dynamic grid at a given moment in time, and may also include attribute values, such as color, texture, etc., to be applied to the reconstructed grid. The sub-grid identifier is included in a tile header of the atlas sub-bitstream that enables selective reconstruction of the sub-grid. A complementary decoder decodes a compressed bitstream for the three-dimensional dynamic grid, such as encoded by the encoder.

Description

Technical Field

[0001] The present disclosure generally relates to compression and decompression of three-dimensional meshes with associated textures or attributes. Background Art

[0002] Various types of sensors (such as light detection and ranging (LIDAR) systems, 3D cameras, 3D scanners, etc.) can capture data indicating the location of points in three-dimensional space (e.g., locations in the X, Y, and Z planes). In addition, such systems can capture attribute information, such as color information (e.g., RGB values), texture attributes, intensity attributes, reflectivity attributes, motion-related attributes, modality attributes, or various other attributes, in addition to spatial information about the corresponding points. In some cases, additional attributes can be assigned to the corresponding points, such as a timestamp when the point was captured. The points captured by such sensors can constitute a "point cloud", which includes a set of points each having associated spatial information and one or more associated attributes. In some cases, a point cloud may include thousands of points, hundreds of thousands of points, millions of points, or even more points. In addition, in some cases, a point cloud can be generated, for example, in software, as opposed to the point cloud being captured by one or more sensors. In either case, such point clouds can include a large amount of data, and storing and transmitting these point clouds can be costly and time-consuming. Furthermore, three-dimensional visual content may also be captured in other ways, such as via 2D images of a scene captured from multiple viewing positions relative to the scene.

[0003] Such three-dimensional visual content can be represented by a three-dimensional mesh, which includes a plurality of polygons with connected vertices, which model the surface of the three-dimensional visual content (such as the surface of a point cloud). In addition, when modeled as a three-dimensional mesh, the texture or attribute values ​​of the points of the three-dimensional visual content can be overlaid on the mesh to represent the attributes or texture of the three-dimensional visual content.

[0004] In addition, a 3D mesh can be generated in the software, for example, without first modeling it as a point cloud or other type of 3D visual content. For example, the software can directly generate a 3D mesh and apply textures or attribute values ​​to characterize the object. Summary of the invention

[0005] In some embodiments, a decoder system includes one or more computing devices storing program instructions that, when executed by the one or more computing devices, cause the one or more computing devices to receive a bitstream representing a compressed mesh. The bitstream includes a base mesh sub-bitstream and an atlas sub-bitstream. The atlas sub-bitstream includes displacement values ​​for a plurality of vertices at subdivided positions of the base mesh to be included in a reconstructed version of the mesh; and a sub-mesh identifier for identifying corresponding sets of the displacement values ​​corresponding to corresponding sub-meshes in a plurality of sub-meshes of the mesh. The decoder system is further configured to parse the atlas sub-bitstream to locate a displacement vector corresponding to a given sub-mesh to be reconstructed; and to reconstruct the given sub-mesh while prohibiting reconstruction of one or more other sub-meshes of the sub-bitstream that were not selected for reconstruction.

[0006] In some embodiments, an encoder system includes one or more computing devices storing program instructions that, when executed by the one or more computing devices, cause the one or more computing devices to receive three-dimensional visual content to be encoded using video-based dynamic mesh compression; and generate an encoded bitstream, the encoded bitstream including: a base mesh sub-bitstream, the base mesh sub-bitstream including encoded mesh information for a base mesh; and an atlas sub-bitstream, the atlas sub-bitstream including displacement values ​​of a plurality of vertices at subdivided positions of the base mesh to be included in a reconstructed version of the mesh. In addition, the atlas sub-bitstream includes a sub-mesh identifier for identifying corresponding multiple sets of the displacement values ​​corresponding to corresponding sub-meshes of a plurality of sub-meshes of the mesh.

[0007] In some embodiments, a system includes one or more sensors configured to capture a point representing an object in the view of the sensor and to capture a texture or attribute value associated with the point of the object. The system also includes one or more computing devices storing program instructions that, when executed, cause the one or more computing devices to generate a three-dimensional mesh that models the point of the object using the vertices of the polygons defining the three-dimensional mesh and the connections between the vertices. In addition, in some embodiments, a three-dimensional mesh can be generated without first being captured by one or more sensors. For example, a computer graphics program can directly generate a three-dimensional mesh with an associated texture or associated attribute value to represent an object in a scene without having to generate a point cloud or other type of three-dimensional visual content representing the object.

[0008] In some embodiments, the encoder system includes one or more computing devices storing program instructions that, when executed by the one or more computing devices, also cause the one or more computing devices to determine multiple tiles of attributes for a three-dimensional mesh and a corresponding attribute map that maps the attribute tiles to the geometry of the mesh.

[0009] The encoder system may also encode the geometry of the mesh by encoding a base mesh and displacements of vertices relative to the base mesh.The compressed bitstream may include the compressed base mesh, the compressed displacement values, and the compressed attribute information.

[0010] In some embodiments, a subgrid may be indicated in the bitstream, where the subgrid corresponds to a subunit of a larger grid encoded in the bitstream. For example, three-dimensional content may include a grid representing a person's face, and as an example, the nose portion of the person's face may be represented using a subgrid, where the subgrid is part of a larger grid (such as a grid representing the person's face). Therefore, several subgrids may be part of a larger grid. The subgrid may be indicated in a base grid sub-bitstream of the bitstream, where different subgrids are signaled separately in the base grid sub-bitstream, or can be identified in the base grid sub-bitstream in other ways. However, in the atlas sub-bitstream, it may be more efficient to use a shared video frame to signal displacement data and / or attribute data for multiple subgrids. For example, slices corresponding to more than one subgrid may be packed into the same video frame.

[0011] Additionally, in some embodiments, only a subset of the sub-grids of the bitstream may need to be reconstructed and / or rendered. In order to selectively render only the subset of sub-grids, it may be necessary to parse the atlas sub-bitstream to separate the tiles corresponding to a given sub-grid from other tiles corresponding to other sub-grids that are included in the same video frame and are not selectively rendered.

[0012] In some embodiments, a tile header included in the atlas sub-bitstream includes a sub-grid identifier, such as a sub-grid ID and / or a sub-grid index reference, that allows a decoder to parse the atlas sub-bitstream and efficiently locate tiles corresponding to a given sub-grid without having to fully reconstruct other sub-grids that were not selected for reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 Example input information for defining a three-dimensional mesh according to some embodiments is illustrated.

[0014] Figure 2Alternative examples of input information for defining a three-dimensional mesh according to some embodiments are illustrated, where the input information is formatted according to an object format.

[0015] Figure 3 An example pre-processor and encoder for encoding a three-dimensional mesh according to some embodiments are illustrated.

[0016] Figure 4 A more detailed view of an example intra encoder is illustrated according to some embodiments.

[0017] Figure 5 An example intra decoder for decoding a three-dimensional mesh according to some embodiments is illustrated.

[0018] Figure 6 A more detailed view of an example inter-frame encoder is illustrated in accordance with some embodiments.

[0019] Figure 7 An example inter-frame decoder for decoding a three-dimensional mesh according to some embodiments is illustrated.

[0020] FIG. 8A to FIG. 8B The segmentation of a grid into multiple tiles is illustrated according to some embodiments.

[0021] Fig. 9 A grid is illustrated that is partitioned into two tiles, each tile comprising a sub-grid, according to some embodiments.

[0022] FIG. 10A to FIG. 10D Adaptive subdivision based on edge subdivision rules for shared edges is illustrated according to some embodiments.

[0023] FIG. 11A to FIG. 11C Adaptive subdivision of adjacent tiles is illustrated according to some embodiments, where shared edges are subdivided in a manner that ensures that vertices of adjacent tiles are aligned with each other.

[0024] Fig.12 An example set of vertices of a base mesh that are displaced by displacement vectors to produce a reconstructed dynamic mesh is illustrated in accordance with some embodiments.

[0025] Fig.13 Example components of a compressed bitstream according to some embodiments are illustrated.

[0026] Fig.14 Example components of an atlas sub-bitstream in a compressed bitstream according to some embodiments are illustrated.

[0027] Fig.15An example tile data unit included in an atlas sub-bitstream according to some embodiments is illustrated, wherein the sub-grid ID is included in a tile header of the tile data unit.

[0028] Fig.16 An example tile data unit included in an atlas sub-bitstream according to some embodiments is illustrated, wherein a single sub-grid ID and a “single sub-grid present” flag are included in a tile header of the tile data unit.

[0029] Fig.17 Another example tile data unit included in an atlas sub-bitstream according to some embodiments is illustrated, wherein sub-grid index position information is included in a tile header of the tile data unit.

[0030] Fig.18 An example parsing of tile data units to identify slice data units corresponding to a given sub-mesh to be reconstructed is illustrated in accordance with some embodiments.

[0031] Fig.19 An example parsing of multiple tile data units to identify path data units corresponding to a given sub-mesh to be reconstructed is illustrated in accordance with some embodiments.

[0032] Fig. 20 is a flow diagram illustrating a process performed by a decoder to reconstruct a given sub-mesh using a sub-mesh identifier included in an atlas sub-bitstream according to some embodiments.

[0033] Fig.21 is a flow diagram illustrating additional process steps performed by a decoder to selectively render a given sub-mesh while inhibiting rendering of other sub-meshes that were not selected for rendering according to some embodiments.

[0034] Fig. 22 is a flow diagram illustrating a process performed by an encoder to generate a bitstream for a compressed representation of three-dimensional visual volumetric content, according to some embodiments, wherein a sub-mesh identifier is included in an atlas sub-bitstream of the bitstream.

[0035] Fig.23 An example computer system that may implement an encoder or decoder according to some embodiments is illustrated.

[0036] This specification includes references to "one embodiment" or "an embodiment." The appearance of the phrase "in one embodiment" or "in an embodiment" does not necessarily refer to the same embodiment. The particular features, structures or characteristics may be combined in any suitable manner consistent with the present disclosure.

[0037] The term "comprising" is open ended. As used in the appended claims, the term does not exclude additional structures or steps. Consider the following cited claim: "an apparatus comprising one or more processor units..." Such a claim does not exclude the apparatus from including additional components (e.g., a network interface unit, graphics circuitry, etc.).

[0038] "Configured to", various units, circuits or other components may be described or described as "configured to" perform one or more tasks. In such contexts, "configured to" is used to imply a structure (e.g., a circuit system) by indicating that the unit / circuit / component includes a structure that performs the one or more tasks during operation. In this way, the unit / circuit / component is said to be configured to perform the task even when the specified unit / circuit / component is currently inoperable (e.g., not turned on). The units / circuits / components used with the "configured to" language include hardware-such as circuits, memories storing executable program instructions to implement operations, etc. Reference to a unit / circuit / component "configured to" perform one or more tasks is explicitly intended not to invoke 35 U.S.C. §112 (f) for the unit / circuit / component. In addition, "configured to" may include a general structure (e.g., a general circuit system) manipulated by software and / or firmware (e.g., an FPGA or a general processor executing software) to operate in a manner capable of performing one or more tasks to be solved. "Configured to" may also include adapting a manufacturing process (eg, a semiconductor fabrication facility) to produce a device (eg, an integrated circuit) suitable for implementing or performing one or more tasks.

[0039] "First," "second," etc. As used herein, these terms act as labels for the nouns that precede them, and do not imply any type of ordering (e.g., spatial, temporal, logical, etc.). For example, a buffer circuit may be described herein as performing a write operation of a "first" value and a "second" value. The terms "first" and "second" do not necessarily imply that the first value must be written before the second value.

[0040] "Based on". As used herein, this term is used to describe one or more factors that influence a determination. This term does not exclude additional factors that may influence the determination. That is, the determination may be based solely on these factors or at least in part on these factors. Consider the phrase "A is determined based on B". In this case, B is a factor that influences the determination of A, and such a phrase does not exclude that the determination of A may also be based on C. In other examples, A may be determined based solely on B. DETAILED DESCRIPTION

[0041] As data acquisition and display technologies become more advanced, the ability to capture volumetric content including thousands of points in 2D or 3D space has increased (such as via LIDAR systems). In addition, the development of advanced display technologies (such as virtual reality or augmented reality systems) has increased the potential uses of volumetric content. However, volumetric content files are typically very large, and storing and transmitting these volumetric content files can be costly and time consuming. For example, communication of volumetric content over a private network or a public network (such as the Internet) can require a considerable amount of time and / or network resources, such that some uses of the volumetric content (such as real-time uses) may be limited. In addition, the storage requirements of the volumetric content files may consume a significant amount of storage capacity of the device storing the volumetric content files, which may also limit potential applications for using the volumetric content data.

[0042] In some embodiments, an encoder may be used to generate compressed volumetric content to reduce the cost and time associated with storing and transmitting large volumetric content files. In some embodiments, a system may include an encoder that compresses attribute information and / or spatial information of volumetric content so that the volumetric content file can be stored and transmitted faster than non-compressed volumetric content and in a manner that the volumetric content file can occupy less storage space than non-compressed volumetric content.

[0043] In some embodiments, additionally or alternatively, such encoders and decoders or other encoders and decoders described herein may be adapted to encode three degrees of freedom plus (3DOF+) scenes, visual volumetric content (such as MPEG V3C scenes), immersive video scenes (such as MPEG MIV), etc.

[0044] In some embodiments, the dynamic mesh to be compressed and / or encoded may include a set of 3D meshes M(0), M(1), M(2)...M(n). Such as multiple representations of the dynamic mesh at different time points (i). Each mesh M(i) may be defined by connectivity information C(i), geometry information G(i), texture coordinates T(i), and texture connectivity TC(i). For each mesh M(i), one or more 2D images A(i,0), A(i,1)...A(i,D-1) describing the texture or attributes associated with the mesh may be included. For example, Figure 1 An example dynamic mesh M(i) including connectivity information C(i), geometry information G(i), texture image A(i), texture connectivity information TC(i) and texture coordinate information T(i) is illustrated. In addition, Figure 2 illustrates an example of a textured mesh stored in object (OBJ) format.

[0045] For example, Figure 2The example texture mesh stored in the object format shown includes geometric information listed as X, Y, and Z coordinates of the vertices and texture coordinates listed as two-dimensional (2D) coordinates of the vertices, where the 2D coordinates identify the pixel position of the pixel storing the texture information for the given vertex. The example texture mesh stored in the object format also includes texture connectivity information indicating the mapping between geometric coordinates and texture coordinates to form polygons such as triangles. For example, a first triangle is formed by three vertices, where the first vertex (1 / 1) is defined as a first geometric coordinate (e.g., 64.062500, 1237.739990, 51.757801), which corresponds to a first texture coordinate (e.g., 0.0897381, 0.740830). The second vertex (2 / 2) of the triangle is defined as a second geometric coordinate (e.g., 59.570301, 1236.819946, 54.899700), which corresponds to a second texture coordinate (e.g., 0.899059, 0.741542). Finally, the third vertex of the triangle corresponds to the third listed geometric coordinate that matches the third listed texture coordinate. However, it should be noted that in some examples, the vertices of a polygon such as a triangle may be mapped to a set of geometric coordinates and texture coordinates that may have different index positions in the corresponding lists of geometric coordinates and texture coordinates. For example, the second triangle has a first vertex that corresponds to the fourth listed set of geometric coordinates and the seventh listed set of texture coordinates. The second vertex corresponds to the first listed set of geometric coordinates and the first listed set of texture coordinates, and the third vertex corresponds to the third listed set of geometric coordinates and the ninth listed set of texture coordinates.

[0046] In some embodiments, the geometric information G(i) may characterize the positions of the vertices of the mesh in 3D space, and the connectivity C(i) may indicate how the vertices are connected together to form polygons that constitute the mesh M(i). In addition, the texture coordinates T(i) may indicate the positions of pixels in the 2D image corresponding to the vertices of the corresponding sub-mesh. The attribute patch information may indicate how to map the texture coordinates defined relative to the 2D bounding box to the three-dimensional space of the 3D bounding box associated with the attribute patch based on the way the points are projected onto the projection plane of the attribute patch. In addition, the texture connectivity information TC(i) may indicate how the vertices represented by the texture coordinates T(i) are connected together to form the polygons of the sub-mesh. For example, each texture or attribute patch of the texture image A(i) may correspond to a corresponding sub-mesh defined using the texture coordinates T(i) and the texture connectivity TC(i).

[0047] In some embodiments, the mesh encoder may perform a tile generation process in which the mesh is subdivided into a set of sub-grids and multiple groups of tiles are used to represent corresponding sub-grids in the sub-grid. These sub-grids may correspond to connected components of texture connectivity, or may be sub-grids that have texture connectivity different from that of the mesh. In some embodiments, the number and size of the sub-grids to be determined may be adjusted to balance discontinuities and the flexibility of updating the mesh, such as via inter-frame prediction. For example, smaller sub-grids may allow for more fine-grained updates to change specific areas of the mesh, such as using an inter-frame prediction process at a subsequent time. However, a larger number of sub-grids may introduce more discontinuities.

[0048] Figure 3 A high-level block diagram of the encoding process in some embodiments is illustrated. Note that a feedback loop during the encoding process enables the encoder to guide the pre-processing steps and change their parameters to achieve the best possible compromise based on various criteria such as: rate distortion, encoding / decoding complexity, random access, reconstruction complexity, terminal capabilities, encoder / decoder power consumption, network bandwidth and latency, and / or other factors.

[0049] A mesh, which may be a static or dynamic mesh, is received at preprocessing module 302. In addition, a property map characterizing how an attribute image (e.g., a texture image) of the static / dynamic mesh is mapped to the mesh is received at preprocessing module 302. For example, the attribute map may include texture coordinates and texture connectivity of the texture image of the mesh. Preprocessing module 302 separates the static / dynamic mesh into a base mesh and a displacement. The displacement characterizes how vertices will be displaced to recreate the original static / dynamic mesh from the base mesh. For example, in some embodiments, vertices included in the original static / dynamic mesh may be omitted from the base mesh (e.g., the base mesh may be a compressed version of the original static / dynamic mesh). As will be discussed in more detail below, the decoder may predict additional vertices to be added to the base mesh, for example, by subdividing the edges between the remaining vertices included in the base mesh. In such examples, the displacement may indicate how the additional vertices will be shifted, wherein the displacement of the added vertices modifies the base mesh to better represent the original static / dynamic mesh. For example, Figure 4 A detailed intra encoder 402 is illustrated that can be used to encode the displacements of the base mesh and the added vertices. For dynamic meshes, a Figure 5 The interframe encoder shown in Figure 5As seen in , instead of signaling a new base grid for each frame, the base grid for the current time frame may be compared to a reconstructed quantized reference base grid m' (e.g., the base grid that the decoder would see from the previous time frame), and a motion vector representing how the current base grid has changed relative to the reference base grid may be encoded instead of encoding a new base grid for each frame. Note that the motion vectors may not be encoded directly, but may be further compressed to exploit the relationship between the motion vectors.

[0050] The separated base grid m and displacement separated by the pre-processing module 302 are provided to the encoder 304, which can be as follows Figure 4 Intra-frame encoder as shown or as Figure 6 The inter-frame encoder shown in FIG. In addition, the attribute map is provided to the encoder 304. In some embodiments, in addition to the separated base grid and displacement, the original static / dynamic grid can also be provided to the encoder 304. For example, the encoder 304 can compare the reconstructed version of the static / dynamic grid (which has been reconstructed by the base grid and displacement) to determine the geometric distortion. In some embodiments, the attribute transfer process can be performed to adjust the attribute value of the attribute image to solve the slight geometric distortion. In some embodiments, feedback can be provided back to the pre-processing 302 by changing the way the original static / dynamic grid is extracted to generate the base grid, for example to reduce distortion. It should be noted that in some embodiments, the intra-frame encoder and the inter-frame encoder can be combined into a single encoder, which includes a logical component for switching between intra-frame encoding and inter-frame encoding. The output of the encoder 304 is a compressed bit stream representing the original static / dynamic grid and its associated attributes / textures.

[0051] With respect to mesh decimation, in some embodiments, a portion of the surface of a static / dynamic mesh may be considered to be an input 2D curve (represented by a 2D polyline), which is referred to as the "original" curve. The original curve may first be downsampled to generate a base curve / polyline, which is referred to as the "decimated" curve. A subdivision scheme (such as those described herein) may then be applied to the decimated polyline to generate the "subdivided" curve. For example, a subdivision scheme using an iterative interpolation scheme may be applied. The subdivision scheme may include inserting a new point in the middle of each edge of the polyline at each iteration. The inserted points represent additional vertices that may be moved by the displacement.

[0052] For example, the subdivided polyline is then deformed to better approximate the original curve. More precisely, a displacement vector is calculated for each vertex of the subdivided mesh so that the shape of the displaced curve approximates the shape of the original curve. The advantage of a subdivided curve is that it has a subdivision structure that allows efficient compression while ensuring a reliable approximation of the original curve. Compression efficiency is achieved through the following properties:

[0053] • Decimated / basis curves have a small number of vertices and require a limited number of bits to encode / transmit.

[0054] • Subdivision curves are automatically generated by the decoder once the base / decimated curves are decoded (eg, no need to signal or hardcode any information other than the subdivision scheme type and subdivision iteration count at the decoder).

[0055] • Generate displacement curves by decoding and applying displacement vectors associated with subdivision curve vertices. In addition to allowing spatial / quality scalability, the subdivision structure enables efficient wavelet decomposition, which provides high compression performance (eg with respect to rate-distortion performance).

[0056] For example, Figure 4 A more detailed view of an example intra encoder is illustrated according to some embodiments.

[0057] In some embodiments, the intra-frame encoder 402 receives the base grid m(i), the displacement d(i), the original static / dynamic grid M(i) and the attribute map A(i). The base grid m(i) is provided to the quantization module 404, wherein the various aspects of the base grid can be (optionally) further quantized. In some embodiments, various grid encoders can be used to encode the base grid. In addition, in some embodiments, the intra-frame encoder 402 can allow customization, wherein different corresponding grid coding schemes can be used to encode the base grid. For example, the static grid encoder 406 can be a selected grid encoder selected from a group of feasible grid encoders, such as a DRACO encoder (or another suitable encoder). The encoded base grid encoded by the static grid encoder 406 is provided to a multiplexer (MUX) 438 to be included in the compressed bit stream b(i). In some embodiments, the sub-grid can be encoded as a separate statically encoded grid in the base grid sub-bitstream. In addition, the encoded base grid is provided to a static grid decoder to generate a reconstructed version of the base grid (which the decoder will see). This reconstructed version of the base grid is used to update the displacements d(i) to account for any geometric distortion between the original base grid and the reconstructed version of the base grid (as seen by the decoder). For example, the static grid decoder 408 generates a reconstructed quantized base grid m'(i) and provides the reconstructed quantized base grid m'(i) to a displacement update module 410, which also receives the original base grid and the original displacements d(i). The displacement update module 410 compares the reconstructed quantized base grid m'(i) (as seen by the decoder) with the base grid m(i) and adjusts the displacements d(i) to account for the differences between the base grid m(i) and the reconstructed quantized base grid m'(i). These updated displacements d'(i) are provided to a wavelet transform 412, which applies a wavelet transform to further compress the updated displacements d'(i) and outputs wavelet coefficients e(i), which are provided to a quantization module 414, which generates quantized wavelet coefficients e'(i). The quantized wavelet coefficients may then be packed into 2D image frames via an image packing module 416, wherein the packed 2D image frames are further video encoded via video encoding 418. Additionally, the encoded video images are provided to a multiplexer (MUX) 438 for inclusion in a compressed bitstream b(i). Additionally, in some embodiments, displacement values ​​(such as indicated in the generated quantized wavelet coefficients e'(i), or indicated using other compression schemes) may be at least partially encoded outside of the video sub-bitstream, such as in their own displacement data sub-bitstream, in a base grid sub-bitstream, or in an atlas data sub-bitstream. Note that the atlas sub-bitstream includes both the atlas data sub-bitstream and the video sub-bitstream, wherein the atlas data sub-bitstream includes information for identifying the tiles included in the video sub-bitstream.

[0058] Furthermore, to account for any geometric distortion introduced relative to the original static / dynamic mesh, an attribute transfer process 430 may be used to modify attributes to account for the differences between the reconstructed deformed mesh DM(i) and the original static / dynamic mesh.

[0059] For example, video encoding 418 may further perform video decoding (or a complementary video decoding module ( Figure 4 )). This produces reconstructed packed quantized wavelet coefficients, which are unpacked via the image unpacking module 420. In addition, inverse quantization can be applied via the inverse quantization module 422, and the inverse wavelet transform 424 can be applied to generate the reconstructed displacement d"(i). In addition, the reconstructed quantized base grid m'(i) generated by the static grid decoder 408 can be inverse quantized via the inverse quantization module 428 to generate a reconstructed base grid m"(i). The reconstructed deformed grid generation module 426 applies the reconstructed displacement d"(i) to the reconstructed base grid m"(i) to generate a reconstructed deformed grid DM(i). It should be noted that the reconstructed deformed grid DM(i) represents the reconstructed grid that the decoder will generate, and solves any geometric deformation caused by losses introduced during the encoding process.

[0060] The attribute transfer module 430 compares the geometry of the original static / dynamic mesh M(i) with the reconstructed deformed mesh DM(i), and updates the attribute map to account for any geometric deformations, which is output as an updated attribute map A'(i). The updated attribute map A'(i) is then filled, where the 2D image including the attribute image is filled so that the space not used to convey the attribute image has been filled. In some embodiments, color space conversion is optionally applied at the color space conversion module 434. For example, the RGB color space used to characterize the color values ​​of the attribute image can be converted to a YCbCr color space, and color space subsampling, such as 4:2:0, 4:0:0, etc., can also be applied. The updated attribute map A'(i), which has been filled and optionally color space converted, is then video encoded via the video encoding module 436 and provided to the multiplexer 438 for inclusion in the compressed bitstream b(i).

[0061] In some embodiments, the controller 400 can coordinate the various quantization and inverse quantization steps and the video encoding and decoding steps so that inverse quantization "cancels" quantization and so that video decoding "cancels" video encoding. In addition, the attribute transfer module 430 can take into account the quantization level being applied based on communications from the controller 400.

[0062] Figure 5An example intra decoder for decoding a three-dimensional mesh according to some embodiments is illustrated.

[0063] The intraframe decoder 502 receives a compressed bit stream b(i), such as Figure 4 The compressed bitstream generated by the intra-frame encoder 402 is shown. The demultiplexer (DEMUX) 504 parses the bitstream into a base grid sub-component, a displacement sub-component, and an attribute map sub-component. It should be noted that the displacement sub-component and the attribute map sub-component may be included together as part of the atlas sub-bitstream of the bitstream. The static grid decoder 506 decodes the base grid sub-component to generate a reconstructed quantized base grid m'(i), which is provided to the inverse quantization module 518, which in turn outputs the decoded base grid m"(i) and provides it to the reconstructed deformed grid generator 520.

[0064] In addition, the displacement sub-component of the bitstream is provided to video decoding 508, where the video-encoded image frames are video-decoded and provided to image unpacking 510. Image unpacking 510 extracts the packed displacements from the video-decoded image frames and provides them to inverse quantization 512, where the displacements are inversely quantized. In addition, the inverse quantized displacements are provided to an inverse wavelet transform 514, which outputs decoded displacements d"(i). A reconstructed deformed mesh generator 520 applies the decoded displacements d'(i) to a decoded base mesh m"(i) to generate a decoded static / dynamic mesh M"(i). In addition, the attribute map sub-component is provided to video decoding 516, which outputs a decoded attribute map A"(i). The decoded mesh M"(i) and the decoded attribute map A"(i) can then be used to render a reconstructed version of the three-dimensional visual content at a device associated with the decoder.

[0065] like Figure 5 As shown, the bitstream is demultiplexed into three or more separate sub-bitstreams:

[0066] ●Base grid sub-bitstream,

[0067] ● a displacement sub-bitstream for positions and potentially for each vertex attribute (also called a geometry atlas sub-bitstream), and

[0068] • A property map sub-bitstream (also called property atlas sub-bitstream) for each property map.

[0069] In some embodiments, the three substreams may be referred to as a base grid sub-bitstream and an atlas sub-bitstream, where the atlas sub-bitstream includes such Fig.13 The displacement shown and also Fig.13 Properties shown.

[0070] The base trellis sub-bitstream is fed to a trellis decoder to generate a reconstructed quantized base trellis m'(i). The decoded base trellis m"(i) is then obtained by applying inverse quantization on m'(i). The proposed scheme is not restricted by which trellis codec is used. The trellis codec used may be explicitly specified in the bitstream or may be implicitly defined / fixed by the specification or the application.

[0071] The displacement sub-bitstream can be decoded by a video / image decoder. The generated image / video is then unpacked and inverse quantization is applied to the wavelet coefficients. In an alternative embodiment, the displacement can be decoded by a dedicated displacement data decoder. The proposed scheme is not restricted by which codec / standard is used. Image / video codecs such as [HEVC][AVC][AV1][AV2][JPEG][JPEG2000] can be used. Dictionary-based decoders such as ZIP or motion decoders for decoding grid motion information can be used, for example, as dedicated displacement data decoders. The decoded displacement d"(i) is then generated by applying an inverse wavelet transform to the unquantized wavelet coefficients. The final decoded grid is generated by applying a reconstruction process to the decoded base grid m"(i) and adding the decoded displacement field d"(i).

[0072] The attribute sub-bitstream is directly decoded by a video decoder and a decoded attribute map A"(i) is generated as output. The proposed scheme is not restricted by which codec / standard is used. Image / video codecs such as [HEVC] [AVC] [AV1] [AV2] [JPEG] [JPEG2000] can be used. Alternatively, the attribute sub-bitstream can be decoded by a non-image / video decoder (e.g., using a dictionary-based decoder such as ZIP). Multiple sub-streams can be decoded, each of which is associated with a different attribute map. Each sub-stream can use a different codec.

[0073] Figure 6 A more detailed view of an example inter-frame encoder is illustrated in accordance with some embodiments.

[0074] In some embodiments, the inter encoder 602 may include similar components as the intra encoder 402, but instead of encoding a base grid, the inter encoder may encode motion vectors that can be applied to a reference grid to generate a base grid at a decoder.

[0075] For example, in the case of dynamic meshes, a temporally consistent remeshing process is used, which may result in the same subdivision structure shared by the current mesh M'(i) and the reference mesh M'(j). Such a coherent temporal remeshing process makes it possible to skip encoding of the base mesh m(i) and reuse the base mesh m(j) associated with the reference frame M(j). This may also enable better temporal prediction for both attribute and geometry information. More precisely, a motion field f(i) may be computed and encoded that describes how to move the vertices of m(j) to match the positions of m(i). Figure 6 Such a process is described in . For example, motion encoder 406 may generate a motion field f(i) that describes how to move the vertices of m(j) to match the positions of m(i).

[0076] In some embodiments, the base grid m(i) associated with the current frame is first quantized (e.g., using uniform quantization) and encoded by a static grid encoder. The proposed scheme is not restricted by which grid codec is used. The grid codec used can be explicitly specified in the bitstream by encoding the grid codec ID, or can be implicitly defined / fixed by the specification or application.

[0077] Depending on the application and target bitrate / visual quality, the encoder may optionally encode a set of displacement vectors associated with the subdivided mesh vertices, which is referred to as a displacement field d(i).

[0078] The displacement field d(i) is then updated (at update displacement module 410) using the reconstructed quantized base grid m'(i) (e.g., the reconstructed output of base grid 408) to generate an updated displacement field d'(i), thereby processing the difference between the reconstructed base grid m'(i) and the original base grid m(i). A wavelet transform is then applied to d'(i) at wavelet transform 412 by utilizing the subdivision surface grid structure, and a set of wavelet coefficients is generated. The wavelet coefficients are then quantized at quantization 414, packed into a 2D image / video (at image packing 416), and compressed by an image / video encoder (at video encoding 418). The encoding of the wavelet coefficients can be lossless or lossy. A reconstructed version of the wavelet coefficients is obtained by applying image unpacking and inverse quantization to the reconstructed wavelet coefficient video generated during the video encoding process (e.g., at 420, 422, and 424). The reconstructed displacement d”(i) is then calculated by applying the inverse wavelet transform to the reconstructed wavelet coefficients. The reconstructed base mesh m”(i) is obtained by applying inverse quantization to the reconstructed quantized base mesh m'(i). The reconstructed deformed mesh DM(i) is obtained by subdividing m”(i) and applying the reconstructed displacement d”(i) to its vertices.

[0079] Since the quantization step and / or the grid compression module may be lossy, a reconstructed quantized version of m(i) is calculated (denoted as m'(i)). If the grid information is losslessly encoded and the quantization step is skipped, then m(i) will exactly match m'(i).

[0080] like Figure 6 As shown, the reconstructed quantized reference base grid m'(j) is used to predict the current frame base grid m(i). Figure 3 The pre-processing module 302 described in can be configured so that m(i) and m(j) share the same:

[0081] ●The number of vertices;

[0082] Connectivity

[0083] ● texture coordinates; and

[0084] ●Texture connectivity.

[0085] The motion field f(i) is computed by considering the quantized version of m(i) and the reconstructed quantized base mesh m'(j). Since m'(j) may have a different number of vertices than m(j) (e.g., vertices may be merged / removed), the encoder keeps track of the transform applied to m(j) to obtain m'(j) and applies it to m(i) to ensure a 1-to-1 correspondence between m'(j) and the transformed quantized version of m(i), denoted as m*(i). The motion field f(i) is computed by subtracting the quantized position p(i,v) of vertex v of m*(i) from the position p(j,v) of vertex v of m'(j):

[0086] f(i,v)=p(i,v)-p(j,v)

[0087] The motion field is then further predicted by the connectivity information of m'(j) and entropy coded (eg, context adaptive binary arithmetic coding may be used).

[0088] Since the motion field compression process may be lossy, a reconstructed motion field (denoted as f'(i)) is calculated by applying the motion decoder module 408. The reconstructed quantized basis grid m'(i) is then calculated by adding the motion field to the position of m'(j). The rest of the encoding process is similar to intra-frame coding.

[0089] Figure 7 An example inter-frame decoder for decoding a three-dimensional mesh according to some embodiments is illustrated.

[0090] The interframe decoder 702 includes Figure 5The inter-frame decoder 702 is similar to the intra-frame decoder 502 shown in FIG. However, instead of receiving a directly encoded base grid, the inter-frame decoder 702 reconstructs the base grid of the current frame based on the motion vector of the displacement field relative to the reference frame. For example, the inter-frame decoder 702 includes a motion field / vector decoder 704 and a reconstruction of the base grid module 706.

[0091] In a similar manner to the intra decoder, the inter decoder 702 splits the bitstream into three separate substreams:

[0092] ● motion substream (e.g. base grid subbitstream for inter-coding),

[0093] a displacement sub-stream (e.g., a displacement sub-bitstream of an atlas sub-bitstream), and

[0094] - Attribute substream (eg, attribute subbitstream of an atlas subbitstream).

[0095] The motion substream is decoded by applying a motion decoder 704. The proposed scheme is not restricted by which codec / standard is used to decode the motion information. For example, any motion decoding scheme can be used. Optionally, the decoded motion is then added to the decoded reference quantization base grid m'(j) to generate a reconstructed quantization base grid m'(i), i.e., the grid that has been decoded at instance j can be used for the prediction of the grid at instance i. The decoded base grid m"(i) is then generated by applying inverse quantization to m'(i).

[0096] About Figure 5 The displacement substream and the attribute substream are decoded in a similar manner to the intra-frame decoding process described above. The decoded grid M" (i) is also reconstructed in a similar manner.

[0097] The inverse quantization and reconstruction process is not standardized and may be implemented in various ways and / or combined with the rendering process.

[0098] Grid division and control coding to avoid cracks

[0099] In some embodiments, the grid may be subdivided into sets of tiles (e.g., sub-portions), and these tiles may potentially be grouped into multiple groups of tiles, such as sets of tile groups / tiles. In such embodiments, different encoding parameters (e.g., subdivision, quantization, wavelet transform, coordinate system, etc.) may be used to compress each tile or tile. In some embodiments, to avoid cracks at tile boundaries, lossless coding may be used for boundary vertices. In addition, quantization of wavelet coefficients for boundary vertices may be disabled, and local coordinate systems for boundary vertices may not be used.

[0100] In some embodiments, scalability can be supported at different levels. For example, temporal scalability can be achieved by temporal subsampling and frame reordering. In addition, different mechanisms can be used for geometry / vertex attribute data and attribute map data to achieve quality and spatial scalability. In addition, region of interest (ROI) reconstruction can be supported. For example, the encoding process described in the previous section can be configured to encode ROI with higher resolution and / or higher quality for geometry, vertex attributes and / or attribute map data. This is particularly useful for providing higher visual quality content (e.g., higher quality of the face relative to the rest of the body) under strict bandwidth and complexity constraints. Priority / importance / space / bounding box information can allow the decoder to adaptively decode a subset of the grid based on the view volume, power budget or terminal capability in association with slices, slice groups, tiles, network abstraction layer (NAL) units and / or sub-bitstreams. It should be noted that any combination of such decoding units can be used together to implement such functionality. For example, NAL units and sub-bitstreams can be used together.

[0101] In some embodiments, temporal and / or spatial random access may be supported. Temporal random access may be achieved by introducing IRAPs (intra-frame random access points) in different substreams (e.g., attribute atlas, video, grid, motion, and displacement substreams). Spatial random access may be supported by defining and using tiles, sub-pictures, slice groups, and / or slices, or any combination of these decoding units. Metadata describing the layout and relationships between different units may also need to be generated and included in the bitstream to help the decoder determine the units that need to be decoded.

[0102] As discussed above, various functions may be supported, such as:

[0103] ● Random access to space;

[0104] ● Adaptive quality allocation (e.g., bump compression, such as assigning higher quality to the face compared to the body of a human model);

[0105] ●Region of interest (ROI) access;

[0106] ● Coding unit level metadata (e.g. object description, bounding box information);

[0107] ● spatial and quality scalability; and

[0108] ● Adaptive streaming and decoding (eg, high priority areas are streamed / decoded first).

[0109] The disclosed compression scheme allows various coding units (e.g., slices, slice groups, and tiles) to be compressed using different encoding parameters (e.g., subdivision schemes, subdivision iteration counts, quantization parameters, etc.), which may introduce compression artifacts (e.g., cracks between slice boundaries). In some embodiments, as further discussed below, efficient strategies (e.g., efficient strategies in terms of computational complexity, compression efficiency, power consumption, etc.) can be used, which allow the scheme to handle different coding unit parameters without introducing artifacts.

[0110] Grid tiles

[0111] The grid may be divided into sets of tiles (eg, parts / segments) which may be encoded and decoded independently. Fig. 8A / Figure 8B As illustrated in , vertices / edges shared by more than one tile are replicated. Note that the mesh 800 is divided into two tiles 850 and 852 by replicating three vertices {V0, V1, V2} and two shared edges {(V0, V1), (V1, V2)}. In other words, when the mesh is divided into two tiles, each of the two tiles includes vertices and edges that are the previous set of vertices and edges in the combined mesh, so these vertices and edges are repeated in the tiles.

[0112] Subgrid

[0113] Each tile can be further split into a set of sub-grids that can exploit dependencies between them during encoding. For example, Fig. 9 An example of a grid 900 is shown, which is divided into two tiles (902 and 904), which contain three sub-grids and four sub-grids, respectively. For example, tile 902 includes sub-grids 0, 1, and 2; and tile 904 includes sub-grids 0, 1, 2, and 3.

[0114] The subgrid structure can be defined in any of the following ways:

[0115] ● explicitly encode a per-face integer attribute indicating the index of the submesh to which each face of the mesh belongs, or

[0116] ● CCs of a mesh are detected implicitly by treating each connected component (CC) as a sub-mesh, either with respect to connectivity of position or connectivity of texture coordinates, or both. Mesh vertices are traversed from neighbor to neighbor, which enables detection of CCs in a deterministic manner. The index assigned to CCs starts at 0 and increases by one each time a new CC is detected.

[0117] Sharding

[0118] In some embodiments, a sub-mesh may belong to one or more tiles (e.g., associating metadata with overlapping portions of the mesh). In another embodiment, a sub-mesh may belong to only a single tile. Vertices on the borders between tiles are not repeated. Tiles are also encoded using dependencies between them, so they cannot be encoded / decoded independently.

[0119] Various strategies can be used to encode the list of subgrids associated with a shard, such as:

[0120] Entropy decoding

[0121] Intra-frame prediction

[0122] Inter-frame prediction

[0123] ● Local subgrid index (i.e., smaller range)

[0124] Shard Group

[0125] A shard group is a group of shards. Shard groups are particularly useful for storing parameters shared by the shards within the group or allowing unique handles that can be used to associate metadata with those shards.

[0126] In some implementations, to support using different encoding parameters for each slice, the following may be used:

[0127] - The relationship between sub-meshes and tiles can be exploited to assign a tile ID to each face of the base mesh.

[0128] ●Vertices and edges belonging to a single sub-mesh are assigned the shard ID to which the sub-mesh belongs.

[0129] ●Vertices and edges that lie on the boundary of two or more shards are assigned to all corresponding shards.

[0130] ●When a subdivision scheme is applied, the decision whether to subdivide an edge depends on the subdivision parameters of all the shards to which the edge belongs.

[0131] For example, FIG. 11A to FIG. 11C Demonstrates adaptive subdivision based on shared edges. FIG. 11A to FIG. 11C The subdivision decision determined in (for example, suppose there is an edge belonging to two shards, shard 1 and shard 2, the subdivision iteration count of shard 1 is 0, and the subdivision iteration count of shard 2 is 2. The shared edge will be subdivided 2 times (i.e., taking the maximum subdivision count)), using FIG. 10A to FIG. 10D The subdivision scheme shown in Figure 2 subdivides the edges. For example, based on the subdivision decisions associated with different edges, apply Figures 10A-10D The adaptive subdivision scheme described in . Fig. 10A , Fig. 10B , Fig. 10C and Fig. 10DIt shows how a triangle should be subdivided when 3, 2, 1 or 0 edges of the triangle are subdivided respectively. The vertices created after each subdivision iteration are assigned to the fragments of their parent edges.

[0132] When applying quantization to a wavelet coefficient, the quantization parameter for the vertex is selected based on the quantization parameters of all the slices to which the vertex belongs. For example, suppose a slice belongs to two slices, slice 1 and slice 2. Slice 1 has a quantization parameter QP1. Slice 2 has a quantization parameter QP2. The wavelet coefficient associated with the vertex will be quantized using quantization parameter QP = min (QP1, QP2).

[0133] Grid tiles

[0134] To support grid tiles, the encoder can divide the grid into a set of tiles and then encode / decode these tiles independently by applying any grid codec, or use a grid codec that natively supports tile grid decoding.

[0135] In both cases, the set of shared vertices that lie on the tile boundary is repeated. To avoid gaps between tiles, the encoder can do either of the following:

[0136] ● Encode the splicing information indicating the mapping between repeated vertices as follows:

[0137] ○ Encode per-vertex labels identifying duplicate vertices (by encoding vertex attributes using the mesh codec)

[0138] ○ Encode for each duplicate vertex the index of the vertex it should be merged with.

[0139] ● Ensure that the decoded positions and vertex attributes associated with duplicate vertices match exactly

[0140] ○ Each vertex tag identifies duplicate vertices, and the mesh codec is used to decode this information into vertex attributes

[0141] ○ Apply the adaptive subdivision scheme described in the previous section to ensure consistent subdivision behavior across tile boundaries

[0142] ○ The encoder needs to maintain the mapping between duplicate vertices and adjust the encoding parameters to ensure matching values

[0143] ■ Encode repeated vertex positions and vertex attribute values ​​in a lossless manner

[0144] ■Disable wavelet transform (transform bypass mode)

[0145] ■ Perform a search in the coded parameter space to determine a set of parameters encoded in the bitstream and ensure that the positions and attribute values ​​match

[0146] ■Do nothing

[0147] ○ The decoder shall decompress each vertex label information to be able to identify duplicate vertices. If the encoder signals different encoding parameters for duplicate vertices compared to non-duplicate vertices, the decoder shall

[0148] Adaptively switches between two sets of parameters.

[0149] ○ The decoder may apply smoothing and automatic stitching as post-processing based on signaling provided by the encoder or based on analysis of the decoded mesh. In a specific embodiment, each vertex or patch includes a flag for enabling / disabling such post-processing. Among other things, the flag may:

[0150] ■Signaled as a SEI message,

[0151] ■ Encode using the trellis codec, or

[0152] ■Signaled in the atlas sub-bitstream.

[0153] In another embodiment, the encoder may replicate regions of the grid and store them in multiple tiles. This may be done to:

[0154] ●Fault tolerance,

[0155] ●Protective tape, and

[0156] ●Seamless / adaptive streaming.

[0157] Fig.12 An example set of vertices of a base mesh that are displaced by displacement vectors to produce a reconstructed dynamic mesh is illustrated in accordance with some embodiments.

[0158] When the total number of vertices of the target mesh and the predicted mesh is N, the vertex position in the target mesh is in (e.g., the X, Y, and Z position values ​​of the kth vertex), and the predicted vertex position in the mesh is in (e.g., the X, Y, and Z position values ​​of the kth vertex), the displacement values ​​of vertex k are conceptually as follows:

[0159] D k ={D k,0 ,D k,1 ,D k,2}={O k,0 -P k,0 ,O k,1 -P k,1,O k,2 -P k,2}

[0160] On the encoder side, the displacement data It can be transformed into displacement coefficients (or coefficients) by transformation methods such as wavelet transform In the case where the displacement data is not transformed, The transformed data may then be quantized.

[0161] After quantization (if applicable), Can be placed on the image. The kth component C of the i-th coefficient corresponding to the k-th coordinate in 3D space i,k Can be placed at pixel position {x i,k ,y i,k}. (i,k) and {x i,k ,y i,k The mapping between} can be expressed as {x i,k ,y i,k}=scanorder(i,k).

[0162] Signaling subgrid identifier

[0163] In some embodiments, several methods may be used to indicate subgrids. For example, the atlas tile layer may convey subgrid information in the atlas sub-bitstream. Figure 4 As shown, the compressed bitstream b(i) includes multiple sub-bitstreams, such as a compressed base grid bitstream, a compressed displacement sub-bitstream, and a compressed attribute sub-bitstream. It should be noted that both the compressed displacement sub-bitstream and the compressed attribute sub-bitstream are video encoded and can therefore be included in one or more atlas sub-bitstreams. As discussed above, the compressed grid can be divided into multiple sub-grids, and these sub-grids can have components in each sub-bitstream in the sub-bitstream. For example, the base grid of the sub-grid can be included in the base grid sub-bitstream. In addition, the atlas sub-bitstream may include compressed displacement values ​​and compressed attribute information of one or more sub-grids. The sub-grid signaling technology disclosed herein can be used in MPEG standard codecs, such as in V-DMC.

[0164] In some cases, for an atlas data sub-bitstream (e.g., a sub-bitstream including compressed displacement data (e.g., Figure 4For example, in the atlas data sub-bitstream, it may be beneficial to indicate a sub-grid ID of a sub-grid corresponding to each tile layer of the atlas data sub-bitstream. This may allow mesh refinement information, such as corresponding to a base mesh subdivision method, to be easily located in the atlas data sub-bitstream. This may also indicate a corresponding area in a 2D image where relevant refinement information may be found for a given sub-grid. Being able to locate the sub-grid information for each tile layer enables partial decoding, such as in V-DMC compression. The index of the sub-grid corresponding to the slice in the tile layer may be explicitly signaled in the atlas tile header. When multiple slices corresponding to the same sub-grid ID are signaled in different tile layers, the slices may be combined to create one sub-grid.

[0165] Fig.13 Example components of a compressed bitstream according to some embodiments are illustrated.

[0166] As an example, a compressed bitstream 1302, such as may be generated by encoder 402 or encoder 406, includes a base grid sub-bitstream 1304 and an atlas sub-bitstream 1306. The base grid sub-bitstream 1304 also includes coded base grids for different sub-grids, such as coded base grids for sub-grids X, Y, and Z. In some embodiments, the coded base grids of the sub-grids may include sub-grid IDs, such as ID XXXX, ID YYYY, and ID ZZZZ. For example, each of the coded sub-grid base grids 1308, 1316, and 1318 may be able to be identified in the base grid sub-bitstream 1304. However, in the atlas sub-bitstream, the displacement video frames 1312 to 1322 and the attribute video frames 1314 to 1324 are not limited to only including slices for a single sub-grid, but may include slices for more than one sub-grid. In addition, atlas data such as atlas tiles and path data 1310 to 1320 may include slice data for multiple video frames. Additionally, in some implementations, multiple tiles may reference the same video frame.Thus, determining which tiles conveyed in the atlas sub-bitstream 1306 correspond to which of the sub-grids conveyed in the base grid sub-bitstream 1304 is not trivial.

[0167] In some embodiments, to identify displacements communicated in tiles packed into video frames 1312 to 1322 and to identify property values ​​communicated in tiles packed into video frames 1314 to 1324, a subgrid ID (or other means for identifying a subgrid, such as a subgrid index position) is communicated in a tile header of atlas tiles and tile data 1310 to 1320. Additionally, in some embodiments, a subgrid identifier may also be signaled for a tile data unit (PDU) included in a tile.

[0168] More specifically, in a base mesh sub-bitstream, such as base mesh sub-bitstream 1304, each sub-mesh may be distinguished by its sub-mesh ID derived from the smh_id signaled in submesh_header(). The sub-mesh ID may be derived based on syntax elements signaled in the base mesh sub-mesh information.

[0169]

[0170] The arrays SubMeshIDToIndex and SubMeshIndexToID provide forward and reverse mappings of the ID associated with each submesh and the ordinal index of each submesh, respectively. The submesh ID is specified as SubMeshIndexToID[smh_id]. For example, in some embodiments, Fig.15 As shown, the sub-grid ID may be signaled in a tile header of a tile data unit of the atlas sub-bitstream 1306 to identify the location where the sub-grid displacement (and / or property value) is signaled in the atlas sub-bitstream 1306. As another example, Fig.17 As shown, a sub-grid index position value may be signaled in a tile header of a tile data unit of an atlas sub-bitstream 1306 to identify the location at which a sub-grid displacement (and / or property value) is signaled in the atlas sub-bitstream 1306 .

[0171] In a data unit corresponding to a slice, such as slice data unit 1410 (PDU 1410), a sub-mesh ID of a sub-mesh corresponding to the slice may be signaled. For example, in a slice data unit, a sub-mesh ID may be signaled as follows:

[0172]

[0173] pdu_submesh_id[tileID][patchIdx] indicates the associated submesh ID specified in the current atlas tile with tile ID equal to tileID, in the current tile with index patchIdx.

[0174] Fig.14 Example components of an atlas sub-bitstream in a compressed bitstream according to some embodiments are illustrated.

[0175] For example, the atlas tiles and slice data 1310 to 1320 may include tile date units, such as tile data units 1402, 1404, 1406, and 1408. Each tile data unit in the tile data units may include a tile header and one or more slice data units, wherein the slice data unit includes information for identifying the slices within a given tile. For example, tile data unit 1402 may correspond to tile 1 of displacement video frame 1312 and tile 1 of attribute video frame 1314. However, it should be noted that in some embodiments, different slice sizes and / or packing may be used to pack attribute slices in a different manner than displacement slices. As another example, tile data unit 1404 may correspond to different tiles in displacement video frame 1312 and attribute video frame 1314, such as tile 4. In addition, the separate slice data units included in the corresponding tile data units may provide information for identifying the slices in the corresponding tile and mapping the slices in the corresponding tile to the 3D volume of the reconstructed volumetric visual content.

[0176] Fig.15 An example tile data unit included in an atlas sub-bitstream according to some embodiments is illustrated, wherein the sub-grid ID is included in a tile header of the tile data unit.

[0177] For example, the tile data unit 1402 may include a tile header 1502 including a subgrid ID indicating a subgrid having a tile included in tile 1 (which corresponds to the tile data unit 1402). In addition, it is also possible to optionally signal in which PDU the corresponding subgrid tile is included. For example, the tile data unit 1402 includes a tile header 1502 including a subgrid ID, and the body 1504 includes a PDU for the tile represented by the tile data unit 1402.

[0178] Fig.16 An example tile data unit included in an atlas sub-bitstream according to some embodiments is illustrated, wherein a single sub-grid ID and a “single sub-grid present” flag are included in a tile header of the tile data unit.

[0179] In some embodiments where only one subgrid has tiles included in the tile corresponding to the tile data unit, a single subgrid present flag may be set. For example, the tile data unit 1404 corresponds to tile 2 and may include only a single tile (or multiple tiles) corresponding to a single subgrid. For example, the tile data unit 1404 includes a tile header 1602 that includes a subgrid ID and a single subgrid present flag, and the body 1604 includes a PDU for the tile represented by the tile data unit 1404.

[0180] Fig.17Another example tile data unit included in an atlas sub-bitstream according to some embodiments is illustrated, wherein sub-grid index position information is included in a tile header of the tile data unit.

[0181] As another option, in some embodiments, the subgrid index position information may be signaled in the tile header instead of signaling the subgrid ID in the tile header. Wherein the base grid sub-bitstream includes information for identifying a given subgrid based on its subgrid index position. For example, the tile data unit 1406 includes a tile header 1702, which includes the subgrid index position and optionally indicates in which PDU of the PDU the given subgrid has a tile. In addition, the body 1704 of the tile data unit 1406 includes a tile data unit, such as Fig.17 shown.

[0182] More specifically, in an atlas tile header, the number of sub-grids corresponding to the tile mentioned in the atlas tile header may be signaled as follows:

[0183]

[0184]

[0185] In the diagram above, the following syntax elements are used:

[0186] aa_use_single_mesh_flag[ath_id] indicates that the tiles mentioned in this atlas tile header with ath_id being ath_id correspond to a single mesh. When it is not present, aa_use_single_mesh_flag is inferred to be true.

[0187] athe_num_submeshes_minus2[ath_id] plus 2 indicates the number of submeshes corresponding to the tile mentioned in this atlas tile header with ath_id as ath_id. When it is not present, athe_num_submeshes_minus2 is inferred to be -1.

[0188] ● athe_signalled_submesh_id_length_minus1[ath_id] plus 1 specifies the number of bits used to represent the syntax element athe_submesh_id[ath_id][i].

[0189] athe_submesh_id[ath_id][i] indicates the ith submesh index of the submesh corresponding to the tile mentioned in the atlas tile header. When it is not present, athe_submesh_id[i] is inferred to be 0. The length of the athe_submesh_id[ath_id][i] syntax element is AsveSubmeshIdBitCount bits, where AsveSubmeshIdBitCount=

[0190] (athe_signalled_submesh_id_length_minus1[ath_id]+1)

[0191] In some implementations, athe_signalled_tile_id_length_minus1[ath_id] is required to be greater than or equal to bmsi_signalled_submesh_id_length_minus1.

[0192] Additionally, in some embodiments, the value of pdu_submesh_id[tileID][patchIdx] signaled in patch_data_unit(tileID, patchIdx) shall be one of athe_submesh_id[ath_id][i], where i is in the range of 0 to AtheNumSubmesh[ath_id]–1, inclusive.

[0193] In some embodiments, ath_submesh_id[i] shall not be equal to ath_submesh_id[j] for all i!=j.

[0194] In some implementations, AsveSubmeshIdBitCount may be derived from the syntax elements signaled in the ASPS (atlas sequence parameter set). For example:

[0195]

[0196] ●asve_signalled_submesh_id_length_minus1 plus 1 specifies the number of bits used to represent the syntax element athe_submesh_id[ath_id][i].

[0197] ●AsveSubmeshIdBitCount=(asve_signalled_submesh_id_length_minus1+1).

[0198] In some implementations, asve_signalled_tile_id_length_minus1 is required to be greater than or equal to bmsi_signalled_submesh_id_length_minus1.

[0199] In some embodiments, the number of submeshes corresponding to the tiles mentioned in the atlas tile header may be explicitly signaled instead of signaling athe_use_single_mesh_flag and athe_num_submeshes_minus1. For example, the tile data unit 1402 may explicitly signal that two submeshes are referenced in the tile header 1502 and the submesh IDs of the two submeshes. However, for Fig.16 The tile data unit 1404 shown may use a single sub-mesh presence flag rather than explicitly signaling the number of sub-meshes referenced in the tile header 1602 .

[0200] In addition, as regards Fig.17 As discussed, a mapping to a submesh ID may be signaled instead of directly signaling the submesh ID itself. For example, athe_submesh_id may be a mapping between a submesh id signaled in a slice and a submesh ID of a submesh. In this case, the signaling method may be as follows:

[0201]

[0202]

[0203] Where the submesh ID of the submesh for the tile pdu_submesh_id[tileID][patchIdx] is one of the values ​​in athe_submesh_id[ath_id].

[0204] In some implementations, when athi_single_tile_in_atlas_frame_flag is true or athi_num_tiles_in_atlas_frame_minus1 is 0, ath_signalled_submesh_id_flag[ath_id] is not signaled but inferred to be 0.

[0205] In some embodiments, the submesh index signaled in the tile data unit and the inter-tile data unit, such as pdu_submesh_id[tileID][patchIdx], may be encoded with athe_signalled_submesh_id_length_minus1[ath_id]+1 or asve_signalled_submesh_id_length_minus1+1 bits.

[0206] In some embodiments, when the number of submeshes in an atlas tile AtheNumSubmesh[ath_id] is 1, the submesh IDs of tile data units and inter-tile data units (such as pdu_submesh_id[tileID][patchIdx]) may be inferred to be athe_submesh_id[ath_id][0] instead of signaling them.

[0207] During reconstruction, when a submesh with a given submesh ID needs to be decoded, the decoder only decodes atlas tile layers (e.g., atlas data units) whose atlas tile header has the ID of that submesh in the atlas_submesh_id (e.g., atlas tile header submesh ID field).

[0208] Fig.18 An example parsing of tile data units to identify slice data units corresponding to a given sub-mesh to be reconstructed is illustrated in accordance with some embodiments.

[0209] For example, suppose there are 3 tiles and 5 submeshes. The tiles in tile[0] correspond to submeshes[0] and submeshes[2]. The tiles in tile[1] correspond to submeshes[3], and the tiles in tile[2] correspond to submeshes[1] and submeshes[4]. The athe_submesh_id in each tile header will be

[0210] athe_submesh_id[0] = {0,2}

[0211] athe_submesh_id[1] = {3}

[0212] athe_submesh_id[2] = {1,4}

[0213] When the decoder needs to decode submesh[1], the decoder parses down to the atlas tile header and searches for submeshID(=1) in athe_submesh_id. In this example, the decoder will find submeshID(=1) in tile[2]. The decoder will then only decode the tiles in tile[2] and not the other tiles (tile[0] and tile[1]).

[0214] Fig.19 An example parsing of multiple tile data units to identify path data units corresponding to a given sub-mesh to be reconstructed is illustrated in accordance with some embodiments.

[0215] In some embodiments, there is no restriction that the sub-mesh ID for the tiles in the atlas frame should be unique. For example, the information of a sub-mesh can be signaled in multiple tiles. In addition, multiple tiles can be signaled in multiple tiles. If multiple tiles contain information of a sub-mesh and are signaled in multiple tiles, this may have an impact on partial decoding because multiple tiles will be required to decode a sub-mesh. When sub-meshes with the same sub-mesh ID are reconstructed (= processed using information from the atlas data sub-bitstream, such as subdivision, adding displacement, etc.) multiple times, the sub-meshes are cascaded to create one sub-mesh. The order of cascading can be based on the tile ID or the tile index or the tile ID and the tile index. In another embodiment, the order of cascading can be explicitly signaled. When sub-meshes are cascaded, the connectivity can be updated based on the updated vertex index.

[0216] For example, suppose there are two tile data units in a tile. For example, pdu[0][0] and pdu[0][1] have the same pdu_submesh_id, 1. This is in the case of tile 1902. Fig.19, both slice data units include submesh 1. This indicates that information such as pdu_subdivision_method, pdu_transform_method, pdu_2d_pos_x, pdu_2d_pos_y, etc. from pdu[0][0] and pdu[0][1] are applied to the same submesh (e.g., for basemesh_submesh[1]) of the decoded basemesh (e.g., the output of the basemesh subbitstream). When submesh[1][0] is reconstructed using information from pdu[0][0] and basemesh_submesh[1], and submesh[1][1] is reconstructed using information from pdu[0][1], then the final submesh[1] is the concatenation of submesh[1][0] and submesh[1][1]. The vertex count of submesh[1][0] is pdu_vertex_count_minus1[0][0], and the vertex count of submesh[1][1] is pdu_vertex_count_minus1[0][1]. The vertex count of submesh[1] is the vertex count of submesh[1][0] + the vertex count of submesh[1][1].

[0217] In some embodiments, subgrid 1 may also be concatenated with subgrid information from tile 1904 .

[0218] Fig. 20 is a flow diagram illustrating a process performed by a decoder to reconstruct a given sub-mesh using a sub-mesh identifier included in an atlas sub-bitstream according to some embodiments.

[0219] At block 2002, a decoder receives a bitstream of three-dimensional (3D) visual volumetric content compressed using video-based dynamic mesh compression (V-DMC), wherein the bitstream includes a base mesh sub-bitstream (such as base mesh sub-bitstream 1304), an atlas sub-bitstream (such as atlas sub-bitstream 1306), and a sub-mesh identifier included in the atlas sub-bitstream, wherein the sub-mesh identifier indicates a tile and / or slice including information related to a given sub-mesh included in the base mesh sub-bitstream. For example, the received bitstream includes a sub-mesh identifier in the atlas sub-bitstream, such as a sub-mesh ID or index value as shown in tile headers 1502, 1602, and 1702.

[0220] At block 2004, the decoder parses the atlas sub-bitstream to locate the displacement vectors (and / or property values) corresponding to a given sub-mesh to be selectively reconstructed. Fig.18As shown, if attempting to reconstruct subgrid 4, the decoder may parse the tile headers for tiles 0, 1, and 2 without parsing the body of the corresponding tiles. Based on parsing the tile headers, the decoder may determine that the displacement and / or attribute information location in the video frame is defined in the slice data unit of tile 2. The decoder may then parse the body of tile 2 to extract the desired slice data unit information for subgrid 4 without having to parse the slice data unit information of other tiles and / or other subgrids that were not selected for reconstruction.

[0221] At block 2006, the decoder reconstructs the given sub-grid using a portion of the base grid sub-bitstream corresponding to the given sub-grid and using the displacement value corresponding to the given sub-grid parsed from the atlas sub-bitstream. For example, a specific one of the encoded base grids 1308, 1316, and 1318 may be used to generate a base grid for the given sub-grid, and the displacement value and the attribute value parsed from the PDU of the tile determined at block 2004 may be used to selectively reconstruct the given sub-grid.

[0222] Fig.21 is a flow diagram illustrating additional process steps performed by a decoder to selectively render a given sub-mesh while inhibiting rendering of other sub-meshes that were not selected for rendering according to some embodiments.

[0223] The reconstructed mesh may also be selectively rendered at block 2102. In some embodiments, the selective rendering may include applying a property value such as a texture or a color to the reconstructed mesh. Additionally, at block 2104, the decoder prohibits rendering of one or more other sub-meshes signaled in the base mesh sub-bitstream and the atlas sub-bitstream of the bitstream. For example, other sub-meshes may not be selected for rendering, and processing steps for those sub-meshes, such as identifying displacement values ​​and property values, may be omitted.

[0224] Fig. 22 is a flow diagram illustrating a process performed by an encoder to generate a bitstream for a compressed representation of three-dimensional visual volumetric content, according to some embodiments, wherein a sub-mesh identifier is included in an atlas sub-bitstream of the bitstream.

[0225] At block 2202, an encoder receives three-dimensional (3D) visual volumetric content to be encoded using video-based dynamic mesh compression (V-DMC).For example, encoders 402 and 406 may receive three-dimensional (3D) visual volumetric content to be encoded.

[0226] At block 2204, the encoder generates an encoded bitstream for the 3D visual volumetric content, wherein the encoded bitstream includes a base grid sub-bitstream, an atlas sub-bitstream, and a sub-grid identifier included in the atlas sub-bitstream, wherein the sub-grid identifier indicates a tile and / or slice including information related to a given sub-grid included in the base grid sub-bitstream. For example, encoder 402 or 406 may output Fig.13 The compressed bitstream 1302 shown in FIG. 1 includes a base grid sub-bitstream 1304 and an atlas sub-bitstream 1306. Figures 15 to 17 As shown, the atlas sub-bitstream 1306 includes a sub-mesh identifier in the tile header (and optionally in the PDU).

[0227] Example Computer System

[0228] Fig.23 An example computer system 2300 is illustrated that can implement an encoder or decoder or any other of the components described herein (e.g., as described above with reference to Figures 1 to 22 ). The computer system 2300 may be configured to perform any or all of the embodiments described above. In various embodiments, the computer system 2300 may be any of various types of devices, including, but not limited to, a personal computer system, a desktop computer, a laptop computer, a notebook computer, a tablet computer, an all-in-one computer, a tablet or netbook computer, a mainframe computer system, a handheld computer, a workstation, a network computer, a camera, a set-top box, a mobile device, a consumer device, a video game controller, a handheld video game device, an application server, a storage device, a television, a video recording device, a peripheral device (such as a switch, a modem, a router), or generally any type of computing or electronic device.

[0229] The various embodiments of the point cloud encoder or decoder described herein may be executed on one or more computer systems 2300, which may interact with various other devices. Figures 1 to 22 Any component, action or functionality described may be implemented in a system configured as Fig.232300. In the illustrated embodiment, the computer system 2300 includes one or more processors 2310 coupled to a system memory 2320 via an input / output (I / O) interface 2330. The computer system 2300 also includes a network interface 2340 coupled to the I / O interface 2330, and one or more input / output devices 2350, such as a cursor control device 2360, a keyboard 2370, and a display 2380. In some cases, it is contemplated that the embodiments may be implemented using a single instance of the computer system 2300, while in other embodiments, multiple such systems or multiple nodes making up the computer system 2300 may be configured to host different parts or instances of the embodiments. For example, in one embodiment, some elements may be implemented by one or more nodes of the computer system 2300 that are different from those nodes that implement other elements.

[0230] In various embodiments, computer system 2300 may be a uniprocessor system including one processor 2310 or a multiprocessor system including a plurality of processors 2310 (e.g., two, four, eight, or another suitable number). Processor 2310 may be any suitable processor capable of executing instructions. For example, in various embodiments, processor 2310 may be a general-purpose or embedded processor that implements any of a variety of instruction set architectures (ISAs), such as x86, PowerPC, SPARC, or MIPS ISAs, or any other suitable ISAs. In a multiprocessor system, each of processors 2310 may typically, but not necessarily, implement the same ISA.

[0231] The system memory 2320 may be configured to store point cloud compression or point cloud decompression program instructions 2322 and / or sensor data accessible by the processor 2310. In various embodiments, the system memory 2320 may be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash type memory, or any other type of memory. In the illustrated embodiment, the program instructions 2322 may be configured to implement an image sensor control application incorporating any of the above functionalities. In some embodiments, the program instructions and / or data may be received, sent, or stored on different types of computer accessible media or similar media separate from the system memory 2320 or the computer system 2300. Although the computer system 2300 is described as implementing the functionality of the functional blocks of the preceding figures, any functionality described herein may be implemented by such a computer system.

[0232] In one embodiment, the I / O interface 2330 may be configured to coordinate I / O communications between the processor 2310, the system memory 2320, and any peripheral devices (including the network interface 2340 or other peripheral interfaces, such as the input / output device 2350) in the device. In some embodiments, the I / O interface 2330 may perform any necessary protocol, timing, or other data conversion to convert data signals from one component (e.g., the system memory 2320) into a format suitable for use by another component (e.g., the processor 2310). In some embodiments, the I / O interface 2330 may include support for devices attached, for example, via various types of peripheral device buses (such as a variation of the peripheral component interconnect (PCI) bus standard or the universal serial bus (USB) standard). In some embodiments, the functions of the I / O interface 2330 may be divided into two or more separate components, such as a north bridge and a south bridge, for example. In addition, in some embodiments, some or all of the functions of the I / O interface 2330 (such as an interface to the system memory 2320) may be directly incorporated into the processor 2310.

[0233] The network interface 2340 may be configured to allow data to be exchanged between the computer system 2300 and other devices (e.g., carriers or proxy devices) attached to the network 2385 or between nodes of the computer system 2300. In various embodiments, the network 2385 may include one or more networks, including, but not limited to, a local area network (LAN) (e.g., an Ethernet or an enterprise network), a wide area network (WAN) (e.g., the Internet), a wireless data network, some other electronic data network, or some combination thereof. In various embodiments, the network interface 2340 may support communication via a wired or wireless general data network (such as any suitable type of Ethernet network), for example; communication via a telecommunications / telephone network (such as an analog voice network or a digital fiber optic communication network); communication via a storage area network (such as a Fiber Channel SAN), or communication via any other suitable type of network and / or protocol.

[0234] In some embodiments, input / output devices 2350 may include one or more display terminals, keyboards, keypads, touch pads, scanning devices, voice or optical recognition devices, or any other device suitable for inputting or accessing data by one or more computer systems 2300. Multiple input / output devices 2350 may be present in computer system 2300, or may be distributed across various nodes of computer system 2300. In some embodiments, similar input / output devices may be separate from computer system 2300 and may interact with one or more nodes of computer system 2300 via a wired or wireless connection (such as via network interface 2340).

[0235] like Fig.23 As shown, memory 2320 may include program instructions 2322, which may be executed by a processor to implement any element or action described above. In one embodiment, the program instructions may implement the method described above. In other embodiments, different elements and data may be included. Note that data may include any data or information described above.

[0236] It will be appreciated by those skilled in the art that computer system 2300 is merely illustrative, and is not intended to limit the scope of the embodiments. Specifically, computer systems and devices may include any combination of hardware or software that can perform the indicated functions, including computers, network devices, Internet devices, PDAs, wireless phones, pagers, etc. Computer system 2300 may also be connected to other devices that are not illustrated, or may otherwise be operated as an independent system. In addition, the functions provided by the illustrated components may be combined in fewer components or distributed in additional components in some embodiments. Similarly, in some embodiments, the functions of some components in the illustrated components may not be provided, and / or other additional functions may be available.

[0237] Those skilled in the art will also recognize that, although various items are illustrated as being stored in memory or on a storage device during use, for the purpose of memory management and data integrity, these items or parts thereof may be transferred between memory and other storage devices. Alternatively, in other embodiments, some or all of these software components may be executed in a memory on another device and communicate with the illustrated computer system via inter-computer communication. Some or all of the system components or data structures may also be stored (e.g., as instructions or structured data) on a computer-accessible medium or portable article to be read by a suitable drive, and various examples thereof are described above. In some embodiments, instructions stored on a computer-accessible medium separated from the computer system 2300 may be transmitted to the computer system 2300 via a transmission medium or signal (such as an electrical signal, an electromagnetic signal, or a digital signal transmitted via a communication medium such as a network and / or a wireless link). Various embodiments may also include receiving, sending, or storing instructions and / or data implemented according to the above description on a computer-accessible medium. Generally speaking, computer-accessible media may include non-transitory computer-readable storage media or memory media, such as magnetic or optical media, for example, disks or DVD / CD-ROMs, volatile or non-volatile media, such as RAM (e.g., SDRAM, DDR, RDRAM, SRAM, etc.), ROM, etc. In some embodiments, computer-accessible media may include transmission media or signals, such as electrical, electromagnetic, or digital signals transmitted via a communication medium such as a network and / or wireless link.

[0238] In different embodiments, the method described herein can be implemented in software, hardware or a combination thereof. In addition, the order of the frames of the method can be changed, and various elements can be added, reordered, combined, omitted, modified, etc. For those skilled in the art who benefit from the present disclosure, various modifications and changes can be made. The various embodiments described herein are intended to be illustrative and not restrictive. Many variations, modifications, additions and improvements are possible. Therefore, multiple examples can be provided for the components described as a single example in this article. The boundaries between various components, operations and data repositories are arbitrary to a certain extent, and specific operations are illustrated in the context of a specific illustrative configuration. Other allocations of functions are expected, and they may fall within the scope of the appended claims. Finally, the structure and function presented as discrete components in the example configuration may be implemented as a combined structure or component. These and other variations, modifications, additions and improvements may fall within the scope of the embodiments defined in the following claims.

[0239] Priority claim

[0240] This application claims the benefit of priority to U.S. Provisional Application Serial No. 63 / 588,967, filed on October 9, 2023, entitled “Mesh Compression with Base Mesh Information Signaled in a First Sub-Bitstream and Sub-Mesh Information Signaled with Displacement Information in an Additional Sub-Bitstream,” and the entire text of that U.S. Provisional Application is incorporated herein by reference.

Claims

1. A non-transitory computer-readable storage medium storing program instructions, wherein the program instructions, when executed using one or more computing devices, cause the one or more computing devices to perform the following operations: A bitstream of a mesh that has been compressed using video-based dynamic mesh compression is received, wherein: The bitstream comprises a base grid sub-bitstream, the base grid sub-bitstream comprising coded grid information for a base grid; The bitstream comprises an atlas sub-bitstream comprising displacement values ​​for a plurality of vertices at subdivision locations of the base mesh to be included in a reconstructed version of the mesh; and The atlas sub-bitstream comprises a sub-grid identifier for identifying respective sets of the displacement values ​​corresponding to respective sub-grids of the plurality of sub-grids of the grid; parsing the atlas sub-bitstream to locate a displacement vector corresponding to a given sub-grid; and The given sub-mesh of the mesh is reconstructed using at least a portion of the base mesh and the displacement values ​​corresponding to the given sub-mesh.

2. The non-transitory computer-readable storage medium of claim 1, wherein: signaling the displacement value in the atlas sub-bitstream using a tile data unit; and The atlas sub-bitstream includes information indicating a relationship between the content of a given tile data unit and a corresponding sub-grid included in the base grid sub-bitstream.

3. The non-transitory computer-readable storage medium of claim 2, wherein: The atlas sub-bitstream comprises a tile identifier of a tile of a video frame signaled in the atlas sub-bitstream, wherein the tile identifier further comprises tile data unit information for locating a corresponding tile included in the tile of the video frame; and The tile identifier includes a tile header indicating, for a corresponding tile in the tiles, a corresponding sub-grid in the sub-grid having a tile included in the corresponding tile in the tiles. 4 . The non-transitory computer-readable storage medium of claim 2 , wherein a corresponding tile header in the tile headers corresponding to a tile having only a single sub-grid included in the corresponding tile includes a single sub-grid existence flag.

5. The non-transitory computer-readable storage medium according to claim 2, wherein for a given sub-mesh, corresponding displacement values ​​are included in tile data units spanning two or more tiles, and The atlas sub-bitstream indicates a group of slice data units spanning the two or more tiles to be concatenated to form a concatenated slice data unit corresponding to a single sub-grid. 6 . The non-transitory computer-readable storage medium of claim 5 , wherein the tile data units to be concatenated are identified using a same sub-grid identifier in respective tile headers of the two or more tiles.

7. The non-transitory computer-readable storage medium according to any one of claims 1 to 6, wherein the sub-grid identifier is included in a tile header of the atlas sub-bitstream and / or a frame header of the atlas sub-bitstream.

8. The non-transitory computer-readable storage medium of claim 7, wherein a given tile header indicates a sub-mesh identifier for each sub-mesh having a displacement value signaled in the tile corresponding to the given tile header.

9. The non-transitory computer-readable storage medium of claim 8, wherein the sub-grid identifier included in the tile header of the atlas sub-bitstream comprises an index value corresponding to a sub-grid index of the base grid sub-bitstream.

10. The non-transitory computer-readable storage medium of any one of claims 1 to 6, wherein reconstructing the given sub-mesh of the mesh using at least a portion of the base mesh and the displacement values ​​corresponding to the given sub-mesh comprises: selectively rendering the given sub-mesh; as well as Rendering of one or more other sub-meshes of the mesh is prohibited, the one or more other sub-meshes being signaled in the base mesh sub-bitstream and the atlas sub-bitstream of the mesh.

11. The non-transitory computer-readable storage medium of any one of claims 1 to 6, wherein the sub-grid identifier further identifies respective sets of property values ​​corresponding to the respective sub-grids of the plurality of sub-grids of the grid, wherein the program instructions, when executed using the one or more computing devices, further cause the one or more computing devices to perform the following operations: parsing the atlas sub-bitstream to locate a property value corresponding to the given sub-mesh; and The given sub-mesh of the mesh is rendered using the attribute value corresponding to the given sub-mesh.

12. The non-transitory computer-readable storage medium of any one of claims 1 to 6, wherein the base-grid sub-bitstream further comprises sub-grid identifiers for the plurality of sub-grids to be used for the grid for a corresponding base grid.

13. A non-transitory computer-readable storage medium according to claim 1, wherein the base grid sub-bitstream or the atlas sub-bitstream includes mapping information for mapping a sub-grid ID to a sub-grid index used in the base grid sub-bitstream and / or the atlas sub-bitstream.

14. A method comprising: A bitstream of a mesh that has been compressed using video-based dynamic mesh compression is received, wherein: The bitstream comprises a base grid sub-bitstream, the base grid sub-bitstream comprising coded grid information for a base grid; The bitstream comprises an atlas sub-bitstream comprising displacement values ​​for a plurality of vertices at subdivision locations of the base mesh to be included in a reconstructed version of the mesh; and The atlas sub-bitstream comprises a sub-grid identifier for identifying respective sets of the displacement values ​​corresponding to respective sub-grids of the plurality of sub-grids of the grid; parsing the atlas sub-bitstream to locate a displacement vector corresponding to a given sub-grid; and The given sub-mesh of the mesh is reconstructed using at least a portion of the base mesh and the displacement values ​​corresponding to the given sub-mesh.

15. The method according to claim 14, further comprising: receiving an indication of the given sub-grid to be reconstructed, The reconstructing the given sub-grid comprises selectively reconstructing the given sub-grid based on the indication, while prohibiting reconstruction of other sub-grids among the plurality of sub-grids that are not indicated for reconstruction.

16. The method of claim 14, wherein: The atlas sub-bitstream comprises tile identifiers of tiles of video frames signaled in the atlas sub-bitstream; and The tile identifier includes a tile header indicating, for a corresponding tile in the tiles, a corresponding sub-grid in the sub-grid having a tile included in the corresponding tile in the tiles.

17. The method according to any one of claims 14 to 16, wherein the sub-grid identifier further identifies respective sets of attribute values ​​corresponding to the respective sub-grids of the plurality of sub-grids of the grid, wherein the method further comprises: Parsing the atlas sub-bitstream to locate the attribute value corresponding to the given sub-grid; as well as The given sub-mesh of the mesh is rendered using the attribute value corresponding to the given sub-mesh.

18. A non-transitory computer-readable storage medium storing program instructions, the program instructions, when executed using one or more computing devices, causing the one or more computing devices to perform the following operations: receiving three-dimensional visual content to be encoded using video-based dynamic mesh compression; as well as Generate a coded bit stream, the coded bit stream comprising: a base grid sub-bitstream, the base grid sub-bitstream comprising coded grid information for a base grid; an atlas sub-bitstream comprising displacement values ​​for a plurality of vertices at subdivision locations of the base mesh to be included in a reconstructed version of the mesh, The atlas sub-bitstream comprises a sub-grid identifier, and the sub-grid identifier is used to identify corresponding multiple groups of displacement values ​​corresponding to corresponding sub-grids among multiple sub-grids of the grid.

19. The non-transitory computer readable storage medium of claim 18, wherein: The atlas sub-bitstream also includes attribute values ​​of the grid; and The sub-grid identifier also identifies respective sets of the property values ​​corresponding to the respective sub-grids of the plurality of sub-grids of the grid.

20. A non-transitory computer-readable storage medium according to claim 18 or claim 19, wherein the base grid sub-bitstream or the atlas sub-bitstream includes mapping information for mapping a sub-grid ID to a sub-grid index used in the base grid sub-bitstream and / or the atlas sub-bitstream.

Images