V3C Syntax Extensions for Mesh Compression

Abstract

In the current implementation of V-DMC, (u,v) coordinates are generated using the Microsoft UVAtlas and carried in the base mesh sub-bitstream, along with 3D position and topology. The high-level syntax constructs described here support projection-based atlas map generation and a means for deriving (u,v) coordinates at the decoder side using V3C syntax construct extensions. Compared to previous implementations, to maintain the current V3C geometry bitstream concept, a separate sub-bitstream, referred to here as the vertex property sub-bitstream, is used to carry displacement information.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority under 35 U.S.C. § 119 to U.S. Provisional Patent Application No. 63 / 380,087, entitled "V3C SYNTAX EXTENSION FOR MESH COMPRESSION," filed October 19, 2022, which is incorporated herein by reference in its entirety for all purposes.

[0002] The present invention relates to three-dimensional graphics, and more particularly to mesh compression. [Background technology]

[0003] Recently, a new method for compressing volumetric content such as point clouds based on 3D-to-2D projections has been standardized. Also known as V3C (visual volumetric video-based compression), this method maps 3D volume data into a number of 2D patches, which are then arranged into an atlas image, which is then encoded by a video encoder. The atlas image corresponds to the geometry of the points, their respective textures, and an occupancy map that indicates which locations should be considered for point cloud reconstruction.

[0004] In 2017, MPEG issued a Call for Proposals (CfP) for the compression of point clouds. After evaluating several proposals, MPEG is currently considering two different techniques for point cloud compression: 3D native coding (based on octrees and similar coding methods) or 3D-to-2D projection followed by traditional video coding. For dynamic 3D scenes, MPEG is using Test Model Software (TMC2), which is based on patch surface modeling, projection of the patches from 3D to a 2D image, and coding of the 2D image with a video encoder such as HEVC. This method has proven more efficient than native 3D coding and can achieve competitive bitrates with acceptable quality.

[0005] Due to the success of coding 3D point clouds with projection-based methods (also known as video-based methods, or V-PCC), it is expected that future versions of this standard will include additional 3D data, such as 3D meshes. However, the current version of this standard is only suitable for transmitting sets of unconnected points, and therefore does not have a mechanism for transmitting point connectivity as required for 3D mesh compression.

[0006] Methods have also been proposed to extend the functionality of V-PCC to meshes. One possible approach is to encode vertices using V-PCC and then encode connectivity using a mesh compression method such as TFAN or Edgebreaker. A limitation of this approach is that the original mesh must be dense so that the point cloud generated from the vertices can be efficiently encoded after projection, rather than being sparse. Furthermore, because the order of vertices affects connectivity coding, different methods have been proposed to reorganize mesh connectivity. An alternative approach to encoding sparse meshes is to use raw patch data to encode the 3D vertex positions. Because raw patches directly encode (x,y,z), in this method, all vertices are encoded as raw data, while connectivity is encoded using a similar mesh compression method, as described above. Note that in raw patches, vertices can be transmitted in any preferred order, so the order generated from connectivity coding can be used. While this method can encode sparse point clouds, raw patches are not efficient for encoding 3D data, and additional data, such as triangular face attributes, may be missing from this method.

[0007] Microsoft's UVAtlas is the state-of-the-art automatic texture map generation, but it takes a significant amount of time and only optimizes for local frames. V-PCC generates patches using orthogonal projection, but only for point clouds, so it does not support patch generation for meshes. Summary of the Invention [Problem to be solved by the invention]

[0008] In the current implementation of V-DMC, the (u,v) coordinates are generated using the Microsoft UVAtlas and carried in the base mesh sub-bitstream, along with the 3D position and topology. The high-level syntax constructs described herein support projection-based atlas map generation and a means for deriving the (u,v) coordinates at the decoder side using V3C syntax construct extensions. Compared to previous implementations, to maintain the concept of the current V3C geometry bitstream, a separate sub-bitstream, referred to herein as the vertex property sub-bitstream, is used to carry displacement information. [Means for solving the problem]

[0009] In one aspect, a method programmed into a non-transitory memory of a device includes implementing an atlas mapping process, performing connectivity processing to divide triangles and generate vertex information, and performing vertex position processing to adjust vertex positions within the vertex information. The method further includes implementing a displacement sub-bitstream to include displacement information, which is utilized by the vertex position processing. The method further includes reconstructing a mesh based on the displaced vertices and the connectivity information. The atlas mapping process includes receiving patch identification information and mapping function parameters, and generating (u,v) coordinates based on the patch identification information and the mapping function parameters. The method further includes utilizing one or more flags to bypass one or more V-DMC methods and use texture attribute map video as a V3C attribute component video. The implementing the atlas mapping process occurs before performing the connectivity processing and the vertex position processing. The implementing the atlas mapping process occurs after performing the connectivity processing and the vertex position processing.

[0010] In another aspect, an apparatus includes: a non-transitory memory for storing an application, the application for implementing an atlas mapping process, performing a connectivity process to split triangles and generate vertex information, and performing a vertex position process to adjust vertex positions within the vertex information; and a processor coupled to the memory and configured to process the application. The application is further for implementing a displacement sub-bitstream to include displacement information, the displacement information being utilized by the vertex position process. The application is further for reconstructing a mesh based on the displaced vertices and the connectivity information. The atlas mapping process includes receiving patch identification information and mapping function parameters, and generating (u,v) coordinates based on the patch identification information and the mapping function parameters. The application is further for utilizing one or more flags to bypass one or more V-DMC methods and use texture attribute map video as a V3C attribute component video. Implementing the atlas mapping process occurs before performing the connectivity process and the vertex position process. Implementing the atlas mapping process is performed after performing the connectivity process and the vertex position process.

[0011] In another aspect, a system includes an encoder configured to encode a 3D mesh to generate patch identification information and mapping function parameters; and a decoder configured to implement an atlas mapping process on the patch identification information and the mapping function parameters, perform a connectivity process to split triangles and generate vertex information, and perform a vertex position process to adjust vertex positions within the vertex information. The decoder is further configured to implement a displacement sub-bitstream to include displacement information, which is utilized by the vertex position process. The decoder is further configured to reconstruct a mesh based on the displaced vertices and the connectivity information. The atlas mapping process includes receiving patch identification information and mapping function parameters and generating (u,v) coordinates based on the patch identification information and the mapping function parameters. The decoder is further configured to utilize one or more flags to bypass one or more V-DMC methods and use texture attribute map video as a V3C attribute component video. Implementing the atlas mapping process occurs before performing the connectivity process and the vertex position process. Implementing the atlas mapping process is performed after performing the connectivity process and the vertex position process. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram of a decoding implementation according to some embodiments. [Figure 2] FIG. 1 is a diagram of a decoding implementation according to some embodiments. [Figure 3] FIG. 1 is a diagram of a modified V-DMC standard according to some embodiments. [Figure 4] FIG. 1 is a block diagram of an exemplary computing device configured to implement a mesh compression method according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0013] A mesh typically consists of a set of polygons that represent the surface of a volume. An efficient way to represent the surface properties (e.g., color properties) of a mesh is to generate a texture atlas that maps the 3D surface properties to a 2D surface. The results of the mapping function are stored in (u,v) coordinates, added to the mesh data, and then further encoded using mesh compression methods. However, the presence of (u,v) coordinates can significantly increase the size of the compressed mesh.

[0014] The latest international point cloud compression standard, V-PCC, uses orthogonal projection to generate depth map images of point clouds. The projection parameters are encoded in a metadata bitstream known as the atlas bitstream, so the decoder only needs to take those parameters and apply a mapping function to each (u,v) coordinate in the depth map to reconstruct the 3D information.

[0015] As an extension to the current V3C standard, we describe the inclusion of syntax elements that allow the transmission of parameters for generating mapping coordinates for meshes using orthogonal projection. The new syntax elements define a displacement video sub-bitstream, reserving the original definition of a geometry image already defined in the V3C standard. The displacement of a point (e.g., the movement of a point in 3D space) is not necessarily in the direction of the projection, and a new bitstream (instead of a geometry bitstream) can be used to indicate the displacement. We also describe how to define the relationship between patches and faces to be coded in a general mesh encoder and indicate the appropriate signaling.

[0016] The new VDMC standard for dynamic mesh compression is a future extension of the V3C standard, the current international standard for video-based volume coding. The new syntax disclosed herein accommodates several important new techniques for efficient coding, such as texture mapping and the use of displacement images, and integrates these new techniques while preserving the existing V3C syntax. The disclosed syntax allows for decoder-side derivation of atlas mapping coordinates, improving integration with current V3C syntax elements currently used for point cloud coding.

[0017] New syntax elements allow sending orthoAtlas parameters to the decoder, which is a syntax for texture map generation using orthogonal projection.

[0018] Flexibility for signaling projection-based atlas parameterization (e.g., "orthoatlas") includes describing and using V3C syntax structures and extensions to allow deriving (u,v) coordinates at the decoder side, rather than carrying them in the base mesh sub-bitstream as is currently done, and keeping the V3C tile header syntax unchanged by assuming a one-to-one correspondence between a single tile and a single sub-mesh (which makes the tile header syntax structure the same as the V3C tile header syntax). Furthermore, signaling the association between tiles and sub-meshes is performed as part of the afps extension. The use of new patch types is similar to those previously introduced (e.g., I_MESH), with minimal changes to the original V3C patch data type information.

[0019] Compared to previous approaches, we separate and explicitly distinguish between the geometry sub-bitstream and the displacement sub-bitstream, but use the geometry sub-bitstream to carry displacement information. The displacement sub-bitstream instead becomes a separate sub-bitstream. In this way, the geometry sub-bitstream remains semantically consistent with the current V3C geometry concept, potentially enhancing the surface reconstruction process.

[0020] A general module for "connectivity" processing has been introduced, of which the "subdivision" method is one instance. Several "flags" can provide options to use / bypass specific V-DMC methods. Texture attribute map video can be considered a V3C attribute component video. V3C atlas tile partition information and patch information data can be used, but the aforementioned mesh-specific tile partitions cannot.

[0021] Patch IDs and face group IDs are signaled in the mesh bitstream as associations between (patchId, facegroupId) pairs, allowing for a more general, not necessarily Draco-specific, signaling method.

[0022] Figure 1 shows a diagram of a decoding implementation according to some embodiments. V3C sub-bitstreams are received. For example, a base mesh sub-bitstream 100, an atlas sub-bitstream 102, a displacement sub-bitstream 104, an attribute sub-bitstream 106, and a geometry sub-bitstream 108 are received. The sub-bitstreams are decoded by a mesh codec 110, a first video codec 112, a second video codec 114, or a third video codec 116. For example, the base mesh sub-bitstream 100 is decoded by the mesh codec 110, the displacement sub-bitstream 104 is decoded by the first video codec 112, and the attribute sub-bitstream 106 is decoded by the second video codec 114. The displacement sub-bitstream 104 is decoded by the video codec 112 and the displacement decoder 122.

[0023] The mesh codec 110 decodes the base mesh sub-bitstream 100 and outputs a decoded base mesh 120 that includes the patch ID.

[0024] The atlas mapping process 130 is an additional implementation. The atlas mapping process 130 receives (patchID, 0) information based on the decoded base mesh. The atlas mapping process 130 also receives parameters (e.g., C, P) from the atlas sub-bitstream 102 in the form of patches and regenerates the u, v coordinates.

[0025] The generated u,v coordinates are used in connectivity processing 132 and vertex position processing 134. A surface processing 140 is also implemented to generate bumps or other features on the surface. The object / mesh 136 is reconstructed based on the processing. Additionally, a texture map 138 is reconstructed based on the decoded attribute sub-bitstream 106.

[0026] Figure 2 shows a diagram of a decoding implementation according to some embodiments. The implementation is similar to the implementation described in Figure 1, except that in this implementation, the atlas mapping process 130 is performed after the vertex position process 134. The generation of the (u,v) coordinates is performed in fragment shading. Because the mesh has already been subdivided, more (u,v) coordinates are generated.

[0027] Figure 3 shows a diagram of the modified V-DMC standard according to some embodiments. Instead of receiving (u,v) coordinates, the atlas mapping process 130 receives simplified information ((x,y,z) and (v1,v2,v3)) and generates (u,v) coordinates. The simplified information and (u,v) coordinates are passed to the connectivity process 132, which splits triangles and generates vertex information. The vertex position process 134 adjusts the positions of the vertices in the vertex information. For example, gray dots 300 represent the original positions of the vertices, and black dots 302 represent the adjusted positions of the vertices. The displaced vertices and connectivity are then used to reconstruct a mesh 304. Texture transfer 306 is also performed.

[0028] V-DMC High-Level Syntax and Semantics V3C unit JPEG2025530443000002.jpg75170 JPEG2025530443000003.jpg141170

[0029] vuh_vertex_property_index indicates the index of the vertex property data carried in the vertex property data unit. The value of vuh_vertex_property_index shall be in the range from 0 to (vps_vdmc_vertex_property_video_count[vuh_atlas_id] - 1), inclusive. If present, vuh_reserved_zero_10bits shall be equal to 0 in bitstreams conforming to this version of this document. Other values ​​of vuh_reserved_zero_10bits are reserved for future use by ISO / IEC. Decoders shall ignore the value of vuh_reserved_zero_10bits.

[0030] JPEG2025530443000004.jpg34170 JPEG2025530443000005.jpg47170

[0031] V3C parameter set extension As a base, the previously introduced extended syntax structure is used, but some syntax elements are added and removed, as shown below. JPEG2025530443000006.jpg141170 JPEG2025530443000007.jpg24170

[0032] vps_dmc_mesh_data_substream_codec_id indicates the identifier of the codec used to compress the base mesh data. This codec can be identified through a profile, a component codec mapping SEI message, or another implementation.

[0033] vps_vdmc_mesh_data_facegroup_id_property_present_flag equal to 1 indicates that one of the property types present in the base mesh data stream is face group ID.

[0034] vps_vdmc_mesh_property_count indicates the number of properties (e.g., textures, normals, etc.) in the base mesh sub-bitstream. vps_ext_mesh_property is in the range 0 to 127, inclusive.

[0035] vps_vdmc_mesh_property_type[ j ][ i ] indicates the type of the ith property present in the mesh of the atlas with atlas ID equal to j.

[0036] TIFF2025530443000008.tif91162

[0037] If vps_vmc_ext_displacement_video_present_flag[ j ] is equal to 1, it indicates that at least one of the vertex property video data is of displacement type.

[0038] vps_vmc_ext_vertex_property_video_count[ j ] indicates the number of vertex properties associated with the atlas with atlas ID j. vps_vmc_ext_vertex_property_video_count[ j ] shall be in the range of 0 to 127, inclusive.

[0039] vps_ext_vertex_property_type[j][i] indicates the vertex property type of the vertex property video data unit with index i of the atlas with atlas ID j. The following table shows the list of supported vertex properties and their relationship to vps_ext_vertex_property_type[j][i].

[0040] TIFF2025530443000009.tif70162

[0041] JPEG2025530443000010.jpg69170

[0042] di_displacement_codec_id[j][i] indicates the identifier of the codec used to compress the vertex property video data unit with index i of the atlas with atlas ID j. di_displacement_codec_id[j][i] shall be in the range of 0 to 255, inclusive.

[0043] di_displacement_prevent_video_conversion_flag[j][i] prevents the output of the vertex property video data unit with index i of the atlas with atlas ID j from being converted from the substream decoder. If the flag is true, the output is used as is without the conversion process from Appendix B of [5]. If the flag is true, the size of the displacement video shall be the same as the nominal video size indicated in the bitstream.

[0044] di_displacement_2d_bit_depth_minus1[j][i] + 1 indicates the nominal 2D bit depth to which a vertex property video data unit with index i of an atlas with atlas ID j is converted. di_displacement_2d_bit_depth_minus1[j] shall be in the range from 0 to 31, inclusive.

[0045] di_displacement_MSB_align_flag[j][i] indicates how decoded displacement video samples associated with the vertex property video data unit with index i of the atlas with atlas ID j are converted to samples of the nominal displacement bit depth.

[0046] di_displacement_frame_width[j][i] indicates the frame width of the displacement video of the vertex property video data unit with index i of the atlas whose atlas ID is atlasID.

[0047] di_displacement_frame_height[j][i] indicates the frame height of the displacement video of the vertex property video data unit with index i of the atlas whose atlas ID is atlasID.

[0048] Atlas Sequence Parameter Set Extension JPEG2025530443000011.jpg75170 JPEG2025530443000012.jpg93170

[0049] asps_vmc_ext_atlas_mapping_present_flag equal to 1 specifies that the atlas_map_processing_information() syntax structure is present. asps_vmc_ext_atlas_mapping_present_flag equal to 0 specifies that the atlas_map_processing_information() syntax structure is not present.

[0050] asps_vmc_ext_connectivity_present_flag equal to 1 specifies that the connectivity_processing_information( ) syntax structure is present. asps_vmc_ext_connectivity_present_flag equal to 0 specifies that the connectivity_processing_information( ) syntax structure is not present.

[0051] asps_vmc_ext_displacement_present_flag equal to 1 specifies that the displacement_processing_information( ) syntax structure is present. asps_vmc_ext_displacement_present_flag equal to 0 specifies that the displacement_processing_information( ) syntax structure is not present.

[0052] asps_vcm_ext_log2_displacement_packing_block_size[j][i] specifies the value of the variable DisplacementPackingBlockSize, which is used for the horizontal and vertical placement of the transformed displacement vectors within the vertex property video data unit with index i of the atlas with atlas ID j, as follows: The values ​​of JPEG2025530443000013.jpg10170di_log2_displacement_packing_block_size[ j ][ i ] must be in the range from 0 to 7, inclusive.

[0053] asps_vmc_ext_num_displacement_video indicates the number of displacement videos. If vps_vmc_ext_displacement_video_present_flag is equal to 1, the default value of asps_vmc_ext_num_displacement_video is 1.

[0054] asps_vmc_ext_property_index_to_displacement_index[ i ] denotes a structure that maps a vertex property video data index to a displacement at index i.

[0055] asps_vmc_ext_patch_mapping_method indicates how to map submesh face group IDs to patches.

[0056] If asps_vmc_ext_patch_mapping_method is equal to 0, all triangles in the corresponding submesh are associated with the current patch. In this case, a submesh can have only one patch associated with it.

[0057] If asps_vmc_ext_patch_mapping_method is equal to 1, the association between face group IDs and patch IDs is explicitly signaled in the base mesh data unit.

[0058] If asps_vmc_ext_patch_mapping_method is equal to 2, the association between face group IDs and patch IDs is explicitly signaled in the mesh patch data unit.

[0059] Otherwise, the triangular faces in the corresponding submesh are divided into face groups according to the method indicated by asps_vmc_ext_patch_mapping_method.

[0060] TIFF2025530443000014.tif89169

[0061] asps_vmc_ext_patch_mesh_data_enable_flag, when equal to 1, specifies that mesh information (i.e., number of triangles, number of vertices, 3D bounding box) of patch with index i in frame with index j is indicated in the bitstream for each patch data unit. If asps_vmc_ext_patch_mesh_data_enabled_flag is equal to 0, mesh information is obtained by processing the information in the base mesh data unit.

[0062] JPEG2025530443000015.jpg94170

[0063] The ampi_projection_enabled_flag indicates that the 2D location where the attribute is projected is explicitly signaled in the mesh patch data unit, so the projection ID and orientation index mentioned above can also be used.

[0064] ampi_atlas_mapping_method indicates the type of atlas mapping method used. If the signaled ampi_atlas_mapping_method is equal to 0, it indicates that the 2D (u,v) positions of the orthogonal projections of the 3D vertices are derived at the decoder side and are not signaled in the mesh bitstream.

[0065] TIFF2025530443000016.tif27160

[0066] ampi_ortho_atlas_gutter: Safeguard space between patches during the patch packing process when using Ortho Atlas.

[0067] ampi_ortho_atlas_lod_width_scale: Width scaling factor used for packing patches for all frames in the sequence. If not present, 1.0 should be used.

[0068] ampi_ortho_atlas_height_scale: Height scaling factor used for packing patches for all frames in the sequence. If not present, 1.0 should be used.

[0069] JPEG2025530443000017.jpg68170

[0070] cpi_connectivity_method indicates the type of connectivity method to be used. TIFF2025530443000018.tif27153

[0071] cpi_subdivision_method indicates the subdivision method. TIFF2025530443000019.tif27153

[0072] cpi_subdivision_iteration_count indicates how many times subdivision needs to be applied recursively.

[0073] cpi_tjunction_removing_method indicates a method for removing t-junctions generated by different subdivision methods or by different subdivision iterations of two triangles that share a side.

[0074] JPEG2025530443000020.jpg96170 JPEG2025530443000021.jpg114170

[0075] If dppi_displacement_coordinate_system[ ltpIndex ] is equal to 0, it indicates that the global coordinate system is used. If dppi_displacement_coordinate_system[ ltpIndex ] is equal to 0, it indicates that a transformation to the local coordinate system is used.

[0076] dp4pi_transform_index[lptIndex][i] indicates the transform applied to the displacement video with index i. The transform index can indicate that no transform has been applied. If the transform is "LINEAR_LIFTING", the required parameters are signaled as "vmc_lifting_transform_parameters".

[0077] TIFF2025530443000022.tif38151

[0078] dppi_extension_transform_parameters_present_flag[ lptIndex ][ i ] equal to 1 indicates the presence of the vmc_lifting_transform_parameters() syntax structure for the displacement video with index i. dppi_extension_transform_parameters_present_flag equal to 0 indicates the absence of the vmc_lifting_transform_parameters() syntax structure.

[0079] JPEG2025530443000023.jpg94170

[0080] ltpIndex indicates the system level, whereby level 0 represents the entire sequence, level 1 represents a frame, and level 2 represents a patch.

[0081] Atlas Frame Parameter Set Extension JPEG2025530443000024.jpg55170 JPEG2025530443000025.jpg219152 JPEG2025530443000026.jpg56170

[0082] afps_num_submesh indicates the number of submeshes in the mesh. In the current high-level syntax, it is assumed to be equal to afti_num_tiles_in_atlas_frame_minus1 + 1.

[0083] afps_submesh_id[ i ] indicates the ID associated with the submesh with index 1.

[0084] afps_vmc_ext_single_submesh_in_frame_flag indicates that there is only one submesh in the mesh frame.

[0085] If afps_vmc_ext_overriden_flag is true, the subdivision method, displacement coordinate system, transformation index, transformation parameters, and attribute transformation parameters override those signaled in asps_vmc_extension().

[0086] afps_vmc_ext_single_attribute_tile_in_frame_flag indicates that there is only one tile per attribute signaled in the video stream.

[0087] Note that previously specified attribute tile information can be used in the case of mapped attributes: atlas_frame_tile_information.

[0088] JPEG2025530443000027.jpg69170 JPEG2025530443000028.jpg228170 JPEG2025530443000029.jpg41170

[0089] Patch Data Unit JPEG2025530443000030.jpg182170 JPEG2025530443000031.jpg254170 JPEG2025530443000032.jpg72170

[0090] mdu_mesh_present_flag[ tileID ][ p ], when equal to 1, specifies that mesh-related syntax elements are present in patch p in the current atlas tile whose tile ID is equal to tileID. mdu_mesh_present_flag, when equal to 0, specifies that mesh-related syntax elements are not present.

[0091] mdu_vertex_count_minus1[tileID][p] + 1 specifies the number of vertices in the patch with index p of the current atlas tile whose tile ID is equal to tileID. This value shall be determined by processing the base mesh data unit, if not present.

[0092] mdu_triangle_count_minus1[ tileID ][ p ] + 1 specifies the number of triangles in the patch with index p of the current atlas tile whose tile ID is equal to tileID. This value shall be determined by processing the base mesh data unit, if not present.

[0093] mdu_3d_bounding_box_min[ tileID ][ p ][ i ] specifies the minimum bounding box along the i-th axis of the patch with index p in the current atlas tile whose tile ID is equal to tileID. The value of mdu_3d_bounding_box_min[ tileID ][ p ][ i ] must be between 0 and 2. asps_geometry_3d_bit_depth_minus1 + 1 It shall be in the range -1, inclusive. The number of bits used to represent mdu_3d_bounding_box_min[ tileID ][ p ] is asps_geometry_3d_bit_depth_minus1 + 1. This value shall be determined by processing the base mesh data unit, if not present.

[0094] mdu_3d_bounding_box_max[ tileID ][ p ][ i ] specifies the maximum bounding box value along the i-th axis of the patch with index p in the current atlas tile whose tile ID is equal to tileID. The value of mdu_3d_bounding_box_max[ tileID ][ p ][ i ] must be between 0 and 2. asps_geometry_3d_bit_depth_minus1 + 1It shall be in the range -1, inclusive. The number of bits used to represent mdu_3d_bounding_box_max[ tileID ][ p ] is asps_geometry_3d_bit_depth_minus1 + 1. This value shall be determined by processing the base mesh data unit, if not present.

[0095] mdu_2d_pos_x[ tileID ][ p ] specifies the x coordinate of the upper-left corner of the patch bounding box size of patch p in the current atlas tile whose tile ID is equal to tileID, expressed as a multiple of PatchPackingBlockSize.

[0096] mdu_2d_pos_y[ tileID ][ p ] specifies the y coordinate of the upper-left corner of the patch bounding box size of patch p in the current atlas tile whose tile ID is equal to tileID, expressed as a multiple of PatchPackingBlockSize.

[0097] mdu_2d_size_x_minus1[ tileID ][ p ] + 1 specifies the width value of the patch with index p in the current atlas tile whose tile ID is equal to tileID.

[0098] mdu_2d_size_y_minus1[ tileID ][ p ] + 1 specifies the height value of the patch with index p in the current atlas tile whose tile ID is equal to tileID.

[0099] mdu_projection_id[ tileID ][ p ] specifies the projection mode value and projection surface normal index value of the patch with index p of the current atlas tile whose tile ID is equal to tileID. The value of mdu_projection_id[ tileID ][ p ] must be in the range from 0 to asps_max_number_projections_minus1 (inclusive).

[0100] The number of bits used to represent mdu_projection_id[tileID][p] is Ceil(Log2(asps_max_number_projections_minus1 + 1)).

[0101] mdu_orientation_index[tileID][p] specifies the patch orientation index of the patch with index p in the current atlas tile whose tile ID is equal to tileID and is used to determine the matrices P, R, and O for reconstruction, as shown in Equation (1) and Table 1.

[0102] If mdu_lod_enabled_flag[ tileID ][ p ] is equal to 1, it specifies that LOD parameters are present in the current patch p of the current atlas tile whose tile ID is equal to tileID. If mdu_lod_enabled_flag[ tileID ][ p ] is equal to 0, LOD parameters are not present in the current patch. If mdu_lod_enabled_flag[ tileID ][ p ] is not present, its value shall be inferred to be equal to 0.

[0103] mdu_lod_scale_x_minus1[tileID][p] specifies the LOD scaling factor that should be applied to the local x-coordinate of the point in the patch with index p of the current atlas tile whose tile ID is equal to tileID, before being added to the patch coordinate TilePatch3dOffsetU[tileID][p]. If mdu_lod_scale_x_minus1[tileID][p] is not present, its value shall be inferred to be equal to 0.

[0104] mdu_lod_scale_y_idc[tileID][p] indicates the LOD scaling factor that should be applied to the local y coordinate of the point in the patch with index p of the current atlas tile whose tile ID is equal to tileID, before being added to the patch coordinate TilePatch3dOffsetV[tileID][p]. If mdu_lod_scale_y_idc[tileID][p] is not present, its value shall be inferred to be equal to 0.

[0105] mesh_inter_data_unit(tileID, patchIdx), mesh_merge_data_unit(tileID, patchIdx), and mesh_skip_data_unit(tileID, patchIdx) can be easily defined. However, the use of mesh_raw_data_unit(tileID, patchIdx) should be discussed further in the group. Note that if tileId = submeshID (which has the same independent decoding function) is used, there is no need to send a different ID to link patches to submeshes, and the tileID can be used for that purpose.

[0106] Rebuild To generate the connectivity associated with the orthoatlas projection patches, the following is applied.

[0107] The variables pIdx, oIdx, lodX, and lodY are assigned as follows: JPEG2025530443000033.jpg51170 JPEG2025530443000034.jpg47170

[0108] Variable BB min (x), Bbmin (y), B.B. min (z), BB max (x), BB max (y), B.B. max(z) is assigned as follows: JPEG2025530443000035.jpg53170

[0109] Then, for each sample with coordinates (x,y,z) belonging to the reconstructed vertex position, we first convert (x,y,z) into a local patch coordinate pair (u * ,v * ), the transformation to the 2D u,v coordinate system is performed. JPEG2025530443000036.jpg31170 where, JPEG2025530443000037.jpg22170 JPEG2025530443000038.jpg21170

[0110] The final (u,v) coordinates are then obtained using the following formula: JPEG2025530443000039.jpg31170 where: [Table 1]

[0111] FIG. 4 illustrates a block diagram of an exemplary computing device configured to implement a mesh compression method according to some embodiments. The computing device 400 can be used to acquire, store, compute, process, communicate, and / or display information, such as images and videos, including 3D content. The computing device 400 can implement any of the encoding / decoding aspects. In general, a hardware architecture suitable for implementing the computing device 400 includes a network interface 402, memory 404, a processor 406, I / O device(s) 408, a bus 410, and storage 412. The selection of the processor is not critical as long as a suitable processor with sufficient speed is chosen. The memory 404 can be any conventional computer memory known in the art. The storage 412 can include a hard drive, CD-ROM, CDRW, DVD, DVDRW, high-definition disk / drive, ultra-high-definition drive, flash memory card, or any other storage device. The computing device 400 can include one or more network interfaces 402. An example of a network interface includes a network card connected to an Ethernet or other type of LAN. The I / O device(s) 408 may include one or more of a keyboard, mouse, monitor, screen, printer, modem, touch screen, button interface, and other devices. The mesh compression application(s) 430 used to implement the mesh compression implementation are likely stored in storage 412 and memory 404 and processed as applications are typically processed. The computing device 400 may include more or fewer components than those shown in FIG. 4 . In some embodiments, mesh compression hardware 420 is included. While the computing device 400 of FIG. 4 includes the application 430 and hardware 420 for the mesh compression implementation, the mesh compression method may be implemented in the computing device as hardware, firmware, software, or any combination thereof.For example, in some embodiments, the mesh compression application 430 is programmed into memory and executed using a processor. As another example, in some embodiments, the mesh compression hardware 420 is programmed hardware logic that includes gates that are specifically designed to implement the mesh compression method.

[0112] In some embodiments, mesh compression application(s) 430 include several applications and / or modules. In some embodiments, a module also includes one or more sub-modules. In some embodiments, fewer or additional modules may be included.

[0113] Examples of suitable computing devices include a personal computer, a laptop computer, a computer workstation, a server, a mainframe computer, a handheld computer, a personal digital assistant, a cellular / mobile phone, a smart appliance, a game console, a digital camera, a digital camcorder, a camera phone, a smartphone, a portable music player, a tablet computer, a mobile device, a video player, a video disc writer / player (e.g., a DVD writer / player, a high-definition disc writer / player, an ultra-high-definition disc writer / player), a television, a home entertainment system, an augmented reality device, a virtual reality device, smart jewelry (e.g., a smart watch), a vehicle (e.g., an autonomous vehicle), or any other suitable computing device.

[0114] To utilize the mesh compression method, a device acquires or receives 3D content (e.g., point cloud content). The mesh compression method can be implemented with user assistance or automatically without user involvement.

[0115] In operation, the mesh compression method provides the flexibility to signal projection-based atlas parameterization methods (e.g., "orthoatlas"), including:

[0116] Description and use of V3C syntax structures and extensions to allow derivation of (u,v) coordinates at the decoder side, rather than carrying them in the base mesh sub-bitstream as is currently done.

[0117] Keeping the V3C tile header syntax unchanged with a one-to-one correspondence between a single tile and a single submesh. Compared to the previous implementation, this change keeps the V3C tile header syntax structure unchanged. Signaling of the association between a tile and a submesh is performed as part of the afps extension.

[0118] Use V3C atlas partition information instead of extensions.

[0119] The use of the new patch type is similar to those previously introduced (e.g., I_MESH), with minimal changes to the original V3C patch data type syntax structure.

[0120] Compared to previous approaches, the mesh compression method separates and explicitly distinguishes between the geometry sub-bitstream and the displacement sub-bitstream, but uses the geometry sub-bitstream to carry displacement information. As described herein, displacement is carried in the vertex property sub-bitstream. The introduction of such a separate sub-bitstream will result in various vertex processing operations in the future (e.g., mesh tracking and vertex correspondence). This change also allows the geometry sub-bitstream to be semantically consistent with the current V3C geometry sub-bitstream, providing the potential for enhancements to the surface reconstruction process.

[0121] The mesh compression method introduces a general module for "connectivity" processing, of which the "refinement" method is one instance.

[0122] Instead of using the Draco-specific term "attribute", we use the general term "property" to name all data associated with a mesh / submesh, as in the "ply" file format for example.

[0123] Several "flags" are used to provide options to use / bypass specific V-DMC methods, and also to treat texture attribute map video as V3C attribute component video.

[0124] The mesh compression method allows patch IDs and face group IDs to be signaled in the mesh bitstream as associations between (patchId, facegroupId) pairs, allowing for a more general, non-Draco-specific signaling method.

[0125] Some embodiments of V3C syntax extensions for mesh compression 1. A method programmed into a non-transitory memory of a device, comprising: Implementing an atlas mapping process; performing connectivity processing to split triangles and generate vertex information; performing vertex position processing to adjust the positions of vertices in the vertex information; A method comprising:

[0126] 2. The method of claim 1, wherein the method further comprises the step of implementing a displacement sub-bitstream for including displacement information, the displacement information being utilized by the vertex position processing.

[0127] 3. The method of claim 2, further comprising the step of reconstructing a mesh based on the displaced vertices and connectivity information.

[0128] 4. The atlas mapping process receiving a patch identification and a mapping function parameter; generating (u,v) coordinates based on the patch identification information and the mapping function parameters; 2. The method of claim 1, comprising:

[0129] 5. The method of claim 1, further comprising utilizing one or more flags to bypass one or more V-DMC methods and use texture attribute map video as V3C attribute component video.

[0130] 6. The method according to claim 1, wherein the step of implementing the atlas mapping process is performed before performing the connectivity process and the vertex position process.

[0131] 7. The method according to claim 1, wherein the step of implementing the atlas mapping process is performed after performing the connectivity process and the vertex position process.

[0132] 8. An apparatus comprising: A non-transitory memory for storing an application, said application comprising: Implement atlas mapping process, Perform connectivity processing to split triangles and generate vertex information; performing vertex position processing to adjust the positions of the vertices in the vertex information; a non-transient memory for a processor coupled to the memory and configured to process the application; An apparatus comprising:

[0133] 9. The apparatus of claim 8, wherein the application is further for implementing a displacement sub-bitstream for including displacement information, the displacement information being utilized by the vertex position processing.

[0134] 10. The apparatus of clause 9, wherein the application is further for reconstructing a mesh based on the displaced vertices and connectivity information.

[0135] 11. The atlas mapping process receiving a patch identification and a mapping function parameter; generating (u,v) coordinates based on the patch identification information and the mapping function parameters; 9. The apparatus of claim 8, comprising:

[0136] 12. The device described in clause 8, wherein the application is further configured to utilize one or more flags to bypass one or more V-DMC methods and use texture attribute map video as V3C attribute component video.

[0137] 13. The apparatus of claim 8, wherein implementing the atlas mapping process occurs before performing the connectivity process and the vertex position process.

[0138] 14. The device of claim 8, wherein implementing the atlas mapping process occurs after performing the connectivity process and the vertex position process.

[0139] 15. A system comprising: an encoder configured to encode the 3D mesh to generate patch identification information and mapping function parameters; implementing an atlas mapping process for the patch identification information and the mapping function parameters; Perform connectivity processing to split triangles and generate vertex information; performing vertex position processing to adjust the positions of the vertices in the vertex information; a decoder configured to A system including:

[0140] 16. The system of clause 15, wherein the decoder is further for implementing a displacement sub-bitstream for including displacement information, the displacement information being utilized by the vertex position processing.

[0141] 17. The system of clause 16, wherein the decoder is further for reconstructing a mesh based on the displaced vertices and connectivity information.

[0142] 18. The atlas mapping process receiving a patch identification and a mapping function parameter; generating (u,v) coordinates based on the patch identification information and the mapping function parameters; 16. The system of claim 15, comprising:

[0143] 19. The system of clause 15, wherein the decoder is further configured to utilize one or more flags to bypass one or more V-DMC methods and use texture attribute map video as V3C attribute component video.

[0144] 20. The system of claim 15, wherein implementing the atlas mapping process occurs before performing the connectivity process and the vertex position process.

[0145] 21. The system of claim 15, wherein implementing the atlas mapping process occurs after performing the connectivity process and the vertex position process.

[0146] The present invention has been described with reference to specific embodiments incorporating details to facilitate an understanding of the principles of construction and operation of the invention. Reference herein to specific embodiments and their details is not intended to limit the scope of the claims appended hereto. Those skilled in the art will readily appreciate that various other modifications can be made to the embodiments chosen for illustration without departing from the spirit and scope of the invention as defined by the claims. [Explanation of symbols]

[0147] 100 Base Mesh Sub-Bitstreams 102 Atlas Sub-Bitstream 104 Displacement Sub-Bitstream 106 Attribute Sub-Bitstream 108 Geometry Sub-Bitstream 110 Mesh Codec 112 The first video codec 114 Secondary Video Codec 116 Third Video Codec 120 decoded base meshes 122 Displacement Decoder 130 Atlas Mapping Processing 132 Connectivity Processing 134 Vertex Position Processing 136 objects / meshes 138 Texture Maps 140 Surface Treatment 300 Gray Dots 302 Black Dot 304 mesh 306 Texture Transfer 400 Computer equipment 402 Network Interface 404 Memory 406 processor 408 I / O Devices 410 Bus 412 Storage device 420 Mesh Compression Hardware 430 Mesh Compression Application

Claims

1. 1. A method programmed into a non-transitory memory of a device, comprising: Implementing an atlas mapping process; performing connectivity processing to split the triangles and generate vertex information; performing vertex position processing to adjust the positions of vertices in the vertex information; A method comprising:

2. The method of claim 1 , further comprising the step of implementing a displacement sub-bitstream for containing displacement information, the displacement information being utilized by the vertex position processing.

3. 3. The method of claim 2, further comprising the step of reconstructing a mesh based on the displaced vertices and connectivity information.

4. The atlas mapping process includes: receiving a patch identification and a mapping function parameter; generating (u,v) coordinates based on the patch identification information and the mapping function parameters; 2. The method of claim 1, comprising:

5. 10. The method of claim 1, further comprising utilizing one or more flags to bypass one or more V-DMC methods and use texture attribute map video as V3C attribute component video.

6. The method of claim 1 , wherein the step of implementing the atlas mapping process occurs before performing the connectivity process and the vertex position process.

7. The method of claim 1 , wherein the step of implementing the atlas mapping process occurs after performing the connectivity process and the vertex position process.

8. 1. An apparatus comprising: A non-transitory memory for storing an application, said application comprising: Implement atlas mapping process, Perform connectivity processing to split triangles and generate vertex information; performing vertex position processing to adjust the positions of the vertices in the vertex information; a non-transient memory for a processor coupled to the memory and configured to process the application; 10. An apparatus comprising:

9. 9. The apparatus of claim 8, wherein the application is further for implementing a displacement sub-bitstream for including displacement information, the displacement information being utilized by the vertex position processing.

10. 10. The apparatus of claim 9, wherein the application is further for reconstructing a mesh based on the displaced vertices and connectivity information.

11. The atlas mapping process includes: receiving a patch identification and a mapping function parameter; generating (u,v) coordinates based on the patch identification information and the mapping function parameters; 9. The apparatus of claim 8, comprising:

12. 10. The apparatus of claim 8, wherein the application is further adapted to utilize one or more flags to bypass one or more V-DMC methods and use texture attribute mapped video as V3C attribute component video.

13. The apparatus of claim 8 , wherein implementing the atlas mapping process occurs before performing the connectivity process and the vertex position process.

14. The apparatus of claim 8 , wherein implementing the atlas mapping process occurs after performing the connectivity process and the vertex position process.

15. 1. A system comprising: an encoder configured to encode the 3D mesh to generate patch identification information and mapping function parameters; implementing an atlas mapping process for the patch identification information and the mapping function parameters; Perform connectivity processing to split triangles and generate vertex information; performing vertex position processing to adjust the positions of the vertices in the vertex information; a decoder configured to A system comprising:

16. 16. The system of claim 15, wherein the decoder is further for implementing a displacement sub-bitstream for including displacement information, the displacement information being utilized by the vertex position processing.

17. 17. The system of claim 16, wherein the decoder is further for reconstructing a mesh based on the displaced vertices and connectivity information.

18. The atlas mapping process includes: receiving a patch identification and a mapping function parameter; generating (u,v) coordinates based on the patch identification information and the mapping function parameters; 16. The system of claim 15, comprising:

19. 16. The system of claim 15, wherein the decoder is further for utilizing one or more flags to bypass one or more V-DMC methods and use texture attribute map video as V3C attribute component video.

20. 16. The system of claim 15, wherein implementing the atlas mapping process occurs before performing the connectivity process and the vertex position process.

21. 16. The system of claim 15, wherein implementing the atlas mapping process occurs after performing the connectivity process and the vertex position process.

Images