3D Mesh Patch Contour Encoding for Low-Bitrate Volumetric Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mesh compression methods for volumetric video result in large bitrates, distortions, and artifacts, particularly at low rate encoding points, and fail to effectively encode patch borders and dynamic meshes.
Innovation Solution
Segment the dynamic mesh into patches, extract and encode patch contours using near-lossless methods like Draco, and encode patch data independently or differentially with respect to a coarse mesh, preserving patch borders and providing a hierarchical encoding structure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If existing mesh compression methods are used for volumetric video, then encoding can be performed, but the bitrate becomes large and distortions and artifacts occur particularly at low rate encoding points
Solution Approach 1:
The mesh is segmented into multiple patches, where each patch is independently encoded. This segmentation allows for more efficient compression by treating different regions separately, reducing the overall bitrate while maintaining quality. The patch-based approach enables selective encoding of important regions and reduces the propagation of errors across the entire mesh.
Solution Approach 2:
Patch borders are extracted and encoded separately from the interior patch data. This extraction ensures that border regions, which are critical for maintaining mesh continuity and quality, receive dedicated encoding resources. By separating border encoding from interior encoding, the method prioritizes preserving geometric accuracy at patch boundaries while using more compressive schemes for interior regions.
2Loss of information
If existing mesh compression methods are used, then encoding can be performed, but distortions and artifacts occur particularly at low rate encoding points
Solution Approach 1:
A hierarchical encoding structure is introduced with a coarse mesh serving as an intermediary between the original fine mesh and the compressed representation. The coarse mesh captures the overall geometry at lower resolution, while detailed features are encoded as residuals or refinements. This intermediary structure enables progressive transmission where the coarse mesh provides a reliable base that reduces artifacts, and additional detail can be added as bandwidth permits.
3Adaptability or versatility
If existing methods are used, then encoding can be performed, but they fail to effectively encode patch borders and dynamic meshes
Solution Approach 1:
The encoding method is designed to be dynamic and adaptive, allowing the mesh to be re-segmented into patches for each frame based on motion and geometric changes. Patch borders are dynamically adjusted to follow moving features, and the encoding parameters are adapted based on local complexity and motion magnitude. This dynamic approach maintains high border encoding quality even for dynamic meshes by concentrating encoding resources on moving and important regions.
Data Source
AI summary
An apparatus includes one or more processors, and one or more memories storing instructions that, when executed by the one or more processors, cause the apparatus to: segment, for a frame of three-dimensional object data, a mesh of the three-dimensional object data into one or more patches; determine contours of the one or more patches, wherein the contours of the one or more patches comprise one or more vertices having one or more connected neighboring vertices belonging to another patch, or one or more vertices on a border of the mesh; generate a contour mesh of the three-dimensional object data using the contours of the one or more patches, wherein the contour mesh comprises a plurality of the one or more patches; and encode the contour mesh of the three-dimensional object data into a bitstream using a coding method.