Volumetric Video Patch Compression with Dynamic Mesh Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current volumetric video compression methods face inefficiencies in compressing dynamic 3D scenes due to poor temporal compression performance and limited 6DOF capabilities, especially when projecting 3D data onto 2D planes for standard video coding, which fails to effectively handle geometry and attribute changes.
Innovation Solution
The method involves generating and signaling 2D patches from parameterized meshes, allowing for projection plane changes, attribute and geometry patch level of detail scaling, and orientation adjustments to support high-resolution attribute patches alongside lower-resolution geometry patches, enabling efficient compression and decompression of volumetric video data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If 3D data is projected onto 2D planes for standard video coding, then compression is enabled, but temporal compression performance deteriorates and 6DOF capabilities are limited
Solution Approach 1:
The patent segments the 3D mesh into multiple patches, each representing a local region of the mesh. This segmentation allows independent processing and encoding of each patch, improving temporal compression by capturing local geometry changes more effectively while maintaining overall 3D structure. The mesh is divided into triangular faces that are grouped into patches, enabling fine-grained control over compression and rendering.
Solution Approach 2:
The patent introduces patch metadata that stores 3D spatial information (center position, normal vector, bounding box) alongside the 2D patch data. This adds dimensional context back to the compressed representation, enabling 6DOF rendering capabilities while maintaining compression efficiency. The patch metadata bridge allows reconstruction of 3D geometry from 2D projections with full six-degree-of-freedom viewing capability.
2Productivity
If mesh is compressed to 2D patch, then compression efficiency improves, but geometry and attribute changes cannot be effectively handled
Solution Approach 1:
The patent implements dynamic patch structures where patch metadata can be updated between frames to reflect geometry changes. The bounding box, center position, and normal vector are recalculated based on current mesh configuration, allowing the compressed representation to adapt to dynamic geometry changes while maintaining compression efficiency. This dynamic updating enables effective handling of animated or deforming meshes.
Solution Approach 2:
The patent applies different levels of detail and compression strategies to different patches based on their local characteristics. Important regions with high attribute variation or complex geometry can be encoded with higher fidelity, while less critical regions use more aggressive compression. This local quality approach allows effective handling of geometry and attribute changes in critical areas while maintaining overall compression efficiency.
3Measurement precision
If high resolution attribute patches are used, then attribute quality improves, but computational complexity increases
Solution Approach 1:
The patent encodes attribute data at a finer resolution than strictly necessary for all patches, but only processes and renders high-resolution attributes for visible or important patches. The patch metadata enables selective processing where high-resolution attributes are decoded only for patches currently in view or containing critical features, reducing overall computational complexity while maintaining attribute quality where needed.
Solution Approach 2:
The patent uses patch metadata parameters (bounding box, normal vector, center position) to dynamically adjust attribute processing based on patch characteristics and viewing conditions. Attributes are processed at high resolution only when patch parameters indicate the patch is visible, important, or contains significant feature variation, otherwise lower resolution processing is used, balancing attribute quality with computational complexity.
Data Source
AI summary
A method includes receiving a parameterized mesh, the mesh defining a shape and attributes of an object in three-dimensional space; compressing the mesh to generate a two-dimensional patch of the mesh; and signaling at least one of: projection plane changes within the patch to support patches wrapping around a three-dimensional model; separation of attribute and geometry patch level of detail scaling to support at least one high resolution attribute patch alongside at least one lower resolution geometry patch, reflecting at least one feature of at least one lower vertex count mesh; or separation of attribute and geometry patch orientation for image packing.