MPI Depth Signaling for Artifact-Reduced Volumetric Video Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing MPI-based volumetric video encoding methods lack efficient signaling of depth quantization parameters, leading to coarse quantization and visual artifacts when rendering 3D scenes from viewpoints other than the center of projection.
Innovation Solution
The method involves generating patch pictures by clustering multiplane image layers based on transparency, packing them into atlases, and encoding metadata that include depth values and quantization laws, allowing for inverse quantization during decoding to enhance rendering quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If depth quantization parameters are not properly signaled in MPI-based volumetric video encoding, then the encoding process remains simple, but rendering quality deteriorates with coarse quantization and visual artifacts
Solution Approach 1:
The depth information is segmented into multiple depth layers, each representing a specific depth range. Each layer is associated with its own quantization parameters, allowing different depth regions to be encoded with appropriate precision. This segmentation enables fine-grained control of quantization precision without requiring uniform high precision across the entire depth range, thus improving rendering quality while managing complexity.
Solution Approach 2:
Different quantization parameters are assigned to different depth layers based on local requirements. The system signals quantization parameters specifically for each depth layer rather than using a single global parameter set. This allows optimized precision for each depth region, improving overall rendering quality while avoiding the complexity of uniformly high-precision encoding throughout.
2Manufacturing precision
If constant depth is used for all patches in MPI encoding, then the encoding process remains simple, but rendering quality deteriorates when viewing from viewpoints other than the center of projection
Solution Approach 1:
The system transitions from static constant depth values to dynamic depth parameters that vary by depth layer. Each depth layer is associated with specific quantization parameters that can be independently signaled and decoded. This dynamic approach allows the system to adapt depth precision to the specific viewing conditions and depth region, improving rendering quality for off-center viewpoints while managing complexity through structured parameter signaling.
3Manufacturing precision
If detailed depth quantization parameters are added to each layer, then rendering quality improves, but the data stream complexity and processing requirements increase
Solution Approach 1:
The depth layer structure serves multiple functions simultaneously: it organizes spatial depth information, provides quantization parameter grouping, and enables efficient rendering for multiple viewpoints. By making the depth layer structure multi-functional, the system achieves detailed depth quantization precision without proportionally increasing complexity, as the same structural elements serve multiple purposes in the encoding and rendering pipeline.
Data Source
AI summary
Methods, apparatus and data stream are described to encode, transmit and decode an atlas-based representation of a 3D scene based on a multiplane image (MPI) representation in which a depth component is encoded in each layer. Layers of the MPI are clustered on a transparency basis to generate texture, transparency and depth patch pictures. Patch pictures are packed in at least one atlas image. Metadata associating each patch to a layer and each layer to a depth and a depth quantization law are encoded in the data stream with the at least one atlas. At the decoding side, the MPI with a depth component is retrieved from the data stream and is used to render a viewport image from a viewpoint in the neighborhood of the center of the MPI.