Volumetric Video Encoding with Adaptive Angular Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for encoding volumetric scenes as multi-plane images (MPI) face challenges such as high data requirements due to large atlases, sensitivity to motion, and issues with alpha spreading, leading to increased pixel rates and inefficient storage.
Innovation Solution
A dynamic mechanism is employed to control the pixel rate of MPI-based atlases by using a perceptually based quantization law and an iterative resolution adjustment, reducing the angular resolution when packing fails to meet target pixel rates, ensuring efficient packing while maintaining perceptual quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If a large atlas size is used to encode multi-plane images, then the completeness of scene representation is improved, but the storage requirement and pixel rate increase significantly
Solution Approach 1:
The patent divides the multi-plane image into multiple patches and packs them into an atlas. By segmenting the scene representation into discrete patches, the system can selectively include only necessary scene parts, reducing overall storage requirements while maintaining representation completeness for the intended viewing frustum.
Solution Approach 2:
The patent applies different quality levels to different regions of the scene by using variable patch sizes and selective inclusion. Important regions within the viewing frustum are represented with higher fidelity, while less critical areas use lower resolution, optimizing the balance between storage and representation quality.
2Manufacturing precision
If high angular resolution is used for patch generation, then the rendering quality is improved, but the atlas size and pixel rate increase
Solution Approach 1:
The patent implements local quality control by generating patches at different angular resolutions based on their importance and visibility. High-resolution patches are generated only for critical regions within the viewing frustum, while peripheral or less important regions use lower angular resolution, thereby reducing overall atlas size while maintaining rendering quality where it matters most.
Solution Approach 2:
The patent applies partial action by generating high-resolution patches only for the necessary portion of the scene that will be visible in the target viewing frustum, rather than uniformly high resolution across the entire multi-plane image. This selective approach reduces computational and storage overhead while maintaining quality for the intended application.
3Loss of information
If all patches are included in the atlas, then the scene coverage is improved, but the packing efficiency and storage utilization deteriorate
Solution Approach 1:
The patent extracts and includes only the necessary patches for the intended viewing frustum, excluding redundant or invisible scene portions. By taking out only the essential patches required for the target application, the system achieves efficient packing while maintaining adequate scene coverage for the intended use case.
Solution Approach 2:
The patent applies partial action by including only the subset of patches necessary for the target viewing frustum rather than all possible patches. This selective inclusion improves packing efficiency and storage utilization while maintaining sufficient scene coverage for the intended application scope.
Data Source
AI summary
A method and a device for generating a data stream representative of a volumetric video generated from a sequence of multi-plane images are disclosed. A quantization law depending on an angular resolution used to generate patches from the multi-plane images is set up. For each intra-periods of the frame sequence, patches are generated for a given angular resolution. If generated patches do not fit in atlas images of a target size, the angular resolution is decreased and patches are generated again according to this new angular resolution.