Volumetric Video Encoding via 3D-to-2D Patch Projection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for compressing volumetric video data are inefficient, particularly for dynamic 3D scenes, as they often discard information, leading to limited six degrees of freedom (6DOF) capabilities and high bit rate requirements, which is inadequate for advanced applications like AR, VR, and MR.
Innovation Solution
The method involves obtaining patches of a 3D scene, forming an attribute parameter set based on visual properties, and signaling a range for which the attribute parameter set is valid, allowing for efficient encoding and decoding of volumetric video by projecting 3D data onto simple geometric surfaces like spheres, cylinders, or planes, and using conventional video codecs for compression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If volumetric video data is compressed using conventional methods, then bit rate requirements are reduced, but reconstruction quality and 6DOF capabilities deteriorate
Solution Approach 1:
The 3D scene is divided into multiple patches that are processed and encoded separately. Each patch is projected onto 2D planes independently, allowing for more efficient compression while maintaining the overall 3D structure and quality of the volumetric video data
Solution Approach 2:
The patent transforms 3D volumetric video data into 2D projections for compression, then reconstructs the 3D structure during decoding. This dimensionality transformation enables the use of conventional 2D video codecs while preserving 6DOF capabilities and reconstruction quality
2Quantity of substance
If information is discarded during compression, then bit rate requirements are reduced, but 6DOF capabilities and scene coverage deteriorate
Solution Approach 1:
The patent performs preliminary projection of 3D scene patches onto 2D planes before compression. This preprocessing step organizes the data in a way that maximizes compression efficiency while preserving all necessary geometric and visual information for full 6DOF reconstruction
Solution Approach 2:
The patent transforms the representation parameters of volumetric video data from 3D coordinates to 2D projected coordinates with associated depth information. This parameter transformation enables more efficient encoding while maintaining the ability to reconstruct the full 3D scene with 6DOF capabilities
Data Source
AI summary
There are disclosed various methods, apparatuses and computer program products for volumetric video encoding and decoding. In some embodiments of a method for encoding, one or more patches comprising information of a three-dimensional scene are obtained. An attribute parameter set is formed on the basis of visual properties of a surface of an object in the three-dimensional scene. Also an indication that the attribute parameter set relates to visual properties of the surface of the object and an indication on a range for which the attribute parameter set is valid.