Volumetric Video Projection Encoding for Bit Rate Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current volumetric video coding technologies suffer from poor spatial and temporal coding performance, particularly in representing dynamic 3D scene models where geometry and attributes change over time, leading to inefficient compression and reconstruction.
Innovation Solution
The method involves projecting volumetric data, such as point clouds, voxel arrays, or mesh models onto simple geometric surfaces like spheres, cylinders, or planes, and then encoding the resulting 2D projections using standard 2D video coding techniques. This includes encoding texture and geometry information, along with projection geometry details, to reconstruct the 3D scene model efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If volumetric video data is represented using traditional formats (triangle meshes, point clouds, voxel arrays), then the data can be stored and processed, but the spatial and temporal coding performance is poor
Solution Approach 1:
The patent projects 3D volumetric data onto 2D projection surfaces (sphere, cylinder, plane, polyhedron), transforming the data representation from three-dimensional to two-dimensional. This dimensionality change enables the use of efficient 2D video coding standards while preserving the ability to reconstruct 3D views, thereby improving spatial and temporal coding performance without significant information loss
Solution Approach 2:
The patent creates 2D projected copies of the 3D volumetric data on projection surfaces. These 2D copies serve as intermediate representations that can be efficiently coded and transmitted, while the original 3D information can be reconstructed at the decoder side, achieving better compression performance
2Productivity
If 3D volumetric data is directly coded without projection, then the complete 3D information is preserved, but the bit rate required is high and coding efficiency is low
Solution Approach 1:
By projecting 3D volumetric data onto 2D surfaces and then onto flat 2D planes, the patent reduces the dimensionality of the data representation. This enables the application of highly efficient 2D video coding standards, significantly improving coding efficiency and reducing the bit rate required to represent the same volumetric content
Solution Approach 2:
The patent divides the 3D volumetric data into multiple 2D projections from different viewpoints and/or time instances. Each projection can be independently coded using efficient 2D codecs, and the segmented projections are then combined at the decoder to reconstruct the full 3D scene, achieving better overall compression
3Adaptability or versatility
If multiple projections are encoded to improve 6DOF viewing capabilities, then viewing quality is enhanced, but the complexity of encoding and decoding increases
Solution Approach 1:
The patent uses a universal projection framework where the same projection machinery can generate multiple 2D projections from a single 3D volumetric representation. These projections serve multiple purposes: they enable 6DOF viewing, provide redundancy for error correction, and allow flexible extraction of views from any viewpoint within the captured volume, reducing overall system complexity
Solution Approach 2:
The patent creates multiple 2D projected copies of the 3D data from different viewpoints. These copies can be independently decoded and combined to reconstruct views from arbitrary positions and orientations, enabling 6DOF viewing without requiring complex real-time 3D reconstruction and re-projection at the decoder
Data Source
Figure 1
Figure 2a~2b
Figure 3a
AI summary
The invention relates to methods, apparatuses, systems and computer program products for coding volumetric video. A first texture picture coded, said first texture picture comprising a first projection of first volumetric texture data of a first source volume of a scene model and a second projection of second volumetric texture data of said first source volume of said scene model, said first projection being from said first source volume to a first projection surface, and said second projection being from said first source volume to a second projection surface, said second volumetric texture data having been obtained by removing at least a part of said first volumetric texture data that has been successfully projected in said first projection. A a first geometry picture is coded,said geometry picture representing a mapping of said first projection surface to said first source volume and a mapping of said second projection surface to said first source volume. Projection geometry information of said first and second projections is coded, said projection geometry information comprising information of position of said first and second projection surfaces in said scene model.