Volumetric Video Projection for 6DOF Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Volumetric video coding technologies face inefficiencies in spatial and temporal compression due to the complexity of 3D scenes, where geometry and attributes change, leading to poor compression performance and limited six degrees-of-freedom viewing capabilities in current 2D-video based approaches.
Innovation Solution
The method involves projecting volumetric video data onto simple geometric surfaces such as spheres, cylinders, or planes, allowing for standard 2D video coding of texture and geometry information, which can be decoded to reconstruct the 3D scene, with relevant projection geometry information transmitted separately or within the bitstream.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If volumetric video data is transmitted at high spatial resolution, then reconstruction quality is improved, but transmission bitrate increases and compression efficiency deteriorates
Solution Approach 1:
The patent transforms 3D volumetric video data into 2D projected images through projection onto planar surfaces. This dimensional reduction allows standard 2D video coding techniques to be applied, achieving efficient compression while maintaining reconstruction quality through inverse projection at the decoder.
Solution Approach 2:
The volumetric scene is divided into multiple 2D projection images, each representing a different viewpoint or region. This segmentation enables independent processing and compression of each projection using efficient 2D coding tools, improving overall compression efficiency while preserving quality.
2Productivity
If 2D-video based approaches are used for compressing volumetric data, then compression efficiency is improved, but six degrees-of-freedom viewing capabilities are limited
Solution Approach 1:
The projection-based approach creates a universal representation that can serve multiple viewing purposes. The same set of projected images can be used for various viewing angles and six degrees-of-freedom navigation, making the system versatile for different application scenarios while maintaining efficient compression.
3Device complexity
If standard 2D video coding is applied to volumetric data, then coding tools and complexity are reduced, but spatial and temporal coding performance deteriorates
Solution Approach 1:
The patent introduces projection images as an intermediary representation between the 3D volumetric data and the 2D video coding system. This intermediary layer enables the use of simple and efficient 2D coding tools while achieving good coding performance through the projection transformation that captures essential 3D information.
Data Source
Figure 1
Figure 2a~2b
Figure 3a~3b
AI summary
Embodiments for volumetric video encoding and decoding relating to one or more three-dimensional objects are disclosed. In encoding, after mapping from 3D space to 2D plane (802) a point in the 2D plane is examined (805) to determine which points of the 3D object are mapped to the same point to obtain a set of candidate points. Candidate points belonging to a same surface can be used to determine a center of mass for the surface (807). A depth value of the centre of mass is mapped to a 2D projection depth plane (808). A colour value for the centre of mass is interpolated from colour values of points of the set of surface points which are nearest neighbours of the center of mass (810), and used as the colour of the surface in the texture plane (812). Corresponding embodiments for decoding are provided.