Volumetric Video Encoding with Density-Aware 2D Parametrization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in efficiently encoding and decoding volumetric video content, particularly 6DoF video, due to the large amount of data required for rendering and the issues associated with varying element densities in 3D representations, leading to potential rendering quality issues and user discomfort.
Innovation Solution
The method involves encoding a point cloud using 2D parametrizations, depth maps, and texture maps, along with density information, to optimize data transmission and enhance rendering quality by applying specific processes to areas with low element density.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If volumetric video content (6DoF) is encoded with complete free navigation capability, then user immersion and perception of scene depth are improved, but the amount of data to be transported to the renderer increases significantly
Solution Approach 1:
The point cloud is divided into multiple regions based on depth information, with each region having its own density threshold. This segmentation allows different processing strategies to be applied to different spatial zones, reducing overall data requirements while maintaining navigation freedom in critical areas.
Solution Approach 2:
Different density thresholds are applied to different regions of the point cloud based on their importance for navigation. Regions closer to the center or containing important objects have higher density thresholds (more points), while peripheral regions have lower thresholds, optimizing the balance between navigation capability and data volume.
2Quantity of substance
If 3DoF video is used for immersive display, then data transmission requirements are reduced, but user experience deteriorates due to lack of parallax and head translation freedom
Solution Approach 1:
The system dynamically adjusts the density threshold for point cloud regions based on the user's head position and viewing direction. As the user moves their head, different regions become important and their density thresholds are adjusted accordingly, enabling 6DoF navigation experience with optimized data transmission by only maintaining high density in currently visible regions.
3Manufacturing precision
If point cloud density is increased throughout the entire scene, then rendering quality is improved, but data transmission requirements increase
Solution Approach 1:
Different density thresholds are applied to different regions of the point cloud based on their importance for navigation. Regions closer to the center or containing important objects have higher density thresholds (more points), while peripheral regions have lower thresholds, optimizing the balance between navigation capability and data volume.
Solution Approach 2:
Instead of uniformly high density throughout the entire point cloud, the system applies high density only to specific regions that are currently important for the user's view and navigation, accepting lower density in other regions. This partial application of high quality rendering only where needed reduces overall data requirements.
Data Source
Figure 1~2B
Figure 3~4
Figure 5
AI summary
Encoding/decoding data representative of a 3D representation of a scene according to a range of points of view can involve generating a depth map associated with a part of the 3D representation according to a parameter representative of a two-dimensional parametrization associated with the part and data associated with a point included in the part, wherein the two-dimensional parametrization can be responsive to geometric information associated with the point and to pose information associated with the range of points of view. A texture map associated with the part can be generated according to the parameter and data associated with the point. First information representative of point density of points in a part of the part can be obtained. The depth map, texture map, parameter, and first information can be included in respective syntax elements of a bitstream.