Volumetric Video Encoding via 2D Projection Planes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current volumetric video coding technologies face inefficiencies in compressing dynamic 3D scenes due to poor temporal compression performance, especially when dealing with rigid objects in motion, as they struggle to identify and encode motion-compensated residuals in 3D space effectively.
Innovation Solution
The method involves projecting 3D volumetric video objects onto 2D planes, identifying rigid elements, and arranging them in a bounding volume hierarchy for encoding, using standard 2D video coding technologies to compress temporal adjacent projections and encode 3D motion, thereby improving coding efficiency and enabling 6DOF capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If standard volumetric video coding methods are used, then 3D scene representation is achieved, but temporal compression performance deteriorates due to inability to effectively handle rigid object motion
Solution Approach 1:
The patent segments the 3D volumetric video data into multiple 2D projection planes (front, back, left, right, top, bottom views). This segmentation allows standard 2D video coding tools to be applied to each plane independently, enabling effective temporal compression while maintaining overall 3D scene representation. The segmentation transforms the complex 3D temporal compression problem into manageable 2D sub-problems.
Solution Approach 2:
The patent converts 3D volumetric video data into 2D projection planes, effectively reducing dimensionality from 3D to 2D. This dimensionality change enables the use of成熟的2D video coding standards (like H.264/AVC or H.265/HEVC) that have optimized temporal compression algorithms, thereby improving temporal compression performance while still representing the original 3D scene through multiple viewing angles.
2Manufacturing precision
If 3D volumetric video coding is implemented, then spatial representation quality is improved, but encoding complexity increases due to lack of motion compensation in 3D space
Solution Approach 1:
The patent transforms 3D volumetric video encoding into 2D projection plane encoding, leveraging existing 2D video coding infrastructure. This dimensionality reduction allows the use of well-optimized 2D motion compensation and prediction algorithms, significantly reducing encoding complexity compared to developing novel 3D motion compensation techniques, while still achieving high spatial representation quality through multiple projection views.
Solution Approach 2:
The patent makes the encoding system universal by using standard 2D video coding tools that can handle multiple functions: temporal compression, motion compensation, and prediction. These universal 2D coding tools are applied to each of the six projection planes, eliminating the need for specialized 3D coding algorithms and reducing overall encoding complexity while maintaining spatial quality.
3Productivity
If volumetric video data is compressed without rigid object identification, then encoding process is simpler, but compression performance deteriorates due to inability to exploit temporal redundancy in rigid object motion
Solution Approach 1:
The patent segments the volumetric video data into six distinct 2D projection planes (front, back, left, right, top, bottom). This segmentation enables independent analysis and compression of each view, allowing rigid object motion to be more effectively identified and exploited in each projection. The segmentation makes it feasible to apply motion estimation and compensation techniques to each plane, improving compression performance by exploiting temporal redundancy.
Data Source
Figure 1
Figure 2a~2b
Figure 3a~3b
AI summary
The invention relates to a method for encoding, wherein the method comprises receiving a video bitstream; identifying rigid elements of a three-dimensional volumetric video object from the video bitstream, said rigid elements being represented by individual bounding volumes; arranging the individual bounding volumes in a tree structure of a bounding volume hierarchy;projecting each of the bounding volumes onto two-dimensional planes for encoding;encoding three- dimensional motion of the rigid objects with respect to a parent node in the bounding volume hierarchy;encoding three-dimensional residual between temporal adjacent projections according to two- dimensional coding; encoding the bounding volume hierarchy; and transmitting the video bitstream, encoded three-dimensional motion and bounding volume hierarchy for rendering.