Volumetric Video Encoding via Patch Priority Projection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding systems face inefficiencies in compressing dynamic 3D scenes due to challenges in identifying correspondences for motion-compensation, leading to inefficient compression and limited six degrees of freedom (6DOF) capabilities, especially when dealing with volumetric video data that describes a 3D scene or object.
Innovation Solution
The method involves projecting three-dimensional scenes onto simple geometric surfaces such as spheres, cylinders, or planes, unfolding these surfaces into 2D planes, and applying standard 2D video coding techniques to encode texture and geometry information, while signaling priority information for patches to optimize decoding and rendering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If volumetric video data is compressed using traditional 3D-space motion-compensation methods, then compression is attempted, but the process becomes inefficient due to difficulty in identifying correspondences when geometry and attributes change between frames
Solution Approach 1:
The patent transforms the volumetric video compression problem from 3D space to 2D projection space. By projecting the 3D scene onto 2D surfaces and applying 2D video coding techniques, the method avoids the complexity of identifying 3D correspondences while maintaining compression efficiency. This dimensional transformation resolves the technical contradiction by simplifying the correspondence identification process.
2Productivity
If 2D-video based approaches are used for compressing volumetric data, then compression efficiency improves, but the full scene coverage and 6DOF capabilities are limited
Solution Approach 1:
The patent segments the volumetric scene into multiple patches that are projected onto 2D surfaces. Each patch can be independently encoded and decoded, allowing the system to maintain high compression efficiency while preserving the ability to reconstruct the full 3D scene for 6DOF viewing. This segmentation approach resolves the contradiction by enabling both efficient compression and full scene coverage.
3Manufacturing precision
If highly detailed volumetric video representations are reconstructed, then scene detail quality improves, but computational resources and system requirements increase significantly
Solution Approach 1:
The patent applies partial action by prioritizing the decoding and rendering of certain patches over others based on their importance or visibility. This allows the system to achieve high scene detail quality for critical regions while reducing computational resources for less important areas, thereby resolving the contradiction between reconstruction detail and power consumption.
4Ease of manufacture
If real-time rendering is achieved with lower hardware performance, then system accessibility improves, but the complexity of managing patch priorities and decoding orders increases
Solution Approach 1:
The patent applies preliminary action by determining and encoding patch priorities at the encoding stage. This pre-established priority information guides the decoding and rendering process, allowing real-time rendering on lower-performance hardware without requiring complex runtime decision-making. The priority management complexity is shifted to the encoding phase, resolving the contradiction between hardware accessibility and device complexity.
Data Source
AI summary
There are disclosed various methods, apparatuses and computer program products for volumetric video encoding and decoding. In some embodiments of a method for encoding, obtaining one or more patches formed from a three-dimensional image information are obtained. The one or more patches represent projection data of at least a part of an object to a projection plane. Priority for at least one of the one or more patches is determined and the one or more patches are projected to a projection plane. Indication of the priority is encoded into or along a bitstream. In some embodiments of a method for decoding, one or more encoded patches formed from a three-dimensional image information are received. Also at least one indication of priority determined for at least one of the one or more patches is received and the patches are reconstructed in the order defined by the at least one indication of priority.