Immersive Video Encoding View Pruning and Patch Atlas Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Immersive video encoding methods fail to effectively handle patches belonging to foreground objects or occluded and edge regions, leading to suboptimal rendering results.
Innovation Solution
The techniques involve a view optimizer that selects basic and additional views, a pruner that prunes additional views based on comparison with basic views, and a patch packer that generates atlases, along with metadata including visibility lists and view percentages, enabling parallelized pruning and rendering, and improving image quality through patch-based multi-pass rendering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional immersive video encoding methods are used, then encoding complexity is reduced, but rendering quality deteriorates due to inability to handle foreground objects and occluded regions effectively
Solution Approach 1:
The encoding process is segmented into multiple passes, with different handling strategies for different types of patches. Foreground patches, background patches, and occluded patches are identified and processed differently across multiple encoding passes, allowing specialized optimization for each patch type while maintaining overall system manageability
Solution Approach 2:
The encoder performs preliminary classification of patches into foreground, background, and occluded regions before the main encoding process. Visibility lists and depth information are pre-computed to identify which patches require special handling, enabling the multi-pass rendering process to focus computational resources on critical regions
2Manufacturing precision
If multi-pass rendering is implemented, then image quality improves through better handling of patches, but processing time increases
Solution Approach 1:
The rendering process is organized into periodic passes, where each pass handles specific patch types systematically. The first pass processes foreground patches, the second pass handles background patches, and the third pass processes occluded regions. This periodic structure allows the system to achieve high image quality while maintaining predictable processing timelines through systematic organization of computational tasks
Solution Approach 2:
Different rendering quality levels and processing strategies are applied to different regions of the immersive video based on their importance and characteristics. Foreground objects receive higher processing priority and more computational resources, while background regions use optimized but less intensive processing, achieving overall high quality without uniformly high processing costs across the entire scene
3Adaptability or versatility
If all additional views are retained, then rendering flexibility is maintained, but data redundancy increases
Solution Approach 1:
The pruner component extracts and removes redundant additional views based on comparison with basic views. By analyzing visibility lists and determining which additional views provide unique information versus those that are redundant, the system removes unnecessary data while preserving rendering flexibility for views that are actually needed, reducing data redundancy without compromising adaptability
Data Source
AI summary
An example apparatus for encoding immersive video includes a view optimizer to receive a plurality of input views from a source camera and select basic views and additional views from the plurality of input views. The apparatus also includes a view pruner to prune the additional views based on a comparison with the basic views. The apparatus further includes a patch packer to generate atlases based on the pruned additional views and the basic views. The apparatus includes a metadata composer to generate metadata including additional metadata. The apparatus also further includes a bitstream generator to generate a bitstream including the encoded video and the metadata.