Immersive Video Atlas Generation with Priority-Based Pruning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current immersive video technologies face challenges in efficiently transmitting and reproducing 6 DoF videos, as most omni-directional videos only support rotational movements, lacking effective methods for transmitting motion parallax with minimal video and metadata.
Innovation Solution
The proposed method involves determining a priority order for pruning source videos, extracting patches, generating atlases, and encoding metadata to support motion parallax, including flags for entire region inclusion, atlas type, and camera coordinate systems, enabling efficient video reproduction and synthesis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all source videos are transmitted to support motion parallax, then viewing quality is improved, but transmission data volume increases
Solution Approach 1:
The source videos are divided into patches that are selectively included in the atlas based on priority order. Only necessary patches are transmitted, segmenting the complete video data into essential and non-essential portions, thereby reducing transmission data volume while maintaining viewing quality for regions of interest
Solution Approach 2:
Different regions of the immersive video are treated with different quality priorities. Regions of interest (such as areas where users are likely to look or move) are preserved with higher fidelity in the atlas, while less important regions are pruned or excluded, achieving local quality optimization that balances overall viewing quality with data reduction
2Measurement precision
If complete source videos are included in the atlas, then motion parallax accuracy is improved, but metadata size increases
Solution Approach 1:
Only the necessary patches from source videos are extracted and included in the atlas, rather than including complete videos. The extraction process identifies and removes redundant or low-priority video data, maintaining motion parallax accuracy for essential regions while significantly reducing the metadata and video data that needs to be transmitted and processed
3Productivity
If priority-based pruning is implemented, then data transmission efficiency is improved, but processing complexity increases
Solution Approach 1:
The priority order of patches is determined and the atlas is constructed in advance during the encoding phase, before transmission. This preliminary organization of data into a prioritized atlas structure enables efficient transmission and decoding without requiring complex real-time processing during playback, as the pruning and prioritization work is completed beforehand
Data Source
AI summary
Disclosed herein is an immersive video processing method. The immersive video processing method includes: determining a priority order of pruning for source videos; extracting patches from the source videos based on the priority order of pruning; generating at least one atlas based on the extracted patches; and encoding metadata. Herein, a first flag indicating whether or not an atlas includes a patch including information on an entire region of a first source video may be encoded into the metadata.