Immersive Video Pruning for 6 DoF Parallax

The patent segments the omnidirectional video into multiple overlapping source videos (e.g., cube map faces or equirectangular segments) that can be independently processed and synthesized. This segmentation allows the system to handle 6 DoF motion parallax by selectively combining segments based on user movement, reducing the complexity of processing the entire omnidirectional video at once while maintaining high realism and immersion.

The patent changes the synthesis parameters dynamically based on user movement type. For rotational movement, it uses angular interpolation parameters; for translational movement, it uses depth-based warping parameters. This parameter adaptation allows efficient processing tailored to the specific motion type, improving realism without uniformly increasing processing complexity for all movements.

The patent applies different weights to different source videos based on their relevance to the current viewport and user movement. Source videos closer to the center of rotation or translation receive higher weights, while peripheral videos receive lower weights. This local quality adjustment improves video synthesis quality by emphasizing relevant content without uniformly increasing complexity across all sources.

The patent processes only the necessary portion of source videos with full detail, while using lower-resolution or lightly processed versions of peripheral videos. For example, during rotational movement, it fully processes videos along the rotation axis while using simplified processing for videos at the periphery. This partial action approach maintains high quality where needed while reducing overall synthesis complexity.