Volumetric Video Saliency Streams for Lower Bandwidth Rendering

The patent segments the volumetric video representation into multiple saliency video streams, each encoding a specific spatial region or object. Instead of transmitting complete high-resolution views from all camera positions, only the salient regions are encoded at high quality in separate streams, while non-salient regions use lower quality or are omitted. This segmentation allows selective transmission of important visual information, reducing overall bandwidth while maintaining perceived quality.

The patent applies local quality by encoding different spatial regions with different quality levels based on their importance. Saliency regions (containing important objects or actions) are encoded at high quality in dedicated saliency video streams, while non-salient background regions use lower quality encoding or are excluded from transmission. This differential quality approach maintains high perceived quality for important content while reducing total bandwidth consumption.

The patent extracts and separates the most important visual information into dedicated saliency video streams, isolating critical content from the complete volumetric dataset. By extracting only the essential salient regions that contribute most to viewer experience, the system reduces the volume of data requiring real-time processing and rendering, thereby decreasing processing time and eliminating perceptible time lags while preserving rendering quality for the extracted important content.

The patent applies partial action by processing and rendering only the necessary salient regions rather than complete volumetric data from all camera positions. The system identifies and processes approximately 20% of the total volumetric data (the salient portions) to achieve 80% of the perceived quality, avoiding the excessive processing time required for complete data while maintaining acceptable rendering quality.