Tile-Based Immersive Video Encoder Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current solutions for streaming immersive 360° videos face challenges in delivering high-quality video over bandwidth-constrained networks, often resulting in reduced video resolution and quality degradation when the user navigates within the scene, as existing methods either provide the full scene or reduce resolution outside the viewport, leading to sudden quality drops.
Innovation Solution
The implementation of an encoder and player system that spatially tiles the video into segments, allowing for seamless switching between tile segments at any time within a segment interval, using complementary frames to ensure continuous decoding and display quality by anticipating user navigation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the full 360° scene is delivered to the end device, then the user can navigate freely within the scene, but the video resolution is decreased to meet bandwidth constraints
Solution Approach 1:
The 360° video is divided into multiple tiles spatially, allowing the system to deliver only the relevant tiles (those containing the viewport and surrounding area) to the end device. This segmentation enables free user navigation within the delivered scene while maintaining high video resolution by avoiding transmission of unnecessary video data.
2Loss of energy
If the resolution of the portion of the 360° scene outside the current viewport is reduced, then bandwidth is conserved, but the displayed video suffers from sudden degradation when the viewport is moved
Solution Approach 1:
The system pre-delivers not only the current viewport tile but also the surrounding region tiles to the end device. This preliminary action ensures that when the user moves the viewport, the new viewport area is already available at full resolution, preventing sudden quality degradation while still conserving bandwidth by limiting delivery to relevant regions.
3Device complexity
If tile switching is restricted to periodic segment boundaries, then the encoding and delivery process is simplified, but switching latency increases when the user navigates within the scene
Solution Approach 1:
The system enables dynamic tile switching at any time within a segment interval, rather than restricting switches to fixed periodic boundaries. When the user moves the viewport, the system can immediately switch to delivering the new viewport tile, significantly reducing switching latency. The encoder is configured to generate complementary frames that enable this flexible, on-demand tile switching while maintaining manageable complexity.
4Manufacturing precision
If the video is delivered at high resolution, then the immersive experience quality is improved, but the throughput requirement increases beyond network constraints
Solution Approach 1:
The system applies different quality levels to different spatial regions of the 360° video. The viewport region and its surrounding area are delivered at high resolution, while other regions are not delivered or delivered at lower resolution. This local quality approach maintains high immersive experience quality where the user is looking while reducing overall throughput requirements.
Data Source
AI summary
During a streaming session of tile based immersive video, upon request to switch from at a current tile segment associated with a current tile to a new tile segment associated with a new tile at a reference time, an encoder (300) is providing to generate one or more complementary frames of the new tile segment allowing a decoding, by a requesting player, of the new tile segment from the reference time.