Spherical Video Tile Encoding for Viewport Quality Upgrades
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing encoding methods for 360-degree content in extended reality (XR) are inefficient, requiring numerous encodings and high bandwidth, especially for high-resolution headsets, and do not effectively utilize bi-directional tiles, leading to increased computational load and storage needs.
Innovation Solution
The proposed technique encodes spherical media content using a combination of intra-tiles, predictive-tiles, and bidirectional-tiles, leveraging scalable High Efficiency Video Coding (SHVC) to optimize encoding for different resolutions and qualities, allowing efficient delivery and decoding based on user gaze and network conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If all block Intra encoding is used for all qualities and resolutions, then encoding simplicity is maintained, but bandwidth and storage requirements increase significantly
Solution Approach 1:
The patent segments the spherical media content into multiple tiles, each independently encodable. This allows selective encoding of only the tiles needed for the current view, reducing overall data requirements while maintaining encoding simplicity through modular processing
Solution Approach 2:
The patent applies different encoding qualities to different tiles based on their spatial relationship with the user's viewpoint. Tiles closer to the viewpoint receive higher quality encoding while peripheral tiles use lower quality, optimizing the balance between visual quality and data efficiency
2Adaptability or versatility
If phased encoding with multiple qualities and phases is used, then quality adaptation is improved, but the number of encodings required increases exponentially
Solution Approach 1:
The patent divides the encoding task into tile-level operations rather than generating complete encodings for all phases. Each tile can be independently processed and combined, reducing the total number of encodings from exponential to linear complexity
Solution Approach 2:
The patent uses a nested encoding structure where base layer tiles are encoded first, then enhancement layer tiles are added to refine specific regions. This allows progressive quality improvement without requiring separate complete encodings for each quality level
3Device complexity
If bi-directional tiles are not utilized, then encoding structure remains simple, but computational load and processing efficiency decrease
Solution Approach 1:
The patent pre-encodes tiles in a way that enables efficient bidirectional prediction during playback. By preparing reference tiles in advance with appropriate encoding, the system can efficiently generate bi-directional predictions without requiring complex real-time encoding operations
Data Source
AI summary
Systems and methods are described for identifying a plurality of versions of a plurality of frames of a spherical media content item, wherein each version of the plurality of versions is associated with one of a plurality of resolutions and one of a plurality of video qualities. The plurality of versions is encoded to obtain encoding data comprising a group of pictures (GOP) comprising intra-tiles, predictive tiles, bidirectional predictive tiles, and/or residual data. A first frame is provided to a computing device based on a region of interest in a viewport of the computing device. Based on a change in the ROI, a second frame is provided to the computing device, the second frame comprising at least a portion of the residual data, used to enable an upgrade of video quality at the changed ROI.