Spherical Video Tile Encoding for Bandwidth Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Streaming spherical video consumes significant system resources due to its high bandwidth and processing requirements, particularly in encoding and decoding processes.
Innovation Solution
The method involves determining a tile position in a spherical video frame based on a viewer's perspective, encoding a two-dimensional tile at a higher quality, and mapping the frame to a two-dimensional representation using a projection algorithm, allowing for efficient streaming by prioritizing visible portions of the video.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If the entire spherical video frame is encoded at high quality, then video quality is improved, but bandwidth consumption and processing resources increase significantly
Solution Approach 1:
The patent applies local quality by encoding different portions of the spherical video frame at different quality levels. Specifically, the visible region (fovea) is encoded at high quality while the peripheral regions are encoded at lower quality. This is achieved by dividing the spherical frame into multiple tiles and selectively encoding only the visible tile(s) at high quality, thereby reducing overall bandwidth consumption while maintaining perceptual video quality.
2Manufacturing precision
If the entire spherical video frame is encoded at high quality, then video quality is improved, but processing and memory resources increase
Solution Approach 1:
The patent reduces processing resources by applying local quality encoding only to the visible region of the spherical video. The encoder divides the spherical frame into tiles and identifies which tile corresponds to the visible region based on view direction parameters. Only this visible tile is encoded at high quality with full processing, while other tiles are encoded at lower quality or skipped, significantly reducing the computational load on encoders and decoders.
Solution Approach 2:
The patent segments the spherical video frame into multiple tiles, allowing independent processing of each tile. This segmentation enables the system to focus computational resources on only the visible tile(s) while using minimal resources for non-visible tiles. The view direction parameters (theta and phi) are used to determine which segmented tile corresponds to the visible region, enabling efficient resource allocation.
3Quantity of substance
If only visible portions are encoded at high quality, then bandwidth usage is reduced, but video quality in non-visible areas deteriorates
Solution Approach 1:
The patent accepts the deterioration of video quality in non-visible areas as a necessary trade-off, since these regions are not currently being viewed by the user. The system encodes only the visible tile at high quality based on the provided view direction parameters, while non-visible tiles are encoded at lower quality or omitted entirely. This approach optimizes bandwidth usage without noticeably affecting the user's viewing experience, as they are not observing the low-quality regions.
4Quantity of substance
If the entire spherical video frame is transmitted, then complete video coverage is provided, but transmission time and bandwidth consumption increase
Solution Approach 1:
The patent extracts and transmits only the necessary portion of the spherical video data - specifically, the visible tile corresponding to the user's current view direction. By using the view direction parameters (theta and phi) to identify and extract only the relevant tile data for transmission, the system dramatically reduces the volume of data that needs to be transmitted over the network, thereby reducing transmission time and bandwidth consumption while still providing complete coverage of the visible field of view.
Data Source
AI summary
In a general aspect, a method includes determining a tile position in a frame of a spherical video based on a view perspective, selecting a portion of the frame of the spherical video as a two dimensional tile based on the tile position, encoding the two dimensional tile at a first quality, mapping the frame of the spherical video to a two dimensional representation of the spherical video based on a two dimensional projection algorithm, and encoding the two dimensional representation of the spherical video at a second quality.