Spherical Video Encoding for Low Bandwidth Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Spherical video streaming over high latency and low bandwidth networks is challenging due to high bitrate requirements, leading to delayed and poor user experiences, especially when the region of interest (ROI) needs to be quickly updated to avoid noticeable delays and sickness feelings.
Innovation Solution
An apparatus with a processor that encodes spherical video streams by selecting a target area for the ROI using high quality parameters and the remaining area using lower quality parameters, allowing for dynamic updates and efficient streaming over low bandwidth networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the entire spherical field of view is streamed at high resolution, then the image quality for the region of interest is maintained, but the bandwidth consumption increases significantly
Solution Approach 1:
The patent applies local quality by encoding different regions of the spherical video at different quality levels. Specifically, the region of interest (ROI) is encoded at high resolution while surrounding areas are encoded at lower resolution. This is achieved through spatially adaptive quantization where quantization parameters are adjusted based on the distance from the ROI center, allowing the system to maintain high image quality for the most important area while reducing overall bandwidth consumption.
2Adaptability or versatility
If the region of interest is updated quickly to track user head movements, then the user experience is improved, but the encoding and transmission delay increases
Solution Approach 1:
The patent implements preliminary action by pre-calculating and preparing encoded video frames with potential ROI positions before they are actually needed. The system anticipates user head movements and pre-encodes multiple possible ROI regions, so when the user moves their head, the corresponding high-quality encoded region is already available for immediate transmission, minimizing the perceived delay.
Solution Approach 2:
The patent applies dynamics by making the encoding parameters dynamic rather than static. The quantization parameters and encoding resolution are continuously adjusted based on the current ROI position, which changes as the user moves their head. This dynamic adaptation allows the system to track user movements while maintaining efficient bandwidth usage at each moment.
3Measurement precision
If high bitrate encoding is used for the entire spherical video, then the image quality is maintained, but the streaming performance over low bandwidth networks deteriorates
Solution Approach 1:
The patent applies segmentation by dividing the spherical video into multiple zones with different encoding priorities. The ROI is segmented as a high-priority region encoded at high quality, while surrounding areas are segmented into medium and low priority zones encoded at progressively lower qualities. This segmentation allows the system to allocate bandwidth efficiently, ensuring high image quality for the most important region while maintaining acceptable overall streaming performance.
Data Source
AI summary
An apparatus comprising an interface and a processor. The interface may be configured to receive (i) a spherical video stream and (ii) data from a playback device to determine a region of interest. The processor may be configured to (a) generate an encoded spherical video stream from the spherical video stream, (b) select a target area for one or more upcoming frames of the encoded spherical video stream corresponding to the region of interest, (c) encode the target area using first parameters, (d) encode a remaining area of the encoded spherical video stream outside of the target area using second parameters and (e) present the one or more upcoming frames to the playback device. Encoding using the first parameters may have a different bitrate than using the second parameters.