Spherical Video Bitrate Allocation for Low Latency Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Spherical video streaming over high latency and low bandwidth networks is challenging due to high bitrate requirements, leading to delayed and poor user experiences, especially when the region of interest (ROI) needs to be quickly updated to avoid noticeable delays and sickness feelings.
Innovation Solution
A method involving an apparatus with a processor that encodes spherical video streams by selecting a target area for high-quality encoding and additional areas for lower-quality encoding, transmitting only the encoded target and additional areas to a playback device, optimizing bitrate and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If the entire spherical field of view is streamed at high resolution, then the image quality is maintained, but the bandwidth consumption increases significantly
Solution Approach 1:
The patent applies local quality by encoding different regions of the spherical video at different bitrates. The region of interest (ROI) is encoded at high bitrate to maintain image quality, while other regions are encoded at lower bitrates to reduce bandwidth consumption. This is achieved through selective encoding parameters applied to specific spatial regions within the video frame.
Solution Approach 2:
The patent segments the spherical video field of view into multiple regions: the region of interest (ROI), additional areas adjacent to the ROI, and the remaining areas. Each segment is encoded independently with appropriate bitrate allocation, allowing the system to prioritize bandwidth for important regions while reducing transmission load for less critical areas.
2Speed
If the region of interest is updated quickly to track user head movements, then the user experience is improved, but the latency increases due to network constraints
Solution Approach 1:
The patent implements preliminary action by pre-buffering and prioritizing the encoding and transmission of the region of interest. The system anticipates that the ROI will need to be updated frequently based on user head movements, so it prepares and transmits this critical region with higher priority and lower latency requirements, while less critical regions are updated less frequently.
Solution Approach 2:
The patent applies dynamics by making the encoding parameters adaptive based on user behavior. The system dynamically adjusts which regions are encoded at high bitrate versus low bitrate based on detected user head movements and attention patterns. This allows the ROI update speed to be optimized in real-time while managing overall latency constraints.
3Manufacturing precision
If high bitrate encoding is used for the entire spherical video, then the image quality is maintained, but the device complexity increases
Solution Approach 1:
The patent reduces encoding device complexity by segmenting the video into regions with different quality requirements. Instead of encoding the entire spherical video at high bitrate, the system divides the field of view into ROI, additional areas, and remaining areas, applying different encoding complexities to each segment. This significantly reduces the overall computational burden on the encoding device.
Solution Approach 2:
The patent applies local quality by concentrating encoding resources on the region of interest where high image quality is most valuable to the user. Other regions are encoded with lower complexity algorithms and lower bitrates, reducing the overall device complexity while maintaining acceptable image quality in non-critical areas.
Data Source
AI summary
An apparatus comprising an interface and a processor. The interface may be configured to receive a spherical video stream. The processor may be configured to (a) generate an encoded spherical video stream from the spherical video stream, (b) select a target area for one or more frames of the encoded spherical video stream corresponding to a pre-determined region of interest, (c) encode entire fields of view of the one or more frames of the encoded spherical video stream using second parameters, (d) select and encode an additional area next to the target area using third parameters, and (e) transmit the encoded spherical video stream comprising the encoded target area, the encoded additional area, and encoded entire fields of view of the one or more frames to a playback device.