360° Video Heterogeneous Quality via Pyramid Level Assignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing solutions for streaming 360° videos are inefficient in bandwidth utilization and latency, particularly for live streaming, as they require high bandwidth to deliver high-quality videos to users who only view a fraction of the content, leading to significant waste and inability to support live 360° video streaming over low-bandwidth networks like 5G.
Innovation Solution
A video generating device that obtains Laplace pyramid levels of 360° video frames, upscales them to recover Gaussian pyramid levels, and assigns these levels based on a bitrate budget and viewport region to compose 360° videos with heterogeneous spatial quality, optimizing bandwidth usage and reducing latency by prioritizing the viewport region.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If entire 360° video is delivered in high quality, then video quality is improved, but bandwidth consumption increases significantly
Solution Approach 1:
The patent applies local quality by delivering different quality levels to different regions of the 360° video. The viewport region (where the user is looking) receives high-quality video with higher bitrate, while non-viewport regions receive lower-quality video with reduced bitrate. This resolves the contradiction by maintaining high video quality where needed while reducing overall bandwidth consumption.
Solution Approach 2:
The patent segments the 360° video into viewport region and non-viewport region, and further segments the video representation into Laplace pyramid levels. By segmenting both spatially (viewport vs non-viewport) and qualitatively (different pyramid levels), the system can optimize bandwidth usage while maintaining perceived video quality.
2Manufacturing precision
If high resolution 12k video is used to display 4k FoV, then video quality is improved, but bandwidth requirement increases to 418 Mbps
Solution Approach 1:
The patent applies partial action by delivering only the necessary portion of video data required for the user's current viewport. Instead of transmitting the entire 12k video at full quality, the system transmits only the viewport region at 4k quality with appropriate Laplace pyramid levels, reducing bandwidth from 418 Mbps to a much lower rate while maintaining sufficient video quality for the user's actual viewing area.
Solution Approach 2:
The patent changes the quality parameter spatially by using heterogeneous spatial quality representation. Different regions of the video are encoded at different quality levels (different Laplace pyramid levels), allowing the system to meet video quality requirements for the viewport while dramatically reducing overall bandwidth requirements through parameter variation across the video sphere.
3Loss of energy
If viewport-adaptive streaming is used, then bandwidth waste is reduced, but live streaming capability is not supported
Solution Approach 1:
The patent applies preliminary action by pre-computing and encoding the Laplace pyramid levels from the original video footage before live streaming. This preprocessing step creates a compact representation that can be rapidly decoded and adapted to different viewport positions and bitrate budgets in real-time, enabling live streaming capability while maintaining bandwidth efficiency.
Solution Approach 2:
The patent implements dynamics by making the video quality adaptive to changing viewport positions and network conditions. The system dynamically selects appropriate Laplace pyramid levels for different regions based on the user's current head orientation and available bandwidth, allowing the same encoded representation to serve multiple quality and resolution requirements during live streaming.
4Loss of energy
If heterogeneous video representation is used, then bandwidth utilization is improved, but encoding complexity increases
Solution Approach 1:
The patent applies copying by creating multiple copies of the video at different Laplace pyramid levels during the encoding phase. Instead of performing complex quality adaptation during decoding, the system pre-creates compressed versions at various quality levels, which can be quickly selected and combined based on viewport and bandwidth requirements, reducing real-time encoding complexity while maintaining bandwidth efficiency.
Data Source
Figure 1~2
Figure 3
Figure 4~5
AI summary
A video generating device obtains Laplace pyramid levels of 360° video frames; upscales each obtained Laplace pyramid level so as to recover a set of corresponding Gaussian pyramid levels for each 360° video frame; determines a recovered Gaussian pyramid level among the set of recovered Gaussian pyramid levels for each pixel in each 360° video frame in dependence on a bitrate budget for live streaming and a viewport region by performing a pyramid level assignment procedure; composes a 360° video with heterogeneous spatial quality based on summing up the determined recovered Gaussian pyramid levels for each pixel in each 360° video frame. Thereby, reduction of bitrate while providing a high video quality to the user when watching 360° videos is possible. Furthermore, the invention also relates to a download proxy comprising such a video generating device, corresponding methods, and a computer program.