Viewport Bitstream Switching with Sub-GOP Anchor Frames
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional viewport-dependent streaming systems suffer from high interactive latency due to the use of large group of pictures (GOP) sizes, which cause delays in viewport switching and session startup, especially in immersive video applications like VR and AR, leading to noticeable delays and reduced user experience.
Innovation Solution
Implementing sub-GOP structures with anchor frames that allow for reduced latency by starting the output bitstream with an I-frame and a selected anchor frame, omitting intermediate frames, thereby reducing the delay in constructing and transmitting new viewport or session content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If large GOP sizes are used in viewport-dependent streaming, then video compression efficiency is improved, but interactive latency increases
Solution Approach 1:
The patent segments a large GOP structure into multiple smaller sub-GOPs, each with its own anchor frame. This segmentation allows the system to maintain the compression efficiency of larger GOPs while enabling faster viewport switching by only needing to decode a subset of sub-GOPs rather than the entire large GOP, thus reducing interactive latency.
Solution Approach 2:
The patent performs preliminary encoding of multiple anchor frames within the GOP structure at different positions. These pre-placed anchor frames serve as potential starting points for viewport switching, so when a viewport change is requested, the system can immediately begin decoding from the nearest anchor frame without waiting for the next I-frame, reducing the time delay.
2Loss of substance
If large GOP sizes are used, then compression ratio is improved, but viewport switching delay increases
Solution Approach 1:
The patent divides the large GOP into multiple sub-GOPs with anchor frames distributed throughout. When viewport switching occurs, the system segments the decoding task by only processing the relevant sub-GOPs containing the new viewport content, rather than decoding the entire large GOP sequence, thus reducing switching delay while maintaining compression efficiency.
Solution Approach 2:
The patent implements partial action by allowing the decoder to stop after decoding only the necessary sub-GOPs required for the new viewport, without needing to decode the entire large GOP. This partial processing approach significantly reduces viewport switching delay while still achieving the compression benefits of large GOP structures.
3Device complexity
If traditional GOP structures are used, then encoding simplicity is maintained, but session startup latency increases
Solution Approach 1:
The patent segments the traditional GOP structure into multiple sub-GOPs with anchor frames, which adds some encoding complexity but enables the decoder to start rendering sooner by utilizing distributed anchor frames. This segmentation allows session startup to begin at the nearest anchor frame rather than waiting for the next I-frame, reducing startup latency.
Solution Approach 2:
The patent performs preliminary placement of multiple anchor frames throughout the GOP structure during encoding. This preliminary action prepares multiple potential starting points in advance, so when session startup or viewport switching occurs, the system can immediately begin decoding from the nearest anchor frame without significant additional delay, balancing encoding complexity with startup performance.
Data Source
AI summary
Methods, systems, mediums, and devices use video processing with low latency bitstream distribution.