Sub-Picture Bitstream Reposition for Low-Latency 360° Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
360° video processing and delivery face challenges due to high video quality and low latency requirements, exacerbated by large video size, which existing video coding standards struggle to address effectively, particularly in devices with varying memory and processing power.
Innovation Solution
Implement methods for video encoding and decoding that signal level information for each sub-picture within a bitstream, allowing for independent encoding and decoding of sub-pictures, and repositioning them based on predefined constraints, using a profile_tier_level() structure to manage video quality and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If video quality and latency requirements are increased for 360° video, then user experience is improved, but video size and processing complexity increase
Solution Approach 1:
The patent divides a 360° video picture into multiple sub-pictures, each independently encoded with its own profile, tier, and level parameters. This segmentation allows different regions to have different quality and complexity characteristics, improving overall video quality while managing processing complexity through localized optimization rather than uniform high-quality encoding across the entire 360° video.
2Reliability
If video quality is improved for 360° video, then user experience is enhanced, but video size increases
Solution Approach 1:
Different sub-pictures within the 360° video can have different quality characteristics through independent profile, tier, and level assignments. Regions that require higher quality for user experience can be encoded with higher bitrates and better compression settings, while other regions use lower quality encoding, thus achieving enhanced user experience without uniformly increasing the entire video size.
3Loss of time
If low latency is achieved for 360° video, then user experience is improved, but processing requirements increase
Solution Approach 1:
By segmenting the 360° video into independent sub-pictures with individual encoding parameters, the system can process and transmit sub-pictures independently. This enables selective decoding and rendering of only the sub-pictures needed for the current viewport, reducing processing power requirements and latency compared to encoding and decoding the entire 360° video at full quality.
4Adaptability or versatility
If device capabilities are accommodated with varying memory and processing power, then adaptability is improved, but system complexity increases
Solution Approach 1:
The patent employs parameter changes by assigning different profile, tier, and level parameters to different sub-pictures. This allows the system to adapt to varying device capabilities by selecting appropriate parameter combinations for each sub-picture, enabling graceful degradation or enhancement based on device memory and processing power without requiring a completely different system architecture.
Data Source
AI summary
Systems and methods described herein employ a high-level syntax design that supports a sub-picture extraction and reposition process. An input video may be encoded into multiple representations, each representation may be represented as a layer. A layer picture may be partitioned into multiple sub-pictures. Each sub-picture may have its own tile partitioning, resolution, color format and bit depth. Each sub-picture is encoded independently from other sub-pictures of the same layer, but it may be inter-predicted from the corresponding sub-pictures from its dependent layers. Each sub-picture may refer to a sub-picture parameter set where the sub-picture properties such as resolution and coordinate is signaled. Each sub-picture parameter set may refer to a PPS where the resolution of the entire picture is signaled.