Subpicture Video Coding for Viewport Extraction and Decoder Load
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies struggle to efficiently encode and decode video data streams comprising multiple subpictures, leading to suboptimal rate-distortion tradeoffs and inefficient use of decoder resources, particularly in viewport-dependent streaming scenarios.
Innovation Solution
The proposed solution involves distinguishing between layers encoded in non-subpicture divided and subpicture divided manners, allowing for the extraction of subpicture-specific video data streams that leverage vector-based prediction and inter-layer prediction tools, while adapting scaling windows and decoder capability requirements to ensure efficient decoding and resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire video data stream is decoded, then complete video content is available, but decoder resource consumption increases and rate-distortion efficiency deteriorates
Solution Approach 1:
The video data stream is segmented into multiple subpicture-specific data streams, where each subpicture represents a distinct spatial region. The encoder divides the picture into multiple subpictures that can be independently decoded and transmitted, allowing the decoder to selectively process only the subpictures needed for the current viewport, thereby reducing computational resources while maintaining complete video content availability in the full data stream
Solution Approach 2:
The patent introduces a new dimension of organization by creating multiple layer sets, each containing layers with specific subpicture configurations. This multi-dimensional structure (layers × subpictures × viewport regions) enables flexible extraction of subpicture-specific data streams tailored to different viewport requirements, optimizing the trade-off between content completeness and decoding efficiency
2Measurement precision
If subpicture-specific video data streams are extracted with high precision, then viewport-dependent streaming quality improves, but signaling overhead increases
Solution Approach 1:
The patent reuses existing video coding structures and parameters across multiple layer sets and subpicture configurations. By defining template-based layer sets that can be instantiated multiple times with different subpicture assignments, the system achieves precise subpicture extraction without repeatedly signaling identical configuration details, thereby reducing signaling overhead while maintaining extraction precision
Solution Approach 2:
The layer set structure serves multiple functions simultaneously: it defines subpicture configurations, establishes reference picture relationships, and enables viewport-specific extraction. This multi-functional design allows a single signaling structure to accomplish what would otherwise require multiple separate signaling mechanisms, reducing overall overhead while preserving precise subpicture control
3Adaptability or versatility
If layers are encoded in subpicture divided manner, then extraction flexibility improves, but inter-layer prediction efficiency deteriorates
Solution Approach 1:
The patent introduces an intermediary structure (the layer set with explicit subpicture division flags) that mediates between the base layer and enhancement layers. This intermediary allows the base layer to be encoded in subpicture-divided manner for extraction flexibility, while the enhancement layers can reference complete base layer pictures through the layer set structure, thereby maintaining inter-layer prediction efficiency despite subpicture division in lower layers
Data Source
AI summary
Video coding concepts are described which relate to encoding, decoding, extracting and mixing video data streams having encoded therein pictures in a manner subdivided into independently coded subpictures. The concepts relate to an extraction of subpicture specific video data streams having a layer of subdivided pictures and a layer of un-subdivided pictures, a handling of inter-layer prediction tools and a handling of scaling windows for inter-layer prediction for such video data streams, a determination of decoder capability requirements for such data streams, layer-specific constraints for such data streams, and mixing of subpictures encoded with different types of independent coding.