Layered Video Encoding for Bandwidth-Adaptive Low-Latency Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Electronic devices face challenges in streaming video data due to varying device bandwidths, leading to decreased image quality and latency as they switch between different video streams based on available bandwidth, especially when using lossy video compression standards.
Innovation Solution
Video data is partitioned into layers and sent as a single stream, with layers being dropped or added based on device bandwidth, using multiple counters for header, luma, and chroma components during encoding to manage bandwidth efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple video streams with different resolutions are sent to accommodate varying device bandwidths, then adaptability to different bandwidths is improved, but latency increases due to stream switching and synchronization requirements
Solution Approach 1:
The video data is segmented into multiple layers within a single stream, where each layer represents a different resolution level. The encoder divides the video into base layer and enhancement layers, allowing the receiver to selectively decode only the necessary layers based on available bandwidth, eliminating the need to switch between multiple complete streams and reducing latency.
Solution Approach 2:
The system dynamically adjusts which layers are decoded and displayed based on real-time bandwidth conditions. The receiver can flexibly switch between decoding only the base layer (low bandwidth), base layer plus first enhancement layer (medium bandwidth), or all layers (high bandwidth), providing adaptive quality adjustment without stream switching overhead.
2Quantity of substance
If lossy video compression is used to reduce data size for streaming, then bandwidth efficiency is improved, but image quality deteriorates due to quantization distortion
Solution Approach 1:
Different quantization parameters are applied to different layers of the video data. The base layer uses coarser quantization for lower quality but smaller size, while enhancement layers use finer quantization to progressively improve quality. This allows the system to optimize the quality-compression ratio for each layer based on its intended bandwidth scenario.
Solution Approach 2:
The video stream is constructed as a composite of multiple layers with different compression characteristics. By combining base layer (heavily compressed) and enhancement layers (less compressed), the system creates a hierarchical structure where the overall quality is determined by the sum of decoded layers, allowing flexible quality adjustment without complete re-encoding.
3Adaptability or versatility
If video data is sent as multiple separate streams for different resolutions, then bandwidth adaptation is improved, but system complexity increases due to multiple encoding and synchronization processes
Solution Approach 1:
Multiple resolution versions are merged into a single bitstream by organizing them as hierarchical layers. Instead of maintaining separate encoding pipelines for each resolution, the system combines all resolution data into one stream with a unified syntax structure, allowing the receiver to selectively process only the necessary portions based on bandwidth, thereby reducing encoder complexity while maintaining adaptability.
Data Source
AI summary
This disclosure is directed to systems and methods of data partitioning in image encoding. A video encoder may receive a single stream of video data that includes multiple layers. The video encoder may encode the one or more layers utilizing multiple counters for each component within the layers. The multiple counters may correspond to the header bits, luma bits, and chroma bits within each slice layer. The encoded layers may then be assembled into a single slice before it is sent to a decoder for decoding of image frame.