Multi-layer encoding bitstream multiplexer for video scalability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video codec technologies face limitations in efficiently encoding and decoding video streams across diverse devices with varying capabilities and network bandwidths, particularly with scalable video coding (SVC) due to hardware constraints, increased computational requirements, and inefficiencies in bandwidth utilization.
Innovation Solution
A generic encoding and decoding framework that combines features of simulcast and SVC, using a bitstream multiplexer to create a multi-layer encoding (MLE) bitstream, which interleaves component bitstreams to provide flexible and efficient coding, and a demultiplexer to decode appropriate layers based on device capabilities and network conditions, while avoiding conflicting parameter values and bit shifting operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If scalable video coding (SVC) is used to provide different video quality versions, then adaptability to different device capabilities is improved, but device complexity and computational requirements increase
Solution Approach 1:
The patent segments the video encoding into multiple independent component bitstreams (e.g., base layer and enhancement layers) that can be independently decoded. Each component bitstream contains video data at a specific quality level, allowing receivers to select and decode only the necessary components based on their capabilities, thus reducing computational requirements while maintaining adaptability.
Solution Approach 2:
The system dynamically selects which component bitstreams to transmit and decode based on real-time network conditions and receiver capabilities. The encoder can adjust the composition of component bitstreams adaptively, and the receiver dynamically chooses the appropriate subset of layers to process, optimizing the balance between quality and computational load.
2Loss of energy
If multiple component bitstreams are multiplexed into a single MLE bitstream, then bandwidth utilization is reduced, but bitstream complexity increases
Solution Approach 1:
The patent merges multiple component bitstreams into a single multi-layer encoding (MLE) bitstream by multiplexing them together. This consolidation reduces the total number of separate transmissions required, improving bandwidth utilization. The MLE bitstream contains all component bitstreams in an organized structure that can be efficiently transmitted over the network.
Solution Approach 2:
The patent introduces an intermediary structure (the MLE bitstream container with layer identifiers and syntax elements) that mediates between the multiple component bitstreams and the transmission channel. This intermediary organization allows the system to manage bitstream complexity through standardized formatting and metadata, making the combined bitstream easier to handle despite containing multiple layers.
3Manufacturing precision
If video is encoded with high quality to maintain video quality, then distortion is reduced, but bitrate increases
Solution Approach 1:
The patent applies local quality by providing different quality levels in different layers of the component bitstreams. The base layer provides essential video content at lower quality with lower bitrate, while enhancement layers provide additional quality improvements at incremental bitrate costs. This allows the system to deliver appropriate quality locally to each layer based on specific needs.
Solution Approach 2:
The system dynamically adjusts the quality-bitrate tradeoff by selectively transmitting different combinations of component bitstreams based on available bandwidth and receiver capabilities. When bandwidth is limited, only the base layer is transmitted; when bandwidth is abundant, enhancement layers are added to improve quality, creating a dynamic adaptation to network conditions.
Data Source
AI summary
Innovations described herein provide a generic encoding and decoding framework that includes some features of simulcast and some features of scalable video coding. For example, a bitstream multiplexer multiplexes component bitstreams into a multi-layer encoding (MLE) bitstream that provides temporal scalability, spatial resolution scalability and/or signal to noise ratio scalability. Each of the component bitstreams provides an alternative version of input video, and a given component bitstream can be a non-scalable bitstream or scalable bitstream. The multiplexer follows composition rules for the MLE bitstream and may rewrite values of certain syntax elements of component bitstreams using an approach that avoids bit shifting operations. A corresponding demultiplexer receives an MLE bitstream that includes component bitstreams and demultiplexes at least part of at least one of the component bitstreams from the MLE bitstream, following decomposition rules for the demultiplexing.