Scalable Video Coding with Residual Layers and Low Decoder Complexity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing scalable video codecs like SVC and SHVC are complex and require significant computational resources, making them difficult to integrate into modern video delivery ecosystems, especially those with decentralized infrastructure and low-resource devices, and they do not efficiently utilize inter-layer information for bandwidth reduction.
Innovation Solution
A low complexity video coding framework that combines a base codec with enhancement layers, using spatial scaling and residual-based encoding tools like Hadamard transforms and zero-motion vector prediction to efficiently encode and decode video data, allowing for flexible and adaptable video delivery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If scalable video codecs like SVC and SHVC are used to provide scalability features, then video quality requirements are satisfied across different resolutions, but computational complexity and resource requirements increase significantly
Solution Approach 1:
The video signal is segmented into a base layer and enhancement layers. The base layer contains essential information for lower resolution decoding, while enhancement layers contain additional details for higher resolution reconstruction. This segmentation allows decoders to process only the base layer for low-resource devices, reducing computational complexity while maintaining adaptability across different quality requirements.
Solution Approach 2:
The patent extracts and separates interlayer prediction information into distinct components that can be selectively applied. By extracting the essential prediction data from enhancement layers and making it available for base layer decoding, the system reduces the computational burden on decoders while preserving scalability features.
2Loss of energy
If interlayer prediction is used to reduce encoded information at upper layers, then bandwidth requirements decrease, but decoder complexity increases due to need to decode lower layers first
Solution Approach 1:
The patent applies preliminary action by pre-processing the base layer and preparing prediction information in advance. The base layer is decoded and its output is used to generate prediction signals for enhancement layers before the actual encoding process. This preliminary preparation reduces the amount of information that needs to be transmitted and simplifies the decoding process by having prediction data ready beforehand.
Solution Approach 2:
The patent introduces an intermediary process where the base layer decoded output serves as a mediator between the source video and enhancement layers. This intermediary base layer reconstruction provides reference signals for interlayer prediction, enabling bandwidth reduction while managing decoder complexity through structured information flow.
3Adaptability or versatility
If multiple parallel codec copies are used for different quality levels, then scalability is achieved, but integration into decentralized infrastructure becomes difficult
Solution Approach 1:
The patent implements universality by designing a codec structure where a single base layer decoder can serve multiple purposes. The base layer decoding process is universal and can be applied regardless of the target quality level. Enhancement layers are added selectively based on available resources, making the system universally applicable across decentralized infrastructure with varying capabilities.
Solution Approach 2:
The patent applies dynamics by making the enhancement layer processing optional and adaptive. The system can dynamically adjust between processing only the base layer for low-resource scenarios or adding enhancement layers for higher-quality requirements. This dynamic behavior facilitates easy integration into decentralized infrastructure where resources vary, as the system adapts to available capabilities rather than requiring fixed complex processing.
Data Source
AI summary
Examples of a low complexity enhancement video coding are described. Encoding and decoding methods are described, as well as corresponding encoders and decoders. The enhancement coding may operate on top of a base layer, which may provide base encoding and decoding. Spatial scaling may be applied across different layers. Only the base layer encodes full video, which may be at a lower resolution. The enhancement coding instead operates on computed sets of residuals. The sets of residuals are computed for a plurality of layers, which may represent different levels of scaling in one or more dimensions. A number of encoding and decoding components or tools are described, which may involve the application of transformations, quantization, entropy encoding and temporal buffering. At an example decoder, an encoded base stream and one or more encoded enhancement streams may be independently decoded and combined to reconstruct an original video.