Complexity Scalable Multilayer Video Coding With Reference Processing Unit
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies face challenges in efficiently delivering video content to diverse client devices with varying processing and display capabilities, as they require independent bitstreams for each device, which is inefficient and lacks scalability.
Innovation Solution
A complexity scalable multilayer video coding system that exploits correlations among video layers to encode video into a single bitstream, using a reference processing unit (RPU) to estimate optimal filters or processes for inter-layer prediction, allowing flexible customization and efficient delivery across different clients.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If independent bitstreams are generated for each client device, then video delivery compatibility across diverse devices is improved, but system complexity and processing overhead increase significantly
Solution Approach 1:
The patent segments the video bitstream into multiple layers (base layer and enhancement layers) with different complexity levels. Each layer can be independently decoded according to client device capabilities, allowing diverse devices to receive appropriate video quality without requiring completely separate bitstream generation for each device.
Solution Approach 2:
The scalable video codec provides multi-functionality by encoding video into a universal scalable bitstream structure that can serve multiple client devices with varying capabilities simultaneously. The single bitstream contains embedded information at multiple complexity levels, making it universally applicable across different device types.
2Manufacturing precision
If multiple independent bitstreams are generated for different client capabilities, then video quality optimization for each device is improved, but encoding time and processing resources increase
Solution Approach 1:
The patent merges multiple encoding operations into a single scalable video encoding process. By combining base layer and enhancement layer encoding into one unified scalable bitstream generation process, the system achieves video quality optimization for multiple device types simultaneously without requiring separate encoding passes for each client capability.
Solution Approach 2:
The encoder performs preliminary action by pre-encoding video into a scalable bitstream structure that contains multiple quality levels. This preliminary encoding prepares the video data in advance with embedded scalability information, allowing efficient delivery to various clients without requiring real-time encoding adjustments.
3Productivity
If a single scalable bitstream is used for multiple clients, then encoding efficiency is improved, but the ability to optimize for specific device capabilities deteriorates
Solution Approach 1:
The scalable bitstream structure implements dynamics by allowing flexible adaptation of video quality based on client device capabilities. The single bitstream contains dynamically selectable layers that can be activated or deactivated depending on the decoding capabilities and resource availability of each client device, enabling device-specific optimization without separate encoding.
4Manufacturing precision
If complex processing is applied to achieve high video quality, then reconstruction quality is improved, but computational cost and power consumption increase
Solution Approach 1:
The patent applies local quality by providing different video quality levels in different layers of the scalable bitstream. Base layer provides essential video content at lower quality with minimal computational requirements, while enhancement layers provide additional quality improvements for devices with higher computational capabilities. Each client device can select the appropriate quality level matching its processing power.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
A multi-layer video system has a first layer encoder that encodes a first layer of video information, at least one second layer encoder that encodes at least one second layer of video information, and an encoder side reference processing unit (RPU) that estimates one or more of an optimal filter or an optimal process that applies on a reference picture that is reconstructed from the first video information layer, and processes a current picture of the second video information layer, based on a correlation between the first layer reconstructed reference picture. The correlation relates to a complexity characteristic that scaleably corresponds to the first video information layer reconstructed reference picture and the second video information layer current picture. A scalable video bitstream is outputted, which may be decoded by a compatible decoder. A decoder side RPU and the encoder side RPU function as an RPU pair.