Backward-Compatible HDR Codec With Temporal Scalability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding technologies lack backward-compatible solutions for efficiently encoding and transmitting high dynamic range (HDR) images with both temporal and dynamic range scalability, which is essential for new television standards that require support for higher resolutions and frame rates while being compatible with legacy systems.
Innovation Solution
The proposed solution involves a four-layer architecture that includes a base layer encoder for standard dynamic range (SDR) signals and an enhancement layer encoder for HDR signals, using techniques like down-sampling, prediction, and non-linear quantization to generate a packed image that can be multiplexed into a single bitstream, allowing for backward-compatible decoding and scalability in both temporal and dynamic range.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a single codec is designed to support both HDR and SDR with temporal scalability, then backward compatibility and scalability are improved, but the device complexity and coding difficulty increase
Solution Approach 1:
The video signal is segmented into multiple layers: a base layer containing SDR video data and enhancement layers containing HDR-specific data (transfer characteristics, tone mapping parameters). This segmentation allows legacy decoders to process only the base layer while advanced decoders can utilize enhancement layers, resolving the contradiction between versatility and complexity by enabling selective processing.
Solution Approach 2:
The SDR base layer is nested within the broader HDR codec structure, with HDR enhancement data wrapped around it. The base layer SDR video stream is embedded as a subset of the full HDR stream, allowing progressive decoding where SDR content serves as the foundation and HDR information is added as optional enhancement, thus achieving backward compatibility without requiring complete system redesign.
2Measurement precision
If HDR video data is encoded with high precision to maintain dynamic range quality, then image quality is improved, but the data rate and bandwidth requirements increase
Solution Approach 1:
The encoding system dynamically adjusts precision parameters based on the specific HDR content characteristics and display capabilities. Transfer characteristics (e.g., PQ, HLG) are selected and applied to optimize the balance between precision and data rate, allowing high dynamic range representation while adapting the bit depth and quantization parameters to minimize unnecessary data transmission.
Solution Approach 2:
The HDR-specific transfer characteristics and tone mapping parameters are extracted and encoded separately from the base video data in enhancement layers. This extraction allows the base SDR layer to maintain standard data rates while HDR precision is preserved in the separated enhancement stream, enabling quality preservation with reduced overall bandwidth requirements through selective decoding.
3Productivity
If a four-layer architecture is implemented to support both temporal and dynamic range scalability, then coding efficiency and flexibility are improved, but the decoding complexity and processing requirements increase
Solution Approach 1:
The four-layer architecture implements dynamic scalability where the decoder can adaptively select which layers to process based on available bandwidth, processing power, and display capabilities. The base layer provides essential SDR content at lower complexity, while enhancement layers (temporal, dynamic range, quality) are progressively added as computational resources permit, making the system dynamically adjustable rather than statically complex.
Solution Approach 2:
The base layer SDR video stream is prepared and encoded in advance with standard compression, creating a ready-to-decode foundation that works independently. Enhancement layers are pre-encoded with references to base layer data structures, allowing decoders to begin processing with minimal complexity and progressively utilize pre-prepared enhancement data as resources become available, reducing initial decoding burden.
Data Source
AI summary
A processor for video coding receives a full-frame rate (FFR) HDR video signal and a corresponding FFR SDR video signal. An encoder generates a scalable bitstream that allows decoders to generate half-frame-rate (HFR) SDR, FFR SDR, HFR HDR, or FFR HDR signals. Given odd and even frames of the input FFR SDR signal, the scalable bitstream combines a base layer of coded even SDR frames with an enhancement layer of coded packed frames, where each packed frame includes a downscaled odd SDR frame, a downscaled even HDR residual frame, and a downscaled odd HDR residual frame. In an alternative implementation, the scalable bitstream combines four signals layers: a base layer of even SDR frames, an enhancement layer of odd SDR frames, a base layer of even HDR residual frames and an enhancement layer of odd HDR residual frames. Corresponding decoder architectures are also presented.