Inter-layer Reference Picture Processing for Video Scalability

Solution Overview

Problem

Existing video compression standards like H.264 lack backward compatibility with newer standards such as HEVC, limiting the adoption of HEVC and hindering efficient coding-standard scalability in multimedia content distribution.

Innovation Solution

The implementation of an inter-layer reference picture processing system that uses a reference processing unit (RPU) to generate inter-layer reference frames and parameters, allowing for coding-standard scalability by enabling encoders and decoders to work across different standards like H.264 and HEVC, and supporting features like spatial, temporal, and SNR scalability.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If H.264 compression standard is used, then backward compatibility is maintained, but coding efficiency and compression capability are limited

Engineering Contradiction:

Improvebackward compatibilityVSAvoidcoding efficiency

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The video stream is segmented into multiple layers: a base layer encoded with H.264 for backward compatibility, and enhancement layers encoded with HEVC for improved compression efficiency. This segmentation allows different parts of the system to operate at different coding standards simultaneously.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The H.264 base layer is nested within the HEVC enhancement layers, forming a scalable video structure where the base layer provides compatibility support while enhancement layers provide advanced compression. The base layer can be decoded independently by legacy devices, while enhanced devices can utilize both layers for superior performance.

Inventive Principle:

Principle #7Nested doll (Nesting)

2Productivity

If HEVC compression standard is used, then coding efficiency is improved, but backward compatibility with existing standards is lost

Engineering Contradiction:

Improvecompression capabilityVSAvoidbackward compatibility

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The video stream is segmented into multiple layers: a base layer encoded with H.264 for backward compatibility, and enhancement layers encoded with HEVC for improved compression efficiency. This segmentation allows different parts of the system to operate at different coding standards simultaneously.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The H.264 base layer acts as an intermediary that bridges legacy H.264 decoders and advanced HEVC decoders. Legacy devices can decode just the base layer, while enhanced devices can decode both base and enhancement layers, providing a smooth transition path for standard adoption.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Productivity

If new coding standards are developed, then compression efficiency is improved, but adoption is hindered by lack of compatibility

Engineering Contradiction:

Improvecompression efficiencyVSAvoidadoption ease

Core Design Contradiction:

ProductivityVSEase of manufacture

Solution Approach 1:

The video stream is segmented into multiple layers: a base layer encoded with H.264 for backward compatibility, and enhancement layers encoded with HEVC for improved compression efficiency. This segmentation allows different parts of the system to operate at different coding standards simultaneously.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The H.264 base layer is prepared in advance as a compatibility foundation, allowing legacy decoders to function immediately without modification. This preliminary compatibility layer removes barriers to adoption by ensuring that existing infrastructure can continue to operate while new technology is introduced.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentEP3748969B1Inter-layer reference picture processing for coding standard scalability

Publication Date: 2024.01.03 DOLBY LABORATORIES LICENSING CORP

Figure 1~3
Figure 2A~2B
Figure 4~5A

AI summary

Video data are coded in a coding-standard layered bit stream. Given a base layer (BL) and one or more enhancement layer (EL) signals, the BL signal is coded into a coded BL stream using a BL encoder which is compliant to a first coding standard. In response to the BL signal and the EL signal, a reference processing unit (RPU) determines RPU processing parameters. In response to the RPU processing parameters and the BL signal, the RPU generates an inter-layer reference signal. Using an EL encoder which is compliant to a second coding standard, the EL signal is coded into a coded EL stream, where the encoding of the EL signal is based at least in part on the inter-layer reference signal. Receivers with an RPU and video decoders compliant to both the first and the second coding standards may decode both the BL and the EL coded streams.