Video Encoder Reference Picture List Optimization

Solution Overview

Problem

Current video encoding standards, such as H.264, face inefficiencies due to the need to transmit multiple reference indices for each macroblock, leading to increased bitrate overhead and complexity, especially when not all reference pictures contribute to compression gain.

Innovation Solution

A method to dynamically select the number of reference pictures for inter coding based on current-picture correlation, motion vectors, residuals, and Rate-Distortion Optimization, allowing for adaptive reduction of reference indices and improved encoding efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If multiple reference pictures are used for motion estimation and compensation, then coding efficiency is improved, but encoder complexity and bitrate overhead increase due to transmitting multiple reference indices for each macroblock

Engineering Contradiction:

Improvecoding efficiencyVSAvoidencoder complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent dynamically changes the parameter of reference picture list size (num_ref_idx) based on scene complexity detection. When low complexity is detected, the reference picture list is reduced to 1 entry, eliminating the need to transmit multiple reference indices. This parameter adaptation resolves the contradiction by adjusting the reference picture usage based on actual encoding needs rather than always using multiple references.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent introduces dynamic adaptation of the reference picture list size based on detected scene characteristics. The encoder transitions from a static multiple-reference approach to a dynamic approach where the number of reference pictures is adjusted per scene or picture basis. This dynamic behavior allows the system to optimize between using multiple references for complex scenes and using single reference for simple scenes, thereby reducing overall complexity.

Inventive Principle:

Principle #15Dynamics

2Productivity

If multiple reference pictures are used for motion estimation and compensation, then coding efficiency is improved, but bitrate overhead increases due to transmitting multiple reference indices for each macroblock

Engineering Contradiction:

Improvecoding efficiencyVSAvoidbitrate overhead

Core Design Contradiction:

ProductivityVSQuantity of substance

Solution Approach 1:

The patent dynamically changes the parameter of reference picture list size (num_ref_idx) based on scene complexity detection. When low complexity is detected, the reference picture list is reduced to 1 entry, eliminating the need to transmit multiple reference indices. This parameter adaptation resolves the contradiction by adjusting the reference picture usage based on actual encoding needs rather than always using multiple references.

Inventive Principle:

Principle #35Parameter changes

3Measurement precision

If multiple reference pictures are examined during motion estimation, then prediction accuracy is improved, but encoder complexity increases

Engineering Contradiction:

Improveprediction accuracyVSAvoidencoder complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent applies different reference picture strategies to different local regions or scenes based on their characteristics. For simple scenes with little motion, only one reference picture is used. For complex scenes with significant motion or multiple moving objects, multiple reference pictures are utilized. This local adaptation resolves the contradiction by matching the reference picture complexity to the actual prediction needs of each region.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent dynamically changes the parameter of reference picture list size (num_ref_idx) based on scene complexity detection. When low complexity is detected, the reference picture list is reduced to 1 entry, eliminating the need to transmit multiple reference indices. This parameter adaptation resolves the contradiction by adjusting the reference picture usage based on actual encoding needs rather than always using multiple references.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS8073048B2Method and apparatus for minimizing number of reference pictures used for inter-coding

Publication Date: 2011.12.06 INTERDIGITAL MADISON PATENT HLDG

AI summary

A method for minimizing and optimizing a list of active reference pictures to be used for (forward or backward) inter coding of a current picture, or of a pixel region (e.g., video object) within the current picture. Setting the number of active reference pictures in a list equal to one based upon one or more decisions related to the pixel data and movement between the current picture and one or more reference pictures. Otherwise, minimizing the number of active reference pictures in the list based upon one or more other decisions related to the pixel data and movement between the current picture and one or more reference pictures. Minimizing the number of active reference pictures in a second reference picture list based upon the utilization of reference pictures in the first reference picture list. A video encoder processes data representing a two-dimensional video image which has been produced by a conventional commercially available video camera. The video encoder is adapted to determine, select, and to minimize the number of active reference pictures to be used to inter code a current picture. The bit stream transmitted by the encoder can be decoded by decoders of the related art.