Reference Picture Resampling for Video Coding

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Traditional video coding schemes face inefficiencies in predicting images with reference pictures of different resolutions, as existing methods do not adequately account for resolution discrepancies, leading to suboptimal prediction quality and increased complexity.

Innovation Solution

The proposed solution involves adaptive resampling and weighting of reference pictures based on their resolutions, allowing for flexible selection of resampling filters and prediction weights, which are signaled or derived based on the current and reference picture resolutions, to improve prediction accuracy and efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If traditional video coding schemes are used to predict images with reference pictures of different resolutions, then the coding process is simpler, but the prediction quality deteriorates due to inadequate handling of resolution discrepancies

Engineering Contradiction:
Improveprediction qualityVSAvoidcoding complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent applies preliminary action by performing resampling of reference pictures before the prediction process. The encoder resamples reference pictures to a common resolution prior to generating predictions, and the decoder performs corresponding resampling operations. This advance preparation resolves the contradiction by ensuring prediction quality through proper resolution handling while maintaining relatively simple coding structures.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent employs parameter changes by dynamically adjusting resolution parameters based on the reference pictures being used. The system determines a common resolution from multiple reference pictures and resamples accordingly. This allows the prediction quality to be optimized for different resolution scenarios without requiring completely different coding approaches, thus improving prediction accuracy while controlling complexity.

Inventive Principle:
Principle #35Parameter changes

2Adaptability or versatility

If reference pictures of different resolutions are used for prediction, then the adaptability of the video coding system is improved, but the computational complexity increases due to resolution discrepancies

Engineering Contradiction:
Improveresolution adaptabilityVSAvoidcomputational energy
Core Design Contradiction:
Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The system performs preliminary determination of a common resolution from multiple reference pictures before conducting resampling operations. By establishing the target resolution in advance based on the specific reference pictures being used, the system optimizes the resampling process and reduces unnecessary computational energy expenditure while maintaining high resolution adaptability.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent applies local quality by performing resampling operations selectively based on the specific reference pictures being used for prediction. Rather than universally resampling all reference pictures, the system determines the appropriate common resolution locally for each prediction operation, optimizing computational energy usage while maintaining adaptability to different resolution requirements.

Inventive Principle:
Principle #3Local quality

3Measurement precision

If existing prediction methods are used without considering resolution differences, then the device complexity is reduced, but the prediction accuracy deteriorates

Engineering Contradiction:
Improveprediction accuracyVSAvoidprocessing complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent changes the resolution parameter of reference pictures dynamically based on the prediction requirements. The encoder determines a common resolution from available reference pictures and resamples to this resolution before prediction. The decoder performs corresponding resolution adjustments. This parameter adaptation improves prediction accuracy by ensuring consistent resolution handling while avoiding the need for completely complex alternative processing architectures.

Inventive Principle:
Principle #35Parameter changes

Data Source

PatentUS12058361B2Reference picture re-sampling
Publication Date: 2024.08.06 APPLE INC
  • US12058361B2 patent drawing
  • US12058361B2 patent drawing
  • US12058361B2 patent drawing

AI summary

Predictive coding techniques may include resampling of reference pictures, where various coding parameters are determined based on the resolution(s) or pixel format(s) of the prediction references. In a first aspect, lists of weights for use in weighted prediction are based on the resolution(s) of prediction references. In a second aspect, resampling filter parameters are selected based on the resolutions of prediction references. In a third aspect, deblocking filter parameters are based on the resolution(s) of prediction references.