Nonlinear Scaling in Hierarchical Motion Estimation

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Traditional video codecs face inefficiencies in motion estimation due to limited search window sizes, particularly in low-energy or flat image content, leading to suboptimal predictors and increased encoding complexity.

Innovation Solution

The implementation of nonlinear image scaling in hierarchical motion estimation (HME) schemes, which allows for dynamic balancing of computing resource usage and motion search speed with motion estimation fidelity by applying scaling factors greater than two, and adaptive source block sizes based on image characteristics.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If larger search windows are employed in motion estimation, then coding gain is improved, but encoding complexity increases

Engineering Contradiction:
Improvecoding gainVSAvoidencoding complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent applies segmentation by dividing the motion estimation process into multiple hierarchical levels. Instead of performing a single exhaustive search over a large search window, the algorithm segments the search into coarse-to-fine stages: first identifying motion vectors at lower resolution levels with larger effective search windows, then refining these vectors at progressively higher resolutions. This segmentation allows the system to achieve coding gains comparable to large search windows while keeping encoding complexity manageable at each individual stage.

Inventive Principle:
Principle #1Segmentation

2Measurement precision

If larger search windows are used, then motion estimation accuracy is improved, but on-chip memory requirements increase

Engineering Contradiction:
Improvemotion estimation accuracyVSAvoidon-chip memory size
Core Design Contradiction:
Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent implements the nested doll principle by creating a hierarchy of downsampled image layers nested within the full-resolution frame. Each layer is a scaled-down version of the previous layer, forming a nested structure where layer n+1 is embedded within the resolution space of layer n. This nested approach allows motion estimation to effectively access a large search window at coarser levels while using progressively smaller windows at finer levels, thereby achieving high motion estimation accuracy without requiring large on-chip memory buffers for all resolution levels simultaneously.

Inventive Principle:
Principle #7Nested doll (Nesting)

3Ease of manufacture

If fixed shape and size source blocks are used in HME, then processing is simplified, but predictor accuracy deteriorates in low energy content regions

Engineering Contradiction:
Improveprocessing simplicityVSAvoidpredictor accuracy
Core Design Contradiction:
Ease of manufactureVSMeasurement precision

Solution Approach 1:

The patent applies dynamics by making the source block size adaptive rather than fixed. The algorithm dynamically adjusts the source block size based on local image characteristics, specifically the energy content of the region being encoded. In low-energy regions, larger source blocks are used to improve predictor accuracy by capturing more contextual information, while in high-energy regions, smaller blocks are used to maintain processing simplicity and adapt to local variations. This dynamic adaptation resolves the contradiction between processing simplicity and predictor accuracy.

Inventive Principle:
Principle #15Dynamics

4Productivity

If downsampling by factors of two is used, then search speed is improved, but motion estimation fidelity decreases

Engineering Contradiction:
Improvesearch speedVSAvoidmotion estimation fidelity
Core Design Contradiction:
ProductivityVSMeasurement precision

Solution Approach 1:

The patent implements parameter changes by varying the downsampling factor across different hierarchical levels and adapting it to local image characteristics. Instead of uniformly downsampling by a factor of two at all levels, the algorithm adjusts the downsampling parameters dynamically. In regions with simple motion patterns, more aggressive downsampling is applied to maximize search speed, while in regions with complex motion or detailed structures, less aggressive downsampling is used to preserve motion estimation fidelity. This parameter adaptation allows the system to optimize the trade-off between search speed and fidelity on a region-by-region basis.

Inventive Principle:
Principle #35Parameter changes

Data Source

PatentUS9532048B2Hierarchical motion estimation employing nonlinear scaling and adaptive source block size
Publication Date: 2016.12.27 INTEL CORP
  • US9532048B2 patent drawing
  • US9532048B2 patent drawing
  • US9532048B2 patent drawing

AI summary

Systems, devices and methods are described including applying nonlinear scaling to a current image frame and a reference image frame to generate at least a corresponding current image layer and a corresponding reference image layer. Hierarchical motion estimation may then be performed using the nonlinearly scaled image layers. Further, source block size may be adaptively determined in a downsampled image layer and hierarchical motion estimation may be performed using the adaptively sized source blocks.