Motion Estimation Using Adaptive Sub-Pixel Interpolation Filters

Solution Overview

Problem

Newer video codecs require higher-complexity sub-pixel interpolation schemes for motion estimation, increasing computational burden and encoding time, which is burdensome for real-time applications like video conferencing and live events.

Innovation Solution

Implementing simpler sub-pixel interpolation filters during motion estimation and reducing or eliminating pre-interpolation of sub-pixel values, while using higher-complexity filters for motion compensation, to decrease computational complexity and increase encoding speed.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If higher-complexity sub-pixel interpolation schemes are used for motion estimation, then motion estimation accuracy is improved, but computational burden and encoding time increase significantly

Engineering Contradiction:

Improvemotion estimation accuracyVSAvoidcomputational burden

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent segments the video frame into active regions and inactive regions, applying different interpolation complexities to each. Active regions use higher-complexity sub-pixel interpolation for accurate motion estimation, while inactive regions use simpler interpolation methods. This segmentation resolves the contradiction by localizing high-precision processing only where needed, reducing overall computational burden while maintaining motion estimation accuracy in important areas.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies local quality by using different interpolation schemes in different spatial locations. In active regions where motion occurs, high-precision sub-pixel interpolation is applied. In inactive regions, simpler integer-pixel or reduced-precision interpolation is used. This local differentiation maintains motion estimation accuracy where required while significantly reducing computational complexity in regions where high precision is unnecessary.

Inventive Principle:

Principle #3Local quality

2Measurement precision

If higher-complexity sub-pixel interpolation schemes are used for motion estimation, then motion estimation accuracy is improved, but encoding speed decreases

Engineering Contradiction:

Improvemotion estimation accuracyVSAvoidencoding speed

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The patent segments the video frame into active regions and inactive regions, applying different interpolation complexities to each. Active regions use higher-complexity sub-pixel interpolation for accurate motion estimation, while inactive regions use simpler interpolation methods. This segmentation resolves the contradiction by localizing high-precision processing only where needed, reducing overall computational burden while maintaining motion estimation accuracy in important areas.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies partial action by using high-precision sub-pixel interpolation only partially - specifically in active regions where motion occurs - rather than applying it uniformly across the entire frame. This partial application maintains encoding speed by avoiding unnecessary high-complexity processing in inactive regions while still achieving accurate motion estimation where it matters.

Inventive Principle:

Principle #16Partial or excessive action

3Measurement precision

If pre-interpolation of sub-pixel values is performed, then motion estimation accuracy is improved, but memory usage increases

Engineering Contradiction:

Improvemotion estimation accuracyVSAvoidmemory usage

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent selectively performs pre-interpolation of sub-pixel values only in active regions before motion estimation, rather than pre-interpolating the entire reference frame. This preliminary action is localized to where motion is expected to occur, maintaining motion estimation accuracy in active regions while significantly reducing memory usage compared to full-frame pre-interpolation.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent applies local quality by using different interpolation schemes in different spatial locations. In active regions where motion occurs, high-precision sub-pixel interpolation is applied. In inactive regions, simpler integer-pixel or reduced-precision interpolation is used. This local differentiation maintains motion estimation accuracy where required while significantly reducing computational complexity in regions where high precision is unnecessary.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS10462480B2Computationally efficient motion estimation

Publication Date: 2019.10.29 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

The detailed description presents innovations in performing motion estimation during digital video media encoding. In one example embodiment, motion estimation is performed using a lower-complexity sub-pixel interpolation filter configured to compute sub-pixel values for two or more candidate prediction regions at a sub-pixel offset, the two or more candidate prediction regions being located in one or more reference frames. For a selected one of the candidate prediction regions at the sub-pixel offset, motion compensation is performed using a higher-complexity sub-pixel interpolation filter.