Motion Vector Interpolation for High Frame Rate Conversion

Solution Overview

Problem

Current frame interpolation methods for increasing frame rates, such as from 24 to 120 frames per second, are computationally intensive and time-consuming due to the need for frequent motion vector field calculations between multiple phases, especially at higher frame rates like 240 frames per second.

Innovation Solution

The approach calculates motion vector fields only between the previous and future frames, dividing the image into blocks, estimating motion vectors for each block, and using candidate selection and correction modules to interpolate intermediate frames efficiently, with strategies for occluded and normal regions, reducing the number of calculations and hardware requirements.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If motion estimation is performed for each interpolated phase to achieve high frame rate conversion, then frame interpolation accuracy is improved, but computational complexity and processing time increase significantly

Engineering Contradiction:

Improveframe interpolation accuracyVSAvoidcomputational complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent divides the image into multiple blocks and performs motion estimation separately for each block rather than for the entire image. This segmentation approach reduces the computational complexity of motion estimation while maintaining interpolation accuracy, as each block can be processed independently with fewer calculations required per block.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs motion estimation only for the input frames before interpolation, rather than for each interpolated phase. The motion vectors obtained from input frames are then used to generate motion vectors for intermediate phases through interpolation. This preliminary action significantly reduces computational complexity while maintaining sufficient accuracy for frame rate conversion.

Inventive Principle:

Principle #10Preliminary action

2Measurement precision

If motion estimation is performed for each interpolated phase to achieve high frame rate conversion, then frame interpolation accuracy is improved, but processing speed decreases

Engineering Contradiction:

Improveframe interpolation accuracyVSAvoidprocessing speed

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The patent performs motion estimation only for the input frames before interpolation, rather than for each interpolated phase. The motion vectors obtained from input frames are then used to generate motion vectors for intermediate phases through interpolation. This preliminary action significantly reduces computational complexity while maintaining sufficient accuracy for frame rate conversion.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent uses motion vectors from input frames as templates or references to generate motion vectors for interpolated phases. Instead of performing new motion estimation for each phase, the system copies and interpolates motion information from the original frames, dramatically reducing processing time while maintaining acceptable accuracy.

Inventive Principle:

Principle #26Copying

3Measurement precision

If frequent motion vector field calculations are performed to achieve high frame rate conversion, then interpolation quality is improved, but hardware resource consumption increases

Engineering Contradiction:

Improveinterpolation qualityVSAvoidhardware resource consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent divides the image into multiple blocks and performs motion estimation separately for each block rather than for the entire image. This segmentation approach reduces the computational complexity of motion estimation while maintaining interpolation accuracy, as each block can be processed independently with fewer calculations required per block.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs motion estimation only for the input frames before interpolation, rather than for each interpolated phase. The motion vectors obtained from input frames are then used to generate motion vectors for intermediate phases through interpolation. This preliminary action significantly reduces computational complexity while maintaining sufficient accuracy for frame rate conversion.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS9357161B1Motion vector interpolation for motion compensation

Publication Date: 2016.05.31 PIXELWORKS INC

AI summary

A method of performing motion compensation includes identifying a set of candidate motion vectors for a block in an intermediate frame from a set of motion vectors from a previous and current frames, performing block-level analysis for each candidate motion vector, selecting an interpolation motion vector, and using the interpolation motion vector to interpolate the intermediate frame between the previous and current frames, wherein the number of intermediate frames depends upon a conversion to a faster frame rate. A method of determining background and foreground motion vectors for a block in an interpolated frame includes comparing a first motion vector that points to a previous frame to a second motion vector that points to a next frame to produce a difference, and designating one of the first and second motion vectors as a background motion vector and the other as a foreground motion vector depending upon the difference.