Frame Interpolation Using Single-Directional Motion Scaling

Solution Overview

Problem

Conventional frame interpolation methods fail when only single-directional motion is available, leading to incomplete frame interpolation operations due to tradeoffs in cost, computation, and power saving.

Innovation Solution

A method for frame interpolation that performs motion estimation on input image frames to obtain single-directional motion, scales it according to the time point of the target frame, and generates an interpolated frame through forward and inverse warping operations, blending results to create a smooth video.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If bi-directional motion is obtained for frame interpolation, then interpolation quality is improved, but computation cost and power consumption increase

Engineering Contradiction:

Improveinterpolation qualityVSAvoidpower consumption

Core Design Contradiction:

Manufacturing precisionVSPower

Solution Approach 1:

The patent extracts and utilizes only the essential forward motion information from I0 to I1, discarding the need to compute backward motion from I1 to I0. By taking out only the necessary motion component (forward motion scaled by time ratio), the system achieves acceptable interpolation quality while significantly reducing computation cost and power consumption compared to full bi-directional motion estimation.

Inventive Principle:

Principle #2Taking out (Extraction)

2Manufacturing precision

If bi-directional motion is obtained for frame interpolation, then interpolation quality is improved, but computation cost increases

Engineering Contradiction:

Improveinterpolation qualityVSAvoidcomputation cost

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent extracts and utilizes only the essential forward motion information from I0 to I1, discarding the need to compute backward motion from I1 to I0. By taking out only the necessary motion component (forward motion scaled by time ratio), the system achieves acceptable interpolation quality while significantly reducing computation cost compared to full bi-directional motion estimation.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

Instead of performing backward warping operations on both I0 and I1 as in conventional methods, the patent inverts the approach by performing forward warping on I0 and then applying inverse operations to generate the final frame. This inversion simplifies the computational流程 while maintaining interpolation effectiveness.

Inventive Principle:

Principle #13The other way round (Inversion)

3Use of energy by stationary object

If only single-directional motion is used, then power consumption is reduced, but frame interpolation cannot be successfully performed

Engineering Contradiction:

Improvepower consumptionVSAvoidinterpolation success

Core Design Contradiction:

Use of energy by stationary objectVSReliability

Solution Approach 1:

The patent changes the parameter representation by scaling the forward motion by the time ratio (t1-t0)/(t1-t0) to effectively represent the intermediate frame's motion requirements. This parameter transformation allows single-directional motion to suffice for successful interpolation, eliminating the need for bi-directional motion while ensuring interpolation success through proper temporal scaling.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

Instead of performing backward warping operations on both I0 and I1 as in conventional methods, the patent inverts the approach by performing forward warping on I0 and then applying inverse operations to generate the final frame. This inversion simplifies the computational流程 while maintaining interpolation effectiveness.

Inventive Principle:

Principle #13The other way round (Inversion)

Data Source

PatentUS12010456B2Method for performing frame interpolation based on single-directional motion and associated non-transitory machine-readable medium

Publication Date: 2024.06.11 MEDIATEK INC

AI summary

A frame interpolation method for generating a third image frame interpolated between a first image frame and a second image frame includes: performing motion estimation upon a first input image frame and a second input image frame, to obtain a single-directional motion, wherein the first input image frame is derived from the first image frame, and the second input image frame is derived from the second image frame; scaling the single-directional motion according to a time point of the third image frame, to generate a scaled motion; deriving a forward-warped result from a result of performing a forward warping operation and a first inverse operation upon the scaled motion; performing a second inverse operation upon the forward-warped result, to generate an inversed result; and generating the third image frame according to the first image frame, the second image frame, the forward-warped result, and the inversed result.