Adaptive Frame Interpolation for Video Motion Accuracy

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current frame interpolation techniques suffer from inaccuracies in motion estimation due to poor pixel correspondences, leading to low-quality or distorted interpolated frames, especially when dealing with video content captured at lower frame rates.

Innovation Solution

The method involves a computerized apparatus that intelligently selects between Lagrangian and Eulerian interpolation techniques, combining them through average interpolation or weighting based on criteria such as distance or color similarity to generate higher-quality interpolated frames.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If traditional frame interpolation techniques are used, then the process is simple, but the accuracy of motion estimation deteriorates leading to low-quality interpolated frames

Engineering Contradiction:
Improvemotion estimation accuracyVSAvoidinterpolation process complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent applies dynamics by making the interpolation approach adaptive rather than static. The system dynamically selects between Lagrangian and Eulerian interpolation methods based on real-time analysis of pixel motion characteristics, motion magnitude, and scene complexity. This dynamic adaptation allows the system to optimize motion estimation accuracy for different video content types while managing computational complexity through selective application of appropriate methods.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent changes the parameter of interpolation method selection based on analyzed video characteristics. By evaluating motion magnitude, pixel correspondence quality, and scene complexity, the system adjusts which interpolation approach (Lagrangian or Eulerian) is applied to different regions or frames, thereby improving overall motion estimation accuracy while maintaining manageable computational requirements.

Inventive Principle:
Principle #35Parameter changes

2Measurement precision

If Lagrangian interpolation is used, then pixel correspondence accuracy is improved, but it fails when pixel displacement is large

Engineering Contradiction:
Improvepixel correspondence accuracyVSAvoidapplicability to large motion
Core Design Contradiction:
Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent makes the interpolation method dynamic by switching between Lagrangian and Eulerian approaches based on motion magnitude. When pixel displacement is small, Lagrangian interpolation is used for high accuracy. When displacement exceeds thresholds or motion is complex, the system transitions to Eulerian interpolation, which handles large motions more effectively. This dynamic method selection resolves the contradiction between accuracy and adaptability.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent introduces an intermediary analysis step that evaluates pixel motion characteristics before selecting the interpolation method. This intermediary assessment of motion magnitude and scene complexity acts as a mediator to determine whether Lagrangian or Eulerian interpolation should be applied, enabling the system to overcome the limitations of each individual method by selecting the appropriate one for each scenario.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Adaptability or versatility

If Eulerian interpolation is used, then large motion is handled better, but pixel correspondence accuracy deteriorates for small motions

Engineering Contradiction:
Improvehandling of large motionVSAvoidpixel correspondence accuracy
Core Design Contradiction:
Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent applies dynamics by making the interpolation method adaptive to motion characteristics. For small motions where pixel correspondence is critical, the system dynamically selects Lagrangian interpolation to maintain high accuracy. For large motions where displacement exceeds reliable correspondence thresholds, the system transitions to Eulerian interpolation. This dynamic selection resolves the contradiction by matching the method to the motion scenario.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent applies local quality by allowing different interpolation methods to be applied to different regions or frames based on local motion characteristics. Rather than using a single global method, the system analyzes local pixel motion and applies Lagrangian interpolation where accuracy is paramount and Eulerian interpolation where large motion occurs, thereby optimizing performance locally across the entire video sequence.

Inventive Principle:
Principle #3Local quality

4Ease of operation

If a single interpolation method is used, then the process is simple, but quality deteriorates when combining different scene types

Engineering Contradiction:
Improveinterpolation process simplicityVSAvoidinterpolated frame quality
Core Design Contradiction:
Ease of operationVSMeasurement precision

Solution Approach 1:

The patent makes the interpolation system dynamic by automatically selecting between Lagrangian and Eulerian methods based on analyzed video characteristics. Rather than requiring manual intervention or complex multi-step processes, the system dynamically adapts the interpolation approach to match scene complexity and motion characteristics, thereby maintaining both operational simplicity and high quality output across diverse video content.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent creates a universal interpolation system that can handle multiple types of video content (different scene complexities, motion magnitudes, and frame rates) through a single integrated framework. The system evaluates video characteristics and selects the appropriate interpolation method, providing multi-functional capability that maintains quality across diverse scenarios without requiring separate processing pipelines for different content types.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Data Source

PatentUS10057538B2Apparatus and methods for the selection of one or more frame interpolation techniques
Publication Date: 2018.08.21 GOPRO INC
  • US10057538B2 patent drawing
  • US10057538B2 patent drawing
  • US10057538B2 patent drawing

AI summary

Methods and apparatus for the generation of interpolated frames of video data. In one embodiment, the interpolated frames of video data are generated by obtaining two or more frames of video data; performing Lagrangian interpolation on one or more portions of the obtained two or more frames of video data to generate a Lagrangian interpolated image; performing Eulerian interpolation on one or more portions of the obtained two or more frames to generate a Eulerian interpolated image; and when the Lagrangian interpolated image and the Eulerian interpolated image should be combined, computing an average interpolated image using the Lagrangian interpolated image and the Eulerian interpolated image; otherwise, selecting either the Lagrangian interpolated image or the Eulerian interpolated image; and generating an interpolated frame of video data using one or more of the average interpolated image, the Lagrangian interpolated image, or the Eulerian interpolated image.