Neural Frame Upsampling Using Learned Alpha Blending

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing frame interpolation methods for computer-generated images are computationally inefficient and power-intensive, particularly on mobile devices, due to the complexity of rendering realistic movements and lighting effects, and often fail to keep pace with advances in display technology.

Innovation Solution

A neural network-based approach that uses reduced-resolution motion vectors and optical flow data, combined with depth information, to predict blending parameters for interpolating frames, reducing computational burden and power consumption while maintaining image quality.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If frame interpolation is performed using traditional methods, then frame rate is improved, but computational complexity and power consumption increase significantly

Engineering Contradiction:

Improveframe rateVSAvoidpower consumption

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The patent replaces traditional mechanical image processing systems with a neural network-based system. The neural network learns to predict interpolated frames from pairs of input frames, substituting complex traditional image processing algorithms with a data-driven approach that achieves superior performance with lower computational cost on mobile devices

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent changes the fundamental parameters of frame interpolation by using learned blending parameters from neural networks instead of traditional fixed or simple interpolation formulas. This allows the system to adapt blending weights based on image content and motion characteristics, achieving high-quality interpolation with reduced computational burden

Inventive Principle:

Principle #35Parameter changes

2Productivity

If frame interpolation is performed using traditional methods, then frame rate is improved, but computational complexity increases

Engineering Contradiction:

Improveframe rateVSAvoidcomputational complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent segments the frame interpolation task into distinct components: extracting motion information from input frames, predicting blending parameters using a neural network, and combining results to generate interpolated frames. This segmentation allows each component to be optimized independently, reducing overall computational complexity

Inventive Principle:

Principle #1Segmentation

3Manufacturing precision

If rendering is performed at higher resolution and frame rate, then image quality is improved, but computational resources and power consumption increase

Engineering Contradiction:

Improveimage qualityVSAvoidpower consumption

Core Design Contradiction:

Manufacturing precisionVSUse of energy by moving object

Solution Approach 1:

The patent performs preliminary action by rendering frames at a lower resolution first, then using neural networks to generate high-quality interpolated frames that appear to be rendered at higher resolution. This preliminary low-resolution rendering reduces computational burden while the neural network compensates to maintain perceived image quality

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent uses copying by creating interpolated frames that replicate the appearance of high-quality rendered frames without actually performing the expensive rendering computation. The neural network learns to copy the characteristics of high-quality rendered images from low-resolution inputs, producing visually similar results with reduced computational cost

Inventive Principle:

Principle #26Copying

Data Source

PatentUS20260030722A1Neural frame rate upsampling via learned alpha

Publication Date: 2026.01.29 ARM LTD

AI summary

A first interpolated optical flow frame is created based at least on a first preceding or following frame and optical flow from the first preceding or following frame. A first interpolated motion vector frame is also created based at least on a second preceding or following frame and motion vectors from the second preceding or following frame. The first interpolated optical flow frame and the first interpolated motion vector frame are provided to a neural network trained to predict blending parameters for blending each of the first interpolated optical flow frame and the first interpolated motion vector frame to generate an interpolated output frame, and predicted blending parameters are generated and output via the neural network. An interpolated output frame is generated by applying the predicted blending parameters to the first interpolated optical flow frame and the first interpolated motion vector frame.