Color Frame Interpolation Using Reduced-Resolution Motion and Optical Flow
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing frame interpolation methods for rendered image sequences are computationally inefficient and power-intensive, particularly on mobile devices, due to the complexity of rendering realistic images in real-time applications like games and augmented reality, and existing solutions are not well-suited for low-power devices.
Innovation Solution
Employing reduced-resolution motion vector and optical flow scattering, combined with a neural network to predict blending parameters, to generate interpolated frames efficiently by using blending coefficients for motion vector and optical flow-based frames, reducing computational burden and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If frame interpolation is performed using traditional rendering techniques, then image quality is improved, but computational complexity and power consumption increase significantly
Solution Approach 1:
The patent uses motion vectors and optical flow to create simplified representations of pixel movement between frames, copying pixel data according to these motion models rather than performing full rendering calculations. This allows interpolated frames to be generated by copying and transforming existing pixel data based on motion information, significantly reducing computational complexity while maintaining image quality.
Solution Approach 2:
The patent changes the approach from full rendering to parameter-based interpolation by using motion vectors and optical flow parameters to guide pixel copying and blending. By changing from rendering-based frame generation to parameter-driven pixel manipulation, the system achieves lower computational complexity while preserving visual fidelity.
2Manufacturing precision
If frame interpolation is performed using traditional rendering techniques, then image quality is improved, but power consumption increases significantly
Solution Approach 1:
The patent replaces computationally intensive rendering operations with simpler pixel copying operations guided by motion vectors and optical flow. By copying pixels from reference frames according to motion information rather than recalculating entire frames, the system dramatically reduces power consumption while maintaining acceptable image quality for mobile devices.
Solution Approach 2:
The patent uses lightweight, disposable data structures and processing methods for frame interpolation, avoiding the need for expensive, power-intensive rendering pipelines. The approach uses simple pixel copying and blending operations that consume minimal power, making the solution suitable for mobile devices with limited power budgets.
3Manufacturing precision
If full-resolution frame interpolation is performed, then image quality is improved, but computational burden increases
Solution Approach 1:
The patent segments the frame interpolation process into distinct stages: motion estimation (motion vectors and optical flow), pixel copying based on motion information, and blending operations. By segmenting the process and using efficient algorithms for each stage, the system reduces overall computational burden while maintaining image quality, enabling faster rendering performance.
Solution Approach 2:
The patent uses efficient pixel copying operations instead of full rendering calculations. By copying pixels from reference frames according to motion vectors and optical flow, the system achieves fast interpolation with minimal computational burden, significantly improving rendering speed compared to traditional rendering-based interpolation methods.
Data Source
AI summary
First interpolated optical flow data is based, at least in part, on an optical flow from a preceding frame, an optical flow from a following frame, or a combination thereof, with a reduced resolution. First interpolated motion vector data based, at least in part, on motion vectors from a preceding frame, a following frame, or a combination thereof, with a reduced resolution. A motion vector nearest in depth is determined from among the first interpolated motion vector data, or an optical flow nearest in depth is determined from among the first interpolated optical flow data, or a combination thereof, for each pixel of an interpolated frame, and are used to selectively gather one or more color signal values for at least some pixels in the interpolated frame from the preceding frame or the following frame, or a combination thereof.