Digital Frame Transformation Using AI-Guided Optical Flow Warping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional techniques for determining optical flow in digital video editing often result in misalignment errors, leading to blurry visual artifacts due to inaccurate motion estimation and repeated interpolation between pixels.
Innovation Solution
A computing device warps optical flows between digital frames using grid warping and generative AI models to maintain detail and accuracy, decoupling pixel propagation from generation to ensure high-resolution and high-quality video content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional techniques are used to determine optical flow by repeatedly warping digital frames, then motion estimation is performed, but misalignment errors occur and visual artifacts such as blurriness are caused
Solution Approach 1:
The patent segments the optical flow determination process into two distinct stages: (1) determining initial optical flow between adjacent frames, and (2) using generative AI models to refine and propagate pixel values across frames. This segmentation allows each stage to specialize - traditional methods for local motion capture and AI for global consistency - thereby resolving the contradiction between motion estimation capability and frame alignment precision.
Solution Approach 2:
The patent introduces an intermediary generative AI model that acts as a mediator between adjacent frames. Instead of directly warping one frame to another (which causes cumulative errors), the AI model serves as an intermediary that predicts and propagates pixel values based on optical flow guidance, maintaining alignment precision while enabling motion estimation across multiple frames.
2Productivity
If digital frames are warped multiple times using conventional techniques, then pixel values are propagated between frames, but detail is lost and blurriness occurs
Solution Approach 1:
The patent applies preliminary action by using the generative AI model to predict pixel values for future frames before actual warping operations are performed. The AI model pre-computes probable pixel values based on optical flow and previous frame information, so that when warping is necessary, high-fidelity values are already available, preventing detail loss and maintaining image sharpness throughout the propagation process.
3Speed
If conventional optical flow techniques are used, then motion between frames is estimated, but misalignment errors cause visual artifacts
Solution Approach 1:
The patent substitutes the mechanical warping system with an AI-based prediction system. Instead of mechanically warping frames based on optical flow (which accumulates errors), the system uses generative AI models to predict pixel values, replacing the error-prone mechanical transformation with a learned probabilistic model that maintains both speed and reliability.
Data Source
AI summary
Techniques for transforming digital frames using relationship between the digital frames are described. In an example, a computing device can receive a set of digital frames and a set of masks. A computing device can obtain relationships between digital frames of the set of digital frames based on respective displacements of attributes between sequential digital frames. A computing device can obtain one or more pixel values in a portion of at least one digital frame that is define by the mask using corresponding pixel values of other digital frames and the relationships. A computing device can transform (e.g., replace, update) the portion of the at least one digital frame using the one or more pixel values.