Bidirectional Optical Flow Synthesis for Mobile Video Frame Rate
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing frame rate enhancement techniques require significant computation, making them unsuitable for devices with limited calculation power, such as mobile devices, and struggle with accuracy in scenes with large motion scales.
Innovation Solution
A video processing method that determines bidirectional optical flows between adjacent video frames and synthesizes an intermediate video frame using these flows, reducing computation through motion search adjustments and anomaly detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional frame rate enhancement techniques are used to generate intermediate frames, then video smoothness is improved, but computational complexity increases significantly making it unsuitable for mobile devices
Solution Approach 1:
The patent divides the video frame into multiple image blocks and processes each block independently to calculate optical flow. This segmentation approach reduces the overall computational complexity by breaking down the large-scale frame processing into smaller, manageable block-level operations that can be executed more efficiently on mobile devices.
Solution Approach 2:
The patent introduces a multi-resolution image pyramid structure with multiple layers (first image set and second image set with different resolutions). By performing optical flow calculation across different resolution layers and then synthesizing intermediate frames, the method reduces computational burden while maintaining video smoothness, making it suitable for mobile devices with limited processing power.
2Measurement precision
If optical flow calculation is performed for high accuracy in scenes with large motion, then measurement precision is improved, but computation time increases
Solution Approach 1:
The patent performs preliminary optical flow calculations on lower resolution layers of the image pyramid before refining on higher resolution layers. This preliminary action on coarser scales provides initial motion estimates that guide subsequent detailed calculations, reducing the overall computation time while maintaining accuracy for scenes with large motion.
Solution Approach 2:
The patent uses intermediate resolution layers as mediators between low-resolution and high-resolution optical flow calculations. The intermediate layers provide a compromise resolution that reduces computational complexity while still capturing sufficient motion detail, serving as a bridge that balances accuracy and computation time.
3Reliability
If bidirectional optical flow calculation is performed to improve accuracy in large motion scenes, then robustness is improved, but device power consumption increases
Solution Approach 1:
The patent segments the bidirectional optical flow calculation into block-level operations rather than processing entire frames at once. This segmentation reduces the energy required for each processing step while maintaining robustness through comprehensive block-wise analysis, making bidirectional calculation feasible on power-constrained mobile devices.
Solution Approach 2:
The patent implements bidirectional optical flow calculation across multiple resolution layers of the image pyramid. By distributing the computational workload across different resolution dimensions rather than performing all calculations at full resolution, the method maintains processing robustness while significantly reducing overall power consumption on mobile devices.
Data Source
AI summary
The present disclosure relate to a video processing method, an apparatus, a device and a medium, the method including: determining a first optical flow of a first image block in a first video frame moving to a second video frame, and a second optical flow of a second image block in the second video frame moving to the first video frame, wherein the first video frame and the second video frame are adjacent video frames, and each of the first image block and the second image block is an image areas including a plurality of pixel points; and synthesizing an intermediate video frame according to the first video frame, the second video frame, the first optical flow, and the second optical flow, the intermediate video frame being an estimated video frame to be inserted between the first video frame and the second video frame.