Variable Frame Rate Video Conversion via Mapping Adjustment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for converting variable frame rate (VFR) video to fixed frame rate video either introduce visual artifacts, require high processing power, or fail to maintain audio synchronization, leading to undesirable playback issues such as jerky motion and audio desynchronization.
Innovation Solution
A technique that determines an optimal fixed frame rate for VFR video by computing the average, highest, or lowest frame rate, and uses mapping rules to select source frames for conversion, adjusting the mapping to improve smoothness and maintain audio synchronization, thereby reducing processing load and memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If motion analysis and estimation system is used to synthesize interpolated source frames, then smooth video motion is improved, but processing power and computational complexity increase significantly
Solution Approach 1:
The patent extracts and removes the computationally intensive motion analysis and estimation components from the frame rate conversion process. Instead of synthesizing interpolated frames through complex motion estimation, the system simply maps existing source frames to output frames at the target frame rate, eliminating the need for sophisticated motion analysis while maintaining acceptable video quality and significantly reducing processing requirements.
2Power
If conventional mapping method is used to convert VFR to fixed frame rate, then processing load is reduced, but audio synchronization is lost
Solution Approach 1:
The patent implements a feedback mechanism that continuously monitors the timing relationship between video frames and audio samples during frame rate conversion. By tracking the cumulative time difference between expected and actual frame presentation times, the system dynamically adjusts the mapping of source frames to output frames, ensuring that audio-visual synchronization is maintained throughout the conversion process without requiring complex processing.
3Stability of the object's composition
If high frame rate is selected to preserve smoothness, then motion smoothness is improved, but memory usage and processing requirements increase
Solution Approach 1:
The patent applies partial action by selectively repeating source frames only when necessary to maintain motion smoothness at the target frame rate, rather than uniformly increasing the frame rate throughout the entire video sequence. The system maps source frames to output frames based on timing information, repeating frames only in regions where the original frame rate was lower, thereby maintaining motion smoothness where needed while minimizing memory usage and processing requirements overall.
Data Source
AI summary
Converting variable frame rate video to fixed frame rate video is described. A variable frame rate (VFR) video having source frames with varying frame durations is obtained. A fixed frame rate for the VFR video is determined. A mapping is generated which maps the source frames of the VFR video to a sequence of result frames of the fixed frame rate. To generate the mapping, a suitable source frame is chosen for each result frame using one or more mapping rules. After the mapping is generated, the mapping is adjusted to improve smoothness of motion of the VFR video by adding at least one skipped source frame to the mapping and removing at least one repeated source frame from the mapping. The VFR video is then converted into, or interpreted as, a fixed frame rate video having the determined fixed frame rate based on the adjusted mapping.