Motion Vector Scaling for Frame Rate Conversion Cadence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Low frame rate videos result in uneven motion vectors due to inconsistent frame dropping, leading to degraded motion estimation and compensation quality, as existing methods fail to accurately calculate motion vectors and correct for high-low motion artifacts.
Innovation Solution
A method that detects the cadence of low frame rate video and uses this information to scale motion vector candidates, correcting motion vector calculations and frame interpolation by adjusting time stamps and scaling motion vectors to ensure consistent time intervals, thereby improving the accuracy of motion estimation and compensation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If frame rate conversion is performed by dropping frames to convert from high frame rate (60 Hz, 30 Hz, 25 Hz, 24 Hz) to low frame rate (15 Hz), then the frame rate is successfully converted, but the motion vectors become uneven and incorrect time stamps are generated
Solution Approach 1:
The patent applies preliminary action by detecting the cadence pattern of frame drops before performing motion estimation. The system identifies whether frames are dropped in a pattern (e.g., every other frame, every third frame) and uses this information to adjust motion vector calculations in advance, preventing the high-low motion artifact from occurring during the actual frame rate conversion process.
Solution Approach 2:
The patent changes the parameter of motion vector scaling based on the detected cadence pattern. When a pattern is detected, the system applies different scaling factors to motion vectors depending on their position in the sequence, adjusting them to compensate for the uneven time intervals caused by frame dropping. This dynamic parameter adjustment corrects the motion vectors to reflect actual temporal relationships.
2Productivity
If motion vectors are calculated from previous frames using standard MEMC methods, then motion estimation is performed, but the three-dimensional recursive motion vector calculation becomes less accurate due to incorrect motion vectors from previous frames
Solution Approach 1:
The patent implements feedback by using the detected cadence pattern to continuously adjust motion vector calculations throughout the video sequence. The system monitors the frame drop pattern and feeds this information back into the motion estimation process, allowing each subsequent motion vector calculation to be corrected based on the known temporal relationships established by the cadence detection.
3Productivity
If frame interpolation is performed using standard MEMC, then new frames are generated, but halo reduction logic becomes invalid because it compares motion vectors from previous or future frames to current frame
Solution Approach 1:
The patent adjusts the temporal spacing parameter used in halo reduction comparisons based on the detected cadence pattern. Instead of assuming uniform frame intervals, the system modifies the time delta used for comparing motion vectors across frames, ensuring that comparisons are made between frames with correct temporal relationships, thereby restoring the validity of halo reduction logic.
4Speed
If frames are dropped unevenly to achieve low frame rate output, then frame rate conversion is achieved, but the time between frames becomes uneven causing high-low motion artifacts
Solution Approach 1:
The patent performs preliminary detection of the frame drop cadence pattern before processing, identifying the stable periodic structure in the otherwise uneven frame intervals. This preliminary analysis allows the system to predict and compensate for time interval variations throughout the sequence, maintaining stable motion vector magnitudes despite uneven sampling.
Data Source
AI summary
A method of performing motion vector correction in a sequence of video frames includes receiving, at a processor, a frame of video frames at a received rate lower than an original frame rate, identifying motion vector candidates for a frame in the sequence of video frames, detecting a cadence of the sequence of video frames using the motion vector candidates, scaling the motion vector candidates according to the cadence to produce scaled motion vector candidates, calculating motion vectors for a frame in the sequence of video frames using the scaled motion vector candidates, and interpolating at least one new frame of video data using the motion vectors.