Frame Rate Up-Conversion Motion Vector Correction for Occlusion Stability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Frame rate up-conversion technologies face challenges in accurately detecting occlusion areas and correcting motion vectors, leading to unstable detection results and data break issues due to multi-match problems during interpolation frame creation.
Innovation Solution
An apparatus and method that includes a motion vector generating circuit, a motion vector correction circuit, and an interpolation frame generating circuit, which compare previous and current original frames to identify occlusion areas and correct motion vectors using motion vectors from multiple frames, thereby selecting stable motion vectors for interpolation frame creation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If de-halo algorithm is performed to detect occlusion areas using unstable motion vectors, then occlusion detection is attempted, but the detection result is not satisfactory due to motion vector instability
Solution Approach 1:
The patent applies preliminary action by performing motion estimation between adjacent frames (F1-F2 and F2-F3) before the final interpolation. This preliminary motion estimation provides more reliable motion vectors that are used to detect occlusion areas and guide the subsequent frame interpolation process, avoiding the instability of directly using motion vectors from the interpolation process itself.
2Adaptability or versatility
If 3DRS algorithm calculates multiple best vectors for some blocks, then motion estimation is performed, but multi-match issues cause data broken problems in the interpolation frame
Solution Approach 1:
The patent applies local quality by differentiating between foreground and background blocks and applying different motion vector selection strategies to each. For foreground blocks, the patent uses background subtraction and foreground object tracking to identify the correct motion vector. For background blocks, standard motion estimation is sufficient. This localized approach resolves multi-match issues by selecting the appropriate motion vector based on the local characteristics of each block.
3Productivity
If motion vectors are used directly for frame interpolation, then interpolation frames are created, but occlusion areas cause incorrect motion vectors and degraded image quality
Solution Approach 1:
The patent applies segmentation by dividing the image into foreground and background regions using background subtraction. Different motion estimation and compensation strategies are applied to foreground and background blocks separately. This segmentation allows the system to handle occlusion areas appropriately by identifying foreground blocks that may have incorrect motion vectors and treating them differently from background blocks, thereby improving interpolation quality while maintaining efficiency.
Data Source
AI summary
An frame rate up-conversion (FRC) apparatus and method are provided. The motion vector generating circuit compares a previous original frame with a current original frame to obtain the first motion vectors of the blocks of the current original frame, and compares the current original frame and a posterior original frame to obtain the second motion vectors of the blocks of the current original frame. The motion vector correction circuit checks whether the blocks of the second original frame are located in an occlusion area, and corrects the motion vectors of the blocks in the occlusion area based on the first motion vectors and the second motion vectors of the first original frame, the second original frame and the third original frame. The interpolation frame generating circuit creates at least one interpolation frame between the first original frame and the second original frame based on the corrected motion vectors.