Motion Vector Interpolation for Occluded Regions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing frame interpolation methods face difficulties in accurately selecting motion vectors, particularly in occluded regions where background areas become covered or uncovered by moving objects, leading to incorrect motion vector selection and artifacts in interpolated images.
Innovation Solution
The proposed solution involves a motion estimation and correction apparatus that uses Phase Plane Correlation and a 3D recursive motion vector search algorithm, combined with motion vector double confirmation and occlusion detection, to accurately select and correct motion vectors for interpolated frames, ensuring smooth motion and reducing artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If motion estimation and motion compensation techniques are used to improve frame interpolation quality, then motion smoothness is improved, but device complexity increases due to the need for accurate motion vector selection in occluded regions
Solution Approach 1:
The patent segments the image into occluded and non-occluded regions using occlusion detection algorithms. By dividing the processing into distinct regions with different motion vector selection strategies, the system maintains high interpolation quality in occluded areas while using standard techniques in non-occluded areas, thus managing complexity effectively.
Solution Approach 2:
The patent performs preliminary occlusion detection and motion vector selection for occluded regions before final frame interpolation. By identifying occluded regions and selecting appropriate motion vectors in advance, the system avoids complex real-time decision-making during interpolation, reducing overall computational complexity while maintaining reliability.
2Productivity
If frame repetition is used to increase frame rate, then productivity is improved, but image quality deteriorates due to flickering artifacts from moving objects
Solution Approach 1:
The patent introduces intermediate interpolated frames between original frames using motion estimation and compensation. These intermediate frames act as mediators that smoothly transition between original frames, eliminating the flickering artifacts caused by simple frame repetition while maintaining high frame rates.
Solution Approach 2:
The patent changes the temporal parameter by generating frames at intermediate time points between original frames. By calculating motion vectors and interpolating at these intermediate time points, the system achieves higher effective frame rates with smooth motion transitions, avoiding the quality degradation of simple repetition.
3Device complexity
If motion vectors are selected from available frames without occlusion detection, then device complexity is reduced, but measurement precision deteriorates due to incorrect motion vector selection in occluded regions
Solution Approach 1:
The patent performs preliminary occlusion detection before motion vector selection. By identifying occluded regions in advance, the system can apply specialized motion vector selection techniques only where needed, rather than complicating the entire process. This preliminary action maintains measurement precision in critical areas while keeping overall device complexity manageable.
Data Source
AI summary
An apparatus has a motion vector estimation module to perform motion vector estimation on a current frame and on a previous frame of image data to produce a current phase and a previous phase, a motion vector generation module to generate motion vector fields for the current and previous phases, and a motion vector interpolation module to find interpolated motion vectors for an interpolated phase that have minimal differences between motion vectors in the current and previous motion vector fields and to product interpolated motion vectors.