Pixel Adaptive Blending for Frame Interpolation Occlusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing frame interpolation techniques face challenges in accurately handling occluded regions and selecting proper motion vectors, leading to flickering artifacts and incorrect motion depiction, especially at object boundaries and in areas where background regions become covered or uncovered.
Innovation Solution
The method employs pixel adaptive blending, using motion vectors, occlusion confidence measures, and background confidence measures to calculate fallback signals and weights for neighbor blocks, ensuring accurate interpolation by blending previous and current frame data with spatial and content-based weighting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If motion vectors are used to shift image data for moving objects to correct positions in interpolated frames, then motion compensation is improved, but difficulties arise in selecting correct motion vectors in occluded regions leading to flickering artifacts
Solution Approach 1:
The patent applies local quality by treating different regions of the image with different processing strategies. Occluded regions are identified and handled separately from non-occluded regions using occlusion detection mechanisms. Motion vectors are selected and blended locally based on region-specific characteristics, allowing accurate motion compensation in visible areas while preventing flickering artifacts in occluded areas through specialized handling.
Solution Approach 2:
The patent implements dynamics by adaptively selecting and blending motion vectors from multiple candidate blocks based on local image content and motion characteristics. The system dynamically adjusts motion vector selection by evaluating pixel differences, gradient magnitudes, and occlusion probabilities for each region, allowing the motion compensation process to adapt to varying motion patterns and occlusion conditions across different parts of the image.
2Area of stationary object
If motion vectors from neighboring blocks are blended at object boundaries, then coverage of boundary regions is improved, but the wrong motion vector may be selected for several pixels
Solution Approach 1:
The patent applies partial action by selectively blending motion vectors only in regions where it is beneficial, rather than uniformly across the entire image. The system evaluates each pixel or small region to determine whether motion vector blending is appropriate based on gradient analysis and occlusion detection. This selective approach ensures that boundary regions receive enhanced coverage while avoiding the introduction of incorrect motion vectors in regions where simple motion vector selection is sufficient.
3Productivity
If simple frame repetition is used to increase frame rate, then frame rate increase is achieved, but moving objects appear to jump and exhibit flickering artifacts
Solution Approach 1:
The patent applies preliminary action by performing motion estimation and occlusion detection before frame interpolation. The system pre-identifies occluded regions and pre-selects candidate motion vectors from multiple blocks before the actual interpolation process. This preliminary analysis allows the interpolation to proceed smoothly with pre-computed information, achieving high frame rates while maintaining motion accuracy by having the correct motion vectors ready in advance for each region.
Data Source
AI summary
An apparatus has a neighbor motion vector module arranged to receive motion vector data for blocks that neighbor a current pixel and select a predetermined number of block motion vectors, at least two pixel motion vector modules arranged to receive the block motion vector data and to produce interpolated pixel motion vector data, a weighting module configured to produce a weighting for the interpolated pixel motion vector data from the motion vector modules, and a blending module to blend the interpolated pixel motion vector data according to the weighting and to produce pixel output data for the current pixel.