Occlusion Region Detection in Frame Rate Up-Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current frame rate up-conversion techniques, such as motion compensated frame interpolation, often result in halo effects or 'ghost replicas' in occlusion regions due to the lack of luminance information in consecutive frames, leading to suboptimal visual performance.
Innovation Solution
A method for detecting and classifying occlusion regions by comparing the sum of absolute differences between bidirectional motion vectors and forward or backward motion vectors with a threshold, allowing for different interpolation strategies based on region classification, and employing de-blocking filtering to prevent artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If motion compensated frame interpolation is used for frame rate up conversion, then motion blur is avoided, but halo effects or ghost replicas appear in occlusion regions
Solution Approach 1:
The patent segments the image into occlusion regions and non-occlusion regions using motion vector analysis. By detecting inconsistencies between bidirectional and unidirectional motion vectors, the method identifies occlusion regions where luminance information is unavailable. Different processing strategies are then applied to each segment: standard motion compensation for non-occlusion regions and alternative interpolation methods for occlusion regions, thereby eliminating halo effects while preserving motion clarity.
Solution Approach 2:
The patent applies different quality levels and processing methods to different regions of the image. In non-occlusion regions, full motion compensation is applied for high quality. In occlusion regions, where luminance information is missing, the method uses alternative approaches such as gradient-based interpolation or copying from reference frames. This local differentiation ensures that each region receives the most appropriate processing, avoiding the halo effects that occur when uniform processing is applied throughout.
2Reliability
If multiple frames are used in motion estimation and compensation to handle occlusion regions, then luminance information availability improves, but computational complexity increases
Solution Approach 1:
The patent applies partial action by using multiple frames only when and where necessary - specifically in occlusion regions where luminance information is missing. In non-occlusion regions, standard two-frame motion compensation is sufficient. The method performs motion vector analysis to identify occlusion regions first, then selectively applies the more computationally intensive multi-frame processing only to those specific regions, thereby balancing information availability with computational efficiency.
Solution Approach 2:
The system uses the motion vector field itself to identify regions that need special processing. By analyzing inconsistencies between bidirectional and unidirectional motion vectors, the algorithm self-identifies occlusion regions and automatically applies appropriate processing strategies without requiring external control or manual intervention, thus managing computational complexity efficiently.
3Ease of manufacture
If simple frame rate up conversion methods (black frame insertion, frame repetition, frame averaging) are used, then computational simplicity is maintained, but visual performance degrades due to motion blur and intensity reduction
Solution Approach 1:
The patent fundamentally changes the processing parameters in occlusion regions by switching from standard motion compensation to alternative interpolation methods. When luminance information is unavailable, the system adjusts parameters such as using gradient-based methods, copying pixel values from reference frames, or applying different weighting schemes. This parameter adaptation allows the system to maintain good visual performance in challenging regions without requiring completely different processing pipelines.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
In frame rate up-conversion for video signals interpolated frames are inserted within the video stream to increase the frame rate for display. Halo effects, observed as ghost replicas, can occur in occlusion regions around the boundaries of moving objects. To enable the occlusion regions to be compensated, they are detected by a method utilizing forward, backward and bidirectional motion vectors for each of a first frame (100), a second frame (104) and an interpolated frame (102). Occlusion regions (106,108) are located by comparing bidirectional motion vectors with forward or backward motion vectors to find inconsistencies. Then blocks of pixels having bi directional motion vectors (112,114) can be compensated unidirectionally by using the pixel information (220,224) in one of the original frames (100,102).