Real-time Occlusion Detection Using Forward-Backward Flow Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional techniques for detecting occluded pixels in video sequences often fail due to inaccurate and noisy optical flow data, leading to unacceptable results, and are either time-consuming or resource-intensive.
Innovation Solution
A lightweight, real-time occlusion detection method using a two-pass approach with a forward-backward check and filtering process, which evaluates flow vectors and differentiates between occlusion and noise by considering chroma values, allowing for accurate identification of occluded pixels without requiring depth data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional techniques are used to detect occluded pixels based on flow vectors, then occlusion detection can be performed, but the results are unacceptable due to inability to distinguish between occlusion and noise
Solution Approach 1:
The patent implements a forward-backward consistency check where occlusion detection results are validated by checking if the detected occluded pixels are consistent when processing frames in reverse order. This feedback mechanism filters out false positives caused by noise while preserving true occlusion detections, thereby improving reliability without requiring more precise flow vectors
Solution Approach 2:
The patent performs preliminary filtering of flow vectors by evaluating their consistency with neighboring vectors and with the optical flow field before using them for occlusion detection. This preliminary action removes noisy and unreliable flow vectors in advance, allowing accurate occlusion detection even when individual flow vectors are imprecise
2Reliability
If existing approaches utilize conventional techniques for detecting occluded pixels, then detection can be performed, but the methods are time consuming
Solution Approach 1:
The patent segments the occlusion detection process into distinct stages: flow vector evaluation, forward occlusion detection, backward occlusion detection, and consistency validation. By segmenting the process, each stage can be optimized independently and only necessary computations are performed, reducing overall processing time while maintaining accuracy
Solution Approach 2:
The patent applies partial action by performing occlusion detection only on regions where flow vectors indicate potential occlusion, rather than processing the entire frame. This selective approach maintains high accuracy for occluded regions while significantly reducing processing time for non-occluded areas
3Reliability
If existing approaches utilize conventional techniques for detecting occluded pixels, then detection can be performed, but the methods are resource intensive
Solution Approach 1:
The patent uses the existing optical flow data and image properties already available in the video processing pipeline to perform occlusion detection, rather than requiring additional sensors, depth cameras, or specialized hardware. The method leverages chroma information and flow vector consistency that are naturally present in standard video data, making the system self-sufficient and resource-efficient
Solution Approach 2:
The patent changes the parameters used for occlusion detection from relying solely on precise flow vector magnitude to using flow vector consistency, chroma differences, and spatial coherence. These parameter changes allow accurate detection using standard computational resources without requiring expensive specialized hardware or additional energy-consuming sensors
Data Source
AI summary
Systems and methods estimate occluded pixels in frames of a video sequence. Optical flow data is received to determine a validity for forward and backward flow vectors for a common pixel location in a first frame and a second frame that are temporally next to one another. Occlusion information for the first frame determines pixels that are hidden in the second frame with respect to playback from the first frame to the second frame. Occlusion information for the second frame determines pixels that are hidden in the first frame with respect to playback from the second frame to the first frame.