Optical Flow Map Refinement via Foreground Background Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing optical flow map generation techniques face difficulties with depth discontinuities, resulting in artifacts in view interpolation results, particularly when dealing with stereo image pairs that include foreground and background segments.
Innovation Solution
The proposed method refines optical flow maps by using foreground and background image segmentation, employing a deep learning-based approach to estimate alpha mattes and combining initial optical flow maps with reference optical flow maps generated for the background stereo image pair, thereby enhancing the accuracy of view interpolation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional optical flow map generation techniques are used, then the processing is simple and fast, but artifacts appear in view interpolation results due to depth discontinuities
Solution Approach 1:
The patent segments the image into foreground and background regions using alpha mattes generated by deep learning models. This segmentation allows different optical flow estimation strategies to be applied to different regions, improving accuracy at depth discontinuities while maintaining overall system efficiency. The foreground-background separation enables precise handling of occluding boundaries without requiring complex global optimization.
Solution Approach 2:
The patent introduces alpha mattes as an intermediary representation that bridges the gap between simple optical flow methods and accurate depth discontinuity handling. The alpha mattes serve as a mediator that guides the optical flow estimation process, allowing the system to achieve high accuracy without the full complexity of traditional multi-view geometry methods.
2Manufacturing precision
If foreground and background segmentation is used to refine optical flow maps, then artifacts are reduced in view interpolation, but additional processing overhead is introduced
Solution Approach 1:
The patent performs foreground-background segmentation and alpha matte generation as preliminary steps before optical flow estimation. By preparing these segmentation maps in advance, the subsequent optical flow refinement process can proceed efficiently without repeated segmentation computations. This preliminary action structure enables high precision optical flow maps while minimizing overall processing time through optimized computation sequencing.
Solution Approach 2:
The patent applies refined optical flow computation selectively to regions where it provides the most benefit, such as boundaries between foreground and background segments. Rather than uniformly applying complex refinement across the entire image, the system focuses computational resources on local regions with depth discontinuities, achieving high precision where needed while maintaining fast processing for homogeneous regions.
Data Source
AI summary
Methods, apparatus, systems and articles of manufacture (e.g., physical storage media) to generate optical flow maps based on foreground and background image segmentation are disclosed. Example apparatus disclosed herein are to generate a reference optical flow map based on a stereo pair of background images corresponding to a first camera field-of-view and a second camera field-of-view, and generate a first optical flow map based on a stereo pair of input images corresponding to the first camera field-of-view and the second camera field-of-view. Disclosed example apparatus are also to combine the reference optical flow map and the first optical flow map based on an alpha matte to generate a second optical flow map associated with the stereo pair of input images, the alpha matte representative of segmentation of at least one of the stereo pair of input images into foreground and background regions.