Multi-level optical flow estimation framework for stereo pairs of images based on spatial partitioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning-based optical flow estimation for high-resolution images is constrained by memory limitations, leading to undesirable artifacts when down sampling and up sampling is employed, which degrades the quality of view synthesis results.
Innovation Solution
A multi-level optical flow framework that partitions input images into regions suitable for deep learning processing at full resolution, followed by merging high-quality optical flow results to generate accurate motion vectors without downsampling artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning-based optical flow estimation is applied to high-resolution images, then estimation accuracy is improved, but memory constraints prevent direct processing
Solution Approach 1:
The patent divides the high-resolution image into multiple lower-resolution patches or regions. Each patch is processed independently through the deep learning optical flow model, allowing the memory-constrained model to handle high-resolution input by breaking it into manageable segments that fit within memory limits while maintaining overall estimation accuracy.
Solution Approach 2:
The patent transforms the problem from processing a single high-resolution image to processing multiple low-resolution images. This dimensional transformation from one high-dimensional input to multiple lower-dimensional inputs enables the model to work around memory constraints while preserving the ability to estimate optical flow accurately across the entire high-resolution image.
2Productivity
If images are downsampled to meet memory constraints, then processing becomes feasible, but artifacts are introduced in view synthesis results
Solution Approach 1:
Instead of downsampling the entire image and risking artifacts, the patent segments the image into patches that can be processed at their native resolution. This avoids the downsampling-upsampling cycle that causes artifacts while still meeting memory constraints through the reduced size of individual patches.
Solution Approach 2:
The patent creates multiple copies of the optical flow estimation process, one for each image patch. Each patch generates its own optical flow map independently, and these copies are then stitched together to form the complete optical flow map for the entire high-resolution image, avoiding the need for lossy downsampling.
3Manufacturing precision
If images are processed at full resolution, then output quality is maintained, but memory constraints are exceeded
Solution Approach 1:
The patent segments the high-resolution image into smaller patches that can be processed at full resolution without exceeding memory constraints. Each patch maintains its full resolution throughout processing, ensuring output quality is preserved while the overall memory usage remains manageable due to the reduced size of individual patches being processed simultaneously.
Data Source
AI summary
Techniques related to multi-level optical flow estimation are discussed. Such techniques include partitioning each pair of input images into one or more partitions, separately performing optical flow estimation on the partitions, and merging the separately generated optical flow results into a final optical flow map for the pair of input images.