Geometric Disparity Upsampling With Selectivity Kernels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing disparity estimation techniques face challenges in achieving higher resolution disparity maps with lower latency, computational complexity, and memory usage, while maintaining accuracy, especially when transitioning from lower to higher resolution images, and lack the ability to dynamically select between disparity upsampling and higher resolution estimation based on scene complexity.
Innovation Solution
A multi-factor selectivity kernel-based machine learning model is employed for geometric upsampling, utilizing a composite of kernels that consider feature similarity, spatial distance, geometric consistency, temporal consistency, and estimation confidence to enhance the upsampling process, allowing dynamic selection between disparity upsampling and higher resolution estimation based on scene complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If disparity estimation is performed on higher resolution multiscopic images, then higher resolution disparity maps are obtained, but computational complexity, memory footprint, power consumption, and latency increase
Solution Approach 1:
The patent segments the disparity estimation process into two distinct stages: (1) performing disparity estimation on lower resolution multiscopic images to obtain a low resolution disparity map, and (2) upsampling the low resolution disparity map to generate a high resolution disparity map. This segmentation allows each stage to operate at optimal resolution levels, reducing overall computational complexity while maintaining high output resolution.
Solution Approach 2:
The patent introduces a low resolution disparity map as an intermediary representation between the input multiscopic images and the final high resolution disparity map. This intermediary allows the system to work with reduced data volumes during computation, then restore detail through upsampling, effectively decoupling computational requirements from output resolution requirements.
2Productivity
If lower resolution multiscopic images are used for disparity estimation, then computational complexity and latency are reduced, but disparity information for finer details is lost
Solution Approach 1:
The patent performs disparity estimation preliminarily on lower resolution images to capture coarse disparity structures and geometric relationships. This preliminary estimation establishes a foundation that guides subsequent upsampling operations, ensuring that fine details are added in a geometrically consistent manner rather than through random interpolation.
Solution Approach 2:
The patent replaces traditional mechanical upsampling methods (such as bilinear or bicubic interpolation) with a learning-based upsampling approach. The machine learning model learns geometric relationships and patterns from training data, enabling it to generate high resolution disparity maps that preserve geometric consistency and capture fine details without requiring direct processing of high resolution input images.
3Measurement precision
If standard upsampling methods are used on lower resolution disparity maps, then higher resolution disparity maps are obtained, but geometric consistency and accuracy are compromised
Solution Approach 1:
The patent changes the fundamental parameters of the upsampling process by transitioning from fixed mathematical interpolation formulas to adaptive, learning-based parameter selection. The machine learning model dynamically adjusts upsampling parameters based on local image characteristics, disparity values, and geometric constraints, enabling preservation of geometric consistency while achieving high resolution output.
Solution Approach 2:
The patent applies different upsampling strategies to different regions of the disparity map based on local characteristics. The machine learning model identifies regions with different geometric properties (such as edges, smooth surfaces, or occluded areas) and applies appropriate upsampling techniques to each region, preserving geometric consistency locally while achieving high overall resolution.
4Productivity
If disparity upsampling is performed on lower resolution disparity maps, then computational efficiency is improved, but accuracy compared to direct high resolution estimation is reduced
Solution Approach 1:
The patent replaces traditional mechanical upsampling algorithms with a machine learning-based upsampling system. The learning-based approach learns optimal upsampling strategies from training data, enabling it to achieve accuracy comparable to or exceeding direct high resolution estimation while maintaining the computational efficiency of processing lower resolution images. The model captures complex patterns and relationships that fixed algorithms cannot replicate.
Data Source
AI summary
Certain aspects of the present disclosure provide techniques for upsampling input data including inputting input data at a first resolution into a machine learning (ML) model comprising a plurality of selectivity kernels, each of the plurality of selectivity kernels configured to perform a different type of selectivity to upsample the input data; and obtaining output data, corresponding to the input data, at a second resolution, from the ML model, the second resolution being higher than the first resolution, wherein the output data is based on a composite of outputs from the plurality of selectivity kernels.