Stereo Image Depth Estimation Through Compressed Feature Correlations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for computing depth information from stereo images require significant processing time and computation, and the maximum disparity used may not be sufficiently large, leading to incorrect disparity predictions and non-scalable computation times.
Innovation Solution
A technique that computes correlations between feature maps by multiplying each column in a left feature map with all columns in a right feature map, followed by compressing and masking the correlation maps to improve computational efficiency and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If feature shifting is used to compute correlations between feature maps, then depth information can be extracted from stereo images, but processing time and computation time increase significantly
Solution Approach 1:
The patent segments the correlation computation into distinct operational phases: extracting feature maps from stereo images, computing correlations between corresponding feature maps, compressing the correlation volume, and selecting maximum correlation values. This segmentation allows for optimized processing at each stage, particularly through the compression step that reduces computational burden while preserving depth information accuracy.
Solution Approach 2:
The patent transforms the 3D correlation volume into a compressed representation by aggregating correlation values across the depth dimension. This dimensional transformation reduces the computational complexity from O(N³) to O(N²) while maintaining the ability to extract accurate depth information through maximum value selection in the compressed correlation map.
2Measurement precision
If maximum disparity is increased to include all corresponding positions in the search window, then disparity prediction accuracy improves, but computation time increases linearly and becomes non-scalable
Solution Approach 1:
The patent resolves this contradiction by introducing a compression dimension that aggregates correlation values across the disparity range. Instead of processing each disparity value independently, the system compresses the correlation volume along the disparity dimension, allowing the maximum disparity parameter to be increased for accuracy without linearly increasing computation time, thus achieving scalability.
Solution Approach 2:
The patent performs preliminary compression of the correlation volume before disparity selection. By pre-aggregating correlation values across all disparities in the compressed correlation map, the system prepares the data structure in advance, allowing rapid disparity selection without re-computing correlations for each maximum disparity value, thereby improving scalability.
3Measurement precision
If feature shifting is used to compute correlations, then depth map can be generated from stereo images, but the method requires significant processing and computation resources
Solution Approach 1:
The patent segments the depth estimation pipeline into distinct modules: feature extraction, correlation computation, compression, and disparity selection. This modular segmentation reduces processing complexity by allowing each module to be optimized independently and processed in parallel, while maintaining the overall accuracy of depth map generation.
Solution Approach 2:
The patent introduces a compression dimension that transforms the 3D correlation volume into a 2D compressed correlation map. This dimensional reduction simplifies the data structure and reduces memory requirements, making the system less complex while preserving the information needed for accurate depth map generation through subsequent maximum value selection.
Data Source
AI summary
In various examples, a technique for computing feature correlations given a stereo image pair (captured using two or more image sensors having at least partially overlapping fields of view) is disclosed. The technique includes, for one or more channels in a set of feature channels, receiving a first feature map for a first image in a stereo image pair and a second feature map for a second image in the stereo image pair and computing a corresponding set of correlation maps. The technique also includes generating a set of compressed correlation maps; masking one or more portions of individual compressed correlation maps of the set compressed correlation maps based at least on a respective correlation filter to generate a corresponding set of masked correlation maps; and generating a depth map associated with the stereo image pair based at least on the set of masked correlation maps.