Depth Image Generation via Recursive Path Cost Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for creating depth images, such as Semiglobal Matching (SGM), require significant computational resources and are inefficient, especially for low-performance hardware.
Innovation Solution
The proposed method reduces the computational effort by calculating path costs for groups of K pixels instead of individual pixels, allowing for a more efficient distribution of path directions and aggregation of costs, which can be performed even on low-resource hardware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If Semiglobal Matching (SGM) is used to calculate depth images with high accuracy, then measurement precision is improved, but device complexity and computational resources increase significantly
Solution Approach 1:
The patent divides the image processing into distinct segments: a first pass that calculates path costs for all pixels, and a second pass that performs cost aggregation only for selected pixels (e.g., every k-th pixel or pixels at specific positions). This segmentation reduces the number of expensive aggregation operations while preserving depth image accuracy through the use of interpolation for intermediate pixels.
Solution Approach 2:
The patent applies partial action by performing complete SGM processing only for a subset of pixels (e.g., boundary pixels, pixels at regular intervals, or pixels in regions of interest), while using simplified interpolation methods for the remaining pixels. This approach achieves acceptable accuracy for most pixels while significantly reducing computational complexity.
2Measurement precision
If recursive path cost calculation is performed for each pixel along multiple smoothness paths, then measurement precision is improved, but productivity decreases due to high computational effort
Solution Approach 1:
The patent performs preliminary calculation of path costs for all pixels in a first pass before performing cost aggregation. The pre-calculated path costs are stored and then reused during the second pass when performing aggregation, eliminating the need to recalculate path costs for each aggregation operation and significantly improving calculation speed.
Solution Approach 2:
The patent creates and stores copies of path cost data during the first pass, which are then referenced and reused during the second pass for cost aggregation. This copying approach avoids redundant calculations and enables faster processing by having pre-computed data readily available.
3Measurement precision
If N smoothness paths are calculated for each pixel, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent merges the path cost calculation and cost aggregation operations into a two-pass framework where path costs are calculated once for all pixels and then aggregated for selected pixels. This merging approach reduces redundant calculations and lowers hardware requirements by avoiding the need to maintain separate computation pipelines for each pixel.
Data Source
Figure 1~2
Figure 3~5
Figure 6~8
AI summary
A method for generating depth images, wherein images of a scene (4) are recorded with at least two cameras (5, 6), the cameras (5, 6) each having an image sensor which is read out pixel by pixel, wherein depth information is derived from content correspondences between the images, and wherein semiglobal smoothness conditions are used to determine the content correspondences. The method is characterized in that the depth calculation for each pixel proceeds along the readout direction of the two cameras (5, 6), that for each integer k>1 along the readout direction of neighboring pixels, the recursive calculation of the path cost of each of the N≥k one-dimensional smoothness paths is performed once at one of these k pixels, and that a cost aggregation for each pixel of this k-tuple is composed of its local cost function as well as of all N path cost functions applicable to the k-tuple (cf.