Sub-pixel Disparity Estimation Using Census Features and Equiangular Fitting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing disparity estimation techniques, such as those disclosed in Japanese Patent No. 4168125 and 'Stereo Processing by Semiglobal Matching and Mutual Information', face challenges in achieving high accuracy for sub-pixel disparity values, particularly due to the random nature of Hamming distance variations and insufficient reference pixels, leading to significant errors in depth measurement.
Innovation Solution
A disparity estimation device and method that calculates census feature amounts for images captured by multiple cameras, determines difference degrees using Hamming distance, and employs equiangular fitting with multiple reference pixels to estimate sub-pixel disparity values, improving accuracy by considering the distribution of pixel values around each pixel.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If census transform and correlation operation are used to associate pixels, then disparity estimation can be performed, but the accuracy in estimating sub-pixel disparity values is low
Solution Approach 1:
The patent changes the parameter of disparity precision from integer pixels to sub-pixel values. It introduces a cost function that evaluates disparity values at sub-pixel precision levels (e.g., 0.5, 0.25, 0.125 pixels) rather than only integer pixel levels, thereby improving measurement precision while maintaining reliability through the use of census transform features that are robust to sub-pixel variations
Solution Approach 2:
The patent adds a new dimension to disparity estimation by introducing multiple reference pixels at different disparity offsets (±1, ±2 pixels) around the optimal disparity position. This creates a multi-dimensional evaluation space where sub-pixel disparity values are estimated by analyzing cost variations across these multiple reference positions, transforming a one-dimensional integer pixel search into a multi-dimensional sub-pixel optimization problem
2Productivity
If Hamming distance is used for census feature comparison, then disparity calculation is efficient, but random variations cause significant errors
Solution Approach 1:
The patent implements a feedback mechanism where the cost function evaluates disparity hypotheses and uses the resulting cost values to refine the disparity estimation. By calculating costs for multiple reference pixels at different disparities and using these cost feedbacks to determine the optimal sub-pixel disparity value, the system converts the random variations in Hamming distance into informative signals that guide the estimation toward more accurate results
Solution Approach 2:
The patent performs preliminary integer-pixel disparity estimation using efficient Hamming distance comparison before proceeding to sub-pixel refinement. This preliminary action establishes a baseline optimal disparity position that serves as the starting point for subsequent sub-pixel optimization, ensuring that the computationally intensive sub-pixel calculation begins from an already refined position rather than searching the entire disparity range
3Device complexity
If integer pixel accuracy is used for disparity values, then calculation is simple, but depth measurement precision is insufficient
Solution Approach 1:
The patent segments the disparity estimation process into two distinct stages: (1) integer pixel accuracy estimation using simple Hamming distance comparison to find the optimal disparity position, and (2) sub-pixel accuracy refinement using cost function evaluation at multiple reference pixels. This segmentation allows the system to maintain calculation simplicity in the first stage while achieving high precision in the second stage, avoiding the need to perform complex calculations across the entire disparity range
Data Source
AI summary
A disparity estimation device calculates, for each of first pixels of a first image and each of second pixels of a second image, a first census feature amount and a second census feature amount, calculates, for each of the first pixels, a first disparity value of the first pixel with integer accuracy, extracts, for each of the first pixels, reference pixels located in positions corresponding to the first disparity value and a near disparity value close to the first disparity value from the second pixels, calculates sub-pixel evaluation values based on the relationship between the pixel values of the first pixel and the neighboring pixel and the pixel values of each of the reference pixels and the neighboring pixel, and estimates a second disparity value of the first pixel with sub-pixel accuracy by equiangular fitting.