SPAD Depth Map Upscaling via Signal Count Factors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for upsampling low resolution depth maps, such as bilinear, weighted average, median, and bicubic methods, result in blurry images or edge artifacts, and are computationally intensive, while super resolution and joint bilateral upsampling have limitations including reliance on motion and synchronized devices, and sensitivity to border pixels.
Innovation Solution
An electronic device with a single-photon avalanche diode (SPAD) array and readout circuitry generates depth and signal count maps, where an upscaling processor calculates upscaling factors based on physical properties between intensity and distance observations, allowing real-time upsampling without calibration, effectively increasing the resolution of depth maps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If spatial upsampling methods (bilinear, weighted average, median, bicubic) are used to increase the number of ranging points in a low resolution depth map, then the resolution is improved, but the image quality deteriorates with blurry images or edge artifacts
Solution Approach 1:
The patent changes the fundamental parameter being upscaled from spatial coordinates to time-of-flight values. Instead of interpolating spatial positions, the invention upsamples the time-of-flight measurements themselves, which represent actual physical distance information. This parameter transformation allows generating multiple plausible time-of-flight values for each pixel while maintaining physical consistency and avoiding traditional interpolation artifacts.
Solution Approach 2:
The patent introduces an intermediary statistical model that acts as a bridge between the low-resolution depth map and the high-resolution output. This model uses the measured time-of-flight values and their uncertainties to generate probability distributions, from which multiple realistic time-of-flight values can be sampled. The intermediary model ensures that upsampled values are physically plausible rather than merely spatial interpolations.
2Reliability
If complex upsampling algorithms with larger kernels are used to improve image quality, then the reliability is improved, but the computational complexity increases
Solution Approach 1:
The patent extracts the essential information needed for upsampling directly from the time-of-flight measurements and their associated uncertainties. Instead of using large convolutional kernels that process extensive neighborhoods of pixels, the invention extracts the key parameters (time-of-flight values and their standard deviations) and uses these to generate upsampled values through statistical sampling. This extraction approach reduces computational complexity while maintaining image quality.
Solution Approach 2:
The patent creates multiple copies of the depth information by generating several plausible time-of-flight values for each pixel through statistical sampling. Rather than using complex algorithms to compute a single high-resolution value, the invention copies the essential measurement information and varies it according to the measured uncertainty, producing multiple realistic outcomes that can be used for rendering or further processing.
3Manufacturing precision
If super resolution methods relying on motion between acquisitions are used to increase ranging points, then the resolution is improved, but the device complexity and calibration requirements increase
Solution Approach 1:
The patent enables the depth sensing system to perform upsampling using only its own measurements and inherent uncertainty information. The system uses the time-of-flight data and standard deviations already captured by the sensor to generate high-resolution output, without requiring additional sensors, motion capture systems, or external calibration data. The upsampling process is self-contained and uses the measurement uncertainty as the sole input parameter.
Solution Approach 2:
The patent creates a universal upsampling method that works for any time-of-flight depth sensor regardless of its specific resolution or field of view. The algorithm is resolution-agnostic and can be applied to depth maps of any size or sensor type. This universal approach eliminates the need for device-specific calibration or synchronization mechanisms, making the solution broadly applicable across different sensor platforms.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution provides clear and accurate upsampling of low resolution depth maps to higher resolutions, improving image quality and reducing computational complexity, while being generic and less sensitive to input data characteristics.
Implementation Method 1
An electronic device with a single-photon avalanche diode (SPAD) array and readout circuitry generates depth and signal count maps
Data Source
AI summary
An electronic device includes a SPAD array and readout circuitry coupled thereto. The readout circuitry generates a depth map having a first resolution, and a signal count map having a second resolution greater than the first resolution. The depth map corresponds to distance observations to an object. The signal count map corresponds to intensity observation sets of the object, with each intensity observation set including intensity observations corresponding to a respective distance observation in the depth map. An upscaling processor is coupled to the readout circuitry to calculate upscaling factors for each intensity observation set so that each distance observation has respective upscaling factors associated therewith. The depth map is then upscaled from the first resolution to the second resolution based on the respective upscaling factors.