SPAD Depth Map Upscaling via Signal Count Intermediary
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for upsampling low resolution depth maps, such as bilinear, weighted average, median, bicubic, super resolution, and joint bilateral upsampling, suffer from issues like adding no real information, producing blurry images, requiring complex algorithms, being costly, or being sensitive to border pixels, and do not effectively leverage known device properties.
Innovation Solution
An electronic device with a single-photon avalanche diode (SPAD) array and readout circuitry generates depth and signal count maps, where an upscaling processor calculates upscaling factors based on physical properties between intensity and distance observations, allowing real-time upsampling from low to high resolution without calibration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If spatial upsampling methods (bilinear, weighted average, median, bicubic) are used to increase the number of ranging points in a low resolution depth map, then the resolution is improved, but the image quality deteriorates due to blurriness and edge artifacts, and no real information is added
Solution Approach 1:
The patent uses the signal count map as an intermediary to guide the upsampling process. The signal count map, which has higher resolution than the depth map, serves as a mediator that provides edge and boundary information to guide the generation of virtual ranging points during upsampling, thereby avoiding the blurriness and edge artifacts produced by traditional spatial upsampling methods.
2Manufacturing precision
If super resolution from multiple acquisitions is used to increase the number of ranging points, then the resolution is improved, but the device complexity and cost increase due to requiring multiple synchronized devices or motion between acquisitions
Solution Approach 1:
The patent employs self-service by using the device's own signal count map (which is already captured simultaneously with the depth map) to guide the upsampling process. This eliminates the need for additional synchronized devices or motion between acquisitions, reducing device complexity while achieving super-resolution效果的depth maps.
3Manufacturing precision
If joint bilateral upsampling is used to upscale the depth map using the signal count map, then the resolution is improved and edge following is enhanced, but the method remains sensitive to border pixels and requires tuning
Solution Approach 1:
The patent applies local quality by generating virtual ranging points with different weights based on their local characteristics. Border pixels are assigned appropriate weights to account for their sensitivity, while interior pixels are processed differently. This localized approach to weight assignment improves robustness to border pixels and eliminates the need for extensive tuning of joint bilateral upsampling parameters.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach effectively upscales low resolution depth maps to higher resolution by using upscaling factors derived from physical properties, improving image quality and reducing computational complexity, while being robust to border pixels and not requiring additional devices or calibration.
Implementation Method 1
The electronic device includes a single-photon avalanche diode (SPAD) array and readout circuitry coupled thereto. The readout circuitry is configured to generate a depth map and a signal count map
Implementation Method 2
The readout circuitry is configured to generate a depth map having a first resolution, and a signal count map having a second resolution greater than the first resolution
Data Source
AI summary
An electronic device includes a single-photon avalanche diode (SPAD) array and readout circuitry coupled thereto. The readout circuitry generates a depth map having a first resolution, and a signal count map having a second resolution greater than the first resolution. The depth map corresponds to distance observations to an object. The signal count map corresponds to intensity observation sets of the object, with each intensity observation set including intensity observations corresponding to a respective distance observation in the depth map. An upscaling processor is coupled to the readout circuitry to calculate upscaling factors for each intensity observation set so that each distance observation has respective upscaling factors associated therewith. The depth map is then upscaled from the first resolution to the second resolution based on the respective upscaling factors.