ToF Image Processing Using Non-Saturated Spot Pixels for Depth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing time-of-flight (ToF) systems face challenges in accurately determining depth information due to saturation of spot peak pixel values, particularly for objects at close distances or with high reflectivity, which can lead to discarded data and increased acquisition time for capturing depth information of the whole scene.
Innovation Solution
An image processing circuitry that generates component data from ToF measurements, calculates phase amplitude values, identifies spot and valley pixel regions, determines a spot peak pixel with a threshold, and uses non-saturated pixels in a predetermined range centered at the spot peak pixel to calculate depth values, thereby extending the dynamic range without discarding saturated regions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If spot ToF systems concentrate light in light spots to illuminate the scene, then depth measurement precision is improved for specific regions, but pixel saturation occurs for objects at close distances or with high reflectivity
Solution Approach 1:
The patent applies parameter changes by utilizing pixels with different phase amplitude values (below threshold vs. above threshold) within the same spot pixel region. Non-saturated pixels (phase amplitude below threshold) are used for objects at close distances or with high reflectivity, while saturated pixels (phase amplitude above threshold) are used for objects at farther distances. This parameter-based selection resolves the contradiction by adapting the measurement approach based on the signal characteristics of each pixel region.
2Measurement precision
If saturated spot pixel regions are discarded to avoid inaccurate depth information, then measurement precision is maintained for non-saturated pixels, but loss of information occurs for saturated regions
Solution Approach 1:
The patent converts the harmful effect of pixel saturation into a beneficial feature by categorizing pixels based on their phase amplitude values. Instead of discarding saturated pixels, the system identifies them as a distinct category (phase amplitude above threshold) and uses them specifically for measuring objects at farther distances. This transforms the previously harmful saturation effect into a useful measurement mode, eliminating information loss while maintaining accuracy.
3Loss of information
If acquisition time is increased to capture depth information of the whole scene including saturated regions, then completeness of depth data is improved, but productivity decreases
Solution Approach 1:
The patent applies partial action by using different pixel subsets for different measurement needs within a single acquisition. Non-saturated pixels (phase amplitude below threshold) handle close-range measurements, while saturated pixels (phase amplitude above threshold) handle far-range measurements. This partial utilization of available pixel data eliminates the need for additional acquisition time or multiple scans, maintaining high productivity while achieving complete scene coverage.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach allows for accurate depth determination across varying distances and reflectivity levels with a single acquisition, maintaining high sensitivity for both low and high signal levels without requiring changes in acquisition time or hardware, thus enhancing the dynamic range and depth information capture.
Implementation Method 1
time-of-flight (ToF) systems are known, which are used for determining a distance to or a depth map of (objects in) a scene that is illuminated with light. Time-of-flight includes a variety of methods that measure the time that a particle or a light wave needs to travel a distance in a medium.
Implementation Method 2
Known ToF systems can obtain depth measurements of objects in a scene for every pixel of the depth image simultaneously, wherein the depth image is captured with an image sensor. For capturing this image, the ToF system typically illuminates the scene with, for instance, a modulated light wave and images the backscattered/ reflected light wave with an optical lens portion on the image sensor having, for example, a pixel array, wherein a gain of the pixel array is modulated accordingly. Signal depth information can be obtained from the resulting modulation.
Data Source
Figure 1
Figure 2
Figure 3A~3C
AI summary
An image processing circuitry for an indirect time-of-flight system configured to: - generate, based on obtained image data of a plurality of pixels representing a time-of-flight measurement of light reflected from a scene that is illuminated with spotted light, corresponding component data including component values of the plurality of pixels; - calculate, based on the generated component data, corresponding phase amplitude image data including phase amplitude values of the plurality of pixels; - determine, based on the calculated phase amplitude image data, spot pixel regions (31, 32) and a valley pixel region (33) among the plurality of pixels; - determine, in each of the spot pixel regions, a spot peak pixel having a maximum phase amplitude value (34, 35) among the phase amplitude values of the respective spot pixel region; - determine a first set of spot pixel regions, wherein the first set of spot pixel regions includes such spot pixel regions (32) in which the determined maximum phase amplitude value of the spot peak pixel is above a predetermined threshold (Zth); - determine, in each of the spot pixel regions in the first set of spot pixel regions, a predetermined number of non-saturated pixels or non-saturated pixels in a predetermined range centered at the spot peak pixel which have a phase amplitude value below the predetermined threshold; and - calculate a depth value for each of the spot pixel regions in the first set of spot pixel regions based on component values of the predetermined number of non-saturated pixels.