Time-of-Flight Camera Dynamic Range Extension via Multi-Exposure Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Time-of-flight cameras face challenges in extending their dynamic range, leading to saturation effects in near-range measurements and noise in far-range due to large intensity differences and low signal strength.
Innovation Solution
A method that combines raw phase images captured at different exposure times, normalizes and weights them using a matrix that sets saturated pixels to zero, and then combines these weighted images to generate a High Dynamic Range (HDR) image, allowing for independent pre-processing without altering subsequent data processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single exposure time is used for capturing raw phase images, then the processing is simple and fast, but saturation effects occur in near-range measurements and noise increases in far-range measurements
Solution Approach 1:
The patent segments the imaging process by capturing multiple raw phase images with different exposure times (short and long exposure times) to handle different distance ranges separately. Saturated pixels from short exposure images and non-saturated pixels from long exposure images are identified and combined, allowing accurate measurement across the full dynamic range without overwhelming the entire processing system
Solution Approach 2:
The patent adds the time dimension by capturing images at multiple exposure times, transforming a single 2D spatial image into a 3D data structure across time/exposure dimensions. This allows the system to select and combine pixel data from different temporal snapshots, effectively extending the dynamic range while maintaining processing feasibility through systematic combination rules
2Illumination intensity
If multiple exposure times are used to extend dynamic range, then saturation effects are prevented, but the processing complexity and computational load increase
Solution Approach 1:
The patent applies local quality by treating different pixels differently based on their saturation status. For each pixel position, the system evaluates whether it is saturated in short exposure images or long exposure images, and selectively combines data from appropriate exposure times. This localized approach extends dynamic range pixel-by-pixel while avoiding unnecessary processing of already-valid pixels
Solution Approach 2:
The patent performs preliminary identification of saturated and non-saturated pixels in each exposure group before combining them. By pre-classifying pixel validity status and creating masks for saturated versus non-saturated regions, the system prepares the data structure in advance, making the subsequent combination process more efficient and less computationally intensive
3Reliability
If multiple raw phase images are combined with different exposure times, then the dynamic range is extended, but the processing time increases
Solution Approach 1:
The patent discards saturated pixel data from inappropriate exposure images and recovers valid measurement information by combining only the non-saturated pixels from each exposure group. This selective discarding and recovering approach extends dynamic range by utilizing valid data from both short and long exposure images without processing redundant or invalid information, thus reducing unnecessary computational overhead
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach effectively extends the dynamic range of time-of-flight cameras, utilizing all pixel information while preventing saturation, and allows for conventional processing of the resulting HDR images, maintaining original resolution and enabling accurate distance calculations.
Implementation Method 1
systems that obtain distances from a phase shift of emitted and received radiation
Implementation Method 2
The term time-of-flight camera or time-of-flight camera system is intended here to cover, in particular, systems that obtain distances from a phase shift of emitted and received radiation
Data Source
AI summary
The invention relates to a method for the dynamic extension of raw phase images of a time-of-flight camera or time-of-flight camera system, in which method at least two depth images are taken using different exposure times.