ToF Imaging Circuitry Multiphase Sub-exposure Accumulation

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current time-of-flight (ToF) imaging systems require multiple exposures to calculate depth images, leading to motion artifacts when objects or the camera are moving, and previous solutions with more taps have not achieved similar quality due to mismatched components, increasing bandwidth and memory requirements.

Innovation Solution

An electronic device with circuitry that accumulates multiple sub-exposures with same phase data in an associated phase memory during a multiphase exposure, allowing for reduced motion artifacts and improved usability in dynamic environments by enabling single-exposure depth imaging.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If multiple exposures are used to calculate depth images, then measurement precision is improved, but loss of time increases and motion artifacts occur

Engineering Contradiction:
Improvedepth image qualityVSAvoidexposure time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent divides the correlation data acquisition into multiple sub-exposures within a single exposure period, where each sub-exposure captures data for a specific phase. This segmentation allows parallel accumulation of phase data (I, Q, Ib, Qb) across multiple sub-exposures while maintaining a single continuous exposure, thereby improving depth measurement precision without increasing total exposure time or causing motion artifacts.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent employs periodic modulation of the light source and synchronous detection across multiple sub-exposures, where each sub-exposure corresponds to a specific phase of the modulation cycle. This periodic action enables the system to accumulate correlation data for all four phases through repeated cycling, achieving high measurement precision while completing all measurements within a single exposure period.

Inventive Principle:
Principle #19Periodic action

2Loss of time

If four-tap modulator is used for single exposure depth imaging, then loss of time is reduced, but measurement precision deteriorates due to component mismatch

Engineering Contradiction:
Improveexposure timeVSAvoiddepth image quality
Core Design Contradiction:
Loss of timeVSMeasurement precision

Solution Approach 1:

The patent extracts only the necessary phase information from each sub-exposure and accumulates it in dedicated phase memory locations. By selectively capturing and storing only the relevant correlation data for each phase (I, Q, Ib, Qb) across multiple sub-exposures, the system achieves single-exposure depth imaging with high measurement precision, avoiding the component mismatch problems of four-tap modulators while maintaining time efficiency.

Inventive Principle:
Principle #2Taking out (Extraction)

Solution Approach 2:

The patent merges the data from multiple sub-exposures by accumulating correlation data for the same phase across different sub-exposures in phase memory. This combining of data from multiple temporal instances allows the system to achieve high signal-to-noise ratio and measurement precision equivalent to multiple exposures, while completing all measurements within a single continuous exposure period.

Inventive Principle:
Principle #5Merging (Combining)

3Device complexity

If sequential correlation data acquisition is used, then device complexity is reduced, but productivity decreases

Engineering Contradiction:
Improvepixel architectureVSAvoidframe rate
Core Design Contradiction:
Device complexityVSProductivity

Solution Approach 1:

The patent implements continuous accumulation of correlation data across multiple sub-exposures within a single exposure period, without interrupting the light source modulation or the measurement process. This continuous action allows the system to accumulate all four phase data points simultaneously across sub-exposures, dramatically improving productivity and frame rate while maintaining relatively simple pixel architecture compared to parallel multi-tap solutions.

Inventive Principle:
Principle #20Continuity of useful action

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This approach reduces motion artifacts and increases the quality of depth frames by allowing for pseudo-parallel correlation data acquisition, enhancing the robustness and efficiency of ToF imaging systems in dynamic conditions.

Implementation Method 1

A typical number of 4 frames is required to calculate the depth image. The modulator is typically a 2-tap electro-optical modulator

Methodology Applied
Scientific EffectPhotoelectric Effect: Photoelectric Effect

Data Source

PatentUS11726207B2Imaging device and method
Publication Date: 2023.08.15 SONY SEMICON SOLUTIONS CORP
  • US11726207B2 patent drawing
  • US11726207B2 patent drawing
  • US11726207B2 patent drawing

AI summary

An electronic device comprising circuitry configured to accumulate, in a multiphase exposure, multiple sub-exposures with same phase data in an associated phase memory.