Shared Storage Node Global-Shutter Image Sensor for High-Resolution Depth Sensing

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Indirect Time-of-Flight (iTOF) depth sensing systems face challenges in achieving high spatial resolution for 2D imaging due to the need for dual storage nodes per cell, which increases the pitch of sensing elements and limits the resolution of 2D images, while also requiring multiple acquisition phases to measure phase shifts in the carrier wave.

Innovation Solution

An optical sensing array with a matrix of sensing elements and storage nodes arranged such that each storage node is shared between neighboring sensing elements, allowing photocharge to be transferred during different detection intervals, enabling high-resolution 2D imaging and depth sensing with reduced pitch and comparable signal-to-noise ratio to conventional iTOF sensors.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If dual storage nodes per cell are used in conventional iTOF systems, then depth sensing capability is achieved, but the pitch of sensing elements increases and spatial resolution for 2D imaging deteriorates

Engineering Contradiction:
Improvedepth sensing capabilityVSAvoidspatial resolution
Core Design Contradiction:
Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

The patent merges the functionality of dual storage nodes into a shared storage node configuration where neighboring sensing elements share common storage nodes. This combining approach reduces the total number of storage nodes required per cell, thereby reducing pitch and improving spatial resolution while maintaining depth sensing capability through temporal multiplexing of the shared nodes.

Inventive Principle:
Principle #5Merging (Combining)

Solution Approach 2:

The patent introduces dynamic switching circuitry that dynamically connects sensing elements to different shared storage nodes based on the detection interval. This dynamic reconfiguration allows the system to achieve depth sensing functionality equivalent to dual storage nodes while using fewer physical storage nodes, thus reducing pitch and improving spatial resolution.

Inventive Principle:
Principle #15Dynamics

2Measurement precision

If multiple acquisition phases are used to measure phase shifts, then depth mapping accuracy is improved, but the complexity of the detection system increases

Engineering Contradiction:
Improvedepth mapping accuracyVSAvoiddetection system complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent makes the shared storage nodes multi-functional by using them for both 2D imaging and depth sensing through temporal multiplexing. The same storage nodes that would traditionally be dedicated to depth sensing are now shared with imaging functions, reducing overall system complexity while maintaining measurement precision through phase-shift detection across multiple acquisition phases.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The switching circuitry automatically manages the allocation of shared storage nodes to different sensing elements based on detection interval requirements. This self-managing system reduces the need for complex external control mechanisms while maintaining the ability to perform multiple acquisition phases for accurate depth mapping.

Inventive Principle:
Principle #25Self-service

3Manufacturing precision

If shared storage nodes are used between neighboring sensing elements, then the pitch of sensing elements is reduced and spatial resolution is improved, but the switching circuitry complexity increases

Engineering Contradiction:
Improvespatial resolutionVSAvoidswitching circuitry complexity
Core Design Contradiction:
Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent segments the switching control into modular units where each sensing element has its own switching circuitry that controls connection to shared storage nodes. This segmentation allows for scalable implementation and reduces overall complexity by distributing the switching control logic across multiple independent units rather than requiring a complex centralized switching system.

Inventive Principle:
Principle #1Segmentation

4Manufacturing precision

If high spatial resolution is achieved through reduced pitch, then 2D imaging quality is improved, but the signal-to-noise ratio may deteriorate due to smaller pixel area

Engineering Contradiction:
Improvespatial resolutionVSAvoidsignal-to-noise ratio
Core Design Contradiction:
Manufacturing precisionVSReliability

Solution Approach 1:

The patent implements continuous integration of photocharge in the shared storage nodes across multiple detection intervals. This continuous accumulation of signal over time compensates for the reduced pixel area, maintaining signal-to-noise ratio while achieving high spatial resolution through reduced pitch. The continuous useful action of charge integration ensures that sufficient signal is collected despite smaller sensing element areas.

Inventive Principle:
Principle #20Continuity of useful action

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

The solution allows for high-resolution 2D imaging and depth mapping with comparable signal-to-noise ratio and spatial resolution to conventional iTOF systems, while maintaining the ability to capture 3D iTOF data, by interleaving storage nodes and using switching circuitry to synchronize detection intervals with the carrier frequency.

Implementation Method 1

a first matrix of optical sensing elements, arranged on the substrate in rows and columns... each of the sensing elements and the first neighboring sensing element are connected together to the respective first storage node

Methodology Applied
Scientific EffectPhotoelectric effect: Photoelectric Effect

Implementation Method 2

an illumination assembly, which is configured to direct optical radiation toward a target scene while modulating the optical radiation with a carrier wave having a predetermined carrier frequency

Methodology Applied
Scientific EffectAmplitude modulation: Phase Modulation

Implementation Method 3

Control circuitry measures the time delay between the transmitted and received light pulses at each point in the scene, which is indicative of the distance traveled by the light beam, and hence of the depth of the object at the point

Methodology Applied
Scientific EffectTime of flight: Time of Flight

Data Source

PatentUS11558569B2Global-shutter image sensor with time-of-flight sensing capability
Publication Date: 2023.01.17 APPLE INC
  • US11558569B2 patent drawing
  • US11558569B2 patent drawing
  • US11558569B2 patent drawing

AI summary

Apparatus for optical sensing includes first matrix of optical sensing elements, arranged on a semiconductor substrate in rows and columns. A second matrix of storage nodes is arranged on the substrate such that respective first and second storage nodes in the second matrix are disposed in proximity to each of the sensing elements within the first matrix. Switching circuitry couples each of the sensing elements to transfer photocharge to the respective first and second storage nodes. Control circuitry controls the switching circuitry in a depth sensing mode such that over a series of detection cycles, each of the sensing elements and a first neighboring sensing element are connected together to the respective first storage node during the first detection interval, and each of the sensing elements and the second neighboring sensing element are connected together to the respective second storage node during the second detection interval.