SPAD Image Persistence Using Pose Data for Low-Light Imaging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional CMOS and CCD image sensors suffer from high read noise and motion blur under low light conditions, particularly in mixed-reality systems like HMDs, which affects pass-through imaging and other operations.
Innovation Solution
Implementing a SPAD image sensor array with a persistence mechanism that combines multiple image frames to generate a composite image, using pose data and signal strength analysis to dynamically adjust the contribution of each frame, thereby reducing noise and motion artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If conventional CMOS or CCD image sensors are used for low light imaging, then the system can capture images in dark environments, but the read noise dominates the detected signal and degrades image quality
Solution Approach 1:
The patent combines multiple sequentially captured image frames into a single composite image. By accumulating photon detections across multiple frames, the signal accumulates linearly while read noise, being random and uncorrelated between frames, accumulates only as the square root of the number of frames. This merging approach significantly improves the signal-to-noise ratio in low light conditions.
Solution Approach 2:
The patent implements continuous imaging at high frame rates, capturing multiple frames in rapid succession without interruption. This continuous capture allows the system to accumulate sufficient photon signals over time while maintaining high temporal resolution, enabling low light imaging without sacrificing frame rate.
2Measurement precision
If the frame rate is reduced to allow conventional sensors to detect enough photons, then the signal to noise ratio improves, but motion blur increases in captured images
Solution Approach 1:
The patent merges multiple short-exposure frames captured at high frame rates into a single composite image. Each individual frame remains sharp and free from motion blur due to the short exposure time, while the combination of multiple frames accumulates sufficient photon signals to achieve high signal-to-noise ratio without requiring long exposures.
Solution Approach 2:
The patent uses periodic shutter operations to capture a sequence of image frames at high frame rates. The shutter opens and closes rapidly to capture multiple frames in succession, each with a short exposure time that freezes motion, while the periodic repetition allows accumulation of signal over time.
3Measurement precision
If multiple image frames are captured and combined to improve signal to noise ratio, then low light imaging quality improves, but the computational complexity and processing time increase
Solution Approach 1:
The patent extracts and utilizes pose data from inertial measurement units (IMUs) to determine persistence terms, rather than relying solely on complex image processing algorithms. This extraction of motion information from a separate sensor system simplifies the image combination process by providing direct motion compensation data.
Solution Approach 2:
The system uses its own IMU sensors to generate persistence terms for image combination, making the system self-sufficient. The pose data from the IMUs directly informs how to weight and combine frames, eliminating the need for external complex processing or additional sensors.
4Measurement precision
If pose data and signal strength analysis are used to dynamically adjust frame contributions, then image quality and noise reduction improve, but the processing complexity increases
Solution Approach 1:
The patent dynamically changes the persistence terms (weighting factors) based on pose data and signal strength analysis. By adjusting these parameters according to actual motion and signal conditions, the system optimizes image quality and noise reduction while maintaining computational efficiency through simple parameter modulation rather than complex processing.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enhances image quality under low light conditions by eliminating read noise and reducing motion artifacts, allowing for high-framerate imaging without signal domination by noise, and improves operations like reprojection and object tracking.
Implementation Method 1
single photon avalanche diode (SPAD) pixels that detect photons
Implementation Method 2
single photon avalanche diode (SPAD) pixels that detect photons and that generate electrical signals
Data Source
AI summary
A system for adding persistence to SPAD imagery is configurable to capture, using a SPAD array, a plurality of image frames. The system is configurable to capture, using an IMU, pose data associated with the plurality of image frames. The pose data includes at least respective pose data associated with each of the plurality of image frames. The system is configurable to determine a persistence term based on the pose data. The system is also configurable to generate a composite image based on the plurality of image frames, the respective pose data associated with each of the plurality of image frames, and the persistence term. The persistence term defines a contribution of each of the plurality of image frames to the composite image.