Low-Bit-Depth Image Sensor Frames for High-Bit-Depth Imaging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Quanta image sensors generate high shot noise due to their short duration, resulting in low bit depth frames with high noise levels, which are not suitable for generating high-quality digital images.
Innovation Solution
A system comprising an image sensor that generates low bit depth frames, processed by a machine learning model with a three-dimensional convolutional layer, a two-dimensional convolutional LSTM layer, and a concatenation layer, which generates high bit depth images by merging information from multiple frames, reducing noise and enhancing image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If quanta image sensors are used to detect individual photons, then read noise is reduced and temporal granularity is improved, but shot noise increases due to short duration frames
Solution Approach 1:
The patent combines multiple low bit depth frames into a single high bit depth image by merging the binary data from multiple short-duration exposures. This accumulation of photons across multiple frames reduces shot noise while preserving the high temporal granularity advantage of quanta sensors.
Solution Approach 2:
The patent creates multiple copies of the scene through repeated short-duration measurements (multiple frames) and then combines them. Each frame captures photon arrivals independently, and the combination process reconstructs a high-quality image that leverages the low read noise characteristic while mitigating the shot noise problem.
2Manufacturing precision
If multiple low bit depth frames are processed to generate high bit depth images, then image quality is improved, but processing complexity increases
Solution Approach 1:
The patent replaces complex traditional image processing algorithms with a trained neural network model. The neural network automatically learns the optimal combination strategy for merging multiple low bit depth frames into high bit depth images, simplifying the processing pipeline while maintaining high image quality.
Solution Approach 2:
The patent transforms the bit depth parameter from low (binary or few bits) to high (8 bits or more) through the neural network processing. This parameter transformation is achieved by learning the statistical relationships between multiple low bit depth frames and generating a high bit depth output that preserves fine intensity variations.
3Duration of action of moving object
If short duration frames are used to capture photon arrivals, then temporal granularity is improved, but shot noise increases
Solution Approach 1:
The patent performs multiple preliminary measurements (short duration frames) before generating the final image. Each preliminary frame captures photon arrivals with high temporal granularity, and the subsequent combination process integrates these preliminary measurements to produce a final image with reduced shot noise while preserving temporal information.
Data Source
AI summary
In accordance with some embodiments, systems, methods, and media for generating digital images using low bit depth image sensor data are provided. In some embodiments, the system comprises: an image sensor; a processor programmed to: receive, from the image sensor, a series of low bit depth frames; provide low bit depth image information to a trained machine learning model comprising: a 3D convolutional layer; a 2D convolutional LSTM layer; a concatenation layer configured to generate a tensor that includes an output of the 2D convolutional LSTM layer and the low bit depth image information; and a 2D convolutional layer configured to generate an output based on the tensor; and generate a high bit depth image of a scene based on an output of the two-dimensional convolutional layer.