Convolutional Neural Network Low-Light Imaging Brightness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional camera settings for low-light environments, such as adjusting shutter speed or ISO, often result in motion blur, noise, or reduced depth of field, making it difficult to capture bright images without significant drawbacks.
Innovation Solution
A system utilizing a trained convolutional neural network that processes raw sensor data to generate illuminated images with enhanced brightness, equivalent to higher illuminance levels, without increasing ISO, aperture, or shutter speed, using techniques like data packing, black level subtraction, and amplification, and employing architectures like multi-scale context aggregation networks or U-Nets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If shutter speed is decreased to capture more photons, then brightness is improved, but motion blur increases
Solution Approach 1:
The system performs preliminary actions by capturing multiple images at different exposure levels (including underexposed, correctly exposed, and overexposed) before final processing. This allows the neural network to selectively combine information from these pre-captured images to produce a bright output without motion blur, as the preliminary captures preserve the sharp details from the correctly exposed image while the overexposed image provides additional brightness information.
Solution Approach 2:
The system changes exposure parameters by capturing images at multiple exposure levels (different shutter speeds or ISO settings) and then uses a neural network to process these variations. The network learns to optimally combine these different parameter settings to achieve high brightness without the motion blur that would result from simply using a single long exposure.
2Illumination intensity
If ISO is increased to increase brightness, then illumination intensity is improved, but noise increases
Solution Approach 1:
The system creates multiple copies of the scene at different exposure levels and uses a neural network to synthesize a final image that copies the brightness characteristics of the overexposed image while copying the sharpness and low-noise characteristics of the correctly exposed image. The network learns to copy desirable features from different source images rather than simply amplifying a single noisy image.
Solution Approach 2:
The final image is composed by combining information from multiple source images with different exposure characteristics, similar to creating a composite material. The neural network synthesizes a composite image that integrates the brightness from overexposed captures with the quality (low noise, sharpness) from correctly exposed captures, achieving properties that neither single image possesses alone.
3Illumination intensity
If aperture is increased to capture more light, then brightness is improved, but depth of field is reduced
Solution Approach 1:
The system captures multiple preliminary images at different aperture settings or uses a single aperture setting that maintains adequate depth of field, then processes these images through a neural network to achieve the brightness of a wide aperture without sacrificing depth of field. The network learns to compensate for the reduced light intake by intelligently processing the preliminary captures.
4Illumination intensity
If traditional low-light settings are used, then brightness is improved, but image quality deteriorates due to motion blur and noise
Solution Approach 1:
The neural network acts as an intermediary between the raw sensor data from multiple exposure-level images and the final output image. It processes the intermediate representations from these images, learning to combine them in ways that achieve high brightness while maintaining image quality. The network mediates between the conflicting requirements of brightness and quality by synthesizing a result that possesses both properties.
Solution Approach 2:
The system transforms the parameter space by working in the domain of multiple exposure levels rather than directly adjusting single image parameters like ISO or shutter speed. The neural network learns optimal parameter combinations and transformations to achieve the desired brightness-quality tradeoff, effectively changing how the brightness parameter is controlled from direct physical adjustment to learned synthesis.
Data Source
AI summary
An example apparatus for imaging in low-light environments includes a raw sensor data receiver to receive raw sensor data from an imaging sensor. The apparatus also includes a convolutional neural network trained to generate an illuminated image based on the received raw sensor data. The convolutional neural network is trained based on images captured by a sensor similar to the imaging sensor.