On-Device Low-Light Video from Sliding-Window Frame Merging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing astrophotography techniques using mobile devices face challenges such as thermal noise due to image sensor heating, require off-device post-processing, and lack on-device video generation capabilities, especially in extreme low-light conditions like the night sky.
Innovation Solution
A mobile device captures a series of long-exposure images, applies a sliding window for merging and alignment, and performs real-time image processing to generate video frames, which can be streamed or stored, while monitoring sensor temperature to prevent thermal noise.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If long-exposure images are captured to improve low-light visibility, then image brightness is improved, but thermal noise increases due to image sensor heating
Solution Approach 1:
The system captures multiple images with different exposure times in a periodic sequence, alternating between longer exposures for brightness and shorter exposures for thermal noise reference, then processes these periodic captures to produce high-quality low-light images
Solution Approach 2:
A second reference image captured with a shorter exposure time serves as an intermediary to measure thermal noise independently, allowing the system to subtract thermal noise from the longer exposure image while preserving the desired brightened scene details
2Manufacturing precision
If multiple images are captured and merged to reduce thermal noise, then image quality is improved, but processing time increases
Solution Approach 1:
The system performs alignment and merging operations on captured images in real-time during the capture sequence itself, rather than as a separate post-processing step, allowing preliminary processing to occur while capturing continues
Solution Approach 2:
The image capture and processing operations continue continuously without interruption - images are captured, aligned, and merged in an ongoing stream, allowing the system to maintain continuous useful action rather than stopping for batch processing
3Manufacturing precision
If professional astrophotography equipment is used to improve image quality, then image quality is improved, but device cost and complexity increase
Solution Approach 1:
The mobile device performs its own image processing, alignment, and merging operations independently without requiring external professional equipment or software, making the device self-sufficient for astrophotography processing
Solution Approach 2:
Complex mechanical astrophotography equipment is replaced with computational methods running on a mobile device - image alignment uses algorithmic approaches rather than mechanical precision mounts, and image merging uses software processing rather than specialized hardware
Data Source
AI summary
An example embodiment may involve capturing a sequence of images, wherein there are 4 or more images in the sequence of images, and wherein each of the sequence of images has an exposure length of 4-100 seconds; applying a sliding window over the sequence of images as downsampled, wherein at least 4 images are encompassed within the sliding window’, and wherein for each position of the sliding window the applying involves: (i) aligning a set of images within the sliding window, and (ii) merging the set of images as aligned into a video frame; combining video frames generated by way of the sliding window into a video file; and storing, by the mobile device, the video file in memory of the mobile device.