Subject-Aware Image Fusion for Low Light Motion Artifacts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In digital image processing, especially in low light conditions and long exposure image capture, existing image fusion techniques struggle to account for local motion and camera shake, leading to artifacts like ghosting and reduced image quality due to the selection of reference images.
Innovation Solution
An adaptive, subject-aware approach for image bracket selection and fusion, where images are selected based on sharpness and blink scores, and exposure times are adjusted relative to the presence of human or animal subjects, combining insights from Still Image Stabilization, Optical Image Stabilization, and High Dynamic Range capture modes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If multiple images are captured with long exposure times to reduce noise and improve signal-to-noise ratio, then image quality is improved, but motion artifacts and ghosting effects worsen due to object motion and camera shake during the extended capture interval
Solution Approach 1:
The system dynamically adjusts exposure times for different regions of the image based on detected motion. Static regions use longer exposure times to reduce noise, while regions with detected motion use shorter exposure times to avoid motion artifacts. This dynamic, region-specific exposure adjustment resolves the contradiction by allowing long exposures where safe and short exposures where needed.
Solution Approach 2:
The image is segmented into multiple regions with different exposure characteristics. The system divides the image into regions of interest (such as detected subjects) and background regions, then applies different exposure strategies to each segment. This segmentation allows the system to capture long exposures for stable background areas while using short exposures for areas with motion, eliminating ghosting artifacts.
2Manufacturing precision
If multiple images are captured and fused together to reduce noise, then signal-to-noise ratio is improved, but image registration accuracy worsens due to camera shake and rotation between captured images
Solution Approach 1:
The system introduces an intermediary stabilization mechanism that compensates for camera shake and rotation before image fusion. Optical Image Stabilization (OIS) or Electronic Image Stabilization (EIS) acts as a mediator that corrects for camera motion, aligning the captured images more accurately. This intermediary stabilization process enables accurate registration even when multiple images are captured in sequence, resolving the contradiction between noise reduction and registration accuracy.
3Illumination intensity
If exposure time is increased to capture more light in low light conditions, then image brightness is improved, but motion blur increases due to the extended capture duration
Solution Approach 1:
The system applies different exposure qualities to different regions of the image. Regions with detected subjects or motion receive shorter exposure times to prevent motion blur, while other regions receive longer exposure times to ensure sufficient brightness. This local quality differentiation resolves the contradiction by providing appropriate exposure characteristics for each region based on its specific requirements.
4Ease of operation
If standard image fusion is applied without subject awareness, then processing simplicity is maintained, but image quality worsens due to inadequate handling of regions with different motion characteristics
Solution Approach 1:
The system performs preliminary subject detection and motion analysis before the image fusion process. By identifying regions with motion or subjects in advance, the system can pre-determine which regions require special handling during fusion. This preliminary action enables the fusion algorithm to apply appropriate weighting and blending strategies for different regions, improving image quality without significantly complicating the overall processing pipeline.
Data Source
AI summary
Devices, methods, and computer-readable media are disclosed, describing an adaptive, subject-aware approach for image bracket selection and fusion, e.g., to generate high quality images in a wide variety of capturing conditions, including low light conditions. An incoming image stream may be obtained from an image capture device, comprising images captured using differing default exposure values, e.g., according to a predetermined pattern. When a capture request is received, it may be detected whether one or more human or animal subjects are present in the incoming image stream. If a subject is detected, an exposure time of one or more images selected from the incoming image stream may be reduced relative to its default exposure time. Prior to the fusion operation, one of the selected images may be designated a reference image for the fusion operation based, at least in part, on a sharpness score and/or a blink score of the image.