Deep Learning Focus Fusion for Extended Depth of Field
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Handheld imaging devices, such as smartphones, struggle with shallow depth of field due to uncontrolled variations in field of view and focal settings, leading to unsatisfactory results from techniques like focus bracketing and focus stacking.
Innovation Solution
A neural network processes a sparse set of input images captured with different focal settings, aligning and combining them to generate an output image with an extended depth of field, using preprocessing to correct geometric and photometric variations, and training on a ground-truth image set.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If multiple images are captured with different focal depths to extend depth of field, then the depth of field is extended, but the device complexity increases due to uncontrolled variations in field of view and focal settings
Solution Approach 1:
The patent replaces complex mechanical control systems for adjusting focal depths and field of view with a neural network-based computational approach. The neural network automatically processes multiple input images with varying focal settings to generate an output image with extended depth of field, eliminating the need for precise mechanical control of camera parameters.
Solution Approach 2:
The neural network acts as an intermediary that processes the uncontrolled variations in field of view and focal settings between multiple input images. It learns to align and combine these variations to produce a consistent output image with extended depth of field, transforming uncontrolled variability into a useful feature.
2Manufacturing precision
If focus bracketing and focus stacking techniques are used in handheld devices, then the depth of field can be extended, but the image quality deteriorates due to uncontrolled variations in field of view and focal settings
Solution Approach 1:
The patent replaces traditional focus bracketing and stacking techniques with a neural network-based approach that automatically handles the complexity of combining multiple images. The neural network learns optimal alignment and fusion strategies, producing superior image quality compared to conventional methods by adapting to the specific characteristics of each input image.
Solution Approach 2:
The neural network dynamically adjusts processing parameters based on the input images' characteristics, including field of view variations and focal depth differences. This adaptive parameter adjustment allows the system to maintain high image quality across diverse shooting conditions, unlike fixed-parameter traditional methods.
3Measurement precision
If a high numerical aperture camera is used to achieve sharper focus, then the focus sharpness is improved, but the depth of field becomes shallow
Solution Approach 1:
The patent merges multiple input images captured at different focal depths into a single output image with extended depth of field. By combining information from images with different focus settings, the system achieves both sharp focus and extended depth of field, resolving the trade-off inherent in using high numerical aperture cameras alone.
Data Source
AI summary
A method for image enhancement includes capturing multiple input images of a scene, including at least a first input image having a first field of view (FOV) captured with a first focal depth and a second input image having a second FOV captured with a second focal depth. The input images in the sequence are preprocessed so as to align the images. The aligned images are processed in a neural network, which generates an output image having an extended depth of field encompassing at least the first and second focal depths.