Multi-frame Depth-of-Field Processing for Selective Image Deblurring
Solution Overview
Problem
Image capture devices with large aperture lenses suffer from limited depth of field, resulting in portions of an image being blurry, especially when capturing scenes with objects at varying distances, making it difficult to keep both foreground and background objects in focus.
Innovation Solution
The implementation of multi-frame depth-of-field (MF-DOF) processing, which involves capturing two image frames at different focal distances and using AI-based deblurring techniques to align and enhance regions of interest, thereby increasing the apparent depth of field without merging the frames, thus reducing blur and improving image clarity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If a large aperture lens is used to capture images, then light gathering capability is improved, but depth of field is reduced causing blur in portions of the image
Solution Approach 1:
The patent segments the image into multiple regions of interest (ROIs) at different depths and processes each region separately using corresponding image frames focused at different focal distances. This allows selective enhancement of different depth planes while maintaining the benefits of large aperture photography.
Solution Approach 2:
The patent introduces a temporal dimension by capturing multiple image frames at different focal distances and combining them through multi-frame depth-of-field (MF-DOF) processing. This transforms a single 2D image limitation into a 3D+time solution where depth information is extracted across multiple frames.
2Manufacturing precision
If multiple image frames are merged to increase depth of field, then focus coverage is improved, but artifacts such as ghosting and broken lines are introduced
Solution Approach 1:
The patent applies local quality by performing deblurring operations selectively on identified regions of interest rather than uniformly processing the entire image. This localized approach reduces artifacts by only enhancing areas that require it while preserving natural appearance in other regions.
Solution Approach 2:
The patent replaces traditional mechanical merging of multiple frames with an AI-based deblurring system. Instead of physically combining frames which causes misalignment artifacts, the system uses neural networks to analytically reconstruct sharp regions from blurred inputs, substituting mechanical fusion with intelligent processing.
3Manufacturing precision
If AI-based deblurring is applied to increase depth of field, then apparent focus is improved, but computational complexity increases
Solution Approach 1:
The patent performs preliminary action by pre-identifying regions of interest and pre-aligning multiple image frames before applying AI deblurring. This preparation work reduces the computational burden during the actual deblurring process by limiting processing to only necessary regions and providing pre-processed input data.
Solution Approach 2:
The patent applies partial action by selectively applying AI deblurring only to identified regions of interest rather than processing the entire image frame. This selective approach significantly reduces computational complexity while maintaining the effectiveness of depth of field enhancement where it matters most.
Data Source
AI summary
The present disclosure provides systems, apparatus, methods, and computer-readable media that support multi-frame depth-of-field (MF-DOF) for deblurring background regions of interest (ROIs), such as background faces, that may be blurred due to a large aperture size or other characteristics of the camera used to capture the image frame. The processing may include the use of two image frames obtained at two different focus points corresponding to the multiple ROIs in the image frame. The corrected image frame may be determined by deblurring one or more ROIs of the first image frame using an AI-based model and/or local gradient information. The MF-DOF may allow selectively increasing a depth-of-field (DOF) of an image to provide focused capture of multiple regions of interest, without causing a reduction in aperture (and subsequently an amount of light available for photography) or background blur that may be desired for photography.