Camera Deblurring with Reconstructed Depth Maps for Extended Depth-of-Field
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image capturing technologies face challenges with depth-of-field issues, leading to low-quality and unrealistic images, especially in low-light conditions and when auto-focus cameras are used, and depth cameras provide unreliable depth information, resulting in inaccurate image correction.
Innovation Solution
A method and system using a reconstructed depth map to apply extended depth-of-field correction by determining a point spread function based on optical depths, allowing for accurate image deblurring without relying on depth cameras.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If the aperture size is significantly smaller to improve low-light capture, then image quality in low-light settings improves, but depth-of-field range becomes limited and images outside focusing distance range become blurred
Solution Approach 1:
The patent creates a virtual copy of the depth information by reconstructing a depth map from 3D model data and 2D image data. This reconstructed depth map is then used to generate virtual out-of-focus blur that matches the optical characteristics of the camera, allowing the system to simulate the depth-of-field effect without being constrained by the physical aperture limitations of the actual camera hardware.
Solution Approach 2:
The patent replaces the mechanical aperture control system with a computational approach. Instead of physically adjusting the aperture to control depth-of-field, the system uses image processing algorithms that simulate the optical blur effect based on reconstructed depth information, thereby decoupling the mechanical aperture from the depth-of-field control.
2Manufacturing precision
If depth cameras are used to capture depth information for image correction, then image deblurring capability improves, but reliability and accuracy of depth information deteriorates due to similar depth-of-field issues
Solution Approach 1:
The patent introduces a 3D model as an intermediary that contains accurate depth information independent of the camera's optical limitations. This 3D model serves as a mediator between the captured 2D image and the desired corrected output, providing reliable depth data that can be used to compute accurate blur kernels without being affected by the depth camera's depth-of-field constraints.
Solution Approach 2:
The system performs preliminary reconstruction of the depth map using 3D model data before applying the deblurring operation. This preliminary action establishes accurate depth information in advance, which is then used to compute the appropriate blur correction for each region of the image, ensuring that the deblurring process benefits from reliable depth data rather than unreliable real-time depth camera output.
3Speed
If auto-focus camera is used to capture images, then focusing speed improves, but image quality deteriorates because only one focusing distance range can be captured sharp at a time
Solution Approach 1:
The patent applies dynamic blur correction where the blur kernel is computed differently for different depth regions. Instead of a uniform blur applied to the entire image, the system dynamically adjusts the blur characteristics based on the reconstructed depth map, allowing each region at different distances to have its blur corrected according to its specific optical characteristics, thereby achieving extended depth-of-field effect.
Data Source
Figure 1~2
Figure 3
AI summary
Disclosed is computer-implemented method comprising: obtaining 3D model of real-world environment; receiving image of real-world environment captured using camera (106, 208), and pose information indicative of camera pose from which image is captured; utilising 3D model of real-world environment to generate reconstructed depth map from perspective of camera pose; and applying extended depth-of-field correction to image segment(s) of image that is/are out of focus, by using point spread function determined for camera, based on optical depths in segment(s) of reconstructed depth map corresponding to image segment(s) of image.