Dual-Pixel Defocus Deblurring and Depth Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Image blurring due to objects being outside the depth of field in camera images is challenging to correct using existing image processing techniques, as they often require ground-truth data and are not scene-independent.
Innovation Solution
Dual-pixel image data is used to generate an in-focus image and depth map by formulating a loss function based on relationships between sub-images and blur kernels, allowing for the training of mathematical models to deblur images and estimate depth without ground-truth data, using scene-independent blur kernels scaled to different size scales.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If existing image processing techniques are used to correct blurring, then image quality may be improved, but ground-truth data is required and the method is not scene-independent
Solution Approach 1:
The system uses itself to solve the problem by utilizing dual-pixel data from the same image capture to generate depth information and guide deblurring, eliminating the need for external ground-truth data or scene-specific calibration
Solution Approach 2:
The invention changes the parameter space by introducing depth as an additional dimension and using dual-pixel defocus cues to estimate depth, allowing the system to adapt to different scenes without requiring ground-truth data
2Adaptability or versatility
If dual-pixel image data is used with loss function optimization, then in-focus image and depth map can be generated without ground-truth data, but computational complexity increases
Solution Approach 1:
The computational problem is segmented into distinct components: dual-pixel data processing, depth map generation, blur kernel estimation, and image deblurring. This modular approach allows each component to be optimized independently and processed in a structured pipeline
Solution Approach 2:
The system performs preliminary depth estimation using dual-pixel defocus cues before executing the computationally intensive deblurring process. This preliminary action guides subsequent processing steps and reduces the overall computational burden by avoiding trial-and-error approaches
3Measurement precision
If blur kernels are applied at multiple size scales to generate cost volume, then depth estimation accuracy is improved, but processing time and computational resources increase
Solution Approach 1:
The system dynamically adjusts the application of blur kernels at multiple scales based on local depth variations and defocus characteristics. Rather than uniformly applying all scales to all regions, the processing adapts to local conditions, reducing unnecessary computations while maintaining depth estimation accuracy
Data Source
AI summary
A method includes obtaining dual-pixel image data that includes a first sub-image and a second sub-image, and generating an in-focus image, a first kernel corresponding to the first sub-image, and a second kernel corresponding to the second sub-image. A loss value may be determined using a loss function that determines a difference between (i) a convolution of the first sub-image with the second kernel and (ii) a convolution of the second sub-image with the first kernel, and/or a sum of (i) a difference between the first sub-image and a convolution of the in-focus image with the first kernel and (ii) a difference between the second sub-image and a convolution of the in-focus image with the second kernel. Based on the loss value and the loss function, the in-focus image, the first kernel, and/or the second kernel, may be updated and displayed.