Single-Image Dual-Pixel Synthesis for Defocus Deblurring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for reducing defocus blur in captured images are limited by the performance of depth of field map estimation and non-blind deconvolution, and require a two-stage approach that is computationally inefficient.
Innovation Solution
A single-encoder multi-decoder deep neural network is trained to perform dual-pixel image synthesis and deblurring using a latent space encoder and dual-pixel view decoders, optimizing for dual-pixel-loss and view difference loss functions, allowing single-image input processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a two-stage approach with defocus map estimation and non-blind deconvolution is used, then defocus blur reduction can be achieved, but processing time increases and performance is bounded by estimation accuracy
Solution Approach 1:
The patent merges the defocus map estimation and deconvolution operations into a single unified neural network model. The network simultaneously performs both tasks in one forward pass, eliminating the sequential two-stage processing and reducing overall computation time while maintaining performance through end-to-end optimization.
Solution Approach 2:
The patent replaces the traditional mechanical two-stage processing pipeline (estimation followed by deconvolution) with a single neural network inference operation. This substitution eliminates multiple processing steps and reduces time consumption while achieving comparable or superior deblurring performance.
2Reliability
If paired data capture is required for training, then accurate defocus deblurring can be achieved, but system complexity and data collection requirements increase
Solution Approach 1:
The patent uses synthetic dual-pixel data generated from single images through the neural network to train the model, rather than requiring actual paired captured data. The network learns to synthesize the relationship between single images and dual-pixel views, enabling accurate deblurring without complex paired data collection systems.
Solution Approach 2:
The patent introduces synthetic data as an intermediary between the simple single-image input and the complex deblurring output. The neural network learns the transformation from single images to dual-pixel views through synthetic training data, eliminating the need for complex paired data capture hardware and procedures.
3Ease of operation
If single-image input is used, then ease of operation improves, but available information for accurate deblurring is reduced
Solution Approach 1:
The patent replaces traditional information-extraction methods with a neural network that directly transforms single images into deblurred outputs through learned features. The network captures defocus information implicitly during training and applies it automatically during single-image inference, eliminating the need for explicit dual-pixel input data.
Solution Approach 2:
The patent changes the input representation from actual dual-pixel images to single images, and the network learns to compensate for the reduced information by synthesizing the necessary defocus characteristics during the transformation process. This parameter change enables ease of operation while maintaining deblurring accuracy through intelligent data synthesis.
Data Source
AI summary
A system and method of determining synthetic dual-pixel data, performing deblurring, predicting dual pixel views, and view synthesis. The method including: receiving an input image; determining synthetic dual-pixel data using a trained artificial neural network with the input image as input to the trained artificial neural network, the trained artificial neural network includes a latent space encoder, a left dual-pixel view decoder, and a right dual-pixel view decoder; and outputting the synthetic dual-pixel data. In some cases, determination of the synthetic dual-pixel data can include performing reflection removal, defocus deblurring, or view synthesis.