Dual-Pixel Depth Mapping for Single-Exposure Background Blur
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image capture devices struggle to selectively blur background portions of images while keeping foreground objects in focus, especially when dedicated depth sensors are unavailable, and traditional depth estimation methods from dual pixel images are challenging due to limited cues and correspondence issues.
Innovation Solution
A machine learning approach using neural networks is employed to estimate depth maps from dual pixel images, leveraging parallax, semantic, and defocus cues, trained on a large dataset from a multi-camera device, to generate accurate depth maps that guide selective blurring of background objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional structured light methods are used for depth mapping, then depth information can be obtained, but the process is time-consuming and requires sequential projection of multiple patterns
Solution Approach 1:
The patent segments the depth information capture into two simultaneous channels: one for texture capture and another for depth capture using dual pixel technology. This allows parallel processing of different image types without sequential pattern projection, resolving the time loss issue while maintaining depth mapping capability.
Solution Approach 2:
The patent performs preliminary action by capturing both texture and depth information in a single exposure using dual pixel imaging. The depth information is pre-captured alongside texture data, eliminating the need for subsequent sequential pattern projections and enabling real-time depth mapping.
2Measurement precision
If multiple image captures are performed for depth estimation, then depth accuracy can be improved, but the frame rate decreases
Solution Approach 1:
The patent merges texture capture and depth capture into a single imaging operation using dual pixel technology. Both types of information are captured simultaneously in one exposure, maintaining frame rate while obtaining sufficient data for depth estimation through unified multi-task processing.
Solution Approach 2:
The patent performs preliminary action by capturing all necessary depth and texture information in a single exposure. This preliminary capture eliminates the need for multiple sequential captures, maintaining high frame rate while providing sufficient data for accurate depth estimation through advanced processing algorithms.
3Reliability
If depth information is captured using traditional methods, then depth mapping can be achieved, but real-time processing is difficult due to computational complexity
Solution Approach 1:
The patent replaces complex mechanical structured light projection systems with a computational approach using dual pixel imaging. Depth information is obtained through image processing of simultaneously captured photos, substituting physical pattern projection with algorithmic depth extraction, thereby reducing device complexity while maintaining depth mapping reliability.
Solution Approach 2:
The patent changes the parameter of information capture from sequential multi-pattern projection to simultaneous dual-channel imaging. By capturing both texture and depth information in a single exposure, the computational complexity is reduced while maintaining depth mapping capability through efficient processing of unified data.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This method enables natural and pleasing depth-dependent blurring of background objects while maintaining foreground clarity, even on mobile devices, without requiring network connectivity, and produces higher-quality selectively blurred images than traditional techniques.
Implementation Method 1
a lens is arranged to receive light from the object and project the light onto a sensor array
Implementation Method 2
the sensor array comprises a plurality of photosensitive elements arranged in a grid pattern
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Apparatus and methods related to using machine learning to determine depth maps for dual pixel images of objects are provided. A computing device can receive a dual pixel image of at least a foreground object. The dual pixel image can include a plurality of dual pixels. A dual pixel of the plurality of dual pixels can include a left-side pixel and a right-side pixel that both represent light incident on a single dual pixel element used to capture the dual pixel image. The computing device can be used to train a machine learning system to determine a depth map associated with the dual pixel image. The computing device can provide the trained machine learning system.