Light Field Camera Array Extended Depth of Field
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems, such as plenoptic cameras and camera arrays, face challenges in producing images with extended depth of field, often resulting in noisy images or requiring complex depth estimation and significant computing power, which limits their ability to achieve realistic eye-to-eye contact simulation and real-time imaging applications.
Innovation Solution
The method involves using an array of image capturing devices to generate synthetically focused images at various depths, integrating these images to form a blurred image, and then deconvolving it using an inverse point spread function to produce a virtual output image with extended depth of field, allowing for flexible virtual camera positions without the need for depth models or complex algorithms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If plenoptic camera or camera array is used to capture light field information, then depth information and multiple viewpoints are obtained, but the spatial resolution is restricted and the depth of field is limited
Solution Approach 1:
The image sensor is divided into multiple sensor elements, each corresponding to a specific microlens. This segmentation allows each sensor element to capture light from a specific angular direction, enabling the reconstruction of depth information and multiple viewpoints while maintaining acceptable spatial resolution through computational processing
Solution Approach 2:
The patent transitions from capturing only spatial information (2D image) to capturing light field information by adding the angular dimension. The microlens array encodes directional information of incoming light rays, transforming the imaging system from capturing intensity distribution to capturing both spatial and angular distribution of light
2Manufacturing precision
If digital photomontage technique is applied to produce extended depth of field, then multiple focused images are combined, but the resulting image quality is dependent on depth map estimation and requires complex processing
Solution Approach 1:
The patent replaces complex mechanical focusing mechanisms and depth map estimation algorithms with a computational approach based on light field integration. By integrating light field data across multiple angular directions, the system achieves extended depth of field through mathematical operations rather than mechanical adjustment or complex image processing
Solution Approach 2:
The system changes the parameter being measured from single-point focus to integrated light field distribution. Instead of attempting to estimate depth maps or mechanically adjust focus, the patent integrates light field information across multiple angular directions, transforming the problem from depth estimation to light field synthesis
3Manufacturing precision
If multiple images are processed to create extended depth of field, then depth information is utilized, but computing power requirements increase significantly
Solution Approach 1:
The microlens array performs preliminary optical processing by spatially separating light rays according to their angular directions before they reach the sensor. This pre-processing encodes depth and viewpoint information in the spatial arrangement of light on the sensor, reducing the computational burden during image reconstruction compared to processing raw multi-view images
4Adaptability or versatility
If conventional camera array is used, then multiple viewpoints are captured, but realistic eye-to-eye contact simulation and real-time imaging are limited
Solution Approach 1:
The light field camera provides universal imaging capability by capturing complete light field information that can be processed to generate images from any virtual viewpoint and with any virtual aperture settings. This multi-functionality enables both eye-to-eye contact simulation in teleconferencing and real-time imaging applications without requiring separate hardware configurations
Data Source
Figure 1
Figure 2A~2B
Figure 3
AI summary
In a method and system for providing virtual output images from an array of image capturing devices image data (Ii(s,t), I2(s,t)) is taken from the devices (Ci, C2). This image data is processed by convolving the image data with a function, e.g. the path (S) and thereafter deconvolving them, either after or before summation (SUM), with an inverse point spread function (IPSF) or a filter (HP) equivalent thereto to produce all- focus image data (I0(s,t)).