4D Light Field Refocusing Without Demosaicking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for refocusing 4D light field data require demosaicking and do not effectively account for the geometry of the scene or multi-focal plenoptic cameras, leading to suboptimal sharpness and color artifacts in the refocused images.
Innovation Solution
A method for obtaining a refocused image from 4D raw light field data without demosaicking, using weighted pixel values from a neighborhood of pixels based on specific functions that adapt to the focus plane and geometry of the scene, ensuring reduced color artifacts and improved sharpness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If demosaicking is performed on 4D raw light field data before refocusing, then color information can be reconstructed, but color artifacts appear in the refocused images
Solution Approach 1:
The patent performs refocusing operations on the raw 4D light field data before demosaicking, rather than the conventional approach of demosaicking first. This preliminary refocusing action preserves color information integrity and avoids the propagation of demosaicking errors that cause color artifacts in the final image
Solution Approach 2:
The patent processes different color channels (R, G, B) separately through the refocusing operation, applying the refocusing formula independently to each color component. This segmentation approach prevents color channel interference and artifact generation that occurs when demosaicking is performed beforehand
2Productivity
If conventional refocusing methods are used on 4D raw light field data, then refocused images can be generated, but sharpness is suboptimal due to ignoring scene geometry
Solution Approach 1:
The patent incorporates scene geometry information (depth map) to adapt the refocusing process locally for different regions of the image. The refocusing formula uses depth values d(x,y) to adjust the weighting function W(k,l,x,y) based on the actual distance of scene objects, providing optimal sharpness for each local region rather than applying a uniform refocusing operation
Solution Approach 2:
The patent makes the refocusing process dynamic by using the depth map to continuously adjust refocusing parameters across different spatial locations and depth planes. The weighting function adapts dynamically to the scene geometry, allowing the system to optimize sharpness for objects at various depths rather than using a fixed refocusing approach
3Productivity
If conventional refocusing methods are used on 4D raw light field data, then refocused images can be generated, but the methods do not account for multi-focal plenoptic camera geometry
Solution Approach 1:
The patent develops a universal refocusing formula that works for both single-focal and multi-focal plenoptic cameras. The formula incorporates camera-specific parameters (focal lengths f_i of different microlens groups) that can be adjusted to match different camera configurations, making the method adaptable to various plenoptic camera types without requiring fundamentally different processing approaches
Data Source
Figure 1
Figure 2
Figure 3~5
AI summary
In one embodiment, it is proposed a method for obtaining a refocused image from 4D raw light field data for a given focus plane value g. The method is executed by an electronic device and is remarkable in that it comprises determining at least one pixel value, for at least one color component, of said refocused image at coordinates (k, l) E N2, said determining comprising: obtaining at least one projected pixel value with coordinates comprised in a neighborhood of said coordinates (k, l), for said at least one color component; weighting said at least one projected pixel value based on a function, said weighting outputting a weighted value; updating said at least one pixel value, for said at least one color component, of said refocused image at coordinates (k, l) with said weighted value.