Light Field Camera Refocus Before Color Interpolation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional light field photography methods suffer from reduced sharpness in composite images due to color interpolation processing, which blurs sub-images and results in a final image with compromised sharpness.
Innovation Solution
An image processing device that generates composite images using multi-viewpoint data before color interpolation, featuring a pixel position determination unit, color derivation unit, and pixel value calculation unit to determine pixel positions and colors based on optical parameters, allowing for sharp image composition without interpolation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If color interpolation processing is performed before refocus processing, then the missing color information in sensor output pixels is completed, but each part of the multi-viewpoint image becomes blurred and sharpness is reduced
Solution Approach 1:
The patent performs refocus processing on multi-viewpoint image data before color interpolation processing. By reversing the conventional sequence and performing the geometric transformation (refocus) first on the raw sensor data, the patent avoids blurring during the refocus operation. Color interpolation is then applied to the already-sharpened refocused image, completing the color information without degrading sharpness. This sequence reversal resolves the contradiction between completing color information and maintaining image sharpness.
2Loss of information
If refocus processing is performed after color interpolation, then color complete images are available for processing, but blurring occurs during interpolation and is reflected in the composite image
Solution Approach 1:
The patent inverts the conventional processing sequence by performing refocus processing before color interpolation processing. Instead of following the traditional approach of interpolating colors first and then performing geometric transformations, the patent applies the refocus operation directly to the raw multi-viewpoint image data from the sensor. This inversion ensures that sharpness is preserved during the geometric transformation, and color interpolation is applied afterward to the already-sharpened image, thereby resolving the blurring issue in composite images.
3Manufacturing precision
If multi-viewpoint image data is captured multiple times for different focus positions, then focused images of multiple subjects at different depths can be obtained, but the number of image capturing operations increases
Solution Approach 1:
The patent employs a light field camera that captures multi-viewpoint image data in a single shot by utilizing a microlens array placed in front of the image sensor. This adds a spatial dimension (multiple viewpoints) to the captured data, enabling post-capture refocus processing to generate sharp images of subjects at different depths from one captured frame. This dimensional enhancement eliminates the need for multiple captures while maintaining focus accuracy for multiple subjects.
Solution Approach 2:
The patent captures all necessary multi-viewpoint image data in advance using a light field camera with a microlens array. By pre-capturing the light field information that contains directional and spatial data from multiple viewpoints, the system enables subsequent refocus processing to generate sharp images of subjects at various depths without requiring additional capture operations. This preliminary capture of comprehensive data improves capturing efficiency while maintaining focus precision.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables the acquisition of sharp composite images by calculating pixel values directly from multi-viewpoint data, preventing blurring and maintaining image sharpness.
Implementation Method 1
a microlens array is placed behind a main lens
Data Source
AI summary
An image processing device for generating a composite image using multi-viewpoint image data before color interpolation captured by a camera array image capturing device, includes a unit configured to acquire information of a pixel value and a pixel position in the multi-viewpoint image data, a pixel position determination unit configured to determine the pixel position in the composite image in the pixel position of each pixel of the multi-viewpoint image data in accordance with an arbitrary focus position based on optical parameters at the time of the image capturing, a color derivation unit configured to derive the color of each pixel of the multi-viewpoint image data, and a pixel value calculation unit configured to calculate the pixel value of each pixel of the composite image using the determined pixel position in the composite image and the pixel value of the multi-viewpoint image data corresponding to the derived pixel color.