Light Field Camera Pixel Segmentation for Resolution Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image pickup apparatuses that capture two-dimensional light intensity distribution and angle information suffer from deteriorated spatial resolution due to the need to store angle information, and the effect of super-resolution from subpixel shift varies with focus position, leading to inconsistent image quality when refocusing.
Innovation Solution
An image processing method and apparatus that reconstructs input images to generate multiple output images with different focus positions by calculating a virtual imaging plane's position within the depth of focus of the imaging optical system, reducing the maximum apparent pixel pitch and stabilizing the super-resolution effect across focus positions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If pixels of the image pickup element are used for storing angle information of the ray in addition to two-dimensional light intensity distribution, then light field information is obtained, but spatial resolution is deteriorated
Solution Approach 1:
The patent divides the image pickup element into multiple regions, with some pixels dedicated to capturing light intensity distribution and other pixels dedicated to capturing angle information. This segmentation allows simultaneous acquisition of both types of information while maintaining spatial resolution by assigning specific functions to different pixel groups rather than requiring all pixels to store both types of data
Solution Approach 2:
The patent introduces a dimensional separation approach where light intensity information and angle information are captured in different spatial dimensions or pixel regions. By mapping angle information to a separate dimensional space (specific pixel regions), the system obtains comprehensive light field data without degrading the spatial resolution of the intensity distribution
2Measurement precision
If a specific focus position is used to obtain super-resolution from subpixel shift, then resolution is improved, but the effect varies with focus position during refocus, leading to inconsistent spatial resolution
Solution Approach 1:
The patent performs preliminary calculation of optimal virtual imaging plane positions for multiple focus positions before actual refocus operations. By pre-determining the depth of focus ranges and corresponding virtual plane positions, the system ensures that super-resolution effects are maintained consistently across all refocus positions, preventing resolution degradation during dynamic refocusing
Solution Approach 2:
The patent dynamically adjusts the position of the virtual imaging plane based on the focus position to maintain optimal resolution. By changing the virtual plane position parameter in response to focus changes, the system compensates for variations in super-resolution effects and maintains consistent spatial resolution across different refocus positions
Data Source
AI summary
An image processing method includes the steps of obtaining an input image that is an image in which information of an object space is obtained from a plurality of points of view using an image pickup apparatus that includes an image pickup element having a plurality of pixels and an imaging optical system, calculating a first position of a virtual imaging plane that corresponds to a specified focus position, setting the virtual imaging plane to a second position that is in a range of a depth of focus of the imaging optical system with reference to the first position so that a maximum value of an apparent pixel pitch that is formed by reconstructing the input image is decreased, and generating an output image in a state where the virtual imaging plane is set to the second position.