Lens Array Image Processing for Low-Memory Parallax Restoration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image sensors with quad Bayer patterns require significant logic and memory resources for parallax restoration processing due to increased pixel counts, leading to inefficiencies in high-definition image acquisition.
Innovation Solution
The implementation of a processor that generates RGB demosaic images using various interpolation methods and parallax images, along with edge detection and aliasing region identification, to minimize the need for additional logic and memory in demosaicing processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the number of pixels in a color filter array is increased to achieve high-definition image acquisition, then image resolution is improved, but the logic and memory required for parallax restoration processing increases
Solution Approach 1:
The patent divides the image processing into separate stages: demosaicing is performed first on the raw color filter array data, then parallax restoration is applied to the demosaiced image. This segmentation allows each processing stage to operate independently with optimized resource requirements, preventing the combinatorial explosion of complexity that would occur if both operations were performed simultaneously on high-resolution data
Solution Approach 2:
The patent performs demosaicing as a preliminary action before parallax restoration. By completing the color interpolation and generating a complete color image first, the subsequent parallax restoration operates on already-processed data with reduced computational requirements, thereby managing logic and memory usage effectively while maintaining high-definition output
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach reduces the computational burden and resource requirements for parallax restoration, enabling efficient high-definition image acquisition while maintaining image quality.
Implementation Method 1
An image sensor may include a red color filter, a green color filter, and a blue color filter arranged in a predetermined pattern on a light sensing pixel for imaging of a color image
Implementation Method 2
The quad Bayer pattern may be used for auto-focusing on a photographing target by using the parallax of light passing through the same microlens
Implementation Method 3
generate RGB demosaic images by interpolating a color value of an empty pixel for each of the plurality of parallax images by using color values of surrounding pixels
Implementation Method 4
generate an output image by weighted-summing one reference upsampled image selected from the upsampled images and the other upsampled images based on a similarity therebetween
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
An image acquisition apparatus includes an image sensor including a microlens and a plurality of pixels arranged adjacent to each other and sharing the microlens; and a processor configured to process an input image acquired by the image sensor, wherein the processor may be further configured to: generate a plurality of parallax images by extracting and combining sub-images having a same parallax from the input image; generate RGB demosaic images by interpolating a color value of an empty pixel for each of the plurality of parallax images by using color values of surrounding pixels; generate a plurality of upsampled images by upsampling each of the RGB demosaic images to an input image size; and generate an output image by weighted-summing one reference upsampled image selected from the upsampled images, and other upsampled images selected from the upsampled images, wherein the one reference upsampled image and the other upsampled images are selected based on a similarity therebetween.