Array Camera Image Restoration for High-Resolution Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing imaging systems with multiple cameras often produce output images with resolutions that are only a small fraction of the total sensor pixels, and existing image restoration methods are inefficient in enhancing image resolution without increasing system volume.
Innovation Solution
An image restoration method using an array camera with a lens array and a sensing array, employing a neural network model to estimate global and local parameters for image alignment and refinement, allowing for high-resolution image synthesis without increasing the camera's physical size.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple cameras are used to capture images from different viewpoints, then the output image resolution is improved, but the device volume increases
Solution Approach 1:
The patent merges multiple camera functions into a single array camera device. Instead of using separate physical cameras positioned at different locations, the invention integrates multiple imaging functions into one compact device with a lens array and sensing array, achieving multi-viewpoint capture without proportionally increasing volume
Solution Approach 2:
The patent transitions from a single viewpoint (2D image capture) to multiple viewpoints (3D spatial information capture) by arranging lens elements and sensing elements in specific geometric patterns. This dimensional approach allows the system to capture depth and multiple perspectives simultaneously within a compact form factor
2Measurement precision
If existing image restoration methods are used to enhance image resolution, then image quality is improved, but processing time and computational complexity increase
Solution Approach 1:
The patent performs image alignment and refinement operations during the image capture process itself, rather than as a separate post-processing step. By pre-aligning images from different viewpoints and preparing depth information in advance, the system reduces the computational burden and time required for subsequent image restoration and enhancement
Solution Approach 2:
The patent creates synthetic or virtual images from multiple captured viewpoints using computational methods, generating high-resolution images without requiring additional physical sensors. This virtual copying approach allows resolution enhancement through software processing rather than hardware expansion, reducing processing time compared to traditional methods
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided is an image restoration method of restoring an image obtained through an array camera, the image restoration method including obtaining a plurality of images through lens elements included in the array camera, obtaining a global parameter of the plurality of images, generating first processed images by transforming a viewpoint of each of the plurality of images based on the obtained global parameter, obtaining a local parameter for each pixel corresponding to each of the first processed images, generating second processed images by transforming the first processed images based on the obtained local parameter, and generating a synthesized image of a target viewpoint based on synthesizing the second processed images.