Multi-Sensor Image Fusion for Spatial Resolution Enhancement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current photographic image acquisition devices face limitations in spatial resolution due to the constraints of image sensor technology and the exponential cost increase associated with increasing sensor size, while post-capture computational methods like pansharpening and super-resolution are computationally expensive and impractical for real-time application.
Innovation Solution
A photographic image acquisition device that combines a primary image sensor with a secondary image sensor, utilizing nonlocal self-similarity and redundancy at the patch level to achieve concurrent super-resolution reconstruction and pansharpening, allowing for a multiplicative resolution enhancement without requiring precise registration or additional hardware for depth estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the spatial density of the focal plane array is increased to improve spatial resolution, then the spatial resolution is improved, but the signal-to-noise ratio and dynamic range deteriorate due to reduced photon collection per element
Solution Approach 1:
The image sensor is divided into multiple focal plane arrays with different spectral sensitivities (e.g., panchromatic and multi-spectral sensors). Each array captures images simultaneously, allowing the panchromatic array to provide high-resolution structural information while multi-spectral arrays capture spectral data. This segmentation enables resolution enhancement without reducing photo element size, thus maintaining signal-to-noise ratio.
Solution Approach 2:
The invention transitions from improving resolution in the spatial domain alone to utilizing the spectral dimension by incorporating multiple spectral bands. By capturing images in different spectral bands simultaneously and fusing them, the system achieves super-resolution without requiring smaller photo elements, thereby preserving photon collection efficiency and signal quality.
2Measurement precision
If the optical format of the image sensor and lens is increased to accommodate more photosensitive elements for improved spatial resolution, then the spatial resolution is improved, but the fabrication cost increases exponentially
Solution Approach 1:
The invention merges multiple image sensors with different spectral characteristics into a single imaging system. Instead of manufacturing one extremely large high-resolution sensor, the system combines several smaller sensors (panchromatic and multi-spectral) that can be produced at lower costs. The fused output achieves super-resolution equivalent to a much larger sensor without the exponential cost increase.
Solution Approach 2:
The invention changes the spectral sensitivity parameter of the image sensors rather than increasing the physical size or spatial density. By using sensors with different spectral responses (panchromatic vs. multi-spectral) at the same spatial resolution, the system achieves enhanced overall resolution through fusion without the need for larger or denser sensor arrays, thus avoiding exponential cost increases.
3Measurement precision
If post-acquisition computational methods like pansharpening are used to increase spatial resolution, then the spatial resolution is improved, but the computational burden and processing time increase significantly
Solution Approach 1:
The system performs image capture in multiple spectral bands simultaneously during the initial acquisition phase, rather than sequentially processing and fusing images after capture. The parallel capture of panchromatic and multi-spectral images at the same moment prepares the data in advance for fusion, reducing post-processing computational burden and enabling real-time or near-real-time resolution enhancement.
4Measurement precision
If pansharpening is applied to images with parallax mismatch between panchromatic and multi-spectral images, then the spatial resolution can be improved, but additional hardware for depth estimation and complex registration are required
Solution Approach 1:
The system performs geometric correction and registration of multi-spectral images to the panchromatic image as a preliminary step before fusion. By pre-aligning the images and correcting for parallax effects using available metadata or simple geometric models, the system eliminates the need for additional depth estimation hardware, enabling pansharpening to proceed with standard computational resources.
Data Source
AI summary
A photographic image acquisition device comprising a primary image sensor optically coupled to a primary imaging lens and at least one secondary image sensor optically coupled to a secondary imaging lens, the optical axes of the primary and the secondary lenses set parallel to each other, both image sensors set in the same geometric plane, such that both focal planes arrays receive optical projections of the same scene.