Blind Multispectral Image Fusion With Blur Kernel Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional multi-spectral imaging methods face challenges in achieving high spatial and spectral resolutions due to the trade-off between sensor bandwidth and image resolution, with existing fusion techniques failing to enhance spatial resolution of multi-spectral images effectively, especially when images are misaligned or require extensive computational resources.
Innovation Solution
A system that fuses low spatial resolution multi-spectral images with high spatial resolution panchromatic images using a Second-Order Total Generalized Variation (TGV2) function and local Laplacian prior (LLP) to iteratively update blur kernels, achieving high spectral and spatial resolution without prior knowledge of misalignment or parametric models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional MS image fusion methods are used, then spectral information is preserved, but spatial resolution remains low
Solution Approach 1:
The patent merges MS images with a PAN image through iterative fusion, combining the spectral information from MS images with the high spatial resolution from the PAN image to produce fused images that achieve both high spectral and spatial resolution
Solution Approach 2:
The patent introduces a blur kernel as an intermediary element that models the relationship between PAN and MS images. This blur kernel serves as a mediator that captures the degradation process, enabling the fusion algorithm to reversibly transform the low-resolution MS images back to high-resolution images while preserving spectral characteristics
2Ease of operation
If conventional fusion methods are used, then processing is simpler, but images are misaligned and resolution is not improved
Solution Approach 1:
The patent performs preliminary alignment and blur kernel estimation before the main fusion process. By pre-processing the images to correct misalignment and estimate the degradation model, the subsequent fusion operations can focus on resolution enhancement without being hindered by registration errors
Solution Approach 2:
The patent implements an iterative fusion algorithm with feedback mechanisms that continuously refine the blur kernel estimates and update the fused image. The algorithm uses the difference between successive iterations and the residual error to adjust parameters, ensuring progressive improvement in spatial resolution while maintaining spectral fidelity
3Manufacturing precision
If deep-learning based methods are used, then performance is improved, but training data requirements and computational resources increase
Solution Approach 1:
The patent enables the system to self-learn the blur kernel and alignment parameters directly from the input PAN and MS images without requiring external training data. The algorithm performs blind estimation of the degradation model, making the system self-sufficient and adaptable to different imaging conditions without extensive pre-training
Solution Approach 2:
The patent replaces complex deep-learning mechanical systems with a streamlined optimization-based approach. By formulating the fusion problem as an inverse problem with explicit mathematical models for blur and alignment, the system achieves high spatial resolution through efficient numerical optimization rather than resource-intensive neural network inference
4Use of energy by moving object
If conventional fusion methods are used, then computational resources are reduced, but theoretical convergence guarantee is lacking
Solution Approach 1:
The patent segments the fusion problem into distinct sub-problems: alignment estimation, blur kernel estimation, and image fusion. Each sub-problem is solved separately using specialized algorithms, allowing the system to maintain computational efficiency while ensuring convergence through proven optimization methods for each individual task
Solution Approach 2:
The patent dynamically adjusts optimization parameters such as regularization weights and iteration limits based on the specific characteristics of the input images. This adaptive parameter tuning ensures reliable convergence across different scenarios while maintaining computational efficiency by avoiding unnecessary iterations
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
Systems, methods and apparatus for image processing for reconstructing a super resolution image from multispectral (MS) images. Receive image data and initialize a fused image using a panchromatic (PAN) image, and estimate a blur kernel between the PAN image and the MS images as an initialization function. Iteratively, fuse a MS image with an associated PAN image of a scene using a fusing algorithm. Each iteration includes: update the blur kernel based on a Second-Order Total Generalized Variation function to regularize a kernel shape; fuse the PAN image and MS images with the updated blur kernel based on a local Laplacian prior function to regularize the high-resolution information to obtain an estimated fused image; compute a relative error between the estimated fused image of the current iteration and a previous estimated fused image from a previous iteration, to a predetermined threshold, to stop iterations stop, to obtain a PAN-sharpened image.