Multi-Modal Image Co-Registration Using Gabor Jet Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image co-registration techniques for multi-modal images, such as SAR and optical images, lack accuracy and efficiency due to the heterogeneous nature of these modalities, and existing deep learning methods fail to effectively exploit matching relations, leading to significant false matches and incomplete consideration of both global and local features.
Innovation Solution
A deep-learning based multi-input-output encoder-decoder network, specifically a Siamese U-Net architecture, is used for feature extraction, combined with a Gabor-Jet layer for similarity matching, which generates Gabor jet graphs to provide sparse feature points and utilize phase and disparity information for accurate image alignment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If conventional template matching techniques are used for multi-modal image co-registration, then the process can be automated, but the matching accuracy deteriorates due to heterogeneous features between SAR and optical images
Solution Approach 1:
The patent transforms the images into a common complex-valued feature space by applying Gabor filters, which convert spatial domain information into frequency domain representations. This parameter transformation allows heterogeneous SAR and optical images to be compared using unified similarity metrics, resolving the accuracy issue while maintaining automation.
Solution Approach 2:
The patent introduces Gabor jet features as an intermediary representation that bridges the gap between heterogeneous SAR and optical images. These features serve as a common language for comparison, enabling automated matching without direct comparison of raw heterogeneous pixel values.
2Productivity
If deep learning-based techniques with dot product are used for template matching, then computational speed is improved, but matching accuracy deteriorates due to significant false matches
Solution Approach 1:
The patent replaces the conventional dot product operation with Gabor jet-based similarity measurement. This substitution maintains computational efficiency while improving accuracy by incorporating phase information and utilizing the structured representation of Gabor jets, which capture both magnitude and orientation information.
Solution Approach 2:
The patent changes the feature representation from simple pixel values or basic feature maps to Gabor jet features that encode directional and frequency information. This parameter enrichment allows for more discriminative matching without sacrificing computational speed, as the Gabor jet structure enables efficient similarity computation.
3Loss of time
If FFT algorithm is used for fast image matching, then processing time is reduced, but matching accuracy deteriorates because only global features are considered
Solution Approach 1:
The patent segments the image matching process into local Gabor jet feature extraction followed by similarity computation. This segmentation allows the method to capture both local structural details through Gabor filters and global correspondence through the similarity measurement, achieving accuracy without sacrificing the speed benefits of efficient algorithms.
Solution Approach 2:
The patent moves the matching problem from purely spatial domain to frequency domain by applying Gabor filters. This dimensional transformation enables the capture of both local and global features simultaneously through the multi-scale and multi-orientation properties of Gabor functions, while maintaining computational efficiency.
4Manufacturing precision
If SIFT feature extraction is used for image matching, then local features can be identified, but matching accuracy deteriorates because SIFT uses homogeneous image features that do not account for fundamental differences between optical and SAR images
Solution Approach 1:
The patent changes the feature extraction approach from intensity-based SIFT to Gabor filter-based feature extraction. This parameter change allows the system to capture texture and orientation information that is modality-invariant, enabling accurate matching between heterogeneous SAR and optical images while preserving local feature identification capabilities.
Data Source
AI summary
The disclosure generally relates to methods and systems for image co-registration of multi-modal temporal sensing. Conventional techniques for image co-registration of multi-modality images focus on either the spatial or temporal domain and thus are not of high accuracy and do not preserve both global and local characteristics for the matching. The present disclosure solves the technical problems in the art for image co-registration of multi-modal temporal sensing using a deep-learning based multi-input-output encoder-decoder network with a Gabor Jet Model. The deep-learning based multi-input-output encoder-decoder network is utilized for the feature extraction. A distinctive Gabor-jet layer of the Gabor Jet Model is utilized for the similarity matching. The Gabor-jet layer generates a Gabor jet graph which provides sparse feature points for matching between matching images and reference images.