Image Registration via Optimal Transport and Epipolar Geometry
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image registration methods are inadequate for registering multimodal images of 3D scenes, as they often require feature matching which is not achievable when images are of different modalities, leading to insufficient accuracy and complexity in computational procedures.
Innovation Solution
The proposed solution utilizes an optimal transport (OT) method that incorporates epipolar geometry as a regularizer to generate a registration map for image registration, allowing for the registration of multi-modal images without the need for camera calibration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional image registration methods are used, then feature matching can be performed, but registration is not possible when images are of different modalities
Solution Approach 1:
The patent introduces an optimal transport plan as an intermediary that operates in a modality-agnostic feature space. Instead of directly matching features between different modalities, the method transforms features from both images into a common latent space where transport plans can be computed, thereby enabling registration across modalities without direct feature comparison
Solution Approach 2:
The patent replaces traditional feature matching mechanisms with an optimal transport framework. Instead of relying on similarity measures between corresponding features, the method formulates registration as a mass transport problem, using entropy-regularized optimal transport to compute correspondence between image regions regardless of modality differences
2Measurement precision
If conventional image registration methods are used, then registration can be performed, but computational procedures become complex
Solution Approach 1:
The patent changes the parameter space by transforming image features into a latent representation where optimal transport can be efficiently computed. By modifying the feature space parameters and using entropy regularization, the method simplifies the computational procedure while maintaining registration accuracy
Solution Approach 2:
The patent performs preliminary feature extraction and transformation into a common latent space before applying optimal transport. This preliminary action prepares the data in a form that enables efficient computation of registration plans, avoiding complex iterative matching procedures
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
Systems and methods for image processing for determining a registration map between a first image of a scene with a second image of the scene, include solving an optimal transport (OT) problem to produce the registration map by optimizing a cost function that determines a minimum of a ground cost distance between the first and the second images modified with an epipolar geometry based regularizer including a distance that quantifies the violation of an epipolar geometry constraint between corresponding points defined by the registration map. The ground cost compares a ground cost distance of features extracted within the first image with a ground cost distance of features extracted from the second image.