Image Alignment Using Warping Maps for Cross-Spectral Fusion

Solution Overview

Problem

Multi-camera image fusion technologies face challenges in achieving effective image alignment due to varying camera settings and characteristics, leading to poor feature matching and fusion image quality.

Innovation Solution

A method for image alignment that involves receiving images with similar and different spectral properties, calculating feature correspondence, and using warping maps to perform alignment, with feature extraction and matching based on pixel features like brightness, color, and texture, and adjusting alignment based on comparison results.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If feature matching is performed on images with different spectral properties (exposure time, receiving spectrum), then image alignment can be achieved across different camera settings, but the feature matching accuracy deteriorates due to great variations in object characteristics

Engineering Contradiction:

Improveimage alignment adaptabilityVSAvoidfeature matching accuracy

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent performs preliminary feature matching between a first image and a second image with similar spectral properties to establish a warping map in advance. This pre-computed alignment relationship is then applied to subsequent images with different spectral properties, avoiding the need to perform feature matching on images with greatly varying characteristics.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces a warping map as an intermediary that stores the alignment relationship between images with similar spectral properties. This warping map serves as a mediator to transfer alignment information to images with different spectral properties, bridging the gap between images that would otherwise be difficult to match directly.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Manufacturing precision

If multiple images with different spectral properties are processed through feature extraction and matching, then comprehensive alignment can be achieved, but the processing time and computational complexity increase

Engineering Contradiction:

Improvealignment precisionVSAvoidimage processing time

Core Design Contradiction:

Manufacturing precisionVSLoss of time

Solution Approach 1:

The patent performs feature extraction, matching, and warping map generation in advance on images with similar spectral properties. This preliminary processing establishes a reusable alignment model that can be applied to multiple subsequent images without repeating the computationally intensive feature matching process.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The warping map generated from one pair of images with similar spectral properties is made universal and can be applied to multiple different image pairs. This single warping map serves multiple alignment purposes, reducing the need to perform feature matching repeatedly for each image pair.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Adaptability or versatility

If direct feature matching is performed on images with different exposure times or receiving spectra, then image alignment can be attempted, but the alignment effect deteriorates due to high probability of feature matching failures

Engineering Contradiction:

Improvecross-spectral alignment capabilityVSAvoidfeature matching reliability

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The patent introduces a warping map as an intermediary that stores the alignment relationship between images with similar spectral properties. This warping map serves as a mediator to transfer alignment information to images with different spectral properties, bridging the gap between images that would otherwise be difficult to match directly.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent performs preliminary feature matching between a first image and a second image with similar spectral properties to establish a warping map in advance. This pre-computed alignment relationship is then applied to subsequent images with different spectral properties, avoiding the need to perform feature matching on images with greatly varying characteristics.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS20240412390A1Method and electronic system for image alignment

Publication Date: 2024.12.12 MEDIATEK INC

AI summary

A method for image alignment is provided. The method for image alignment includes the following stages. A first image with a first property from a first sensor is received. A second image with a second property from a second sensor is received. The first property is similar to the second property. The first feature correspondence between the first image and the second image is calculated. A third image with a third property from the first sensor and a fourth image with a fourth property from the second image sensor are received. The third property is different from the fourth property. Image alignment is performed on the third image and the fourth image based on the first feature correspondence between the first image and the second image.