Image Fusion via Jacobian Principal Vector Projection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image fusion methods, such as those using the Di Zenzo structure tensor, often introduce hallucinated details and artefacts like halos and bending artefacts due to the ill-posed nature of reintegration in the derivative domain, and struggle to preserve information from multiple spectral channels.
Innovation Solution
A method that determines the Jacobian matrix of input image channels, calculates the principal characteristic vector of its outer product, and generates an output image by projecting input channels in the direction of this vector, avoiding reintegration and using techniques like bilateral filtering to ensure accurate and natural image representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If image fusion is carried out in the derivative domain using structure tensor methods, then image detail information is preserved, but the reintegration step hallucinates new details and introduces artefacts like halos and bending artefacts
Solution Approach 1:
The patent extracts only the gradient magnitude information from the structure tensor decomposition, discarding the gradient direction information that causes integration ambiguities. This selective extraction preserves essential edge information while avoiding the hallucination problem inherent in full gradient field reintegration.
Solution Approach 2:
Instead of integrating gradients to reconstruct the image (the conventional approach), the patent inverts the problem by directly computing the fused image from gradient magnitudes without requiring integration. This reversal eliminates the ill-posed integration step that generates artefacts.
2Loss of information
If multiple spectral channels are captured beyond the visible spectrum, then information from additional modalities is preserved, but the output image cannot be directly visualized by human observers
Solution Approach 1:
The patent maps information from non-visible spectral channels into the visible color space by computing gradient magnitudes that are then displayed using standard color visualization. This allows invisible spectral information to be represented in a form human observers can perceive while maintaining the integrity of the multi-channel data.
Solution Approach 2:
The patent introduces gradient magnitude as an intermediary representation that bridges multi-spectral data and human visual perception. Instead of directly displaying raw multi-channel data or simple false-color composites, the gradient magnitude serves as a mediator that preserves information while enabling natural visual interpretation.
3Ease of operation
If the sign of the derived gradient is defined heuristically, then the gradient field can be processed, but the solution remains ill-posed and introduces artefacts
Solution Approach 1:
The patent extracts only the magnitude component of the gradient, deliberately excluding the sign information that creates ambiguity. By working with magnitude alone, the method avoids the need for heuristic sign assignment and the resulting integration problems entirely.
Data Source
AI summary
A method and system for generating an output image from a plurality, N, of corresponding input image channels is described. A Jacobian matrix of the plurality of corresponding input image channels is determined. The principal characteristic vector of the outer product of the Jacobian matrix is calculated. The sign associated with the principal characteristic vector is set whereby an input image channel pixel projected by the principal characteristic vector results in a positive scalar value. The output image as a per-pixel projection of the input channels in the direction of the principal characteristic vector is generated.