Thermal-Visual Image Fusion Parallax Compensation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional methods for generating contrast-enhanced combined images from thermal and visual light images face challenges such as parallax error, lack of edges or corners in the scene, out-of-focus issues, and varying lighting conditions, which result in reduced image quality and interpretability.

Innovation Solution

A method and apparatus that combine thermal images with High Spatial Frequency Content (HSFC) extracted from visual light images, using a blending measure to determine the quality of each image and adjust luminance weights for improved contrast enhancement, while accounting for focus settings and parallax errors.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If thermal images are combined with visual light images for contrast enhancement, then image clarity and object recognition are improved, but parallax error causes misalignment and ghosting artifacts

Engineering Contradiction:

Improveimage alignment precisionVSAvoidimage quality degradation due to ghosting

Core Design Contradiction:

Measurement precisionVSLoss of information

Solution Approach 1:

The patent applies preliminary action by performing parallax compensation through image registration before combining the thermal and visual light images. The system calculates parallax offsets based on optical parameters and applies corrective transformations to align the images, preventing misalignment and ghosting artifacts in the final contrast-enhanced image.

Inventive Principle:

Principle #10Preliminary action

2Measurement precision

If High Spatial Frequency Content is extracted from visual light images to enhance thermal images, then contrast and edge definition are improved, but out-of-focus visual images introduce noise and reduce overall image quality

Engineering Contradiction:

Improvecontrast enhancement qualityVSAvoidimage quality reliability under varying focus conditions

Core Design Contradiction:

Measurement precisionVSReliability

Solution Approach 1:

The patent applies dynamics by dynamically adjusting the blending measure based on focus quality assessment. The system evaluates the sharpness and focus quality of the visual light image, and adaptively modulates the contribution of HSFC to the final combined image. When the visual image is out of focus, the blending measure reduces HSFC contribution to prevent noise propagation, maintaining reliable image quality across varying focus conditions.

Inventive Principle:

Principle #15Dynamics

3Illumination intensity

If luminance components from visual light images are used to enhance thermal images, then brightness and visibility are improved, but variations in lighting conditions affect the accuracy of the enhancement

Engineering Contradiction:

Improveimage brightnessVSAvoidenhancement accuracy under varying lighting

Core Design Contradiction:

Illumination intensityVSMeasurement precision

Solution Approach 1:

The patent applies parameter changes by incorporating lighting condition parameters into the blending measure calculation. The system assesses lighting variations and adjusts the weighting of luminance components accordingly, ensuring that the contrast enhancement remains accurate and reliable across different lighting conditions while maintaining appropriate brightness levels.

Inventive Principle:

Principle #35Parameter changes

4Productivity

If a fixed blending measure is used to combine thermal and visual light images, then the processing is simple and fast, but it cannot adapt to varying scene conditions and focus quality

Engineering Contradiction:

Improveimage processing speedVSAvoidadaptability to varying scene conditions

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The patent applies feedback by implementing a quality assessment mechanism that evaluates focus sharpness, parallax alignment, and lighting conditions, then uses this feedback to dynamically adjust the blending measure. This closed-loop approach enables the system to adapt to varying scene conditions and focus quality while maintaining efficient processing through optimized calculation methods.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS10694101B2Contrast-enhanced combined image generation systems and methods

Publication Date: 2020.06.23 FLIR SYST AB

AI summary

Various techniques are disclosed to generate a contrast-enhanced combined image based on a thermal image and high spatial frequency content extracted from a visual light image, said images depicting the same scene. In one example, a method comprises: determining a blending measure indicating the quality of at least one of said images; combining luminance components of pixel values comprised in the thermal image and of pixel values representing high spatial frequency content comprised in the visual light image based on the blending measure; and generating a contrast-enhanced combined image based on the thermal image and the combined luminance components.