Gradient-Based Cross-Spectral Stereo Matching for Low-Light Conditions

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current stereo matching algorithms are limited by their reliance on visual cameras, which perform poorly in low-light conditions and adverse weather, and fail to effectively process cross-spectral image pairs due to significant differences in pixel intensities, hindering applications like search and rescue and deep space navigation.

Innovation Solution

A gradient-based cross-spectral stereo matching system that generates disparity maps using environment-specific input parameters, combining visual and infrared images by creating oriented gradients and histograms, and selecting optimal input parameters for accurate disparity map generation across varying conditions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If visual cameras are used for stereo matching, then the system works well in day time and ideal weather conditions, but it performs poorly in low-light conditions and adverse weather

Engineering Contradiction:

Improvestereo matching performanceVSAvoidenvironmental condition adaptability

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

The patent combines visual and infrared image data into a cross-spectral image pair for stereo matching. By merging information from both spectral domains, the system achieves reliable disparity map generation in diverse environmental conditions including low-light and adverse weather scenarios where single-spectral approaches fail

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The stereo matching system is designed to process multiple types of image pairs (visual-visual, infrared-infrared, and cross-spectral visual-infrared). This multi-functionality enables the system to adapt to various environmental conditions using the appropriate spectral combination, making it universally applicable across different operating scenarios

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Adaptability or versatility

If pixel-based stereo matching is used for cross-spectral images, then the algorithm can process the images, but significant differences in pixel intensities result in poor disparity maps

Engineering Contradiction:

Improvecross-spectral processing capabilityVSAvoiddisparity map accuracy

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent transforms the stereo matching approach from direct pixel-based comparison to gradient-based comparison. By changing the parameter from pixel intensity values to gradient magnitudes and orientations, the system overcomes the problem of significant intensity differences between cross-spectral images, enabling accurate disparity map generation

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent replaces the mechanical pixel-based matching mechanism with a gradient-based computational approach. Instead of directly comparing pixel intensities between visual and infrared images, the system computes gradients and uses histogram comparisons, substituting the flawed mechanical approach with a more robust computational method

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Adaptability or versatility

If two full infrared cameras are used for night or adverse weather conditions, then the system can capture images, but the pixel-based stereo matching approach still fails due to intensity differences when matching with visual images

Engineering Contradiction:

Improveoperational condition coverageVSAvoiddisparity map quality

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent merges infrared and visual image data into a cross-spectral stereo pair, combining the complementary strengths of both spectral domains. This merging enables the system to operate effectively in both daytime and nighttime/adverse weather conditions while maintaining high disparity map quality through gradient-based matching

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS11222229B1System and method for multidimensional gradient-based cross-spectral stereo matching

Publication Date: 2022.01.11 THE CHARLES STARK DRAPER LABORATORY INC

AI summary

A hardware system is configured for, and a method of, generating detail-rich gradient-based disparity maps in real-time using an automated gradient-based disparity map classification process that is scalable, can be used under different environment conditions with little to no restrictions, and whose level of precision can be adjusted in a scalable manner. Highly accurate cross-spectral stereo matching methods may be used for search and rescue operations and work at day time and night time using current and past visual and full infrared imaging to generate, classify, and identify scenes in real-time with minimum constraints. Such system and methods may be used to improve operations of existing search and rescue equipment.