Visual Localization Mapping Using Twilight-Aligned Day-Night Features

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current methods for autonomous mobile robots struggle to efficiently create and utilize maps for navigation during night-time conditions due to the limitations of conventional cameras in low light environments, leading to unreliable visual features for localization and mapping.

Innovation Solution

A method that generates a unified map by determining the relative positions of day-time and night-time features using images captured during twilight, leveraging twilight straight lines and urban lights to align and merge day-time and night-time visual features, allowing for efficient creation and use of a single map for both daytime and nighttime navigation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If conventional cameras are used for visual feature extraction, then the system is simple and easy to operate, but the reliability of visual features deteriorates in low light conditions

Engineering Contradiction:

Improvereliability of visual featuresVSAvoidcomplexity of mapping system

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent uses twilight images as an intermediary to bridge daytime and nighttime features. Twilight images capture both daytime features (buildings, streets) and nighttime features (lights, illuminated signs) simultaneously, serving as a mediator to align and merge features from both temporal conditions into a unified map coordinate system

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system performs preliminary alignment of nighttime features to daytime map coordinates using twilight images before merging. By pre-registering the spatial relationship between daytime and nighttime features during twilight conditions, the system prepares the data structure in advance, avoiding complex real-time adjustments and simplifying the overall mapping process

Inventive Principle:

Principle #10Preliminary action

2Measurement precision

If separate daytime and nighttime maps are created, then localization accuracy for each condition is improved, but the complexity of map management and integration increases

Engineering Contradiction:

Improvelocalization accuracyVSAvoidcomplexity of map management

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent merges daytime and nighttime maps into a single unified map structure by aligning nighttime features to the daytime map coordinate system using twilight images as reference. This combination maintains the precision benefits of separate mapping while eliminating the complexity of managing and integrating multiple separate maps

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The unified map structure serves multiple functions: it provides accurate localization for both daytime and nighttime operations, stores features from different temporal conditions, and enables single-map navigation regardless of lighting conditions. This universal map replaces the need for condition-specific map management

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Adaptability or versatility

If multiple maps for different lighting conditions are maintained, then adaptability to various environments is improved, but the effort and time required for map creation and maintenance increases

Engineering Contradiction:

Improveadaptability to light conditionsVSAvoidefficiency of map creation

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Inventive Principle:

Principle #5Merging (Combining)

4Reliability

If visual features are extracted only during daytime, then the quality of daytime features is high, but the system cannot perform localization during nighttime

Engineering Contradiction:

Improvequality of visual featuresVSAvoidoperational capability across light conditions

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent segments visual features by temporal characteristics, identifying and extracting daytime features (buildings, streets, natural landmarks) and nighttime features (lights, illuminated signs, artificial structures) as distinct categories. This segmentation allows the system to process and match features appropriate to current lighting conditions while maintaining a comprehensive unified map

Inventive Principle:

Principle #1Segmentation

Data Source

PatentEP3704555B1Visual localization and mapping in low light conditions

Publication Date: 2023.07.19 STARSHIP TECH OU

Figure 1
Figure 2
Figure 3

AI summary

The present invention relates to a method comprising generating a map comprising day-time features and night-time features, wherein the position of night-time features relative to the day-time features is determined by at least one image captured during twilight. The present invention also relates to a corresponding processing unit configured to execute such a method.