Dual-Camera Robot Localization by Straight-Line Feature Matching

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Localization of mobile robots in unstructured outdoor environments, such as cities and villages, is challenging due to the lack of precise publicly available maps and the dynamic nature of these environments, where existing solutions like GPS provide low precision and are prone to outdated information.

Innovation Solution

A mobile robot equipped with at least two cameras and a processing component that extracts straight lines from visual images and compares them to stored map data using an iterative probabilistic algorithm, such as a particle filter, to determine its location and navigate autonomously, even in dynamic environments.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If GPS is used for localization, then the robot can operate in unstructured outdoor environments, but the localization precision is low (1-10 meters)

Engineering Contradiction:
Improvelocalization precisionVSAvoidenvironment adaptability
Core Design Contradiction:
Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The system segments the localization task into multiple components: GPS provides coarse location, while visual features (straight lines extracted from images) provide fine-grained precision. This hierarchical approach combines the broad coverage of GPS with the precision of visual recognition, achieving centimeter-level accuracy in unstructured environments.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent merges GPS localization with visual-based localization by integrating data from both sources. The system combines the global positioning capability of GPS with the precision of visual feature matching, creating a hybrid localization system that achieves both broad environmental adaptability and high measurement precision.

Inventive Principle:
Principle #5Merging (Combining)

2Measurement precision

If visual techniques are used for localization, then localization precision can be improved, but the system complexity increases

Engineering Contradiction:
Improvelocalization precisionVSAvoidsystem complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The system extracts only the most relevant visual features (straight lines) from images for localization, rather than processing all visual information. This selective extraction simplifies the computational burden while maintaining high localization precision, as straight lines are sufficient for matching with pre-stored map data.

Inventive Principle:
Principle #2Taking out (Extraction)

Solution Approach 2:

The system performs preliminary actions by pre-storing visual features (straight lines) and their geometric relationships in map data before actual localization. During localization, the robot only needs to extract straight lines from current images and match them against the pre-computed map data, significantly reducing real-time computational complexity.

Inventive Principle:
Principle #10Preliminary action

3Reliability

If multiple cameras are used for localization, then robustness and precision are improved, but the device complexity and cost increase

Engineering Contradiction:
Improvelocalization robustnessVSAvoidcamera system complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

Multiple cameras serve multiple functions: they provide redundant visual information for improved robustness, capture images from different perspectives for better feature extraction, and enable stereo vision for depth estimation. This multi-functionality justifies the increased complexity by delivering superior localization performance in diverse conditions.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system uses multiple cameras to provide beforehand cushioning against potential failures or limitations of individual cameras. By having redundant sensing capabilities, the system ensures that localization can continue even if one camera is blocked, fails, or provides insufficient features, thereby improving overall reliability.

Inventive Principle:
Principle #11Beforehand cushioning (Prior cushioning)

4Adaptability or versatility

If SLAM is used for simultaneous localization and mapping, then the robot can operate without pre-stored maps, but the computational complexity and time consumption increase

Engineering Contradiction:
Improveenvironment adaptabilityVSAvoidlocalization time
Core Design Contradiction:
Adaptability or versatilityVSLoss of time

Solution Approach 1:

The system performs preliminary mapping actions by pre-storing visual features and their geometric relationships in map data before the robot needs to localize. This pre-computation of map data eliminates the need for real-time SLAM during localization, significantly reducing computational time while still allowing operation in previously unknown environments.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS11042165B2Mobile robot system and method for autonomous localization using straight lines extracted from visual images
Publication Date: 2021.06.22 STARSHIP TECH OU
  • US11042165B2 patent drawing
  • US11042165B2 patent drawing
  • US11042165B2 patent drawing

AI summary

A mobile delivery robot has at least one memory component containing at least map data; at least two cameras adapted to take visual images; and at least one processing component. The at least one processing component is adapted to at least extract straight lines from the visual images taken by the at least two cameras and compare them to the map data to at least localize the robot. The mobile robot employs a localization method which involves taking visual images with at least two cameras; extracting straight lines from the individual visual images with at least one processing component; comparing the extracted features with existing map data; and outputting a location hypothesis based on said comparison.