Mobile Robot Image Annotation for Adaptive Obstacle Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Mobile robots face challenges in accurately navigating their environments due to the limitations of classifiers trained on generic data, which may not account for specific obstacles or features unique to their operating environments, leading to incorrect predictions and reduced performance in obstacle detection and traversal decisions.

Innovation Solution

The method involves mobile robots capturing images and annotating them with ground truth information as they explore their environments, using machine vision sensor systems and additional sensors to update their training datasets specifically, allowing for continuous retraining of classifiers to optimize their performance for the particular environment, including the use of confidence metrics and map information to refine classifications.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If classifiers are trained on generic data, then the training process is simpler and faster, but the accuracy and reliability of obstacle detection deteriorates in specific operating environments

Engineering Contradiction:

Improveaccuracy of obstacle detectionVSAvoidcomplexity of training data collection

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The mobile robot performs self-training by autonomously capturing images during its operation, annotating them with ground truth information from its sensors, and retraining its classifiers without external intervention. This self-service approach enables the robot to adapt to its specific operating environment while maintaining operational simplicity

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system performs preliminary actions by continuously capturing and annotating images during normal operation, building up environment-specific training data before it is needed for improved classification. This preliminary data collection ensures that when retraining occurs, the classifier already has relevant environmental context to improve accuracy

Inventive Principle:

Principle #10Preliminary action

2Adaptability or versatility

If classifiers are retrained continuously with environment-specific data, then the adaptability and performance improve, but the loss of time for data collection and processing increases

Engineering Contradiction:

Improveadaptability to operating environmentVSAvoidtime for data collection and retraining

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The image capturing and annotation process occurs continuously during the robot's normal operation rather than requiring separate training sessions. The robot captures images, annotates them with ground truth from its sensors, and retrains classifiers in an ongoing manner, ensuring that data collection and adaptation are integrated into the operational workflow without significant time loss

Inventive Principle:

Principle #20Continuity of useful action

Solution Approach 2:

The system applies partial action by selectively retraining classifiers based on confidence metrics and environmental changes rather than continuously retraining all classifiers at full capacity. This approach maintains adaptability while reducing the time and computational resources required for data processing

Inventive Principle:

Principle #16Partial or excessive action

3Measurement precision

If the robot captures and processes more images for training, then the classifier accuracy improves, but the use of energy for data processing increases

Engineering Contradiction:

Improveclassifier accuracyVSAvoidenergy for image processing

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The system dynamically adjusts processing parameters such as image resolution, annotation frequency, and retraining thresholds based on environmental conditions and classifier confidence levels. By changing these parameters adaptively, the robot maintains high classifier accuracy while optimizing energy consumption for image processing according to the specific operational context

Inventive Principle:

Principle #35Parameter changes

4Reliability

If the robot uses additional sensors for ground truth annotation, then the reliability of training data improves, but the device complexity and cost increase

Engineering Contradiction:

Improvereliability of training dataVSAvoidcomplexity of sensor system

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The mobile robot utilizes its existing multi-functional sensors for both operational tasks and ground truth annotation. The same sensors used for navigation and obstacle detection during normal operation are repurposed to provide ground truth information for training data, eliminating the need for additional specialized sensors and maintaining reliability without increasing device complexity

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentEP3234839B1Systems and methods for capturing images and annotating the captured images with information

Publication Date: 2024.08.21 IROBOT CORP

Figure 1
Figure 2
Figure 3

AI summary

The present teachings provide an autonomous mobile robot that includes a drive configured to maneuver the robot over a ground surface within an operating environment; a camera mounted on the robot having a field of view including the floor adjacent the mobile robot in the drive direction of the mobile robot; a frame buffer that stores image frames obtained by the camera while the mobile robot is driving; and a memory device configured to store a learned data set of a plurality of descriptors corresponding to pixel patches in image frames corresponding to portions of the operating environment and determined by mobile robot sensor events.