Robot Training via Virtual Deictic Markers

Solution Overview

Problem

Existing robot manipulator control systems require expensive retraining for modifications to objects or changes in the work environment, and existing software is not easily retooled to meet changing flexibility requirements, making it difficult to adapt to new tasks without significant programmer interaction.

Innovation Solution

A method and system that uses human-assisted task demonstration and virtual deictic markers assigned to perceptual features of the robot's work environment to dynamically assign work tasks, allowing the robot to learn and adapt to new tasks with minimal training or reprogramming by recording motor schema and sensory data during human demonstration and using these markers to guide automated behavior.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If conventional manipulators are designed to operate in a highly structured environment with specific movement trajectories, then control precision is improved, but adaptability to new tasks deteriorates

Engineering Contradiction:

Improvecontrol precisionVSAvoidadaptability to new tasks

Core Design Contradiction:

Manufacturing precisionVSAdaptability or versatility

Solution Approach 1:

The system performs preliminary action by capturing and storing data sequences of operator-controlled manipulator movements before actual task execution. These captured sequences serve as pre-learned examples that the robot can reference and reproduce, eliminating the need for reprogramming when adapting to new tasks while maintaining precise control through the stored movement patterns.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system uses copying by capturing operator manipulation sequences and creating digital replicas of these movement patterns. The robot copies the human operator's demonstrated trajectories and actions, storing them as reusable data sequences that can be reproduced across different tasks and environments, thereby achieving both precision and adaptability.

Inventive Principle:

Principle #26Copying

2Manufacturing precision

If manual programming is used for each new robotic task, then task execution accuracy is improved, but time consumption deteriorates

Engineering Contradiction:

Improvetask execution accuracyVSAvoidtime consumption

Core Design Contradiction:

Manufacturing precisionVSLoss of time

Solution Approach 1:

The system performs preliminary action by capturing and storing data sequences of operator-controlled manipulator movements before actual task execution. These captured sequences serve as pre-learned examples that the robot can reference and reproduce, eliminating the need for reprogramming when adapting to new tasks while maintaining precise control through the stored movement patterns.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system enables self-service by allowing the robot to automatically learn and store movement sequences from operator demonstrations. The captured data sequences are automatically processed and stored in memory, enabling the robot to independently reproduce tasks without requiring manual reprogramming, thereby reducing time consumption while maintaining accuracy.

Inventive Principle:

Principle #25Self-service

3Reliability

If expensive retraining is performed for modifications to objects or changes in work environment, then robot performance is improved, but cost increases

Engineering Contradiction:

Improverobot performanceVSAvoidretraining cost

Core Design Contradiction:

ReliabilityVSEase of manufacture

Solution Approach 1:

The system uses copying by capturing operator manipulation sequences and creating digital replicas of these movement patterns. The robot copies the human operator's demonstrated trajectories and actions, storing them as reusable data sequences that can be reproduced across different tasks and environments, thereby achieving both precision and adaptability.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The system achieves universality by creating a reusable library of captured movement sequences that can be applied across multiple tasks and environments. The stored data sequences serve universal purposes, allowing the robot to adapt to different objects and work conditions without requiring task-specific reprogramming, thereby reducing retraining costs while maintaining performance.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS8843236B2Method and system for training a robot using human-assisted task demonstration

Publication Date: 2014.09.23 GM GLOBAL TECHNOLOGY OPERATIONS LLC

AI summary

A method for training a robot to execute a robotic task in a work environment includes moving the robot across its configuration space through multiple states of the task and recording motor schema describing a sequence of behavior of the robot. Sensory data describing performance and state values of the robot is recorded while moving the robot. The method includes detecting perceptual features of objects located in the environment, assigning virtual deictic markers to the detected perceptual features, and using the assigned markers and the recorded motor schema to subsequently control the robot in an automated execution of another robotic task. Markers may be combined to produce a generalized marker. A system includes the robot, a sensor array for detecting the performance and state values, a perceptual sensor for imaging objects in the environment, and an electronic control unit that executes the present method.