Remote Assistance Distribution for Robotic Object Manipulation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Robotic systems face challenges in efficiently manipulating objects of varying types and sizes, as they often require remote assistance to determine accurate object segmentation and manipulation tasks, especially when confidence levels are low, and existing methods may not prioritize tasks effectively or utilize feedback efficiently.

Innovation Solution

A system that identifies tasks requiring remote assistance, builds a priority queue based on expected task times, and requests assistance from remote assistor devices, using feedback to guide the robotic manipulator in object manipulation, allowing for efficient distribution and utilization of human feedback.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If the robotic system requests remote assistance for all tasks with low confidence levels, then the accuracy of object manipulation is improved, but the time required to complete tasks increases due to waiting for human responses

Engineering Contradiction:

Improveaccuracy of object segmentationVSAvoidtask completion time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system performs preliminary actions by pre-processing object data, generating confidence level assessments, and preparing task requests before actual manipulation is needed. This allows the system to identify which tasks require human assistance in advance, reducing waiting time during critical manipulation phases.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements feedback mechanisms where human operators provide corrections and confirmations that are fed back into the robotic system. This feedback loop allows the system to learn from human decisions, improving future confidence levels and reducing the frequency of required human interventions over time.

Inventive Principle:

Principle #23Feedback

2Extent of automation

If the robotic system uses predetermined knowledge of object locations and types, then the automation level is maintained, but the system cannot handle objects outside its predetermined knowledge base

Engineering Contradiction:

Improveautonomous operation capabilityVSAvoidability to handle unknown objects

Core Design Contradiction:

Extent of automationVSAdaptability or versatility

Solution Approach 1:

Human operators serve as intermediaries between the robotic system and unknown objects. When the system encounters objects outside its predetermined knowledge, it queries human operators who provide descriptive information about object characteristics, enabling the robotic system to adapt to new object types while maintaining autonomous operation.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system dynamically adjusts its level of automation based on confidence levels. For familiar objects, it operates autonomously with high automation. For unknown or ambiguous objects, it transitions to a semi-autonomous mode where human assistance is solicited, thereby adapting its automation extent to match the situation's complexity.

Inventive Principle:

Principle #15Dynamics

3Manufacturing precision

If the system processes all object manipulation tasks sequentially without prioritization, then task accuracy is maintained, but productivity decreases due to inefficient task scheduling

Engineering Contradiction:

Improvetask execution accuracyVSAvoidobject manipulation throughput

Core Design Contradiction:

Manufacturing precisionVSProductivity

Solution Approach 1:

The system segments the task queue into priority levels based on object characteristics, confidence levels, and manipulation complexity. High-priority tasks (those with low confidence or complex manipulation requirements) are processed separately from routine tasks, allowing efficient resource allocation and maintaining accuracy for critical tasks while improving overall throughput.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system applies partial human assistance only to tasks that require it, rather than seeking human input for all tasks. By identifying and focusing human attention on specific high-priority tasks that benefit most from human expertise, the system maintains task execution accuracy for critical operations while avoiding unnecessary delays in processing routine tasks autonomously.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS9649767B2Methods and systems for distributing remote assistance to facilitate robotic object manipulation

Publication Date: 2017.05.16 INTRINSIC INNOVATION LLC

AI summary

Methods and systems for distributing remote assistance to facilitate robotic object manipulation are provided herein. Regions of a model of objects in an environment of a robotic manipulator may be determined, where each region corresponds to a different subset of objects with which the robotic manipulator is configured to perform a respective task. Certain tasks may be identified, and a priority queue of requests for remote assistance associated with the identified tasks may be determined based on expected times at which the robotic manipulator will perform the identified tasks. At least one remote assistor device may then be requested, according to the priority queue, to provide remote assistance with the identified tasks. The robotic manipulator may then be caused to perform the identified tasks based on responses to the requesting, received from the at least one remote assistor device, that indicate how to perform the identified tasks.