Scored Candidate Object Display for Predictable Robot Interaction

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Autonomous robots may act against user intentions due to misinterpretation of instructions or surroundings, making it difficult for users to predict their behavior.

Innovation Solution

A control system that includes an interface to receive recognition information from the robot, displaying candidate target objects with associated scores on a display, allowing users to provide feedback and select preferred actions, thereby enhancing prediction and control of the robot's behavior.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Extent of automation

If the autonomous robot acts autonomously according to user instructions, then the robot can perform tasks independently, but the robot may act against user intentions due to misinterpretation of instructions or surroundings

Engineering Contradiction:
Improveautonomous actionVSAvoidbehavior prediction
Core Design Contradiction:
Extent of automationVSReliability

Solution Approach 1:

The system displays candidate target objects with scores to the user, who then provides feedback by selecting the actual target. This feedback loop allows the robot to learn from user corrections and improve its target recognition accuracy over time, resolving the contradiction between autonomous operation and reliable behavior prediction.

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

The robot performs preliminary recognition of candidate target objects and presents them to the user before executing the final action. This preliminary action allows the user to verify and correct the robot's interpretation, ensuring reliable behavior while maintaining autonomous operation.

Inventive Principle:
Principle #10Preliminary action

2Productivity

If the robot autonomously selects target objects based on its judgment, then task execution speed is improved, but accuracy of target selection deteriorates due to misinterpretation

Engineering Contradiction:
Improvetask execution speedVSAvoidtarget selection accuracy
Core Design Contradiction:
ProductivityVSMeasurement precision

Solution Approach 1:

The system calculates scores for multiple candidate target objects and displays them to the user for verification. The user's selection feedback corrects potential misinterpretations while maintaining efficient task execution, as the robot only requires minimal user input rather than full manual control.

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

The display interface acts as an intermediary between the robot's autonomous judgment and the user's final decision. It presents candidate targets with scores, allowing the user to quickly verify or correct the robot's selection, thus improving accuracy without significantly reducing productivity.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Productivity

If the robot operates completely autonomously without user intervention, then operational efficiency is maximized, but user control and prediction of robot behavior is lost

Engineering Contradiction:
Improveoperational efficiencyVSAvoiduser control
Core Design Contradiction:
ProductivityVSEase of operation

Solution Approach 1:

The system implements partial user intervention by displaying only the top candidate target objects with scores rather than requiring full user control. This partial action maintains high operational efficiency while providing sufficient user control and predictability, as users only need to verify or select from pre-filtered options.

Inventive Principle:
Principle #16Partial or excessive action

Data Source

PatentUS11654549B2Display control device, display control method, computer program product, and communication system
Publication Date: 2023.05.23 SONY GROUP CORP
  • US11654549B2 patent drawing
  • US11654549B2 patent drawing
  • US11654549B2 patent drawing

AI summary

A control system, method and computer program product cooperate to assist control for an autonomous robot. An interface receives recognition information from an autonomous robot, said recognition information including candidate target objects to interact with the autonomous robot. A display control unit causes a display image to be displayed on a display of candidate target objects, wherein at least two of the candidate target objects are displayed with an associated indication of a target object score.