Robot Target Display Interface for Predictable Autonomous Control

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Autonomous robots may act against the user's intention due to misinterpretation of instructions or surroundings, making it difficult for users to predict their behavior.

Innovation Solution

A system that includes an interface receiving recognition information from an autonomous robot, displaying candidate target objects with associated target object scores, and allowing user feedback to guide the robot's behavior.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Extent of automation

If the autonomous robot autonomously selects target objects based on its own judgment, then the robot can operate independently without constant user input, but the robot may act against the user's intention due to misinterpretation of instructions or surroundings

Engineering Contradiction:
Improveautonomous operationVSAvoidbehavior prediction accuracy
Core Design Contradiction:
Extent of automationVSReliability

Solution Approach 1:

The system displays recognized target objects with confidence scores to the user, who then provides feedback by selecting the actual target. This feedback loop allows the robot to learn from user corrections and improve its target recognition accuracy over time, resolving the contradiction between autonomous operation and behavior predictability.

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

The display interface acts as an intermediary between the robot's autonomous decision-making and the user's intentions. By showing candidate targets and their scores, the interface mediates the interaction, allowing the user to guide the robot's behavior without completely taking over control, thus maintaining autonomy while ensuring reliability.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Adaptability or versatility

If the robot recognizes multiple candidate target objects, then the robot has more options to fulfill user requests, but it becomes difficult for the user to predict which object the robot will select

Engineering Contradiction:
Improvetarget selection flexibilityVSAvoiduser predictability
Core Design Contradiction:
Adaptability or versatilityVSLoss of information

Solution Approach 1:

The system uses visual differentiation (analogous to color changes) by displaying candidate targets with different visual indicators and confidence score levels. This helps users quickly understand the robot's recognition state and predict which target is most likely to be selected, maintaining information transparency while preserving selection flexibility.

Inventive Principle:
Principle #32Color changes

Solution Approach 2:

By displaying confidence scores for multiple candidate targets, the system provides feedback to the user about the robot's internal state. This transparency allows users to predict the robot's behavior and intervene if necessary, resolving the information loss while maintaining adaptability.

Inventive Principle:
Principle #23Feedback

3Loss of information

If the system displays detailed information about candidate target objects, then the user can better understand and predict robot behavior, but the interface complexity increases

Engineering Contradiction:
Improvebehavior transparencyVSAvoidinterface complexity
Core Design Contradiction:
Loss of informationVSDevice complexity

Solution Approach 1:

The information display is segmented into essential elements: candidate target images, confidence scores, and selection indicators. This segmentation presents comprehensive information in a structured, easy-to-understand format that minimizes interface complexity while maximizing behavior transparency.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The system extracts and displays only the most critical information needed for user understanding: the candidate targets and their confidence scores. By taking out only the essential elements rather than displaying all possible data, the interface remains simple while providing sufficient transparency into robot behavior.

Inventive Principle:
Principle #2Taking out (Extraction)

Data Source

PatentUS12370673B2Display control device, display control method, computer program product, and communication system
Publication Date: 2025.07.29 SONY GROUP CORP
  • US12370673B2 patent drawing
  • US12370673B2 patent drawing
  • US12370673B2 patent drawing

AI summary

A control system, method and computer program product cooperate to assist control for an autonomous robot. An interface receives recognition information from an autonomous robot, said recognition information including candidate target objects to interact with the autonomous robot. A display control unit causes a display image to be displayed on a display of candidate target objects, wherein at least two of the candidate target objects are displayed with an associated indication of a target object score.