Robot Remote Control Interface with Dynamic Behavior Display

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current robot control systems are cumbersome for users to operate remotely due to incomplete environmental recognition and the need to manage complex operations, especially when the robot moves out of the user's field of vision, making it difficult to control the robot effectively.

Innovation Solution

A robot device equipped with a behavior plan unit that detects its status and decides on future actions, and a display control unit that adjusts the remote control interface based on this information, allowing for a more intuitive and user-friendly remote control experience by displaying relevant environment maps and task content.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Extent of automation

If the robot is equipped with autonomous behavior planning capability, then the robot can perform various actions autonomously, but the user still needs to remotely control the robot for actions that cannot be autonomously performed

Engineering Contradiction:
Improveautonomous behavior planningVSAvoidremote control operation
Core Design Contradiction:
Extent of automationVSEase of operation

Solution Approach 1:

The display control unit provides visual feedback to the user by displaying icons representing the robot's behavior plan on the remote control device. This feedback mechanism allows the user to understand what the robot plans to do next, enabling better decision-making when remote control is needed, thus resolving the contradiction between autonomous capability and ease of remote operation

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

The invention introduces an intermediary display interface that translates the robot's internal behavior plan into visual icons shown to the user. This intermediary representation bridges the gap between autonomous robot operations and user comprehension, making remote control more intuitive and easier to operate

Inventive Principle:
Principle #24Intermediary (Mediator)

2Adaptability or versatility

If the remote control device displays only the robot's field of view, then the robot can navigate to places outside user vision, but the user finds it difficult to operate without seeing the broader environment

Engineering Contradiction:
Improvenavigation to unseen locationsVSAvoiduser operation difficulty
Core Design Contradiction:
Adaptability or versatilityVSEase of operation

Solution Approach 1:

The invention adds a new dimension to the display by showing environmental map information alongside or instead of the robot's camera view. This dimensional addition provides the user with spatial context about the broader environment, making it easier to understand the robot's location and navigate effectively even when moving outside the user's direct field of vision

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

3Adaptability or versatility

If the robot can perform various actions such as movement, gripping, and transferring, then the robot's functionality increases, but the remote control device requires many operations making it complicated to use

Engineering Contradiction:
Improverobot functionalityVSAvoidremote control operations
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The behavior plan unit autonomously determines the robot's next actions based on its current state and environment, reducing the need for the user to manually control each action. The system serves itself by automatically planning sequences of actions, while the user only needs to review the displayed behavior plan and intervene when necessary, thus maintaining high functionality while reducing operational complexity

Inventive Principle:
Principle #25Self-service

Data Source

PatentUS10105843B2Robot device, remote control method of robot device, and program
Publication Date: 2018.10.23 SONY GROUP CORP
  • US10105843B2 patent drawing
  • US10105843B2 patent drawing
  • US10105843B2 patent drawing

AI summary

There is provided a robot device including a behavior plan unit that detects a current status of the robot device or an external environment and that decides one behavior plan of a plurality of behavior plan candidates as a future behavior plan, based on the current status, and a display control unit that decides one display pattern of a plurality of previously prepared display pattern candidates as display content to be displayed on a remote control device that remotely controls the robot device based on the decided behavior plan and that cause the remote control device to display the display pattern decided by the behavior plan unit.