Robot Intention Estimation Using Gaze and Hand Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing robot remote operation systems struggle to accurately estimate the actions of an operator from sensor values, leading to unstable time-series estimation, especially in complex environments, during tasks like Pick & Place operations.
Innovation Solution
A robot remote operation control system that utilizes a trained model to integrate environment sensor values and operator sensor values, such as gaze points and hand trajectories, to estimate operator intentions and generate control commands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If time-series estimation is performed using only sensor values of line of sight or hand movement, then the system can operate with simple sensor input, but the estimation becomes unstable in complicated environments
Solution Approach 1:
The patent combines multiple sensor types (line of sight sensors, hand movement sensors, and environment sensors) into a unified estimation system. By merging these different sensor inputs, the system achieves more stable and accurate operator action estimation, particularly in complicated environments where single-sensor approaches fail.
Solution Approach 2:
The patent introduces a trained model as an intermediary that processes and integrates sensor values from multiple sources. This mediator transforms raw sensor data into reliable action estimates by learning the complex relationships between different sensor inputs and operator intentions, thereby stabilizing the estimation process.
2Measurement precision
If the system waits for complete sensor data before estimating operator action, then estimation accuracy improves, but response time increases
Solution Approach 1:
The patent performs preliminary processing of sensor data through a trained model that can provide initial action estimates even with incomplete information. This preliminary action allows the system to respond quickly while continuing to refine the estimation as more sensor data becomes available, thus reducing response time without sacrificing ultimate accuracy.
Solution Approach 2:
The estimation system dynamically adjusts its processing based on data availability. The trained model can operate with varying degrees of input completeness, adapting its estimation process in real-time to balance between response speed and accuracy based on the current sensor data state.
Data Source
AI summary
In a robot remote operation which recognizes a movement of an operator and transmits the movement of the operator to a robot to operate the robot, a robot remote operation control device includes: an information acquisition part, acquiring an environment sensor value acquired by an environment sensor provided in the robot or a surrounding environment of the robot and an operator sensor value, which is information indicating the movement of the operator that is detected; and an intention estimation part, estimating a motion of the operator, which is a motion instruction with respect to the robot, by using a trained model from the operator sensor value.