Robotic Teleoperation Latency Compensation With Predicted Virtual Views
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Teleoperation of robots often experiences latency due to network congestion, leading to user disorientation and inability to control the robot effectively during periods of lag, resulting in undesirable interactions with the environment.
Innovation Solution
A method that presents a virtual representation of the robot's environment, allows user input during latency to predict the environment's state, and reconciles the predicted view with current data once latency ends, ensuring continuous and coherent control.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If real-time teleoperative control is implemented, then user control over the robot is achieved, but latency during network congestion causes loss of user awareness and control
Solution Approach 1:
The system performs preliminary actions by predicting the robot's environment state and generating a predicted virtual representation during latency periods. This allows the user interface to continue displaying plausible environment states based on previous data and robot motion models, maintaining user awareness and control capability despite network delays.
Solution Approach 2:
The system implements feedback by continuously updating the predicted virtual representation with new sensor data once it becomes available. The reconciliation process compares predicted states with actual states and adjusts the display accordingly, providing the user with accurate feedback about the robot's actual environment while maintaining continuous control during latency.
2Loss of information
If continuous environment data is maintained, then user awareness is preserved, but network congestion causes data unavailability and control loss
Solution Approach 1:
The system creates a copy of the environment representation in the form of a predicted virtual representation that can be displayed and updated locally during latency periods. This predicted copy is generated using robot motion models and previous sensor data, allowing the user interface to maintain environment awareness without requiring continuous real-time data transmission over the network.
Solution Approach 2:
The system performs preliminary computation of the predicted virtual representation using stored robot motion models and historical sensor data before new data arrives. This allows the interface to display plausible environment states during latency, preserving user awareness and enabling continuous control operations.
3Ease of operation
If latency compensation is implemented, then user control during lag is maintained, but system complexity increases
Solution Approach 1:
The system pre-computes predicted virtual representations using stored robot motion models and historical sensor data during latency periods. This preliminary action allows the user interface to maintain continuous display updates without requiring complex real-time processing during network congestion, thereby improving ease of operation with manageable system complexity.
Data Source
AI summary
A method includes presenting a virtual representation of an environment of a robot, receiving a first user command to control the robot within the environment, rendering a predicted version of the virtual representation during a period of latency in which current data pertaining to the environment of the robot is not available, updating the predicted version of the virtual representation based upon a second user command received during the period of latency, and upon conclusion of the period of latency, reconciling the predicted version of the virtual representation with current data pertaining to the environment of the robot.