Robot Remote Operation With 3D Mixed Views Under Communication Delay
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Remote operation of robots experiences operability issues due to communication delays caused by the large volume of visual information and limited bandwidth, leading to impaired performance in environments like international space stations and the moon.
Innovation Solution
A method that converts operation inputs into joint angle commands, acquires actual visual information, generates virtual visual information using a three-dimensional representation, and projects a mixed image combining actual and virtual information, ensuring operability is maintained even with delays by using a neural radiance fields representation and time-series networks for interpolation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If actual visual information is transmitted in real-time, then the operator can ascertain the surrounding environment accurately, but communication delay occurs due to large data volume and limited bandwidth
Solution Approach 1:
The system creates a virtual copy (three-dimensional representation) of the actual environment where the robot is located. This virtual model allows the operator to view the space from arbitrary viewpoints without transmitting large volumes of actual visual data in real-time, thereby reducing communication delay while maintaining environmental awareness accuracy.
Solution Approach 2:
The system pre-processes visual information to create a three-dimensional representation model before transmission. By preparing this virtual model in advance, the system enables the operator to interact with a pre-rendered environment, eliminating the need for continuous real-time transmission of high-volume visual data and thus reducing communication delay.
2Ease of operation
If virtual visual information is generated using three-dimensional representation, then operability is maintained without communication delay, but the data volume and processing complexity increase
Solution Approach 1:
The system transmits only essential parameters and key visual information needed to construct the three-dimensional representation, rather than transmitting complete high-resolution visual data from all angles. This partial transmission approach maintains operability while reducing both data volume and processing complexity compared to full visual field transmission.
Data Source
AI summary
A remote operation method of a robot includes: a conversion step of converting an operation input to a joint angle command value of the robot when an operator performs an operation; an acquisition step of acquiring actual visual information of a space in which the robot is present; a generation step of generating virtual visual information of an arbitrary viewpoint in the space in which the robot is present for the operator using a three-dimensional representation; and a projection step of projecting the virtual visual information to a screen viewed by the operator when the actual visual information is not newly acquired and projecting a mixed image in which an image based on the newly acquired actual visual information is combined with the virtual visual information when the actual visual information is newly acquired.