VR Teleoperation Interface for Multi-Robot Remote Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing robotic systems lack an intuitive and immersive virtual reality-based interface for teleoperation, do not support global teleoperation or multiuser-multirobot access, and are not optimized for automating complex tasks, with limited mobility and agility.
Innovation Solution
A robotic system with a VR-based intuitive and immersive human interface, enabling hand tracking with six degrees of freedom, modular design for various networks, and cloud-based telepresence for controlling multiple robots, utilizing machine learning and reinforcement learning for enhanced automation and mobility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional monitor-mouse-keyboard interfaces are used for teleoperation, then the system is simple to implement, but the operator cannot achieve human-level dexterity and the interface lacks intuitiveness
Solution Approach 1:
The patent replaces traditional mechanical input devices (mouse, keyboard) with a virtual reality interface that uses hand tracking and full-body motion capture. This substitution enables natural, intuitive gestures and movements to control the robotic system, achieving human-level dexterity while maintaining system manageability through software-based tracking and rendering.
Solution Approach 2:
The system creates a virtual copy of the operator's hands and environment in the VR interface. Hand tracking captures the operator's actual hand movements and replicates them in the virtual environment, allowing intuitive control without requiring the operator to learn complex control schemes. This copying mechanism bridges the gap between simple operation and precise control.
2Adaptability or versatility
If robotic systems are designed for high mobility and agility to perform tasks in challenging locations, then the system can access difficult environments, but the system becomes more complex and harder to control remotely
Solution Approach 1:
The robotic system is divided into modular components including separate mobility modules (wheeled or legged), manipulation arms, and sensor suites. This segmentation allows the system to be configured for different environments and tasks while maintaining manageable complexity through standardized interfaces and independent control of each module.
Solution Approach 2:
The system employs dynamic mobility solutions that can adapt to different terrains and operational requirements. The robotic platform can switch between different locomotion modes and the manipulation system can dynamically adjust its degrees of freedom based on task requirements, providing high adaptability without requiring a completely different system for each scenario.
3Adaptability or versatility
If known teleoperated robotic systems are used, then the system can perform remote operations, but the system lacks global teleoperation capability and multiuser-multirobot access
Solution Approach 1:
The system implements a universal cloud-based architecture that enables multiple users to access and control multiple robotic systems simultaneously from any location. The centralized server manages user authentication, robot allocation, and data synchronization, providing global teleoperation capability through a single platform that handles diverse users and robots with standardized protocols.
4Productivity
If robotic systems perform complex tasks manually, then the tasks can be completed with flexibility, but the operations take more time compared to automated execution
Solution Approach 1:
The system records and stores the operator's manual manipulation actions during teleoperation as training data. This preliminary recording phase allows the system to later automate similar tasks by replaying the recorded sequences or using machine learning models trained on this data, significantly reducing execution time for repetitive or complex tasks while maintaining the flexibility of manual control when needed.
Data Source
AI summary
A computer-implemented method for controlling the operation of at least one robotic system, the method comprising: receiving state information from at least one robotic system in an environment, the state information created by the at least one robotic system; processing the state information to generate a virtual robotic system model of at least part of the at least one robotic system in the environment; causing a visualisation of the virtual robotic system model to be displayed to at least one user; receiving a user instruction for performing an operation while the visualisation is being displayed to the at least one user; and issuing a command based on the received user instruction to the at least one robotic system to cause the at least one robotic system to perform the operation.