Robot Teleoperation via 3D Pose Mapping Without Exoskeletons
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current teleoperation methods for robots with multiple degrees of freedom are non-intuitive, slow, and require significant training due to the lack of direct mapping between controllers and robot movements, and existing solutions like exoskeletons and motion capture systems are cumbersome, expensive, and not portable.
Innovation Solution
A system that uses an image capturing device to capture an operator's pose and maps it to a three-dimensional skeleton model, generating kinematic parameters for a robot to mimic the operator's movements, allowing wireless and intuitive control without the need for external motion capture systems or exoskeletons.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If traditional joysticks or remote controllers are used for teleoperation, then the control method is relatively cheap, but the teleoperation becomes non-intuitive and slow due to lack of direct mapping to robot degrees of freedom
Solution Approach 1:
The patent replaces mechanical control interfaces (joysticks, buttons) with an optical-based pose estimation system using computer vision. The system captures images of the operator's body, estimates pose through image processing, and maps it to robot control signals, eliminating the need for physical mechanical controllers and achieving intuitive whole-body interaction.
Solution Approach 2:
The system creates a virtual 3D skeleton model that copies and represents the operator's physical pose from 2D images. This virtual model is then mapped to the robot's configuration space, allowing the robot to replicate the operator's intended movements without direct mechanical coupling.
2Ease of operation
If exoskeletons are used for teleoperation, then the control becomes more intuitive with direct mapping, but the system becomes complex, expensive, and not portable
Solution Approach 1:
The patent extracts the pose estimation function from complex wearable systems (exoskeletons, motion capture suits) and implements it using only standard image capturing devices. By separating the sensing function (camera-based pose estimation) from the control execution, the system achieves exoskeleton-level intuitiveness without the mechanical complexity and cost.
Solution Approach 2:
The system uses a universal image capturing device (standard camera) that can perform multiple functions: capturing operator pose, tracking movement, and providing visual feedback. This single device replaces the specialized sensors and actuators required in exoskeleton systems, achieving the same control intuitiveness with much simpler hardware.
3Measurement precision
If motion capture systems with optical cameras and reflective markers are used, then pose tracking is accurate, but the system requires room retrofitting and is not portable
Solution Approach 1:
The patent introduces an intermediary pose estimation algorithm that processes standard 2D images to extract 3D pose information. This intermediary layer allows the system to achieve accurate pose tracking without requiring specialized motion capture infrastructure, bridging the gap between simple camera input and precise robot control.
4Device complexity
If wearable IMUs are used for motion capture, then the system is more portable, but the devices require precise calibration and are susceptible to drifting
Solution Approach 1:
The patent replaces inertial measurement units (accelerometers, gyroscopes) with vision-based pose estimation. By using external camera observation instead of onboard inertial sensors, the system eliminates drift accumulation and calibration requirements while maintaining portability, as standard cameras do not suffer from the same reliability issues as wearable IMUs.
Data Source
AI summary
A system enables teleoperation of a robot based on a pose of a subject. The system includes an image capturing device and an operator system controller that are remotely located from a robotic system controller and a robot. The image capturing device captures images of the subject. The operator system controller maps a processed version of the captured image to a three-dimensional skeleton model of the subject and generates body pose information of the subject in the captured image. The robotic system controller communicates with the operator system controller over a network. The robotic system controller generates a plurality of kinematic parameters for the robot and causes the robot to take a pose corresponding to the pose of the subject in the captured image.