Robot Handover Simulation Using Motion-Captured Hand Interactions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for training robots to perform human-robot object handovers are expensive, hazardous, and lack standardized experimental settings, making cross-study comparison and optimization difficult due to the need for real-world interactions with humans.
Innovation Solution
A simulation-based framework, such as HandoverSim, uses motion capture data to simulate realistic human hand and object interactions, allowing robots to learn safe and seamless handover techniques in a virtual environment, with collision detection adjustments and separate controllers for the hand and object.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If real human-in-the-loop training is used for robot handover tasks, then training fidelity and safety are improved, but cost, hazard level, and difficulty of reproduction increase
Solution Approach 1:
The patent creates virtual copies of human hands, objects, and interaction scenarios in a simulated environment. Motion capture data from real human hands is used to generate realistic virtual hand models that replicate human behavior patterns, allowing robots to train with high-fidelity virtual representations without requiring actual human participants.
Solution Approach 2:
The patent introduces motion capture technology as an intermediary between real human movement and virtual simulation. The motion capture system records real human hand movements and translates them into virtual hand animations, serving as a mediator that transfers realistic human behavior patterns to the simulated environment without direct human-robot interaction.
2Measurement precision
If standardized experimental settings are implemented across studies, then cross-study comparison and optimization are improved, but adaptability to different research approaches may be reduced
Solution Approach 1:
The patent creates a universal simulation framework that can accommodate multiple research objectives, object types, and evaluation metrics within a single standardized environment. The virtual handover simulator supports diverse experimental configurations while maintaining consistent baseline conditions, allowing different studies to compare results meaningfully without sacrificing their unique research questions.
3Manufacturing precision
If collision detection is enabled between hand and object in simulation, then physical accuracy is improved, but simulation stability and training speed deteriorate due to noise and variability
Solution Approach 1:
The patent applies different collision detection settings to different regions of the simulation. Collision detection is selectively enabled or disabled depending on the specific interaction context - for example, allowing collisions between the robot gripper and object while preventing collisions between the virtual hand and object during certain phases of motion, thereby maintaining stability without sacrificing overall physical accuracy.
Data Source
AI summary
A system may simulate human motion for human-robot interactions, such as may involve a handover of an object. Motion capture can be performed for a hand grasping and moving an object to a location and orientation appropriate for a handover, without a need for a robot to be present or an actual handover to occur. This motion data can be used to separately model the hand and the object for use in a handover simulation, where a component such as a physics engine may be used to ensure realistic modeling of the motion or behavior. During a simulation, a robot control model or algorithm can predict an optimal location and orientation to grasp an object, and an optimal path to move to that location and orientation, using a control model or algorithm trained, based at least in part, using the motion models for the hand and object.