Grasping Hand Model Refinement for Human-Like Robotic Grasps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing robotic grasping methods struggle to predict human-like grasping of objects, especially in cluttered scenes, due to the complexity of hand-object interactions and occlusions, and often require RGB-D cameras which lack versatility.
Innovation Solution
A deep generative network using a coarse-to-fine approach with a classification task and regression task to predict a grasping hand model, employing a generative adversarial network (GAN) to refine hand poses and minimize interpenetration, using RGB images and the MANO model for increased degrees of freedom.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If simple grippers with reduced contact points are used, then device complexity is reduced, but grasping accuracy and versatility deteriorate
Solution Approach 1:
The patent uses a neural network to learn and copy human hand pose configurations from demonstration data. The system captures human hand poses during demonstrations and uses this learned knowledge to generate appropriate hand configurations for robotic grasping, enabling complex grasping behaviors without complex physical hardware
Solution Approach 2:
The patent employs a neural network that takes object properties and task requirements as inputs and outputs hand pose parameters (joint angles, finger positions). By changing the input parameters (object geometry, task type) and having the network adjust the output parameters (hand configuration), the system achieves versatile grasping with a fixed gripper structure
2Measurement precision
If RGB-D cameras are used for hand pose estimation, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent uses a neural network to learn the mapping from RGB images to hand pose parameters by copying training data from human demonstrations. The network learns to infer 3D hand poses from 2D RGB images without requiring depth information, effectively copying the visual processing capabilities needed for accurate pose estimation
Solution Approach 2:
The patent replaces the mechanical/physical depth sensing system (RGB-D camera) with a computational approach using a neural network that processes only RGB images. The neural network substitutes the physical depth measurement mechanism with learned visual inference, achieving similar functionality with simpler hardware
3Measurement precision
If deep learning algorithms are used for pose estimation, then grasping accuracy is improved, but computational time and complexity increase
Solution Approach 1:
The patent performs preliminary action by training the neural network offline using large datasets of human hand demonstrations. During online operation, the pre-trained network quickly infers hand poses from RGB images without requiring iterative optimization or complex computations, transferring the computational burden from runtime to training time
Solution Approach 2:
The patent copies training data from human demonstrations to train the neural network. By learning from extensive offline training data, the network acquires pre-learned knowledge that enables fast, accurate pose estimation during actual grasping tasks without requiring complex real-time computations
Data Source
AI summary
Method for determining a grasping hand model suitable for grasping an object by receiving an image including at least one object; obtaining an object model estimating a pose and shape of the object from the image of the object; selecting a grasp class from a set of grasp classes by means of a neural network, with a cross entropy loss, thus, obtaining a set of parameters defining a coarse grasping hand model; refining the coarse grasping hand model, by minimizing loss functions referring to the parameters of the hand model for obtaining an operable grasping hand model while minimizing the distance between the finger of the hand model and the surface of the object and preventing interpenetration; and obtaining a mesh of the hand represented by the enhanced set of parameters.