Robot Grasping Strategy Selection via Machine Learning Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Robots struggle to determine appropriate grasp strategies for manipulating objects due to the complexity of grasping diverse objects, as existing methods lack effective machine learning-based solutions for selecting optimal grasp strategies considering object features and environmental context.
Innovation Solution
A system utilizing trained machine learning models processes sensor data to generate grasp regions and semantic indications, enabling robots to select grasp strategies that include grasp directions, types, and manipulations, and determine end effector poses for successful grasping by processing vision data through models like object detection and classification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional programming methods are used to control robot grasping, then the robot can execute predefined grasp actions, but it cannot adapt to diverse objects and complex environmental contexts
Solution Approach 1:
The patent replaces traditional mechanical control systems with machine learning models that process sensor data and automatically determine grasp strategies. The system uses trained neural networks to analyze object features and environmental context, substituting complex mechanical programming with intelligent data-driven decision-making algorithms.
Solution Approach 2:
The patent changes the control parameters from fixed programmed instructions to dynamic variables derived from machine learning model outputs. The system adjusts grasp pose, approach direction, and manipulation actions based on real-time sensor data processing, allowing adaptive response to different objects and contexts without reprogramming.
2Reliability
If multiple candidate grasp strategies are evaluated to find the optimal grasp, then grasping success rate improves, but computational time and processing complexity increase
Solution Approach 1:
The patent performs preliminary actions by pre-training machine learning models on extensive datasets of objects and grasp scenarios. The models learn optimal grasp strategies in advance during the training phase, so that during actual operation, the system can quickly retrieve and apply pre-learned strategies without performing exhaustive real-time evaluation of multiple candidates.
Solution Approach 2:
The patent uses copying by training the machine learning models on simulated training examples and simulated data from virtual robots attempting simulated manipulations. The models learn from copied experiences in simulation environments, which are then transferred to real-world robot operation, reducing the need for extensive real-time computation.
3Productivity
If machine learning models are trained using only simulated data, then training efficiency improves, but the model performance degrades when applied to real-world scenarios
Solution Approach 1:
The patent implements a dynamic training approach where the system transitions from static simulated training to dynamic real-world learning. The machine learning models are initially trained on simulated data for efficiency, then continuously adapted and refined using real sensor data from actual robot operations, creating a dynamic learning pipeline that improves real-world performance over time.
Solution Approach 2:
The patent incorporates feedback mechanisms where the system uses real sensor data from actual grasping attempts to evaluate model performance and update training examples. Successful and unsuccessful real-world grasps provide feedback that is used to refine the machine learning models, closing the loop between simulation and reality and improving transferability.
Data Source
Figure 1
Figure 2A~2B
Figure 3
AI summary
Grasping of an object, by an end effector of a robot, based on a grasp strategy that is selected using one or more machine learning models. The grasp strategy utilized for a given grasp is one of a plurality of candidate grasp strategies. Each candidate grasp strategy defines a different group of one or more values that influence performance of a grasp attempt in a manner that is unique relative to the other grasp strategies. For example, value(s) of a grasp strategy can define a grasp direction for grasping the object (e.g., "top", "side"), a grasp type for grasping the object (e.g., "pinch", "power"), grasp force applied in grasping the object, pre-grasp manipulations to be performed on the object, and/or post-grasp manipulations to be performed on the object.