Robot Grasp Region Selection Using Holding Success Probability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Robot systems face inefficiencies in holding target objects, especially when objects are wrapped in plastic bags or cushioning materials, due to changes in light reflection and surface shape, leading to inaccurate estimation of holding positions and increased failure rates.
Innovation Solution
An information processing apparatus that estimates holding success possibility using a pre-trained model to determine optimal holding regions for a robot, allowing it to adjust and improve holding operations based on probability distributions and feedback from holding attempts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the robot performs three-dimensional measurement and CAD model generation to estimate holding position, then the holding position can be estimated for objects stacked in bulk, but the work time increases and efficiency decreases
Solution Approach 1:
The system performs three-dimensional measurement and generates CAD models in advance before the actual picking operation. This preliminary preparation allows the holding position to be pre-calculated and stored, so that during actual operation, the robot can directly retrieve and use the pre-computed holding position information without performing time-consuming measurements and calculations in real-time, thus resolving the contradiction between measurement precision and work time
Solution Approach 2:
The system pre-computes multiple candidate holding positions and evaluates their success probabilities beforehand. By preparing multiple pre-calculated holding position candidates with their respective success probabilities stored in advance, the system can quickly select from pre-prepared options during operation rather than calculating in real-time, thereby reducing work time while maintaining accurate holding position estimation
2Reliability
If the robot repeats holding operations multiple times when initial holding fails, then the target object can eventually be held, but work efficiency decreases
Solution Approach 1:
The system introduces feedback mechanisms where the actual holding results are fed back to update and refine the holding success probability estimates. This feedback loop allows the system to learn from successful and failed holding attempts, continuously improving the accuracy of probability predictions. As a result, the system can make more informed decisions about which holding positions to attempt first, reducing the number of failed attempts and improving both reliability and productivity
Solution Approach 2:
The system dynamically adjusts the selection of holding positions based on real-time feedback from holding attempts. Instead of following a fixed sequence, the system updates the holding success probability distribution after each attempt and dynamically re-ranks candidate positions. This dynamic adaptation allows the robot to learn from each attempt and optimize subsequent attempts, improving both holding success rate and work efficiency by avoiding repeated failures at the same positions
3Object-affected harmful factors
If the target object is wrapped in plastic bags or cushioning material, then the object is protected, but light reflection and surface shape changes make holding position estimation difficult
Solution Approach 1:
The system changes the parameters used for holding position estimation from relying on visual appearance (light reflection and surface shape) to using three-dimensional geometric information and pre-computed CAD models. By transforming the estimation approach to use structural parameters from 3D measurements rather than surface optical properties, the system can accurately estimate holding positions for wrapped objects where light reflection and surface shape are obscured or distorted by packaging materials
Solution Approach 2:
The system introduces CAD models as an intermediary representation between the physical wrapped object and the holding position estimation process. Instead of directly analyzing the obscured visual appearance of wrapped objects, the system uses 3D measurement data to generate or update CAD models, which serve as accurate digital twins. These intermediary models provide reliable geometric information for calculating holding positions, bypassing the problem of light reflection and surface shape changes caused by packaging materials
Data Source
AI summary
The information processing apparatus includes an estimation unit to estimate information indicating a holding success possibility from an image of a plurality of the target objects by using a pre-trained model that estimates the information indicating the holding success possibility in at least one or more partial regions, a determination unit to determine a holding region for the robot to hold the target object among the partial regions based on the information indicating the holding success possibility, and a control unit to move the robot based on the holding region and cause the robot to perform a holding operation on the target object. In a case where the holding operation performed on the target object by the robot is failed, the determination unit determines a partial region, among the partial regions, satisfying a predetermined condition as a next holding region based on the information indicating the holding success possibility.