Robotic Gripping Point Detection for Irregular Objects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in efficiently identifying and gripping objects with arbitrary shapes, particularly in unpacking machines, due to unpredictable scenarios caused by flexible shapes, surfaces, and colors, necessitating improved methods for robotic interaction.
Innovation Solution
A computer vision pipeline utilizing zero-shot learning semantic segmentation and morphological skeletonization to determine gripping points on irregular surfaces, employing object prompts and transformer-based encoding to generate precise gripping points without extensive training data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional object recognition methods are used, then the system can identify objects, but it fails to handle arbitrary shapes and surfaces reliably
Solution Approach 1:
The patent segments the object detection task into multiple stages: first detecting object boundaries using a segmentation model, then identifying gripping points on the segmented regions. This segmentation approach allows the system to handle arbitrary shapes by breaking down the complex recognition task into manageable components, improving both adaptability to various shapes and reliability of identification.
Solution Approach 2:
The patent introduces an intermediary representation (the segmented object mask) that mediates between the raw image input and the gripping point detection. This intermediary segmentation layer provides a simplified, structured representation of the object that facilitates reliable gripping point identification even for arbitrary shapes, resolving the contradiction between handling shape variety and maintaining recognition reliability.
2Measurement precision
If extensive training data is used for object detection, then the system achieves high accuracy, but the system becomes complex and difficult to deploy
Solution Approach 1:
The patent employs a universal segmentation model that can handle multiple object types and shapes without requiring extensive retraining. This multi-functional segmentation approach provides high accuracy for gripping point detection across diverse objects while maintaining relatively simple system architecture, reducing the trade-off between precision and complexity.
Solution Approach 2:
The patent changes the approach from training-specific detectors for each object type to using a general segmentation model with adjustable parameters. This parameter-based adaptation allows the system to achieve high accuracy for gripping point detection on various objects without increasing system complexity through extensive training data requirements.
3Adaptability or versatility
If the segmentation mask covers the entire object, then the object is fully captured, but the gripping point identification becomes less precise
Solution Approach 1:
The patent segments the object mask into multiple regions of interest based on the segmented object structure. This segmentation allows the system to maintain full object coverage through the initial segmentation while then focusing on specific regions to identify gripping points with high precision, resolving the contradiction between overall coverage and localized precision.
Solution Approach 2:
The patent applies local quality analysis by examining different regions of the segmented object mask with different levels of detail. The system identifies gripping points in specific local regions where mechanical interaction is most effective, maintaining both comprehensive object coverage through the segmentation and precise gripping point localization through targeted analysis of critical regions.
Data Source
AI summary
A method for providing a gripping point for mechanical interaction on an object includes (i) providing at least one image of the object, the image being a result of an acquisition by at least one sensor, (ii) providing object prompts based on the provided image by utilising an object detector and/or an unsupervised or semi-supervised segmentation method, (iii) performing a segmentation on the provided images using the provided object prompts to generate an object mask representing the object, (iv) performing a morphological skeletonization in the provided image based on the generated object mask to obtain a polyline, (v) reducing the polyline to a defined number of points, and (vi) selecting at least one of the points to provide the gripping point for mechanical interaction on the object. A computer program, an apparatus, and a storage medium for this purpose are also disclosed.