Gripping Point Selection via Skeletonized Object Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in efficiently identifying and gripping objects with arbitrary shapes, particularly in unpacking machines, due to unpredictable scenarios caused by flexible shapes, surfaces, and colors, necessitating robust and safe mechanical interaction.
Innovation Solution
A computer vision pipeline utilizing zero-shot learning semantic segmentation and morphological skeletonization to determine gripping points on objects, employing a Segment Anything Model (SAM) and transformer-based encoding, which generates object prompts and polylines for robotic interaction without requiring extensive training data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional object recognition methods are used for arbitrary-shaped objects, then the system can process objects with fixed shapes, but it fails to handle flexible shapes, surfaces, and colors reliably
Solution Approach 1:
The patent segments the object detection task into multiple stages: first detecting object boundaries using a segmentation model, then identifying critical points on the segmented regions, and finally selecting gripping points based on geometric properties. This segmentation approach allows the system to handle arbitrary shapes by breaking down the complex recognition task into manageable steps, improving both adaptability and reliability.
Solution Approach 2:
The patent performs preliminary object boundary detection and segmentation before identifying critical points and selecting gripping points. By pre-segmenting the object into distinct regions and identifying key geometric features in advance, the system establishes a reliable foundation for subsequent gripping point selection, enabling reliable manipulation of arbitrary-shaped objects.
2Ease of operation
If the system attempts to localize critical points on arbitrary objects, then it can identify gripping points, but the complexity of coordinating visual recognition and mechanical control increases
Solution Approach 1:
The patent replaces complex mechanical coordination systems with a computer vision-based approach. Instead of relying on sophisticated mechanical sensors and actuators to directly perceive and grasp objects, the system uses image processing, segmentation algorithms, and geometric analysis to automatically identify gripping points, simplifying the overall system architecture while maintaining high operational effectiveness.
Solution Approach 2:
The patent introduces an intermediary computational layer between visual perception and mechanical control. This intermediary layer processes images, segments objects, identifies critical points, and selects optimal gripping locations, serving as a bridge that translates visual information into actionable mechanical commands. This intermediary processing simplifies the coordination between visual recognition and mechanical control by providing a standardized interface.
3Measurement precision
If the system uses complex segmentation models to handle arbitrary shapes, then it can accurately identify object boundaries, but the computational time and processing resources increase
Solution Approach 1:
The patent segments the object detection process into distinct stages: boundary detection, region segmentation, critical point identification, and gripping point selection. By dividing the complex computation into sequential steps, each step can be optimized independently, reducing overall computational time while maintaining high measurement precision through focused processing at each stage.
Solution Approach 2:
The patent applies partial action by focusing computational resources on the most critical aspects of object analysis - specifically, identifying boundaries and key geometric features - rather than processing every pixel and detail. This selective processing approach maintains sufficient measurement precision for reliable gripping while significantly reducing computational time and resource requirements.
Data Source
Figure 1
Figure 2
AI summary
The invention relates to a method (100) for providing a gripping point for mechanical interaction on an object (1), comprising the following steps: - Providing (101) at least one image of the object (1), the image being a result of an acquisition by at least one sensor (2), - Providing (102) object prompts based on the provided image by utilising an object detector and/or an unsupervised or semi-supervised segmentation method, - Performing (103) a segmentation on the provided images using the provided object prompts to generate an object mask representing the object (1), - Performing (104) a morphological skeletonization in the provided image based on the generated object mask to obtain a polyline, - Reducing (105) the polyline to a defined number of points, - Selecting (106) at least one of the points to provide the gripping point for mechanical interaction on the object (1). Furthermore, the invention relates to a computer program, an apparatus, and a storage medium for this purpose.