Robotic Picking Vision Model for Overlapping Object Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Robotic gripper systems face challenges in precisely identifying and handling products that are touching or overlapping on a conveyor belt, leading to slower picks, incorrect picks, or potential damage, and struggle with variability in product size, shape, and weight, requiring adaptable gripping mechanisms.
Innovation Solution
A multi-headed machine learning model is used to identify and segment target objects, track their location, and determine pose and orientation, allowing for efficient and accurate picking by separating object detection and tracking tasks, and utilizing synthetic training data for quick model updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional sensor and controller systems are used to identify products on a conveyor belt, then the system can operate with standard hardware, but the system struggles to precisely identify and handle products that are touching or overlapping, leading to slower picks and incorrect picks
Solution Approach 1:
The patent replaces traditional mechanical sensor systems with a vision-based machine learning system. A camera captures images of products on the conveyor belt, and a trained machine learning model processes these images to identify product locations, orientations, and characteristics. This substitution enables precise identification of overlapping products without the physical limitations of traditional sensors, achieving both high measurement precision and maintained productivity.
2Adaptability or versatility
If the gripper uses fixed gripping mechanisms, then the hardware is simple, but the system cannot adapt to variability in product size, shape, and weight
Solution Approach 1:
The patent implements dynamic gripping mechanisms that can adapt to different product characteristics. The gripper force and configuration are adjusted in real-time based on machine learning model predictions about product size, shape, and weight. This allows the same gripper hardware to handle varied products effectively without requiring multiple specialized grippers for each product type.
Solution Approach 2:
The system changes gripping parameters (force, position, orientation) based on machine learning analysis of product characteristics. The machine learning model predicts optimal gripping parameters from product images, and these parameters are dynamically adjusted to match each specific product, enabling versatile handling without complex mechanical reconfiguration.
3Reliability
If multiple separate models are used for object detection, segmentation, and tracking, then each task can be optimized independently, but the system complexity and computational requirements increase
Solution Approach 1:
The patent combines multiple machine learning functions (object detection, segmentation, and tracking) into a single integrated model. This unified model processes product images and outputs all necessary information for robotic picking in one computational pass, reducing system complexity while maintaining the reliability of each individual function through joint training and optimization.
Data Source
AI summary
Exemplary embodiments relate to a multi-headed machine learning model for a robotic pick-and-place station. The machine learning model works with the robot's vision system to identify, segment, and track moving objects for pickup by a robotic gripper. Due to the nature of the model, it can be quickly and generically adapted to a variety of different target objects in a robotic pick-and-place station. This allows the same model to be used in different contexts, which means that the same hardware and software can be applied even if the objects being picked change. Because the model is multi-headed, a single model can be trained to perform a variety of tasks, such as object detection, classification, orientation recognition, etc.