Vision-Guided Robotic Picking of Mixed Palletized Cuboidal Items
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Robotic manipulators struggle to efficiently handle and manipulate objects of varying sizes and arrangements, particularly when objects are not uniformly packed or oriented, which limits their effectiveness in production and storage tasks.
Innovation Solution
A system utilizing a combination of RGB and depth cameras, a neural network, and a robotic manipulator to detect perimeter edges and generate instructions for picking up and placing cuboidal items from pallets, regardless of uniform or non-uniform packing arrangements, by predicting pixel locations and generating three-dimensional coordinates for the robotic manipulator.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a robotic manipulator uses a specialized end effector for a particular class of objects arranged in a predefined manner, then the repeatability and efficiency of tasks are improved, but the ability to handle different objects and arrangements deteriorates
Solution Approach 1:
The system employs a universal vision-guided robotic manipulator that can handle multiple classes of objects and arrangements through software-based object recognition and adaptive path planning, rather than requiring specialized hardware for each object type. The vision system identifies object characteristics and the controller adjusts manipulation strategies accordingly.
Solution Approach 2:
The system uses dynamic adaptation through real-time vision feedback and predictive modeling to adjust manipulation strategies based on detected object positions, orientations, and arrangements. The robotic manipulator dynamically modifies its motion paths and grasping approaches based on the actual configuration of objects on the pallet.
2Speed
If the robotic manipulator is designed for predefined object arrangements, then the task execution speed is improved, but the system's ability to handle less defined arrangements deteriorates
Solution Approach 1:
The system performs preliminary vision-based identification and predictive modeling of object arrangements before manipulation begins. The vision system captures images and predicts object positions and orientations in advance, allowing the robotic manipulator to plan efficient paths while adapting to the actual arrangement detected.
Solution Approach 2:
The system replaces fixed mechanical positioning mechanisms with vision-based detection and software-based path planning. Instead of relying on predefined mechanical guides for object placement, the system uses computer vision to detect object positions and generates appropriate manipulation paths dynamically.
3Manufacturing precision
If the system uses specialized end effectors for specific object classes, then the manipulation precision is improved, but the ease of operation for different object types deteriorates
Solution Approach 1:
The system maintains a single end effector but changes operational parameters dynamically based on detected object characteristics. The vision system identifies object properties such as size, shape, and material, and the controller adjusts grasping forces, velocities, and paths to achieve precise manipulation appropriate for each object type.
Data Source
AI summary
Image data from an image sensor is input into a predictive model to identify perimeter edges of a set of cuboidal items held on a pallet. Depth data from a depth sensor is used to identify a top surface of a first cuboidal item of the set of cuboidal items. Coordinates associated with corners of the first cuboidal item are identified using information about the perimeter edges and the top surface. The coordinates are used to generate instructions for a robotic manipulator to pick up the first cuboidal item.