3D Object Modeling for Stable Robotic Pick Order Determination
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Robots face challenges in picking objects from a stack due to incomplete information from two-dimensional images, as they cannot accurately detect the dimensions of objects with different sizes and orientations, leading to potential collisions or instability when placing objects.
Innovation Solution
A method that generates a 3D model of objects by matching two-dimensional faces with stored prototype objects, creating new prototypes if necessary, and using image processing techniques to determine unknown dimensions, allowing the robot to select and orient objects for grasping and placement efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the robot uses two-dimensional images to detect objects in the stack, then the detection process is simple and fast, but the robot cannot accurately determine the dimensions and orientation of objects with different sizes
Solution Approach 1:
The patent transitions from two-dimensional image data to three-dimensional object models by matching detected object faces with stored prototype objects that contain full dimensional information. This dimensionality change enables the robot to infer depth and orientation information that cannot be directly observed from 2D images alone.
Solution Approach 2:
The patent creates simplified prototype objects that represent common object types with known dimensions. These prototypes serve as templates that can be matched against detected object faces, allowing the robot to quickly infer complete object properties without complex real-time 3D scanning.
2Stability of the object's composition
If the robot picks objects from the top of the stack first, then it is unlikely to cause other objects to shift or fall, but it may not be possible to remove all top objects before moving to lower objects
Solution Approach 1:
The patent performs preliminary analysis of the stack structure using 3D models to identify objects that can be safely removed. By pre-determining the pick order based on simulated object interactions and stability constraints, the robot can efficiently select objects for removal without causing stack collapse, combining safety with productivity.
3Productivity
If the robot removes objects closest to itself first from the stack, then the picking process is efficient, but objects may not always be stacked in a way that allows removal without knocking other objects over
Solution Approach 1:
The patent uses feedback from 3D model analysis and simulated object interactions to dynamically adjust the pick order. The system continuously evaluates which objects can be safely removed based on their position, orientation, and interaction with neighboring objects, allowing the robot to adapt its picking strategy to maintain stack stability while maximizing efficiency.
4Reliability
If the robot places objects without knowing their full dimensions, then the placement process is faster, but the robot cannot determine the optimal orientation to prevent objects from falling or being damaged
Solution Approach 1:
The patent uses stored prototype objects that contain pre-defined dimensional and orientation information. By matching detected object faces with prototypes, the robot quickly retrieves the necessary dimensional data without performing time-consuming measurements, enabling safe and efficient object placement.
Data Source
AI summary
Methods and apparatus for object detection and pick order determination for a robotic device are provided. Information about a plurality of two-dimensional (2D) object faces of the objects in the environment may be processed to determine whether each of the plurality of 2D object faces matches a prototype object of a set of prototype objects stored in a memory, wherein each of the prototype objects in the set represents a three-dimensional (3D) object. A model of 3D objects in the environment of the robotic device is generated using one or more of the prototype objects in the set of prototype objects that was determined to match one or more of the 2D object faces.