Automated Labeling for Pick-and-Place Using 3D Synthetic Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional labeling methods for pick-and-place systems are inefficient due to the need for manual capture of two-dimensional object images and the requirement for large datasets for machine learning, making the process time-consuming.
Innovation Solution
An automated method and device that generate three-dimensional pictures, capture two-dimensional images, recognize object regions, calculate exposed ratios, and define pick-and-place regions, allowing for automated labeling and pick-and-place operations using a robotic arm and controller.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual capture of two-dimensional object images is used for labeling, then the labeling process can be performed, but the process is time-consuming and inefficient
Solution Approach 1:
The patent uses a three-dimensional model as a copy of the physical object to generate synthetic two-dimensional images. This virtual copy eliminates the need for manual photographing while preserving all necessary visual characteristics for labeling, thereby dramatically improving labeling efficiency and eliminating time loss associated with manual image capture
Solution Approach 2:
The patent replaces the manual mechanical process of photographing physical objects with an automated computer-generated imaging system. The system uses three-dimensional modeling and rendering to substitute the mechanical act of manual photography, achieving automated image generation without human intervention and thus resolving the efficiency and time loss problems
2Reliability
If a large number of two-dimensional object images are collected for machine learning, then the learning accuracy can be improved, but the data collection process becomes time-consuming
Solution Approach 1:
The patent generates synthetic two-dimensional images from three-dimensional models, creating unlimited copies of object views from different angles and conditions. This virtual copying provides sufficient training data for machine learning without requiring physical photographing, thereby maintaining learning accuracy while eliminating time-consuming data collection
Solution Approach 2:
The patent performs preliminary action by pre-establishing accurate three-dimensional models of objects before image collection is needed. These models serve as a ready source for generating any required two-dimensional training images on demand, eliminating the need for time-consuming field data collection while ensuring sufficient data availability for machine learning
Data Source
AI summary
A method further includes the following steps. Firstly, a three-dimensional picture under a generated background condition is generated, wherein the three-dimensional picture includes a three-dimensional object image. Then, a two-dimensional picture of the three-dimensional picture is captured, wherein the two-dimensional picture includes a two-dimensional object image of the three-dimensional object image. Then, an object region of the two-dimensional object image is recognized. Then, an exposed ratio of an exposed area of an exposed region of the object region to an object area of the object region is obtained. Then, whether the exposed ratio is greater than a preset ratio is determined. Then the exposed region is defined as the pick-and-place region when the exposed ratio is greater than the preset ratio.