Synthetic Pile Image Training for Workpiece Pose Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating inference models for workpieces from captured images are cumbersome and inefficient.
Innovation Solution
A system that generates virtual pile images from multiple workpiece images viewed from different viewpoints, using machine learning to train an inference model for inferring workpiece information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If real captured images of piled workpieces are used for training, then the inference model can learn from actual data, but it becomes extremely difficult to acquire training data with accurate ground truth labels for workpiece positions and orientations
Solution Approach 1:
The patent creates virtual copies of workpieces through 3D modeling and renders synthetic pile images from these models. Instead of capturing real workpieces and manually labeling them, the system generates training images computationally, automatically knowing the ground truth positions and orientations of all workpieces in each synthetic pile image.
Solution Approach 2:
The system makes the training data generation self-service by automatically creating synthetic pile images with embedded ground truth information through computational rendering. The process eliminates manual labeling by having the simulation environment itself provide the annotated training data, where the renderer naturally knows the pose and position of each virtual workpiece.
2Quantity of substance
If manual labeling of captured images is performed to obtain ground truth data, then training data can be created, but the process becomes extremely time-consuming and labor-intensive
Solution Approach 1:
The patent replaces manual labeling with automated synthetic image generation. By copying workpiece geometries into virtual scenes and rendering them with known transformations, the system produces training images with automatic ground truth labels, eliminating the time-consuming manual annotation process entirely.
3Measurement precision
If multiple captured images from different viewpoints are processed to create three-dimensional models, then workpiece position information can be obtained, but the process becomes complex and requires multiple imaging devices
Solution Approach 1:
The patent creates simplified 3D model copies of workpieces that capture essential geometric features without requiring full multi-view reconstruction. These simplified models are sufficient for generating diverse training images while avoiding the complexity of precise multi-camera calibration and three-dimensional reconstruction pipelines.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system includes: an image generation unit configured to generate a plurality of workpiece images each of which shows a workpiece viewed from a different viewpoint; an image synthesis unit configured to generate, based on the plurality of workpiece images, one or more virtual pile images showing a plurality of piled workpieces; and a training unit configured to generate, by machine learning using the one or more virtual pile images, an inference model configured to infer workpiece information regarding one or more of the workpieces shown in the pile image.