Object Pose Dataset Generation for Reflective and Transparent Objects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for annotating object poses in images, especially for photometrically challenging objects like shiny or transparent objects, are inefficient due to difficulties in determining key points and unreliable depth information from common sensors.
Innovation Solution
A method involving an acquisition device that physically contacts objects to determine key points, derive pose estimations, and calibrate camera poses, producing high-quality training datasets for machine learning models by capturing images with a camera that includes polarization modality to handle reflective and transparent surfaces accurately.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual annotation of object pose is used, then pose ground truth can be obtained, but the process is extremely time-consuming and difficult
Solution Approach 1:
The patent replaces manual mechanical annotation processes with an automated system combining a robotic arm for physical contact measurement and a polarization camera for optical capture. The robotic arm equipped with a contact sensor automatically touches key points on objects to obtain precise spatial coordinates, eliminating the need for manual pose annotation while maintaining high accuracy.
Solution Approach 2:
The system enables objects to annotate themselves by having the robotic arm physically contact key points on the object surface. The object's own geometric features serve as the annotation targets, and the automated measurement system captures the pose information without requiring external manual intervention for each annotation task.
2Measurement precision
If common depth sensors are used for photometrically challenging objects, then depth information can be obtained, but the sensors fail to return correct depth information due to reflection and refraction artifacts
Solution Approach 1:
The patent introduces a polarization camera as an intermediary device that captures optical properties (polarization states) of light reflected from or transmitted through objects. This intermediary measurement provides complementary information that helps distinguish between actual object geometry and artifacts caused by reflection or refraction, enabling more reliable depth estimation for photometrically challenging materials.
Solution Approach 2:
The system changes the measurement parameter from traditional intensity-based depth sensing to polarization-state-based optical property measurement. By measuring the polarization state of light interacting with the object, the system can differentiate between light affected by reflection/refraction and light carrying true geometric information, thereby improving measurement reliability for shiny and transparent objects.
3Ease of manufacture
If no texture is used on photometrically challenging objects, then the objects maintain their natural appearance, but key points cannot be determined for pose annotation
Solution Approach 1:
The patent replaces texture-based visual key point detection with a mechanical contact-based measurement system. The robotic arm with contact sensor directly touches potential key points on the object surface, and the contact sensor detects the precise location through physical interaction rather than visual texture analysis, enabling key point determination on textureless photometrically challenging objects.
Data Source
Figure 1
Figure 2(a)~2(e)
Figure 3
AI summary
A method for producing a training dataset comprising at least one image of a scene and an associated pose annotation of at least one object (40) within the scene, the method comprising: - providing an acquisition device and defining a reference frame relative to a fixed point; - obtaining a digital model of the object (40); - determining a position of key points of the object (40) by physical contact between the acquisition device and each key point; - locating the key points on the digital model and deriving a pose estimation of the object (40); - calibrating a pose of a camera (62) mounted to the acquisition device; - acquiring an image of the scene with the camera (62), wherein the image includes the object (40); - outputting the image and a corresponding relative pose of the object (40), wherein the corresponding relative pose is determined based on the pose of the camera (62) and the pose estimation of the object (40).