Synthetic 3D Model Annotation for Hand Tracking Datasets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current hand tracking systems face challenges such as a large dimensional configuration space, homogeneous skin color, frequent self-occlusion, and quick hand movement, making it difficult to create substantial and varied training datasets for machine learning models, especially in AR/VR applications where manual annotation is time-consuming and prone to inconsistencies.
Innovation Solution
A method using 3D synthetic models with accurately placed annotation points to generate thousands of 2D images with diverse poses, brightness conditions, and backgrounds, allowing for efficient creation of training datasets by inserting synthetic objects into 2D images based on brightness matching, thereby improving the precision and speed of annotation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual annotation is used to create training datasets, then annotation precision can be controlled, but annotation time becomes extremely long (hours to minutes per image)
Solution Approach 1:
The patent uses 3D synthetic models to generate 2D images that copy and replicate hand poses, backgrounds, and lighting conditions. These synthetic images serve as training data substitutes for manual annotation, dramatically reducing the time required while maintaining annotation quality through controlled generation parameters.
Solution Approach 2:
The system changes parameters such as brightness conditions, poses, and backgrounds by generating multiple synthetic images from a single 3D model. This allows efficient creation of diverse training datasets without manual intervention, transforming the annotation process from time-consuming manual work to automated parameter-based generation.
2Manufacturing precision
If manual annotation of 22 points is performed on 3D images, then annotation detail is achieved, but the process becomes time-consuming without physical constraints
Solution Approach 1:
The patent performs preliminary action by pre-defining annotation points on 3D synthetic models before generating 2D images. This ensures that annotation details are already established in the 3D space, and when 2D images are generated, the annotation points are automatically positioned correctly without requiring manual annotation during the 2D processing stage.
Solution Approach 2:
The system replaces the mechanical manual annotation process with automated computational generation. Instead of manually placing points on images, the system uses 3D rendering and image synthesis to automatically create annotated 2D images, substituting human manual operations with automated algorithms.
3Reliability
If training datasets are created for each new 2D camera and point of view, then camera-specific accuracy is achieved, but substantial time is required for data creation
Solution Approach 1:
The patent creates a universal training dataset generation system that can produce data for multiple camera viewpoints and configurations from a single 3D model. The synthetic image generator can adapt to different camera angles, distances, and lighting conditions by modifying generation parameters, eliminating the need to create separate datasets for each camera configuration.
Data Source
AI summary
The embodiments are directed to generating synthetic data. For example, in some examples, a method for creating 2D images of an object with annotation points is carried out by a processing unit. The method includes considering a synthetic 3D model of the object, named synthetic object, with annotation points correctly placed on the synthetic object. The method also includes generating several synthetic objects with different poses and different brightness conditions. Further, the method includes considering several 2D images with different backgrounds and different brightness conditions. For each 2D image, the method includes insertion of a generated synthetic object in the 2D image.