RGBD Robot Hand Dataset Generation with 21-Keypoint Auto Annotation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The generation of high-quality data sets for robot applications is tedious, requiring manual annotation and inconvenient data acquisition, especially for 3D poses, which is time-consuming and inefficient.
Innovation Solution
A data generation method using a game engine to import a robot model, simulate an RGBD camera, control a human hand within the camera's field of view, and generate an annotated data set with 21 key points coordinates, leveraging the game engine's high restoration modeling for quick and accurate data set creation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual annotation is used to generate data sets, then annotation accuracy can be ensured, but the time consumption and labor intensity increase significantly
Solution Approach 1:
The patent uses virtual copying of real-world scenes into a game engine environment to create synthetic data sets. By replicating real scene geometries, textures, and lighting conditions in a virtual environment, the system generates annotated data without manual annotation while preserving the accuracy characteristics of real-world scenarios. This allows automated generation of large-scale annotated data sets that would otherwise require extensive manual labeling time.
2Quantity of substance
If additional sensors and devices are used to acquire 3D pose data, then data acquisition completeness is improved, but system complexity and cost increase
Solution Approach 1:
The patent introduces a game engine as an intermediary between the physical world and data acquisition. Instead of directly using multiple sensors in the physical environment, the system captures scene data once and uses the game engine to generate infinite variations of 3D pose data through virtual camera movements, object transformations, and scene manipulations. This intermediary approach eliminates the need for additional physical sensors while maintaining data completeness.
Solution Approach 2:
The patent transitions from physical 3D space measurement to virtual 3D space manipulation. By importing real-world scene data into a game engine, the system gains access to an additional dimension of control where virtual objects can be positioned, rotated, and scaled without physical constraints. This allows comprehensive 3D pose data acquisition from multiple viewpoints and configurations without adding physical sensors.
3Quantity of substance
If large scale data sets are generated manually, then data quantity is sufficient for machine learning, but the annotation process becomes prohibitively tedious
Solution Approach 1:
The patent implements self-service annotation through automated rendering in the game engine. The system automatically generates annotated data sets by rendering virtual scenes with pre-defined object models and their corresponding 3D pose information. The game engine itself performs the annotation function by automatically tracking and recording object positions, orientations, and other attributes during virtual camera movements, eliminating the need for manual annotation processes while generating large-scale data sets.
Data Source
AI summary
The present disclosure discloses a data generation method and apparatus, and a computer-readable storage medium, the method including: importing a robot model by using a game engine; simulating a Red-Green-Blue Depth (RGBD) camera by a scene capture component in the game engine; controlling a human hand of the imported robot model to move within a field of view of the RGBD camera by using a joint control module in the game engine; acquiring RGBD image data by using the RGBD camera; and generating an annotated data set with coordinates of 21 key points according to the RGBD image data and coordinate information of a 3D pose of the 21 key points.