3D Structure Annotation for Faster Multi-Modal Perception Labeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The manual process of creating annotated training examples for machine learning perception components, particularly in multiple perception modalities, is time-consuming and costly, limiting the number of training images and affecting performance.
Innovation Solution
An annotation tool that facilitates fast and efficient annotation of perception inputs by generating 3D models from reference frames, allowing for the propagation of these models into target frames, thereby enabling quick and accurate creation of 2D and 3D annotation data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual annotation process is used for creating annotated training examples, then annotation accuracy can be maintained, but time consumption and cost increase significantly
Solution Approach 1:
The system performs preliminary actions by automatically generating initial 3D models from reference frames before manual annotation is needed. These pre-generated models serve as starting points that reduce the time required for manual annotation while maintaining accuracy through subsequent refinement.
Solution Approach 2:
The system creates copies of 3D models from reference frames and propagates them to target frames. This copying approach allows rapid generation of annotation data across multiple frames while maintaining consistency with the source data, significantly reducing time consumption compared to manual annotation of each frame.
2Adaptability or versatility
If manual annotation is performed for multiple perception modalities, then comprehensive training data can be created, but the process becomes excessively time-consuming and costly
Solution Approach 1:
The annotation tool implements multi-functionality by handling multiple perception modalities (2D images, 3D point clouds, depth maps) through a unified 3D model propagation framework. This universal approach enables comprehensive multi-modality annotation while maintaining high productivity through automated model generation and propagation across different data types.
3Productivity
If 3D models are generated and propagated from reference frames, then annotation speed increases significantly, but system complexity increases
Solution Approach 1:
The system introduces 3D models as intermediary representations that mediate between reference frames and target frames. These intermediaries enable rapid annotation propagation while managing complexity by providing a standardized intermediate format that simplifies the transformation process between different frames and modalities.
4Reliability
If more training images are annotated, then perception component performance improves, but resource requirements increase
Solution Approach 1:
The system efficiently generates large volumes of annotated training data by copying and propagating 3D models across multiple frames and modalities. This approach enables the creation of extensive training datasets with consistent annotations, improving perception component reliability while managing resource requirements through automated generation rather than manual annotation of each individual frame.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer-implemented method of creating one or more annotated perception inputs, the method comprising, in an annotation computer system: receiving a plurality of captured frames, each frame comprising a set of 3D structure points, in which at least a portion of a common structure component is captured; computing a reference position within at least one reference frame of the plurality of frames; generating a 3D model for the common structure component by selectively extracting 3D structure points of the at least one reference frame based on the reference position within that frame; determining an aligned model position for the 3D model within a target frame of the plurality of frames based on one or more manual alignment inputs received in respect of the target frame at a user interface whilst rendering the 3D model for manually aligning the 3D model with the common structure component in the target frame; and storing annotation data of the aligned model position in computer storage, in association with at least one perception input of the target frame for annotating the common structure component therein.