Cross-Sensor Data Annotation Through 3D-to-2D Projection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing approaches to generating annotations for ground-truth datasets require independent labeling of each sensor's output data, leading to significant duplication of labor and substantial review overhead.
Innovation Solution
Leverage existing annotations from higher-dimensional sensor data, such as labeled point clouds, to automatically annotate lower-dimensional data, such as images, using a machine-learned boundary recognition model to project and process three-dimensional point cloud data into a two-dimensional space for context-based labeling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If independent labeling of each sensor's output data is performed, then comprehensive ground-truth datasets can be generated, but significant duplication of labor and substantial review overhead occur
Solution Approach 1:
The patent merges the annotation process across multiple sensor types by using annotations from a first sensor (e.g., LIDAR point cloud) to automatically generate annotations for a second sensor (e.g., camera image). This combining approach eliminates the need for independent manual labeling of each sensor while maintaining comprehensive ground-truth quality through cross-sensor contextual relationships.
Solution Approach 2:
The patent creates annotated data for the second sensor by copying and transforming annotations from the first sensor. The annotation system projects three-dimensional annotations from point cloud data into two-dimensional image space, effectively copying the semantic information and adapting it to the target sensor's dimensionality and coordinate system.
2Extent of automation
If three-dimensional point cloud data is projected into two-dimensional space for annotation, then automated annotation of images can be achieved, but processing complexity increases
Solution Approach 1:
The patent addresses the dimensionality mismatch between three-dimensional point cloud data and two-dimensional images by projecting the 3D annotations into 2D space. This dimensionality transformation enables the automated annotation of images using point cloud data while managing processing complexity through established projection mathematics and coordinate system transformations.
Data Source
AI summary
Example aspects of the present disclosure relate to an example computer-implemented method for data annotation for training machine perception models. The example method can include (a) receiving source sensor data descriptive of an object, the source sensor data having a source reference frame of at least three dimensions, wherein the source sensor data includes annotated data associated with the object; (b) receiving target sensor data descriptive of the object, the target sensor data having a target reference frame of at least two dimensions; (c) providing an input to a machine-learned boundary recognition model, wherein the input includes the target sensor data and a projection of the source sensor data into the target reference frame; and (d) determining, using the machine-learned boundary recognition model, a bounded portion of the target sensor data, wherein the bounded portion indicates a subset of the target sensor data descriptive of the object.