Multi-Sensor Label Association via 3D Projection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional user interface technologies cannot associate sensor measurements from multiple sensors of an autonomous vehicle, such as lidar and camera sensors, which limits the generation of accurate labels for training machine learning models and affects the performance of downstream tasks like object detection and sensor fusion.
Innovation Solution
A system that receives sensor data from multiple sensors and generates accurate associations between object tracks by projecting three-dimensional data from one sensor into the coordinate system of another, allowing users to identify corresponding objects across different sensor measurements, and provides a user interface to confirm these associations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional user interface technologies are used for labeling sensor data, then labeling can be performed for single sensor measurements, but associations between measurements from multiple sensors cannot be established
Solution Approach 1:
The system separates the labeling process into distinct phases: single-sensor labeling for each sensor type, followed by a separate association phase that links corresponding objects across different sensors. This segmentation allows conventional labeling techniques to be applied to each sensor independently while still achieving multi-sensor associations through the subsequent matching process.
Solution Approach 2:
The system introduces an intermediary association module that acts as a mediator between different sensor labeling systems. This module receives labeled measurements from multiple sensors and establishes correspondences between them using matching algorithms, enabling cross-sensor associations without requiring direct integration of the original labeling systems.
2Loss of information
If multiple sensors are used to capture sensor measurements, then more comprehensive data is available, but the complexity of associating labels across sensors increases
Solution Approach 1:
The system divides the complex multi-sensor labeling task into independent single-sensor labeling subtasks, each handled by conventional simpler systems. The segmentation reduces the complexity of individual labeling operations while preserving the completeness of multi-sensor data through the subsequent association phase that links results across sensors.
3Reliability
If accurate label associations between sensors are generated, then downstream tasks like object detection and sensor fusion improve, but the time and computational resources required for labeling increase
Solution Approach 1:
The system performs preliminary single-sensor labeling for each sensor type before conducting the association process. This preliminary action allows conventional labeling techniques to be applied efficiently to each sensor independently, reducing the overall computational burden compared to attempting to label and associate all sensors simultaneously.
Solution Approach 2:
The system implements a two-stage process where basic labeling is performed on each sensor separately (partial action), and then selective association is applied to link corresponding objects across sensors. This approach achieves sufficient labeling coverage without requiring exhaustive processing of all possible sensor combinations, thereby reducing time and computational resource requirements.
Data Source
AI summary
Methods and systems for associating labels between multiple sensors. One of the methods includes generating user interface presentation data that, when presented on a user device, causes the user device to display a user interface that: displays the images in the sequence of images and data identifying the first and second object tracks, and is configured to receive user inputs that associate first object tracks with second object tracks; and providing the user interface presentation data for presentation on the user device.