Projected Lidar Annotation for Autonomous Vehicle Image Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating training data for object detection systems in autonomous vehicles are time-consuming and inefficient, particularly due to the manual labeling of large volumes of image data, which is error-prone.
Innovation Solution
An object manager system is used to identify lidar points associated with static objects from sparsely annotated two-dimensional image datasets, projecting and annotating non-annotated images using lidar points to enhance the training data for machine learning models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual labeling of image data is used to generate training data, then accuracy of object detection can be improved, but time consumption and labor effort increase significantly
Solution Approach 1:
The system enables automated self-labeling of image data by using lidar points to automatically annotate objects in images. The lidar system captures 3D spatial information and automatically generates labels for objects without requiring manual human annotation, allowing the system to serve itself in the data labeling process.
Solution Approach 2:
The patent replaces the mechanical manual labeling process with an automated system that uses lidar sensors and image processing algorithms. Instead of human annotators manually marking objects, the system uses computational methods to automatically identify and label objects based on lidar data and image recognition models.
2Measurement precision
If manual labeling of image data is performed, then training data quality can be improved, but productivity of data generation decreases
Solution Approach 1:
The system continuously generates training data by processing lidar scans and images in real-time during vehicle operation. The automated labeling process operates continuously without interruption, converting every lidar scan and image pair into annotated training data, thereby maintaining continuous productive action throughout the vehicle's operational lifecycle.
Solution Approach 2:
The patent replaces slow manual labeling with high-speed automated processing using lidar sensors and image recognition algorithms. The system processes multiple images and generates labeled training data at speeds far exceeding manual capabilities, dramatically improving productivity while maintaining data quality through algorithmic analysis.
3Measurement precision
If more image data is collected for training, then model performance can be improved, but the complexity of processing and managing data increases
Solution Approach 1:
The system segments the data processing task by separating lidar-based object detection from camera-based image capture. By processing lidar points and images independently before combining them for annotation, the system reduces the complexity of handling combined sensor data. The lidar system handles 3D spatial information while the camera system handles 2D image data, dividing the complex processing into manageable segments.
Solution Approach 2:
The patent introduces lidar points as an intermediary that bridges the gap between raw sensor data and final annotations. The lidar points serve as a mediator that automatically identifies objects and provides spatial information that can be projected onto image data, simplifying the overall processing pipeline by providing a clear intermediate representation that eases the transition from raw data to annotated training data.
Data Source
AI summary
Techniques for densifying annotations associated with sensor data are discussed herein. For example, techniques may include projecting a subset of lidar points into an unannotated two-dimensional image. Based on the projecting, the annotations of lidar points projected into the image can be associated with the image data. A subset of pixels of pixels can be dilated, and the dilated pixels can be determined to intersect (e.g., overlap, touch, etc.). Based on the pixels intersecting (or being adjacent) and being associated with the same segment identifier, a contour can be rendered around the corresponding non-dilated pixels to identify an object in the image data.