Autonomous Driving Perception Model Training for Accurate Auto-Labeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high cost and inefficiency of manually annotating training data for machine learning models in autonomous driving systems, particularly for spatiotemporal models, necessitates a more automated approach to generate high-fidelity annotations.
Innovation Solution
A two-stage training process using a first model to auto-label data for an auxiliary task, followed by pre-training a larger model to predict future or past scenarios, and then fine-tuning it for the main perception task, enabling the generation of accurate auto-labels for a production model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual annotation is used for training data, then annotation quality is high, but annotation cost and time consumption are extremely high
Solution Approach 1:
The patent uses a foundation model to generate synthetic training data that copies and transforms real-world driving scenarios. The foundation model processes input images and generates annotated output images with bounding boxes, segmentation masks, and depth information, effectively copying the annotation task from manual human annotation to automated model-based annotation.
Solution Approach 2:
The patent replaces the mechanical manual annotation process with an automated machine learning system. Instead of human annotators manually labeling data, the foundation model automatically processes images through multiple stages (image processing, object detection, segmentation, depth estimation) to generate annotations, substituting human cognitive work with computational processes.
2Productivity
If automated annotation is used, then annotation efficiency is high, but annotation accuracy may be insufficient
Solution Approach 1:
The patent segments the annotation process into multiple independent stages: image processing, object detection, segmentation, and depth estimation. Each stage is handled by specialized modules within the foundation model, allowing each component to optimize for its specific task while contributing to the overall annotation accuracy.
Solution Approach 2:
The patent performs preliminary processing of images through the foundation model before final annotation generation. The model pre-processes input images by extracting features, detecting objects, segmenting regions, and estimating depths in advance, which then informs the final annotation output and improves overall accuracy.
3Reliability
If large amounts of training data are collected, then model performance improves, but data collection and annotation cost increases
Solution Approach 1:
The foundation model serves itself by using its own capabilities to generate training data. The model processes input images through its internal mechanisms (image processing network, object detection network, segmentation network, depth estimation network) to automatically generate annotated training data, eliminating the need for external manual annotation services.
Solution Approach 2:
The patent changes the parameters of data generation by using the foundation model to transform input images into annotated output images. The model adjusts parameters such as bounding box coordinates, segmentation masks, and depth values based on its learned representations, enabling efficient generation of diverse training data with varying complexity levels.
Data Source
AI summary
The present invention relates to a computer-implemented method and a computing device. The method includes obtaining a second dataset including a set of sensor data sequences with associated annotations generated by a first machine learning model trained to perform a perception. Each sensor data sequence includes sensor data samples depicting a physical environment over a plurality of time instances. Then training a second machine learning model, using the second dataset, to perform an augmented perception task. The method also includes fine-tuning, using a third dataset, the second machine learning model, to perform the perception task, wherein the third dataset includes sensor data samples depicting a physical environment and that are annotated for the perception task. The method also includes providing the fine-tuned second machine learning model as a model for annotating training data for subsequent training of a production model, of an automated driving system, to perform the perception task.