Autonomous Driving Perception Model Self-Learning via Virtual Scenario Hard Samples
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current autonomous driving perception technologies face high costs in real vehicle data collection and labeling, and these data cannot be effectively used for constructing LiDAR and visual data sets or iterating perception algorithms, limiting the efficiency of autonomous driving perception algorithms.
Innovation Solution
A vehicle autonomous driving perception self-learning method that involves acquiring artificially constructed and real scenario data, reconstructing scenarios to create virtual scenarios, rendering these scenarios, and using the differences between true and predicted data to identify hard sample data and scenarios for continuous model training until a preset learning condition is met, thereby improving algorithm iteration efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If real vehicle data collection and labeling is performed, then perception algorithm training data is obtained, but costs are high
Solution Approach 1:
The patent uses simulation technology to create virtual scenario data that copies real-world driving scenarios. Instead of collecting and labeling real vehicle data, the system generates synthetic training data through simulation environments, maintaining data quality while eliminating the high costs associated with real data collection and manual labeling.
Solution Approach 2:
The system implements automated data generation and labeling through simulation environments. The simulation platform automatically creates diverse driving scenarios, renders sensor data, and generates labeled training sets without requiring manual intervention, thereby reducing both time and labor costs while maintaining data reliability.
2Quantity of substance
If existing simulation data methods are used, then data accumulation is possible, but they cannot be used for perception algorithm iteration
Solution Approach 1:
The patent implements a systematic approach to varying simulation parameters including environmental conditions (weather, lighting), scenario types (corner cases, common scenarios), and sensor configurations. By dynamically adjusting these parameters, the system generates diverse training data that is specifically tailored for perception algorithm iteration and validation across different operating conditions.
Solution Approach 2:
The simulation system dynamically adapts scenario generation based on algorithm performance feedback. It automatically adjusts scenario difficulty, frequency, and types based on model predictions and ground truth comparisons, enabling continuous iteration and improvement of perception algorithms through progressive challenge escalation.
3Reliability
If massive data are required for training perception algorithm model, then model performance is improved, but costs are huge
Solution Approach 1:
The system employs hard sample mining to identify and focus on specific challenging cases where the model performs poorly. Instead of uniformly processing all training data, it selectively generates and prioritizes difficult scenarios (corner cases, edge cases) that provide the most learning value, thereby achieving efficient model improvement with reduced computational costs.
Solution Approach 2:
The patent implements a closed-loop training system where model predictions are continuously compared with ground truth from simulation data. The feedback mechanism identifies performance gaps and automatically generates targeted hard samples for retraining, enabling iterative model improvement without requiring ever-increasing data volumes.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A vehicle autonomous driving perception self-learning method and apparatus, and an electronic device are provided. The method includes: performing scenario reconstruction according to artificially constructed scenario data and real scenario data to obtain a virtual scenario, and obtaining true value data according to a real position and category of each target in the virtual scenario; rendering the virtual scenario to obtain a rendering result, and inputting the rendering result into an initial perception model to obtain perception prediction data; comparing the true value data and the perception prediction data on the basis of an overall recognition error policy and a local recognition error policy, when it is determined that there is a missing recognition target and/or a misrecognition target in the perception prediction data according to a comparison result, taking the missing recognition target and/or the misrecognition target as hard sample data, taking a scenario where the missing recognition target and/or the misrecognition target is located as a hard sample scenario, and performing driving perception self-learning on the initial perception model according to the hard sample data and the hard sample scenario until the initial perception model satisfies a preset learning end condition.