Vehicle Perception Training Using Self-Supervised Bird's-Eye Views
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing autonomous driving systems require large amounts of annotated data for high-performance deep learning, which is costly and inefficient, especially as the number of vehicles in production fleets grows, and there is a need for cost-effective and efficient development of perception features without significant impact on size, power consumption, or cost.
Innovation Solution
A self-supervised continuous learning method for a perception-development module using vehicle-mounted sensors to generate training data, fuse data from multiple sources, and update perception models to provide a bird's eye view of the environment, enabling local training and model updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large amounts of annotated data are used for deep learning training, then model performance is improved, but cost and time consumption increase significantly
Solution Approach 1:
The system uses the vehicle's own sensors and existing perception models to generate training data autonomously. The perception model processes sensor data to create annotated training samples without external intervention, enabling self-service data generation that reduces dependency on expensive manual annotation while maintaining model performance
Solution Approach 2:
The system performs preliminary data processing and annotation during normal vehicle operation. By continuously generating and storing training data in the background using existing perception capabilities, the system prepares training datasets in advance, eliminating the need for time-consuming post-processing and enabling faster model retraining when needed
2Measurement precision
If manual annotation of sensor data is performed, then training data quality is improved, but cost increases significantly
Solution Approach 1:
The system automatically generates annotated training data using its own perception model. The model processes raw sensor data to produce labeled outputs, which are then stored as training datasets. This self-service approach eliminates manual annotation costs while maintaining consistent data quality standards
Solution Approach 2:
The perception model acts as an intermediary between raw sensor data and training data requirements. Instead of direct manual annotation, the system uses the perception model to transform sensor data into annotated training samples, providing a scalable automated pathway that maintains quality without human intervention
3Measurement precision
If data is uploaded from production vehicles for external annotation and training, then model improvement is achieved, but data security and privacy concerns arise
Solution Approach 1:
The system performs data processing and model training locally within the vehicle using onboard computing resources. Training data is generated and processed in-vehicle, eliminating the need to upload sensitive data to external servers. This self-service approach maintains model performance improvements while keeping data security and privacy within the vehicle's secure environment
Solution Approach 2:
Instead of the conventional approach of uploading data from vehicles to external servers for processing, the system inverts the data flow by bringing processing capabilities to the vehicle. Training data is generated and models are updated locally, reversing the traditional centralized processing model to prioritize data security and privacy
4Adaptability or versatility
If the perception system is continuously updated with new data, then model adaptability is improved, but system complexity increases
Solution Approach 1:
The system continuously generates and processes training data during normal vehicle operation. By maintaining continuous data generation and model updating, the system achieves ongoing adaptability to new scenarios and conditions. This continuous process integrates seamlessly with normal vehicle operation without requiring discrete complex update cycles
Solution Approach 2:
The perception model serves multiple functions: it processes sensor data for real-time perception, generates annotated training data, and enables continuous model updates. This multi-functionality reduces system complexity by consolidating multiple operations into a single versatile model that handles diverse tasks without requiring separate specialized systems
Data Source
Figure 1
Figure 2(a)~2(d)
Figure 3
AI summary
The present invention relates to methods and systems that utilize the production vehicles to develop new perception features related to new sensor hardware as well as new algorithms for existing sensors by using self-supervised continuous training. To achieve this the production vehicle's own perception output is fused with other sensors in order to generate a bird's eye view of the road scenario over time. The bird's eye view is synchronized with buffered sensor data that was recorded when the road scenario took place and subsequently used to train a new perception model to output the bird's eye view directly.