Federated Self-Supervised Perception Updates for Automated Driving
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The development of Automated Driving System (ADS) perception functionality faces challenges in cost-effectiveness and efficiency due to the need for large volumes of annotated training data, which is labor-intensive and costly, and the difficulty in sharing and annotating data collected by vehicles, especially concerning data privacy and bandwidth requirements.
Innovation Solution
A method for updating the perception function of ADS using a computer-implemented federated learning approach that combines self-supervised machine-learning algorithms, where locally updated model parameters from remote vehicles are consolidated to form a global algorithm, fine-tuned with annotated datasets, and then distilled into a more efficient machine-learning algorithm for in-vehicle perception modules, reducing the need for transferring large datasets and incorporating rare scenarios for improved performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If large volumes of annotated training data are used to improve perception functionality, then performance is improved, but cost and time for annotation increase significantly
Solution Approach 1:
The system uses self-supervised learning where the model learns from raw, unannotated sensor data by finding patterns and structures autonomously. The perception algorithm performs self-training by generating its own training signals from raw data, eliminating the need for manual annotation while improving performance on rare scenarios
Solution Approach 2:
The system performs preliminary self-supervised pre-training on large volumes of unannotated data before fine-tuning with small annotated datasets. This preliminary learning phase allows the model to develop robust features from raw data, reducing the subsequent annotation burden
2Reliability
If data is shared across vehicles to improve model training, then performance is improved, but data privacy concerns and bandwidth requirements increase
Solution Approach 1:
The system extracts only the essential model parameters and gradients needed for training improvement, rather than sharing complete raw sensor data. This extraction approach allows collaborative learning across vehicles while maintaining data privacy, as the actual sensitive data never leaves the local vehicle systems
Solution Approach 2:
The system uses federated learning as an intermediary mechanism where a central server coordinates model updates across vehicles without directly sharing raw data. The server aggregates parameter updates from multiple vehicles to create improved global models, acting as a mediator that enables collaboration while preserving data privacy
3Reliability
If complex machine-learning algorithms are deployed for rare scenario detection, then performance on rare scenarios is improved, but system size and computational requirements increase
Solution Approach 1:
The system applies different processing strategies to different data characteristics: self-supervised learning handles common scenarios efficiently, while the same framework is adapted to detect rare scenarios by leveraging the learned representations. This local optimization allows complex rare scenario detection without uniformly increasing overall system complexity
Data Source
AI summary
A computer implemented method and related aspects for updating a perception function of a plurality of vehicles having an Automated Driving System (ADS) are disclosed. The method includes obtaining one or more locally updated model parameters of a self-supervised machine-learning algorithm from a plurality of remote vehicles, and updating one or more model parameters of a global self-supervised machine-learning algorithm based on the obtained one or more locally updated model parameters. Further, the method includes fine-tuning the global self-supervised machine-learning algorithm based on an annotated dataset in order to generate a fine-tuned global machine-learning algorithm comprising one or more fine-tuned model parameters. The method further includes forming a machine-learning algorithm for an in-vehicle perception module based on the fine-tuned global machine-learning algorithm, and transmitting one or more model parameters of the formed machine-learning algorithm for the in-vehicle perception module to the plurality of remote vehicles.