ML Network Training for Vehicle Sensor Dropout Robustness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in maintaining accurate environmental perception when one or more sensors experience image frame dropouts, leading to reduced accuracy and robustness in navigation.
Innovation Solution
The development of techniques for training machine learning networks to predict the vehicle's surroundings even with missing image frames by simulating sensor drop rates during training, allowing the networks to learn and adapt to scenarios with incomplete data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If machine learning networks are trained using complete sensor data without simulating dropouts, then training simplicity is maintained, but the network's robustness to missing data deteriorates
Solution Approach 1:
The patent applies preliminary action by simulating sensor dropouts during the training phase before actual deployment. Training data is intentionally modified to include missing frames that mimic real-world sensor failures, allowing the network to learn robust features in advance. This preparatory training with synthetic dropouts ensures the network can handle missing data during operation without requiring complex redesign.
2Reliability
If machine learning networks are trained with simulated sensor dropouts, then robustness to missing data is improved, but training complexity increases
Solution Approach 1:
The patent implements parameter changes by systematically varying dropout parameters during training, including dropout rates, patterns, and temporal configurations. The training process dynamically adjusts these parameters to create diverse missing data scenarios, enabling the network to generalize across different failure modes. This parameter-based approach provides structured complexity rather than uncontrolled complexity.
Solution Approach 2:
The patent achieves universality by designing a training framework that handles multiple sensor types and dropout scenarios through a unified approach. The same training pipeline processes data from various sensors (cameras, LIDAR, radar) and applies consistent dropout simulations, making the solution broadly applicable across different autonomous vehicle configurations without requiring separate training systems.
3Quantity of substance
If all sensor data is used for training, then training data availability is maximized, but the network's ability to handle incomplete data deteriorates
Solution Approach 1:
The patent applies the taking out principle by deliberately extracting or removing portions of sensor data during training to create incomplete datasets. Instead of using all available sensor frames, the system intentionally excludes certain frames to simulate dropouts, forcing the network to learn from partial information. This extraction of data during training prepares the network to operate effectively when data is missing during deployment.
Data Source
AI summary
Provided are methods for training and evaluating machine learning networks. The methods can include obtaining ground truth data representing images of an environment of a vehicle. The methods can include determining a first plurality of subsets of the ground truth data. The methods can include mapping the first plurality of subsets to a plurality of sensors. The methods can include determining a second plurality of subsets of the ground truth data by removing at least one selected subset from the first plurality of subsets. The methods can include inputting the second plurality of subsets to at least one machine learning network. The methods can include predicting a surrounding view of the environment using the at least one machine learning network. Vehicles and non-transitory computer-readable storage media are also provided.