Patching Deep Neural Networks for Autonomous Vehicles
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous and semi-autonomous vehicles face lengthy validation and revalidation processes due to failure modes in deep neural networks (DNNs), leading to extended downtime and user dissatisfaction, as conventional approaches require retraining and deactivation of DNNs, which are time-consuming and costly.
Innovation Solution
The introduction of patch neural networks (PNNs) that can be deployed to address specific failure modes without deactivating the DNN, allowing for rapid resolution of issues by generating additional signals and outputs, and a fusion mechanism to evaluate outputs from both DNNs and PNNs, maintaining the original performance of the DNN while addressing new failure modes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional retraining and revalidation approaches are used to address DNN failure modes, then the DNN can be updated to fix identified issues, but the process takes six months to a year or more and requires deactivation of the DNN during this period
Solution Approach 1:
The system segments the DNN into multiple versions (original DNN and retrained DNN) that can operate independently and simultaneously. The original DNN remains active while the retrained DNN undergoes validation, allowing continuous operation without deactivation during the validation period.
Solution Approach 2:
A fusion mechanism acts as an intermediary between the original DNN and retrained DNN, evaluating outputs from both systems and determining which output to rely on. This mediator enables smooth transition and comparison without requiring complete deactivation of either system.
2Reliability
If the DNN is deactivated for retraining and validation, then the failure mode can be addressed through extensive testing, but autonomous functionality is lost during this period causing negative user experience
Solution Approach 1:
The system maintains segmented operational capability by keeping the original DNN active while developing the retrained DNN. This segmentation ensures that autonomous functionality remains available through the original DNN even while improvements are being developed and validated.
Solution Approach 2:
The retrained DNN is developed and validated in advance while the original DNN continues to provide service. This preliminary action allows the improved system to be ready for deployment without interrupting current autonomous operations.
3Reliability
If retrained DNNs are deployed to address failure modes, then the identified issues can be fixed, but new or additional failure modes may surface that were not identified during training and validation
Solution Approach 1:
The fusion mechanism provides feedback by evaluating outputs from both the original and retrained DNNs. This feedback loop allows the system to identify when the retrained DNN introduces new failure modes or performs poorly on unseen scenarios, enabling continuous improvement and adaptation.
Solution Approach 2:
The system dynamically switches between the original and retrained DNNs based on real-time performance evaluation by the fusion mechanism. This dynamic adaptation allows the system to respond to unforeseen failure modes by reverting to or switching between versions, maintaining robustness against diverse failure scenarios.
Data Source
AI summary
In various examples, rapid resolution of deep neural network (DNN) failure modes may be achieved by deploying patch neural networks (PNNs) trained to operate effectively on the failure modes of the DNN. The PNNs may operate on the same or additional data as the DNN, and may generate new signals in addition to those generated using the DNN that address the failure modes of the DNN. A fusion mechanism may be employed to determine which output to rely on for a given instance of the DNN/PNN combination. As a result, failure modes of the DNN may be addressed in a timely manner that requires minimal deactivation or downtime for the DNN, a feature controlled using the DNN, and/or semi-autonomous or autonomous functionality as a whole.