DNN Watermark Fault Detection for Real-Time Autonomous Driving
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems for detecting faults in deep neural networks (DNNs) used in autonomous driving are computationally intensive and often require multiple instances of the DNN to be executed, which hinders real-time or near real-time fault detection, compromising the safety and reliability of autonomous systems.
Innovation Solution
The use of motifs, watermarks, and signatures as inputs or modifications to inputs within a single instance of a DNN to detect transient and permanent faults, reducing computational expense and enabling real-time fault detection by comparing predicted outputs to expected outcomes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple instances of DNN are executed to detect faults, then fault coverage is improved, but computational expense and processing time increase significantly
Solution Approach 1:
The patent uses watermark images as simplified copies or representations of actual driving scenes. Instead of processing multiple full DNN instances on complete images, the system embeds watermarks in images and processes only the watermark detection portion, dramatically reducing computational load while maintaining fault detection capability.
Solution Approach 2:
The patent changes the parameter being processed from full-image semantic segmentation to watermark-specific detection. By modifying the input to include detectable watermarks and changing the output to focus on watermark integrity verification, the system achieves fault detection with minimal computational expense.
2Measurement precision
If multiple instances of DNN are executed concurrently, then fault detection accuracy is improved, but memory and hardware utilization increase
Solution Approach 1:
The patent makes the DNN multi-functional by enabling it to perform both normal semantic segmentation tasks and fault detection through watermark verification using a single instance. The same network processes regular images for driving decisions and simultaneously detects watermarks for fault detection, eliminating the need for separate dedicated fault detection instances.
Solution Approach 2:
The patent merges the fault detection function with the primary semantic segmentation function. By embedding watermarks in the input images and having the DNN process both the scene understanding and watermark verification in a unified framework, the system combines multiple functions into one execution stream, reducing memory requirements.
3Reliability
If conventional fault detection methods are used, then comprehensive fault analysis is achieved, but real-time fault detection capability is lost
Solution Approach 1:
The patent applies preliminary action by embedding watermarks in images before they are processed by the DNN. These watermarks contain pre-encoded information that enables rapid fault detection. The watermark is prepared in advance and integrated into the image stream, allowing the DNN to perform quick verification without requiring complex post-processing or additional analysis time.
Data Source
AI summary
In various examples, motifs, watermarks, and/or signature inputs are applied to a deep neural network (DNN) to detect faults in underlying hardware and/or software executing the DNN. Information corresponding to the motifs, watermarks, and/or signatures may be compared to the outputs of the DNN generated using the motifs, watermarks and/or signatures. When a the accuracy of the predictions are below a threshold, or do not correspond to the expected predictions of the DNN, the hardware and/or software may be determined to have a fault-such as a transient, an intermittent, or a permanent fault. Where a fault is determined, portions of the system that rely on the computations of the DNN may be shut down, or redundant systems may be used in place of the primary system. Where no fault is determined, the computations of the DNN may be relied upon by the system.