Neural Network Rule Visualization for Interpretable Vehicle Guidance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks used in systems like vehicle guidance operate as 'black boxes,' making it difficult to understand how predictions are made, leading to decisions that may be unacceptable to human interpretation, such as excessive acceleration or lane changes, which can result in unsafe vehicle operations.
Innovation Solution
An explanation system is developed to generate hierarchical rule-based systems that model neural network operations using decision trees and binary logic, allowing for the interpretation of complex decisions and identifying unacceptable scenarios, which can be addressed by adding hardware and software to prevent unsafe operations or re-training the neural network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If neural networks are used for vehicle guidance decisions, then automation and decision-making speed are improved, but interpretability and safety are worsened due to black box operations
Solution Approach 1:
The patent introduces an explanation system as an intermediary between the neural network and the vehicle control system. This explanation system generates human-understandable explanations for neural network predictions, allowing safety validators to interpret and verify decisions before execution, thus resolving the contradiction between automation and safety
Solution Approach 2:
The patent implements a feedback loop where the explanation system continuously provides interpretable information about neural network decisions back to the control system. This feedback enables real-time validation and adjustment, ensuring that automated decisions meet safety requirements while maintaining automation benefits
2Measurement precision
If complex neural networks are used for accurate predictions, then prediction accuracy is improved, but interpretability and understanding of decisions are worsened
Solution Approach 1:
The patent extracts interpretability information from the complex neural network by creating a separate explanation system. This explanation system takes the neural network's inputs and outputs and generates human-understandable explanations without requiring modification to the original high-accuracy neural network model, thus preserving prediction accuracy while gaining interpretability
Data Source
AI summary
A computer that includes a processor and a memory, the memory including instructions executable by the processor to train a neural network to input data and output a prediction. A policy can be generated based on the data. Force features can be generated based on the policy. Decision nodes can be trained based on force features and a binary vector from the trained neural network. A decision tree can be generated based on the decision nodes. A decision can be generated by inputting a policy to the decision tree. The decision can be compared to the prediction and the neural network re-trained based on a difference between the decision and the prediction.