Neural Network Assessment System for Explainable AI
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks and other machine-learning algorithms are complex and often treated as 'black boxes,' making it difficult to understand why specific predictions are made, which is a challenge in applications like autonomous train control where transparency and reliability are crucial.
Innovation Solution
A computer-implemented method and system for assessing neural networks by processing datasets through a data structure that compares input and output datapoints, using neighborhood criteria to determine the reliability of predictions and visualize the processing pipeline, enabling explainable AI (XAI) and monitoring during inference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If deep neural networks are used for complex tasks like object detection and predictive maintenance, then processing speed and pattern recognition accuracy are improved, but interpretability and understanding of decision-making processes deteriorate
Solution Approach 1:
The patent introduces an assessment system that acts as an intermediary between the neural network and the user. This system includes a data structure that stores information about the neural network's processing pipeline, intermediate layers, and prediction confidence levels. The assessment system queries this data structure to generate explanations about why the neural network made certain predictions, thereby mediating the loss of interpretability while preserving the high processing speed of the original network.
2Measurement precision
If deep neural networks are used for complex tasks like object detection and predictive maintenance, then pattern recognition accuracy is improved, but understanding of decision-making processes deteriorates
Solution Approach 1:
The patent implements a feedback mechanism where the assessment system continuously queries the data structure to retrieve information about the neural network's intermediate processing steps. The system provides feedback in the form of explanations that reveal which intermediate layers contributed most to a prediction, what features were most important, and how confident the network is in its prediction. This feedback loop restores decision-making understanding without compromising the network's high pattern recognition accuracy.
Solution Approach 2:
The patent segments the neural network's processing pipeline into multiple intermediate layers and stores information about each layer in the data structure. The assessment system can then analyze specific segments of the processing pipeline to provide detailed explanations about decision-making at different stages. This segmentation allows the system to maintain high pattern recognition accuracy by preserving the complete processing chain while simultaneously improving decision-making understanding by allowing granular analysis of intermediate steps.
3Device complexity
If neural networks are treated as black boxes, then device complexity is reduced, but reliability and transparency in critical applications deteriorate
Solution Approach 1:
The patent introduces an assessment system as an intermediary layer between the neural network and the application. This system includes a data structure that captures information about the processing pipeline without requiring modifications to the neural network itself. The assessment system queries this data structure to provide transparency and reliability assessments, thereby maintaining the simplicity of the original network while enhancing trustworthiness for critical applications like autonomous train control.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Techniques are disclosed that enable assessment of the processing of datasets in a machine-learning algorithm such as a deep convolutional neural network. Layer-based assessment is possible. The processing of images can be assessed. Explainable artificial intelligence is possible. Safe control of autonomous vehicles is possible.