Artificial Neural Network Prediction Explanation via Input Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural network (ANN) models, particularly in natural language processing (NLP) and natural language understanding (NLU), lack the ability to explain their predictions, making it difficult for system architects and designers to understand their behavior, which hinders improvement and optimization.
Innovation Solution
A method and system that determine relevant portions of input data, fetch partial predictions and scores from ANN models, and calculate influence scores to provide explanations for predictions, enabling a deeper understanding of the prediction process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If AI systems are used for complex tasks in NLP or NLU, then their capability and versatility are improved, but their ability to explain predictions deteriorates
Solution Approach 1:
The patent segments the input data into multiple portions and processes each portion separately through the ANN model to generate individual predictions. This segmentation allows the system to track and explain which specific portions of input data influenced the final prediction, thereby resolving the contradiction between handling complex tasks and providing explanations.
2Measurement precision
If large amount of data is used for training NLU applications, then the model's performance is improved, but the complexity of determining data reliance increases
Solution Approach 1:
The patent introduces an intermediary explanation generation component that acts as a mediator between the ANN model and the user. This intermediary processes the model's internal decisions and translates them into human-understandable explanations, reducing the complexity of determining data reliance while maintaining high prediction accuracy.
3Ease of manufacture
If system architects want to understand AI system behavior, then they can adjust training data and parameters, but the black box nature of AI systems prevents this understanding
Solution Approach 1:
The patent implements a feedback mechanism where the system generates explanations for its predictions and provides them back to users. This feedback loop allows system architects to understand the AI system's behavior by analyzing the generated explanations, enabling them to make informed adjustments to training data and parameters while maintaining the system's black box operational nature.
Data Source
AI summary
This disclosure relates to method and system for providing an explanation for a prediction generated by an artificial neural network (ANN) model for a given input data. The method may include receiving the given input data and the prediction generated by the ANN model. The ANN model may be built and trained for a target application. The method may further include determining a plurality of relevant portions of the given input data. For each of the plurality of relevant portions, the method may further include fetching a portional prediction and a portional prediction score generated by the ANN model, and determining a degree of influence score based on the portional prediction score and a comparison between the portional prediction and the prediction. The method may further include providing the explanation for the prediction based on the degree of influence score of each of the plurality of relevant portions.