Deep Neural Network Interpretation via Decision Boundary Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep neural networks (DNNs) lack transparency and reliability, making it difficult for human users to understand their decision-making processes, which is crucial for critical applications like healthcare and finance, as current interpretation methods like Grad-CAM, LIME, Anchor, and Mask approaches are unreliable, provide limited information, and only offer single-factor explanations.
Innovation Solution
The method involves extracting rules that define decision boundaries between different outcomes in DNNs, allowing for the generation of human-understandable representations of these rules, enabling comprehensive and robust interpretation of DNNs, including the identification of multiple contributing factors to a decision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deep neural networks are used for predictive modeling, then accuracy and capability are improved, but transparency and interpretability deteriorate
Solution Approach 1:
The patent introduces decision boundary visualizations as an intermediary representation that bridges the gap between the opaque DNN internal workings and human understanding. By visualizing the decision boundaries in the input space, the system provides an interpretable view of how the network makes decisions without exposing the complex internal neuron activations and weight configurations that are impossible for humans to comprehend
2Loss of information
If complex interpretation methods are applied to DNNs, then interpretation depth is improved, but reliability and robustness deteriorate
Solution Approach 1:
The patent extracts the essential decision-making information from the complex DNN by visualizing decision boundaries in the input space rather than attempting to interpret internal neuron activations. This extraction approach focuses on the most relevant information (decision boundaries) while eliminating the unreliable and incomprehensible internal representations, thereby providing reliable and robust interpretations
3Device complexity
If single-factor explanation methods are used, then simplicity is improved, but comprehensiveness deteriorates
Solution Approach 1:
The patent segments the complex decision-making process into multiple decision boundaries, each representing a distinct factor or rule that the DNN uses for classification. By visualizing multiple decision boundaries in the input space, the system reveals multiple contributing factors to the decision while maintaining visual clarity and interpretability, thus providing comprehensive explanations without excessive complexity
Data Source
AI summary
Methods, systems and media for deep neural network interpretation via rule extraction. The interpretation of the deep neural network is based on extracting one or more rules approximating classification behavior of the network. Rules are defined by identifying a set of hyperplanes through the data space that collectively define a convex polytope that separates a target class of input samples from input samples of different classes. Each rule corresponds to a set of decision boundaries between two different decision outcomes. Human-understandable representations of rules may be generated. One or more rules may be used to generate a classifier. The representations and interpretations exhibit faithfulness, robustness, and comprehensiveness relative to other known approaches.