Deep Neural Network Decision Boundary Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks are challenging to interpret due to numerous parameters in hidden layers, making it difficult to understand how input data is processed, and existing explainability methods are ambiguous and inaccurate, especially in mission-critical systems like healthcare and autonomous vehicles.
Innovation Solution
A method and apparatus for extracting data in deep learning models by determining a decision boundary set based on distribution probability, using Bernoulli parameters, and employing a Rapidly-Exploring Random Tree (RRT) algorithm to identify decision regions and retrain the model for improved performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep neural networks are used for complex recognition tasks, then model performance is improved, but interpretability and understanding of internal decision mechanisms deteriorate
Solution Approach 1:
The patent introduces decision boundary extraction as an intermediary mechanism that mediates between the complex internal representations of deep neural networks and human interpretable explanations. By extracting and visualizing decision boundaries in the input space, the system provides a bridge that allows users to understand how the network makes decisions without directly examining the complex hidden layer parameters
Solution Approach 2:
The patent extracts specific critical information (decision boundaries) from the complex deep neural network system. Instead of trying to interpret all parameters in hidden layers, the method extracts only the essential decision-making information by identifying boundaries that separate different class regions in the input space, thereby simplifying the explanation while maintaining accuracy
2Ease of operation
If existing explainability methods are applied to deep neural networks, then some interpretation is provided, but accuracy and reliability of explanations deteriorate
Solution Approach 1:
The patent replaces traditional explanation methods (such as LIME or SHAP that use perturbation-based approaches) with a more reliable mechanism based on decision boundary extraction. By using the actual decision boundaries learned by the network during training, the method provides explanations that are more faithful to the network's true decision-making process rather than approximations derived from perturbed inputs
3Loss of information
If all decision boundaries in deep neural networks are analyzed, then complete understanding is achieved, but computational complexity and time consumption increase
Solution Approach 1:
The patent extracts only the essential decision boundaries that are most relevant to the classification task, rather than analyzing all possible boundaries in the network. By identifying and extracting critical boundaries that separate different classes, the method achieves complete understanding of the decision-making process with significantly reduced computational effort
Solution Approach 2:
The patent segments the complex set of all decision boundaries into manageable subsets that can be analyzed independently. By dividing the analysis into separate boundary regions and focusing on critical boundaries, the method makes the analysis process more efficient while maintaining completeness of understanding
Data Source
AI summary
Disclosed is a method and apparatus for extracting data in a deep learning model. The method includes receiving an input query, determining a first decision boundary set being a subset of a decision boundary set corresponding to a target layer of the deep learning model, extracting a decision region including the input query based on the first decision boundary set, and extracting data included in the decision region.