Hyperdimensional Computing Activation Map Analysis for DNN Interpretability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI systems, particularly deep neural networks (DNNs), lack transparency in decision-making processes, making it difficult to explain their binary computed decisions, which is a barrier to their acceptance and usefulness in critical areas like medicine, law, defense, and banking.
Innovation Solution
The Activation Map Analysis (AMA) system is integrated with DNNs to provide explainability by employing a calibration and inferencing process using dimensionality reduction techniques like PCA and Hyperdimensional Computing (HDC) to extract and encode activation maps, calculating credibility scores and confidence for DNN decisions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If deep neural networks are used for complex computational problems, then performance is improved, but transparency and explainability of decisions deteriorate
Solution Approach 1:
The patent introduces an Activation Map Analysis (AMA) system as an intermediary component that attaches to the DNN to extract and analyze activation maps from intermediate layers. This mediator captures the internal reasoning process of the network by extracting activation maps that represent feature detections at different processing stages, making the black-box decisions transparent without altering the original network's high performance.
2Loss of information
If activation map-based techniques are used to explain DNN decisions, then interpretability is improved, but memory demands increase
Solution Approach 1:
The patent extracts only the essential information needed for interpretation by selecting specific intermediate layers of the DNN and capturing their activation maps. Rather than storing or processing the entire network's internal state, the system extracts activation maps from strategically chosen layers that provide sufficient interpretability while minimizing memory consumption.
Solution Approach 2:
The patent transforms high-dimensional activation maps into more compact representations by projecting them into lower-dimensional spaces or encoding them in alternative formats. This dimensionality reduction maintains the interpretability of activation maps while significantly reducing the memory resources required to store and process them.
3Power
If traditional dimensionality reduction methods like PCA are used, then processing capability is maintained, but memory efficiency deteriorates
Solution Approach 1:
The patent employs computationally efficient dimensionality reduction techniques that require minimal memory resources, accepting that these methods may be less powerful than traditional PCA but are sufficient for the interpretability task. The system uses lightweight algorithms that can be executed with limited memory, making the overall system more memory-efficient while maintaining adequate processing capability for analysis.
Data Source
AI summary
An Activation Map Analysis (AMA) system attaches to an existing deep neural network (host network) as an observer that analyzes the internal activities of the host network to provide additional information, e.g., metrics for resilience, interpretability, and adversarial defense, to the end-user. The AMA system is developed using deep k-nearest neighbor (DkNN) coupled with principal component analysis (PCA) for data dimensionality reduction or Hyperdimensional Computing (HDC) to encode activation map data of the host for data reduction.