Memory-Augmented Neural Network Using Hypervector Sharpening
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional classification methods using von-Neumann architectures face limitations, particularly in memory-intensive operations, leading to bottlenecks in conventional CPUs and GPUs, especially for tasks requiring thousands to millions of memory entries, and struggle with device variability and noise in non-von Neumann approaches like resistive memory devices.
Innovation Solution
The use of high-dimensional computing with hypervectors, which are vectors with dimensions higher than 2000, enables efficient representation and classification by transforming similarity scores with a sharpening function to enhance classification accuracy and robustness, particularly in memory-augmented neural networks with explicit memory systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional von-Neumann architectures are used for classification, then conventional CPUs and GPUs can process data, but memory-intensive operations create bottlenecks when handling thousands to millions of memory entries
Solution Approach 1:
The patent transitions from traditional low-dimensional vector representations to high-dimensional hypervectors (dimensions > 2000). This dimensional expansion enables the system to handle millions of memory entries efficiently by leveraging the geometric properties of high-dimensional space, where hypervectors representing different classes become nearly orthogonal, thereby resolving the memory bottleneck in conventional architectures.
2Productivity
If non-von-Neumann approaches like resistive memory devices are used, then memory-intensive operations can be accelerated, but device variability and noise degrade performance
Solution Approach 1:
The patent converts the harmful effect of device noise and variability into a beneficial feature. By using high-dimensional hypervectors with sufficient dimensionality, the system ensures that even with noisy readings, the nearly orthogonal geometry of hypervectors from different classes maintains separability. The noise affects individual components but cannot bridge the large geometric gaps between class representations, thus transforming device imperfections into robust classification performance.
3Measurement precision
If high-dimensional hypervectors are used for representation, then classification accuracy improves through better separation of different classes, but computational complexity increases
Solution Approach 1:
The patent replaces complex iterative computational mechanisms with simple geometric operations in high-dimensional space. Instead of using complex neural network training and inference processes, the system leverages the inherent geometric properties of hypervectors—where similarity is determined by simple dot products or angular measurements. This substitution of complex computational mechanics with straightforward geometric operations reduces computational complexity while maintaining high classification accuracy.
4Adaptability or versatility
If more memory entries are stored to improve few-shot learning, then classification performance on limited data improves, but memory resource requirements increase
Solution Approach 1:
The patent changes the fundamental parameter of vector dimensionality from traditional low dimensions to high dimensions (> 2000). This parameter change enables the system to pack millions of hypervector representations into available memory resources while maintaining their separability and informational content. The high-dimensional parameter allows efficient few-shot learning by providing sufficient geometric separation between classes even with limited training examples, without proportionally increasing physical memory requirements.
Data Source
AI summary
The present disclosure relates to a method for classifying a query information element using the similarity between the query information element and a set of support information elements. A resulting set of similarity scores is transformed using a sharpening function such that the transformed scores are decreasing as negative similarity scores increase and the transformed scores are increasing as positive similarity scores increase. A class of the query information element is determined based on the transformed similarity scores.