Neural Network Memoization Using Shifted Value Localization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks, particularly on edge devices, face challenges due to substantial memory and processing requirements, hindering their adoption.
Innovation Solution
Implement memoization techniques to store input and output values of cacheable instruction blocks in a value cache, utilizing locality of values in neural networks to reduce redundant processing operations and conserve power.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If memoization techniques are implemented to store input and output values in a value cache, then cache hits increase and redundant processing operations are reduced, but memory requirements increase
Solution Approach 1:
The patent extracts only the essential input and output values of cacheable instruction blocks into a value cache, separating these from the full neural network data structure. This selective extraction enables cache hits for redundant operations while minimizing memory consumption by storing only necessary values rather than complete datasets.
Solution Approach 2:
The patent performs preliminary identification of cacheable instruction blocks and pre-stores their input and output values in the value cache before execution. This preliminary action ensures that when redundant operations are detected during neural network processing, the required values are already available in the cache, eliminating the need for recomputation and reducing processing time.
2Power
If substantial memory and processing resources are allocated to neural networks, then processing capability is improved, but power consumption increases
Solution Approach 1:
The patent implements a feedback mechanism that monitors neural network processing operations to identify cacheable instruction blocks and redundant operations. Based on this feedback, the system dynamically stores input and output values in the value cache, creating a self-optimizing system that reduces power consumption by leveraging previously computed results without requiring manual intervention or reconfiguration.
Solution Approach 2:
The patent creates simplified copies of input and output values from cacheable instruction blocks and stores them in the value cache. These copies serve as substitutes for the original data, enabling the system to retrieve pre-computed results instead of performing redundant processing operations, thereby reducing power consumption while maintaining processing capability.
3Speed
If value adjustment is performed to increase cache hits, then processing speed is improved, but computational overhead increases
Solution Approach 1:
The patent applies value adjustment locally only to input and output values of identified cacheable instruction blocks, rather than adjusting all values in the neural network. This localized approach increases cache hits for the most beneficial operations while minimizing the computational overhead associated with value adjustment, as only specific data points require modification.
Data Source
AI summary
A system and method for dynamically adjusting value similarity thresholds to increase cache hits in a value cache utilizing memoization is presented. The method includes: receiving an input matrix comprising a plurality of values; selecting a portion of the input matrix; generating a similarity threshold based on a determined number of least significant bits (LSBs) for comparing a first value of the portion to a second value of the portion; determining that the first and second values are identical in all but the determined number of LSBs; adjusting the first value based on the second value in response to determining that the first value and the second value are within the similarity threshold; generating a new input matrix based on the adjusted first value; and processing the new input matrix with a convolutional neural network (CNN), wherein dynamically adjusting the number of LSBs cache hits in the value cache.