Neural Network Memoization Using Shifted Value Localization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks require substantial compute resources and memory, hindering their adoption on edge devices where power and memory are limited.
Innovation Solution
Implement memoization techniques to store input and output values of cacheable instruction blocks in a value cache, utilizing value locality to reduce redundant processing operations and increase cache hits, thereby reducing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If neural networks are implemented on edge devices, then AI functionality is improved, but power consumption and memory usage increase
Solution Approach 1:
The patent applies preliminary action by pre-processing the input matrix to identify and adjust values before the main neural network computation. This preparation work is done in advance to enable more efficient processing during inference, reducing the computational burden and power consumption during actual operation on edge devices.
Solution Approach 2:
The patent changes parameters by adjusting specific values in the input matrix based on their relationship with other values. By modifying certain input values to match others when appropriate, the system reduces the diversity of computations needed, thereby reducing power consumption while maintaining acceptable AI functionality on resource-constrained edge devices.
2Adaptability or versatility
If neural networks are implemented on edge devices, then AI functionality is improved, but memory resources are exceeded
Solution Approach 1:
The system performs preliminary analysis of the input matrix to identify values that can be adjusted or cached. This pre-processing step enables the system to optimize memory usage during inference by preparing data in a format that requires less memory during the actual neural network computation on edge devices.
Solution Approach 2:
By adjusting input matrix values to reduce diversity and identify redundant computations, the patent reduces the amount of unique data that needs to be stored in memory during neural network operations. This parameter adjustment strategy decreases memory requirements while preserving essential AI functionality on devices with limited memory resources.
3Productivity
If memoization is used to cache input-output pairs, then processing speed is improved, but memory usage increases
Solution Approach 1:
The patent applies local quality by selectively adjusting only specific portions of the input matrix rather than uniformly processing all values. This localized approach identifies regions where value adjustment can enable caching opportunities without requiring memory resources for the entire input matrix, thus improving processing speed through strategic memoization while controlling overall memory usage.
Data Source
AI summary
A system and method for increasing cache hits in a value cache utilizing memoization is disclosed. The method includes receiving an input matrix for a parallel processing circuitry, wherein the parallel processing circuitry configured to process the input matrix with a second matrix; selecting a portion of the input matrix, wherein the portion includes a plurality of values; adjusting a first value of the plurality of values based on a second value of the plurality of values; generating a new input matrix based on the input matrix and the adjusted first value; and configuring the parallel processing circuitry to process the new input matrix with the second matrix.