Neural Network Memoization Caching for Edge Power Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks require substantial compute resources and power consumption, hindering their adoption on edge devices with limited memory and power constraints.
Innovation Solution
Implement memoization techniques to cache input and output values of cacheable instruction blocks in neural networks, reducing power consumption by retrieving from a value cache when it is more efficient than executing the block.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If neural networks are implemented on edge devices, then artificial intelligence capabilities are improved, but power consumption and memory requirements increase
Solution Approach 1:
The patent applies preliminary action by pre-computing and caching instruction blocks that are likely to be executed multiple times with different inputs. The system identifies cacheable instruction blocks during code analysis and pre-stores their outputs in a value cache, so that when the same block is encountered again, the cached result can be retrieved instead of re-executing the block, thereby reducing power consumption on edge devices.
Solution Approach 2:
The patent uses copying by creating and storing copies of computation results in a value cache. When an instruction block is identified as cacheable, its output is copied and stored in the cache under the block's identifier. Subsequent executions can retrieve these copied results instead of performing the full computation again, reducing energy consumption while maintaining AI functionality.
2Productivity
If compute resources are increased to handle neural networks, then processing capability is improved, but device complexity and cost increase
Solution Approach 1:
The patent applies segmentation by dividing the neural network code into discrete instruction blocks that can be individually analyzed and cached. The system identifies specific cacheable instruction blocks within the larger codebase, allowing selective caching of frequently executed segments rather than requiring increased overall compute resources. This segmentation enables efficient resource utilization without increasing device complexity.
3Quantity of substance
If memory is increased to store more neural network data, then model capacity is improved, but power consumption and device requirements increase
Solution Approach 1:
The patent applies local quality by implementing a targeted caching strategy that stores only the results of cacheable instruction blocks in the value cache, rather than increasing overall memory capacity. The cache stores localized computation results that are frequently reused, allowing the system to maintain low memory usage while still providing fast access to repeated computations, thereby reducing power consumption without requiring increased memory capacity.
Data Source
AI summary
A system and method for reducing power consumption in processing artificial neural networks utilizes memoization techniques. The method includes receiving computer code representing a neural network model, the neural network model including an input layer having a first plurality of nodes, and an output layer having a second plurality of nodes; detecting in the computer code a cacheable block of instructions, the cacheable block of instructions including an input and an output, wherein the input and the output are local to the cacheable block of instructions; determining a first power consumption corresponding to retrieving a value from a value cache; determining a second power consumption corresponding to executing the cacheable block of instructions; and storing in the value cache an input value corresponding to the input and an output value corresponding to the output, in response to determining that the second power consumption is higher than the first power consumption.