Neural Network Computing Method Reducing Memory Usage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network computations require large working memory capacities, leading to increased die area and costs due to the need to store entire layers of data before calculations can proceed, which is inefficient and costly.
Innovation Solution
Implement a 'multi-layer jump' approach where only specific elements from previous layers are selected and computed, reducing the memory requirements by storing only the necessary data for each calculation, thereby decreasing memory usage and costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the whole intermediate data of a certain layer is calculated and stored to the working memory, then the neural network computation can proceed layer by layer, but the required working memory capacity becomes quite large
Solution Approach 1:
The patent segments the computation process into multiple passes. In each pass, only the data needed for computing a specific output element is loaded into working memory, rather than storing all intermediate data of a layer. This segmentation of the computation workflow allows the system to process large neural networks with limited working memory capacity.
Solution Approach 2:
The patent changes the computation dimension from traditional layer-by-layer processing to element-wise processing across multiple passes. Instead of completing all computations for a layer before moving to the next layer, the system processes one output element at a time by selectively loading only the necessary input elements and their corresponding weight values into working memory.
2Productivity
If larger working capacity is used to improve NN calculation efficiency, then the computation can handle more data, but the die area and cost increase
Solution Approach 1:
The computation is divided into multiple passes where each pass handles a subset of computations. This segmentation allows the use of smaller working memory units that can be reused across passes, reducing the total die area required for memory while maintaining the ability to process large neural networks.
Solution Approach 2:
The system discards intermediate data from previous layers after it has been used for computation, and recovers/reloads only the necessary data for the next computation pass. This approach allows small working memory units to handle large computational tasks by repeatedly loading and discarding data, eliminating the need for large static memory structures.
Data Source
AI summary
A neural network computing method and a neural network computing device are provided. The neural network computing method includes the following steps. At least one chosen layer is decided. A plurality of front layers previous to the chosen layer are decided. A selected element is selected from a plurality of chosen elements in the chosen layer. A front computing data group related to the selected element is defined. The front computing data group is composed of only part of a plurality of front elements in the front layers. The selected element is computed according to the at least one front computing data group.