FILO Buffer for Neural Network Weight Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network training algorithms, such as backpropagation, require significant data movement and storage, which is inefficient due to the conventional memory/cache hierarchy design optimized for different data reuse patterns.
Innovation Solution
Implementing a first-in, last-out (FILO) buffer system that stores data in a layer-after-layer fashion during the forward pass and retrieves it in the output-layer to input-layer direction during the backward pass, reducing address decoding and arbitration, and thus lowering energy costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If conventional memory/cache hierarchy is used for data storage during neural network training, then data can be stored and retrieved, but energy consumption and latency are high due to inefficient data reuse patterns
Solution Approach 1:
The patent inverts the conventional memory access pattern by implementing a FILO (First-In-Last-Out) buffer that stores data in layer-after-layer fashion during forward pass and retrieves it in reverse order during backward pass. This inversion matches the natural reuse pattern of neural network gradients, eliminating the need for complex address decoding and arbitration, thereby reducing energy consumption while maintaining high training efficiency
2Loss of time
If conventional memory/cache hierarchy is used for data storage during neural network training, then data can be stored and retrieved, but latency is high due to address decoding and arbitration overhead
Solution Approach 1:
The patent inverts the conventional memory access pattern by implementing a FILO (First-In-Last-Out) buffer that stores data in layer-after-layer fashion during forward pass and retrieves it in reverse order during backward pass. This inversion matches the natural reuse pattern of neural network gradients, eliminating the need for complex address decoding and arbitration, thereby reducing latency
Solution Approach 2:
The patent extracts the data storage and retrieval function from the conventional memory hierarchy and implements a dedicated FILO buffer specifically optimized for neural network training. This extraction allows the system to bypass the complex address decoding and arbitration mechanisms of general-purpose memory, reducing both latency and device complexity
3Adaptability or versatility
If regular cache/memory hierarchy is used, then data storage is possible, but it is not efficient for neural network training data reuse patterns
Solution Approach 1:
The patent inverts the conventional memory access pattern by implementing a FILO (First-In-Last-Out) buffer that stores data in layer-after-layer fashion during forward pass and retrieves it in reverse order during backward pass. This inversion matches the natural reuse pattern of neural network gradients, dramatically improving data reuse efficiency while reducing energy waste compared to conventional memory hierarchies
Data Source
AI summary
Systems, apparatuses, and methods for optimizing neural network training with a first-in, last-out (FILO) buffer are disclosed. A processor executes a training run of a neural network implementation by performing multiple passes and adjusting weights of the neural network layers on each pass. Each training phase includes a forward pass and a backward pass. During the forward pass, each layer, in order from first layer to last layer, stores its weights in the FILO buffer. An error is calculated for the neural network at the end of the forward pass. Then, during the backward pass, each layer, in order from last layer to first layer, retrieves the corresponding weights from the FILO buffer. Gradients are calculated based on the error so as to update the weights of the layer for the next pass through the neural network.