Narrow Block Floating-Point Activation Compression for Backpropagation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models, particularly neural networks, are computationally expensive and inefficient for real-time processing due to their high computational demands, especially when trained and executed on general-purpose processors.
Innovation Solution
Storing activation values from neural networks in a compressed block floating-point format during forward and backward propagation, using quantized or narrow block floating-point formats to reduce precision and memory requirements, and employing specialized hardware like FPGAs for accelerated processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If activation values are stored in full precision block floating-point format, then accuracy is maintained, but memory usage and computational overhead increase
Solution Approach 1:
The patent changes the precision parameter of activation values by converting them from full precision block floating-point format to a compressed format with lower precision. This is achieved through quantization and compression operations that reduce the number of bits required to represent each activation value, thereby reducing memory usage while maintaining sufficient accuracy for neural network training operations.
2Adaptability or versatility
If general-purpose processors are used for neural network training, then flexibility is maintained, but computational speed and energy efficiency deteriorate
Solution Approach 1:
The patent segments the neural network training process into distinct phases (forward propagation, backward propagation) and applies different precision formats to different phases. During forward propagation, full precision is used to maintain accuracy, while during backward propagation, compressed lower-precision formats are used to accelerate computation and reduce memory bandwidth requirements, thereby improving overall computational speed while maintaining flexibility.
3Reliability
If full precision formats are used during forward and backward propagation, then training accuracy is maintained, but computational complexity and energy consumption increase
Solution Approach 1:
The patent dynamically adjusts the precision format of activation values based on the phase of training. During forward propagation, full precision block floating-point format is used to ensure accurate computation of activations. During backward propagation, the system transitions to a compressed lower-precision format for gradient computations, thereby reducing energy consumption while maintaining sufficient training accuracy through selective precision management.
Data Source
AI summary
Apparatus and methods for training a neural network accelerator using quantized precision data formats are disclosed, and in particular for storing activation values from a neural network in a compressed format for use during forward and backward propagation training of the neural network. In certain examples of the disclosed technology, a computing system includes processors, memory, and a compressor in communication with the memory. The computing system is configured to perform forward propagation for a layer of a neural network to produced first activation values in a first block floating-point format. In some examples, activation values generated by forward propagation are converted by the compressor to a second block floating-point format having a narrower numerical precision than the first block floating-point format. The compressed activation values are stored in the memory, where they can be retrieved for use during back propagation.