Neural Network Activation Compression for Memory-Efficient Backpropagation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Machine learning models, particularly neural networks, are computationally expensive and challenging to perform feature extraction in real-time using general-purpose processors, necessitating improvements in hardware and software for efficient neural network processing.

Innovation Solution

Storing activation values from neural networks in a compressed format, using block floating-point representation, allows for efficient storage and retrieval during forward and backward propagation training, utilizing general-purpose and special-purpose processors, bulk memory, and block floating-point compressors.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If activation values are stored in full precision format during forward propagation, then accuracy is maintained, but memory storage requirements and computational complexity increase

Engineering Contradiction:

Improveactivation value precisionVSAvoidmemory storage and computational complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent applies parameter changes by transitioning activation values from full precision floating-point format to block floating-point format with reduced precision. This format change reduces the number of bits required to represent each activation value while maintaining sufficient accuracy for backpropagation, thereby reducing memory storage requirements and computational complexity without significantly compromising training accuracy

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent extracts and stores only the essential precision information needed for backpropagation by using block floating-point representation. This extraction approach separates the critical precision components from redundant full-precision data, allowing the system to retain only the necessary information for gradient computation while discarding excess precision that does not contribute meaningfully to training accuracy

Inventive Principle:

Principle #2Taking out (Extraction)

2Quantity of substance

If block floating-point compression is applied to activation values, then memory efficiency improves, but conversion complexity increases

Engineering Contradiction:

Improvememory storage efficiencyVSAvoidformat conversion complexity

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

The patent applies preliminary action by pre-computing and storing block floating-point representation parameters during the forward propagation phase. The system prepares the activation values in compressed block floating-point format before backpropagation begins, including pre-calculating the necessary scaling factors and exponent information. This preliminary preparation reduces the computational burden during backpropagation by having the compression structure already in place

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces an intermediary block floating-point representation system that acts as a bridge between full-precision forward propagation and gradient computation. This intermediary format includes auxiliary data structures such as block exponent tables and scaling factors that mediate the conversion process, simplifying the transformation between formats by providing pre-computed reference information

Inventive Principle:

Principle #24Intermediary (Mediator)

3Quantity of substance

If quantized or block floating-point format is used for activation values, then storage space is reduced, but retrieval and conversion time increases

Engineering Contradiction:

Improvestorage spaceVSAvoidretrieval and conversion time

Core Design Contradiction:

Quantity of substanceVSLoss of time

Solution Approach 1:

The patent applies segmentation by dividing activation values into blocks that share common exponent information in the block floating-point format. Instead of processing each activation value independently, the system groups multiple values into blocks with shared representation parameters, allowing efficient batch retrieval and conversion. This segmentation reduces the total number of conversion operations required while maintaining individual value precision

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS20260017520A1Compression and storage of neural network activations for backpropagation

Publication Date: 2026.01.15 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

Apparatus and methods for training a neural network accelerator using quantized precision data formats are disclosed, and in particular for storing activation values from a neural network in a compressed format for use during forward and backward propagation training of the neural network. In certain examples of the disclosed technology, a computing system includes processors, memory, and a compressor in communication with the memory. The computing system is configured to perform forward propagation for a layer of a neural network to produced first activation values in a first block floating-point format. In some examples, activation values generated by forward propagation are converted by the compressor to a second block floating-point format having a narrower numerical precision than the first block floating-point format. The compressed activation values are stored in the memory, where they can be retrieved for use during back propagation.