Neural Network Activation Map Compression Using Sparse Block Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks require significant memory for activation maps, which limits their deployment on embedded devices with limited memory, and existing compression techniques do not effectively leverage sparsity in activation maps to reduce memory footprint.
Innovation Solution
A system and method for compressing activation maps using a processor to sparsify, quantify, and encode non-zero values into blocks using various lossless compression modes, such as Exponential-Golomb and Golomb-Rice encoding, to generate a bitstream, which can be decompressed and reformatted to reduce memory requirements without accuracy loss.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If activation maps are stored in full precision, then model accuracy is maintained, but memory consumption increases significantly
Solution Approach 1:
The patent divides the activation map into multiple blocks and applies different quantization schemes to different blocks based on their sparsity characteristics. This segmentation allows the system to optimize memory usage for each block individually while maintaining overall model accuracy, resolving the contradiction between memory efficiency and accuracy preservation.
Solution Approach 2:
The patent changes the precision parameter of activation map storage by applying quantization techniques that convert high-precision floating-point values to lower-precision representations. By dynamically adjusting the number of bits used to represent activation values based on their importance and sparsity patterns, the system reduces memory consumption while preserving the accuracy needed for model performance.
2Quantity of substance
If compression techniques are applied to activation maps, then memory footprint is reduced, but existing techniques do not effectively leverage sparsity
Solution Approach 1:
The patent applies different compression strategies to different regions of the activation map based on local sparsity characteristics. By analyzing the sparsity pattern of each block and applying tailored quantization schemes, the system achieves more effective compression compared to uniform approaches, thereby improving productivity in terms of compression effectiveness while reducing memory footprint.
Solution Approach 2:
The patent performs preliminary analysis of activation map sparsity patterns before applying compression techniques. By pre-processing the activation maps to identify sparse regions and non-sparse regions, the system can apply appropriate compression strategies in advance, maximizing memory footprint reduction while maintaining compression effectiveness.
3Quantity of substance
If sparsification is applied to activation maps, then memory requirements are reduced, but the number of non-zero values decreases
Solution Approach 1:
The patent applies quantization parameter changes that preserve the essential information in non-zero activation values while reducing their representation size. By carefully selecting quantization levels and precision for non-zero values, the system reduces memory requirements without significant loss of information, maintaining the integrity of the activation map's meaningful data.
Solution Approach 2:
The patent extracts and separately handles the non-zero values from the activation map, applying specialized compression techniques only to these extracted values. By separating the sparse non-zero elements from the zero elements, the system can optimize storage for the informative non-zero values while ignoring the redundant zeros, thereby reducing memory requirements with minimal information loss.
Data Source
AI summary
A system and a method provide compression and decompression of an activation map of a layer of a neural network. For compression, the values of the activation map are sparsified and the activation map is configured as a tensor having a tensor size of H×W×C in which H represents a height of the tensor, W represents a width of the tensor, and C represents a number of channels of the tensor. The tensor is formatted into at least one block of values. Each block is encoded independently from other blocks of the tensor using at least one lossless compression mode. For decoding, each block is decoded independently from other blocks using at least one decompression mode corresponding to the at least one compression mode used to compress the block; and deformatted into a tensor having the size of H×W×C.