Neural Network Weight Block Compression for Embedded Memory Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks require significant memory for activation maps and weights, which hinders their execution on embedded devices with limited memory, necessitating a reduction in memory footprint.
Innovation Solution
A system and method for lossless encoding and decoding of neural network weights using various compression modes such as Exponential-Golomb, Sparse-Exponential-Golomb, Golomb-Rice, and Fixed length encoding, allowing for independent compression and decompression of weight blocks to reduce memory requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If weights are stored in full precision format, then model accuracy is maintained, but memory consumption increases
Solution Approach 1:
The weight tensor is divided into multiple blocks, where some blocks are marked as sparsified (containing only non-zero values) and others as non-sparsified (stored in full). This segmentation allows the system to apply different storage strategies to different portions of the weights, reducing overall memory consumption while maintaining accuracy for critical weight blocks.
Solution Approach 2:
The system changes the storage parameter of weight blocks from full precision to sparse representation by identifying and marking blocks with low importance or high sparsity. This parameter change reduces the amount of data stored while maintaining the essential functionality of the neural network through selective precision reduction.
2Quantity of substance
If memory footprint is reduced through compression, then embedded device compatibility is improved, but decompression complexity increases
Solution Approach 1:
During the training phase, the system performs preliminary analysis to identify which weight blocks can be sparsified and marks them accordingly. This preliminary action creates a compression map that guides the decompression process, allowing the decompression algorithm to efficiently reconstruct only the necessary blocks without requiring complex analysis during inference.
Solution Approach 2:
The decompression process is segmented into two simple phases: first, reading the compression map to identify which blocks are sparsified, and second, selectively decompressing only those blocks. This segmentation avoids the need for complex global optimization during decompression, reducing device complexity while maintaining compression efficiency.
3Ease of manufacture
If all weight blocks are compressed uniformly, then implementation simplicity is maintained, but compression efficiency decreases
Solution Approach 1:
Instead of applying uniform compression to all weight blocks, the system applies different compression strategies to different blocks based on their local characteristics. Blocks identified as having low importance or high sparsity are compressed using sparse representation, while other blocks maintain full precision. This local quality approach maximizes compression efficiency without requiring complex adaptive algorithms.
Data Source
AI summary
A system and a method provide compression and decompression of weights of a layer of a neural network. For compression, the values of the weights are pruned and the weights of a layer are configured as a tensor having a tensor size of H×W×C in which H represents a height of the tensor, W represents a width of the tensor, and C represents a number of channels of the tensor. The tensor is formatted into at least one block of values. Each block is encoded independently from other blocks of the tensor using at least one lossless compression mode. For decoding, each block is decoded independently from other blocks using at least one decompression mode corresponding to the at least one compression mode used to compress the block; and deformatted into a tensor having the size of H×W×C.