Neural Network Activation Compression for Lower Memory Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) require high computational complexity, leading to high power consumption, making them unsuitable for long-duration operations in low-power devices such as smartphones and power-over-Ethernet security cameras, where reduced power consumption is critical.
Innovation Solution
A DNN module that compresses and decompresses activation data to reduce memory bus bandwidth utilization, utilizing a compression unit to generate a mask and data portion of a compressed output chunk, and a decompression unit to restore the original data, thereby reducing power consumption and enabling faster processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If general-purpose processors like GPUs are used to provide high computational performance for DNNs, then processing speed and accuracy are improved, but power consumption increases making them unsuitable for battery-powered portable devices
Solution Approach 1:
The patent segments the data processing function into two distinct units: a compression unit that processes neuron output data and a decompression unit that processes input data. This segmentation allows each unit to be optimized for its specific function, with the compression unit reducing data size before storage and the decompression unit restoring data when needed, thereby reducing overall power consumption while maintaining processing performance.
Solution Approach 2:
The patent changes the parameter of data representation by compressing neuron output data into a compact format with mask portions and data portions. This parameter change reduces the amount of data that needs to be stored and transmitted, directly reducing the energy required for memory operations and enabling lower power consumption in portable devices.
2Use of energy by moving object
If compression and decompression units are added to reduce memory bandwidth utilization, then power consumption is reduced and processing speed is improved, but device complexity increases
Solution Approach 1:
The patent extracts the compression and decompression functions as separate, dedicated units from the general processing architecture. By taking out these specific functions and implementing them as specialized hardware units, the patent reduces the complexity burden on the main processor while still achieving the benefits of reduced memory bandwidth utilization and lower power consumption.
Data Source
AI summary
A deep neural network (“DNN”) module compresses and decompresses neuron-generated activation data to reduce the utilization of memory bus bandwidth. The compression unit receives an uncompressed chunk of data generated by a neuron in the DNN module. The compression unit generates a mask portion and a data portion of a compressed output chunk. The mask portion encodes the presence and location of the zero and non-zero bytes in the uncompressed chunk of data. The data portion stores truncated non-zero bytes from the uncompressed chunk of data. A decompression unit receives a compressed chunk of data from memory in the DNN processor or memory of an application host. The decompression unit decompresses the compressed chunk of data using the mask portion and the data portion.