Activation Data Compression in Neural Network Processors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) require high computational complexity, leading to high power consumption, making them unsuitable for long-duration operations in low-power devices such as smartphones and power-over-Ethernet security cameras, where reduced power consumption is critical.
Innovation Solution
A DNN module that compresses and decompresses activation data to reduce memory bus bandwidth utilization, using a compression unit to generate a mask and data portion of a compressed output chunk, and a decompression unit to restore the original data, thereby reducing power consumption and enabling faster processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If general-purpose processors like GPUs are used to provide high computational performance for DNNs, then computational performance is improved, but power consumption increases making them unsuitable for low-power devices
Solution Approach 1:
The patent segments the data processing pipeline into separate compression and decompression units that operate independently from the main neural network processing. The compression unit compresses activation data before it leaves the neuron output, and the decompression unit decompresses data before it enters neuron input, allowing the neural network to process compressed data and reduce memory bandwidth requirements without sacrificing computational performance.
Solution Approach 2:
The patent introduces compression and decompression units as intermediary components between the neural network neurons and the memory system. These intermediaries transform the data format (compressing activation data) to reduce the volume of data transferred over the memory bus, thereby reducing power consumption while maintaining the computational integrity of the DNN operations.
2Use of energy by moving object
If compression and decompression units are added to reduce memory bandwidth utilization, then power consumption is reduced, but device complexity increases
Solution Approach 1:
The patent changes the parameter of data representation by compressing activation data from its original format to a compressed format. The compression unit transforms the data parameters (reducing bit width while preserving essential information), and the decompression unit restores them, allowing the system to operate with reduced memory bandwidth requirements without fundamentally changing the neural network architecture.
3Productivity
If memory bus bandwidth is reduced through compression, then processing speed is improved, but the complexity of data management increases
Solution Approach 1:
The patent applies preliminary action by compressing the activation data before it is written to memory or transmitted over the bus. This pre-compression step reduces the amount of data that needs to be managed during subsequent read/write operations, thereby improving processing speed while the complexity is confined to the initial compression stage rather than being distributed throughout the entire data management process.
Data Source
AI summary
A deep neural network (“DNN”) module compresses and decompresses neuron-generated activation data to reduce the utilization of memory bus bandwidth. The compression unit receives an uncompressed chunk of data generated by a neuron in the DNN module. The compression unit generates a mask portion and a data portion of a compressed output chunk. The mask portion encodes the presence and location of the zero and non-zero bytes in the uncompressed chunk of data. The data portion stores truncated non-zero bytes from the uncompressed chunk of data. A decompression unit receives a compressed chunk of data from memory in the DNN processor or memory of an application host. The decompression unit decompresses the compressed chunk of data using the mask portion and the data portion.