Deep Neural Network Feature Map Compression via Multi-Bank Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks require large storage space and memory bandwidth due to the large number of parameters and feature maps, which is a bottleneck for real-time applications like video surveillance and automated driving, as existing compression methods focus primarily on neural network parameters rather than feature maps.
Innovation Solution
A compression method for deep neural networks that involves reading parameters and feature maps, generating a second feature map based on these, and then compressing and storing the feature maps with varying compression ratios depending on the region of interest, reducing storage space and memory bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the deep neural network uses a large number of layers and nodes to improve inference accuracy, then the inference accuracy is improved, but the storage space and memory bandwidth requirements increase significantly
Solution Approach 1:
The patent segments the feature map storage into multiple banks (first feature map storage bank, second feature map storage bank, etc.) and processes different channels of the feature map in segments. This allows parallel processing of different channel groups while reducing the memory bandwidth requirement for each individual access, thereby supporting large-scale neural networks with improved accuracy while managing storage requirements through distributed storage across multiple banks
Solution Approach 2:
The patent introduces a bank dimension to the traditional feature map storage structure by creating multiple feature map storage banks. Each bank stores a portion of the feature map channels, transforming the single-dimension storage problem into a multi-dimension structure. This enables parallel access patterns and reduces the memory bandwidth bottleneck while maintaining the ability to store and process large numbers of parameters required for high-accuracy inference
2Measurement precision
If the deep neural network increases the number of layers and nodes to improve inference accuracy, then the inference accuracy is improved, but the memory bandwidth requirement increases
Solution Approach 1:
The patent divides the feature map channels into multiple groups and assigns each group to a different feature map storage bank. During inference, different banks can be accessed in parallel, significantly increasing the effective memory bandwidth. This segmentation strategy allows the system to handle the large data throughput requirements of deep neural networks with many layers and nodes without being bottlenecked by memory bandwidth limitations
Solution Approach 2:
The patent implements dynamic channel allocation where the assignment of channels to different banks can be adjusted based on the specific inference task and data characteristics. This dynamic configuration optimizes memory access patterns in real-time, ensuring maximum utilization of available bandwidth while processing the large volumes of data required for high-accuracy inference in deep networks
3Measurement precision
If the feature map storage space is increased to accommodate more parameters, then the inference accuracy is maintained, but the storage cost increases
Solution Approach 1:
The patent merges multiple feature map storage banks into a unified memory structure that can be shared across different computation stages. By consolidating storage resources and using intelligent channel allocation, the system maintains sufficient storage capacity for high-accuracy inference while reducing redundant storage and lowering overall storage costs compared to having separate dedicated storage for each layer and channel
4Measurement precision
If the number of feature map channels is increased to improve inference accuracy, then the inference accuracy is improved, but the memory bandwidth and storage space requirements increase
Solution Approach 1:
The patent segments the large number of feature map channels into smaller groups, with each group assigned to a dedicated storage bank. This segmentation reduces the complexity of memory management by breaking down the monolithic feature map storage into manageable units, while still supporting a large total number of channels for high-accuracy inference through parallel access to multiple banks
Solution Approach 2:
The patent implements selective channel allocation where not all channels are actively used in every computation stage. By dynamically activating only the necessary subset of channels based on the current inference task, the system maintains the capability for high-accuracy inference with many channels while reducing the actual memory bandwidth and storage requirements at any given moment, thereby managing system complexity
Data Source
AI summary
Disclosed is a compression method for a deep neural network. The deep neural network comprises a plurality of layers. The method includes the following steps for each of at least one layer of the plurality of layers other than an input layer: reading parameters of that layer from a parameter storage space; reading a first feature map stored by an immediate previous layer from a feature map storage space; generating a second feature map on the basis of the read parameters and the first feature map; compressing the second feature map so as to obtain a third feature map; and storing the third feature map into the feature map storage space. Further, a chip based on the compression method and a medium are also provided.