Bank-Balanced-Sparse Activation Feature Maps for Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep Neural Networks (DNNs) are computationally expensive, leading to high energy consumption, storage costs, and latency issues, especially on edge devices like mobile phones and surveillance cameras.
Innovation Solution
The implementation of bank-balanced-sparse activation feature maps in deep neural network models, which involves segmenting activation feature maps into banks and increasing sparsity to reduce computational costs while maintaining model performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If DNN model parameters are stored in memory space, then model accuracy is maintained, but storage costs and memory space requirements increase significantly
Solution Approach 1:
The patent segments the DNN model parameters into multiple banks (e.g., bank0, bank1, bank2, bank3) organized in a structured manner. This segmentation allows for efficient memory management and retrieval while reducing the overall storage footprint by enabling selective loading and processing of parameter banks rather than storing all parameters simultaneously in memory.
2Measurement precision
If the number of neurons in DNN model is increased, then model performance improves, but memory space consumption and arithmetic operations increase
Solution Approach 1:
The patent divides the neural network into multiple stages with each stage containing a subset of neurons and parameters organized in banks. This segmentation allows the system to process information in smaller chunks, reducing the peak memory space requirements while maintaining overall model performance through multi-stage processing.
Solution Approach 2:
The patent introduces a temporal dimension to the processing architecture by using multiple stages that process data sequentially. This transforms the problem from a spatial memory constraint (all parameters in memory simultaneously) to a temporal processing approach (parameters loaded and processed in sequence), effectively reducing memory space consumption.
3Measurement precision
If dense vector representations are used in search engines, then search accuracy is maintained, but computational cost increases
Solution Approach 1:
The patent segments the high-dimensional dense vector representation into multiple lower-dimensional banks. This segmentation reduces the computational complexity of operations such as similarity search while maintaining accuracy by distributing the computational load across multiple smaller banks that can be processed more efficiently.
Data Source
AI summary
Embodiments disclose bank-balanced-sparse activation neural network models and methods to generate the bank-balanced-sparse activation neural network models. According to one embodiment, a neural network sparsification engine determines a first deep neural network (DNN) model having two or more hidden layers. The engine determines a bank size, a bank layout, and a target sparsity. The engine segments the activation feature maps into a plurality of banks based on the bank size and the bank layout. The engine generates a second DNN model by increasing a sparsity for each bank of activation feature map based on the target sparsity, wherein the second DNN model is used for inferencing.