DNN Weight Block Matrix Compression for Mobile Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep neural network (DNN) implementations for keyword detection and speech recognition in mobile and wearable devices face challenges due to high memory and computation requirements, making it difficult to support these applications without sacrificing accuracy.
Innovation Solution
The method involves dividing a fully connected weight matrix into weight blocks of a determined block size, randomly designating active weight blocks based on a target sparsity ratio, and updating only these blocks during training to generate a sparsified weight block matrix, which reduces memory footprint and computation power while maintaining accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a fully connected weight matrix is used in a deep neural network for keyword detection and speech recognition, then the accuracy of the DNN application is maintained, but the memory footprint and computation power requirements increase significantly
Solution Approach 1:
The fully connected weight matrix is divided into multiple weight blocks, where each block corresponds to a subset of neurons in the hidden layer. This segmentation allows selective activation of specific weight blocks based on the input data, reducing the overall memory footprint while maintaining the necessary computational accuracy for keyword detection and speech recognition tasks.
Solution Approach 2:
Instead of activating all weight blocks in every computation, the system activates only a subset of weight blocks (partial action) based on the specific DNN application requirements. This selective activation reduces computation power consumption and memory usage while preserving the accuracy needed for the intended applications.
2Reliability
If a fully connected weight matrix is used in a deep neural network for keyword detection and speech recognition, then the accuracy of the DNN application is maintained, but the power consumption increases
Solution Approach 1:
The weight matrix is segmented into multiple weight blocks, enabling the system to activate only the necessary blocks for each computation. This reduces the number of active computational units, thereby lowering power consumption while maintaining the accuracy required for keyword detection and speech recognition.
Solution Approach 2:
The system performs partial computation by activating only a subset of weight blocks rather than the entire weight matrix. This partial action reduces the energy required for each inference operation while preserving the accuracy needed for the DNN application.
3Quantity of substance
If the second number of columns in the weight block matrix is compressed, then the memory footprint is reduced, but the accuracy of the DNN may be compromised
Solution Approach 1:
The weight matrix columns are segmented into multiple blocks, allowing selective compression of specific column blocks while preserving others. This enables targeted memory optimization where less critical columns can be compressed or approximated, while critical columns maintain full precision to preserve DNN accuracy.
Solution Approach 2:
Different compression levels are applied to different column blocks based on their importance to the specific DNN application. Critical columns that contribute more to accuracy are preserved with higher precision, while less critical columns undergo greater compression, achieving a balance between memory footprint reduction and accuracy maintenance.
Data Source
AI summary
Aspects disclosed in the detailed description include memory compression in a deep neural network (DNN). To support a DNN application, a fully connected weight matrix associated with a hidden layer(s) of the DNN is divided into a plurality of weight blocks to generate a weight block matrix with a first number of rows and a second number of columns. A selected number of weight blocks are randomly designated as active weight blocks in each of the first number of rows and updated exclusively during DNN training. The weight block matrix is compressed to generate a sparsified weight block matrix including exclusively active weight blocks. The second number of columns is compressed to reduce memory footprint and computation power, while the first number of rows is retained to maintain accuracy of the DNN, thus providing the DNN in an efficient hardware implementation without sacrificing accuracy of the DNN application.