Deep Neural Network Compression via Sparse Matrix Pruning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks, particularly large-scale models like LSTM, face challenges in compression due to their dense matrix representation, which hinders deployment on resource-constrained devices like mobile terminals, leading to increased storage and computational costs, and limited acceleration benefits from conventional CPU and GPU architectures.
Innovation Solution
A multi-iteration compression method is proposed, involving sensitivity analysis, pruning, and fine-tuning, where matrices are divided into submatrices, compressed based on initial compression ratios, and then fine-tuned to maintain accuracy while reducing the number of parameters, along with a load-balanced approach to ensure efficient parallel processing across multiple processing elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If dense matrix representation is used in deep neural networks, then computational accuracy is maintained, but storage requirements and computational costs increase significantly
Solution Approach 1:
The patent applies sparse matrix representation where most elements are zero (empty spaces), similar to porous materials. By representing the weight matrix in sparse format instead of dense format, storage requirements are dramatically reduced while maintaining the same computational accuracy for non-zero elements. The sparse representation stores only non-zero elements and their positions, creating 'porosity' in the data structure.
2Measurement precision
If dense matrix representation is used in deep neural networks, then computational accuracy is maintained, but computational speed decreases due to processing unnecessary zero elements
Solution Approach 1:
The patent extracts and removes the zero elements from the weight matrix, keeping only the non-zero elements for computation. By taking out the unnecessary zero elements that contribute to computational overhead but provide no value to the result, the computational speed is significantly improved while maintaining accuracy through the preserved non-zero elements.
3Device complexity
If uniform compression ratio is applied to all matrices in the neural network, then implementation is simplified, but load balance across processing elements deteriorates
Solution Approach 1:
The patent applies different compression ratios to different weight matrices within the neural network based on their specific characteristics and the computational requirements of their corresponding processing elements. Instead of uniform compression, each matrix is compressed locally according to its needs, optimizing load balance across PEs while maintaining overall network performance.
4Quantity of substance
If high compression ratio is applied to neural network matrices, then storage and computational requirements are reduced, but network accuracy deteriorates
Solution Approach 1:
The patent employs an iterative compression process with dynamic adjustment of compression ratios. The compression ratio for each matrix is not fixed but is dynamically determined through sensitivity analysis and adjusted based on the impact on network accuracy. This dynamic approach allows achieving high compression while maintaining accuracy by adapting the compression level to each matrix's importance and characteristics.
Data Source
AI summary
The present invention relates to artificial neural networks, for example, deep neural networks. In particular, the present invention relates to a compression method considering load balance for deep neural networks and the device thereof. More specifically, the present invention relates to how to compress dense neural networks into sparse neural networks in an efficient way so as to improve utilization of resources of the hardware platform.