Neural Network Weight Parameter Compression via Sparse Row Formatting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Processing and storage of large neural network models in resource-constrained devices like mobile devices are resource-demanding, leading to increased processing time, storage demands, and inefficiencies due to redundancy in weight parameters.
Innovation Solution
The solution involves grouping neural network weight parameters and pruning redundant ones, followed by retraining to maintain accuracy, and storing them in a grouped compressed sparse row format to reduce memory overhead and improve computational efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If neural network weight parameters are stored in standard format, then storage is simple, but memory usage and processing resources increase due to redundancy
Solution Approach 1:
The patent segments the weight parameter storage by dividing weight groups into sparse rows and column indices, storing only non-zero elements and their positions. This segmentation eliminates redundant zero elements and reduces memory usage while maintaining the ability to reconstruct the original weight matrix for neural network operations.
Solution Approach 2:
The patent extracts and removes redundant zero-weight parameters from the storage format, keeping only the essential non-zero weight parameters and their corresponding indices. This extraction principle directly reduces memory usage by eliminating unnecessary data while preserving the functional integrity of the neural network model.
2Productivity
If all weight parameters are retained, then neural network accuracy is maintained, but processing time and computational resources increase
Solution Approach 1:
The patent extracts and removes redundant zero-weight parameters from computation, processing only the essential non-zero weight parameters. This extraction improves processing efficiency by reducing the number of computational operations while maintaining neural network accuracy through the preservation of all meaningful weight parameters.
Solution Approach 2:
The patent changes the representation parameters of weight parameters from a dense format to a sparse format, transforming how weights are stored and processed. This parameter change enables more efficient computation by focusing only on non-zero elements while maintaining the mathematical integrity of neural network operations.
3Quantity of substance
If weight parameters are pruned to reduce redundancy, then memory usage decreases, but neural network accuracy may deteriorate
Solution Approach 1:
The patent extracts only the redundant zero-weight parameters for removal, carefully preserving all non-zero weight parameters that contribute to neural network accuracy. This selective extraction reduces memory usage while maintaining accuracy by keeping all essential computational parameters intact.
Solution Approach 2:
The patent changes the storage parameter from dense to sparse format, which reduces memory usage by representing only non-zero elements. This parameter transformation does not prune meaningful weights but rather optimizes the representation of existing weights, thereby maintaining accuracy while reducing memory consumption.
Data Source
AI summary
Subject matter disclosed herein may relate to storage of signals and/or states representative of parameters in a computing device, and may relate more particularly to storage of signals and/or states representative of neural network parameters in a computing device.