Neural Network Coefficient Compression Using Grouped Sparsity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks have large memory footprints and computational demands due to highly parameterized sets of coefficients, making them difficult to implement on devices with limited resources.
Innovation Solution
Implement a method involving sparsity and compression techniques, such as structured and unstructured sparsity, applied through pruner logic and compression logic, to reduce the memory footprint and computational demands of neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If highly parameterized sets of coefficients are used in neural networks, then accuracy and functionality are improved, but memory footprint and computational demands increase
Solution Approach 1:
The patent extracts and removes redundant or less important coefficients from the neural network model through pruning techniques. By identifying and eliminating coefficients that contribute minimally to the overall accuracy, the model achieves reduced memory footprint while maintaining acceptable performance levels.
Solution Approach 2:
The patent applies partial sparsity to coefficient sets, where only a portion of coefficients are retained while others are set to zero or removed. This partial action approach allows the system to achieve significant memory savings by keeping only the most critical coefficients necessary for maintaining acceptable accuracy.
2Reliability
If highly parameterized sets of coefficients are used in neural networks, then accuracy and functionality are improved, but computational demands increase
Solution Approach 1:
The patent extracts and removes redundant or less important coefficients from the neural network model through pruning techniques. By identifying and eliminating coefficients that contribute minimally to the overall accuracy, the model achieves reduced computational demands while maintaining acceptable performance levels.
Solution Approach 2:
The patent applies partial sparsity to coefficient sets, where only a portion of coefficients are retained while others are set to zero or removed. This partial action approach allows the system to achieve significant computational savings by keeping only the most critical coefficients necessary for maintaining acceptable accuracy.
3Productivity
If sparsity and compression techniques are applied to reduce memory footprint, then resource efficiency is improved, but model complexity increases
Solution Approach 1:
The patent applies sparsity and compression techniques during the model training and preparation phase rather than during inference. By performing coefficient pruning and compression in advance, the system reduces the complexity of the deployed model without adding computational overhead during actual operation.
Solution Approach 2:
The patent changes the parameters of coefficients by applying sparsity transformations and compression algorithms. This transforms the original dense coefficient sets into compressed representations that require fewer resources while maintaining the essential functional characteristics of the neural network.
Data Source
Figure 1
Figure 2a~2b
Figure 3a~3b
AI summary
A method of compressing a set of coefficients for subsequent use in a neural network, the method comprising: applying sparsity to a plurality of groups of the coefficients, each group comprising a predefined plurality of coefficients; and compressing the groups of coefficients according to a compression scheme aligned with the groups of coefficients so as to represent each group of coefficients by an integer number of one or more compressed values.