Neural Network Weight Compression for Memory-Accuracy Tradeoffs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As neural networks handle increasing amounts of data, they become complex, leading to over-fitting and excessive memory allocation, which hinders miniaturization and commercialization, while accuracy for new input data decreases.
Innovation Solution
A method and apparatus for compressing neural network weights by determining modified weight sets, calculating compression efficiency values, and replacing weights with neighboring values to achieve a target weight that maximizes compression efficiency, thereby reducing the size of the weight set without significantly impacting inference accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the neural network processes increasing amounts of data to improve accuracy, then the learning capability improves, but the connection complexity between layers increases excessively
Solution Approach 1:
The patent extracts and removes redundant or less important connections between neurons in the neural network. By identifying and eliminating unnecessary weight connections, the network maintains its accuracy on important data while reducing overall connection complexity and memory requirements.
Solution Approach 2:
The patent changes the parameters of the neural network by adjusting weight values, sparsity levels, and connection thresholds. These parameter modifications allow the network to maintain performance while reducing complexity through controlled changes to the weight distribution and connection density.
2Productivity
If the neural network complexity increases to handle more data, then the data processing capability improves, but the memory allocation increases excessively
Solution Approach 1:
The patent extracts only the essential weight parameters needed for data processing while removing redundant information. This extraction approach maintains the network's data processing capability by preserving critical connections while eliminating memory-consuming redundant weights.
Solution Approach 2:
Instead of storing all original weights, the patent inverts the approach by storing only differential values or compressed representations of weights. This inversion strategy reduces memory allocation while maintaining the ability to reconstruct and use the full weight set for data processing.
3Quantity of substance
If weight compression is applied to reduce memory requirements, then the memory allocation decreases, but the inference accuracy may decrease
Solution Approach 1:
The patent employs feedback mechanisms where the compression process continuously monitors inference accuracy and adjusts compression parameters accordingly. When accuracy degradation is detected, the system feedback-adjusts the compression level or selects different compression strategies to maintain performance while achieving memory reduction.
Solution Approach 2:
The patent applies parameter changes by adjusting compression ratios, quantization levels, and weight precision dynamically. These parameter modifications allow the system to find optimal compression settings that minimize memory usage while maintaining inference accuracy within acceptable thresholds.
Data Source
AI summary
A method of compressing weights of a neural network includes compressing a weight set including the weights of a the neural network, determining modified weight sets by changing at least one of the weights, calculating compression efficiency values for the determined modified weight sets based on a result of compressing the weight set and results of compressing the determined modified weight sets, determining a target weight of the weights satisfying a compression efficiency condition among the weights based on the calculated compression efficiency values, and determining a final compression result by compressing the weights based on a result of replacing the determined target weight.