Neural Network Weight Compression for Memory and Accuracy Balance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As neural networks handle increasing amounts of data, they become complex, leading to over-fitting and excessive memory allocation, which hinders miniaturization and commercialization, while also reducing the reliability of predictions for new input data.
Innovation Solution
A method and apparatus for compressing neural network weights by determining modified weight sets, calculating compression efficiency values, and replacing weights with neighboring values to achieve a target weight that maximizes compression efficiency, thereby reducing the size of the weight set without compromising inference accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the neural network processes increasing amounts of data to improve accuracy, then the learning capability improves, but the connection complexity between layers increases excessively
Solution Approach 1:
The patent extracts and removes redundant or less important connections between neural network layers, keeping only the essential connections needed for accurate prediction. This pruning approach reduces connection complexity while preserving the network's learning capability and prediction reliability.
Solution Approach 2:
The patent applies different processing treatments to different connections within the neural network based on their importance. Important connections are preserved with high precision, while less important connections are reduced or removed, creating a non-uniform structure that optimizes the balance between complexity and performance.
2Productivity
If the neural network complexity increases to handle more data, then the data processing capability improves, but the memory allocation increases excessively
Solution Approach 1:
The patent removes unnecessary weight parameters from the neural network by pruning redundant connections. This extraction of essential elements reduces the total quantity of weight data that needs to be stored in memory, thereby reducing memory allocation requirements while maintaining data processing capability.
Solution Approach 2:
Instead of reducing network complexity to save memory, the patent inverts the approach by selectively removing only the least important connections while preserving the core functional structure. This allows the network to maintain high productivity with reduced memory requirements.
3Quantity of substance
If weight compression is applied to reduce memory requirements, then the storage efficiency improves, but the inference accuracy may deteriorate
Solution Approach 1:
The patent changes the precision parameters of weight representations by rounding weights to fewer decimal places or using quantized values. This parameter change reduces the storage size required for weights while carefully selecting the compression level to maintain inference accuracy within acceptable ranges.
Solution Approach 2:
The patent creates simplified copies of the original weight set with reduced precision or pruned connections. These copied weight sets are used for inference operations, providing a balance between storage efficiency and accuracy by using approximate rather than exact weight values.
Data Source
AI summary
A method of compressing weights of a neural network includes compressing a weight set including the weights of a the neural network, determining modified weight sets by changing at least one of the weights, calculating compression efficiency values for the determined modified weight sets based on a result of compressing the weight set and results of compressing the determined modified weight sets, determining a target weight of the weights satisfying a compression efficiency condition among the weights based on the calculated compression efficiency values, and determining a final compression result by compressing the weights based on a result of replacing the determined target weight.