Neural Network Weight Tensor Compression for Low-Memory Deployment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks require large memory and computational resources, making them inefficient for devices with limited resources like mobile phones and IoT devices, and their transmission is cumbersome due to high bandwidth requirements.
Innovation Solution
A method involving a loss function that increases the sparsity of neural network weight tensors, entropy coding, and quantization to compress neural networks, allowing for efficient representation and transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If neural networks are deployed to devices with limited resources, then the device can perform advanced tasks, but the memory and computational requirements cannot be met
Solution Approach 1:
The patent extracts and removes redundant information from neural network weight tensors through entropy coding. By identifying and eliminating redundant patterns in the weight data, the system reduces the memory footprint while preserving the essential functional capabilities of the neural network, enabling deployment on resource-constrained devices
Solution Approach 2:
The patent transforms the weight tensor parameters into a compressed representation through entropy coding. This parameter transformation reduces the storage requirements and computational overhead while maintaining the neural network's ability to perform advanced tasks, resolving the contradiction between capability and resource consumption
2Ease of operation
If neural networks are transmitted among devices, then model sharing and updates are enabled, but the bandwidth requirements become prohibitively high
Solution Approach 1:
The patent extracts redundant information from the weight tensor before transmission. By applying entropy coding to remove redundant patterns in the weight data, the system significantly reduces the amount of data that needs to be transmitted between devices, enabling efficient model sharing and updates over limited bandwidth connections
Solution Approach 2:
Instead of transmitting the full weight tensor, the patent inverts the approach by transmitting only the essential compressed representation. The entropy-coded weight tensor contains only the necessary information to reconstruct the original weights, dramatically reducing transmission bandwidth requirements while maintaining model sharing functionality
Data Source
AI summary
There is provided an apparatus comprising means for training a neural network, wherein the training comprises applying a loss function configured to increase sparsity of a weight tensor of the neural network and to cause a plurality of non-zero elements of the weight tensor to be substantially equal to each other; and means for entropy coding the weight tensor to obtain a compressed neural network.