Neural Network Compression via Layered Quantization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks occupy significant storage space, leading to issues like slow performance and downtime on devices, especially when multiple applications are installed on mobile devices.
Innovation Solution
A method and apparatus for compressing neural networks by selecting layers, quantifying parameters, and training using a machine learning method to achieve effective compression while maintaining original accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If neural networks are installed on mobile devices to provide AI functionality, then the application's functional capabilities are improved, but the storage space consumption increases significantly
Solution Approach 1:
The patent segments the neural network into multiple layers and processes each layer independently through quantization. By dividing the network structure into manageable layers and applying compression techniques layer by layer, the system achieves significant storage reduction while maintaining overall network functionality.
Solution Approach 2:
The patent changes the precision parameters of neural network weights from high-precision floating-point numbers to low-precision quantized values. This parameter transformation reduces the storage requirements for each weight while preserving the essential computational functionality of the network.
2Measurement precision
If high-precision parameters are used in neural networks, then the calculation accuracy is improved, but the storage space requirement increases
Solution Approach 1:
The patent systematically changes the precision parameter of neural network weights from high-precision floating-point representation to low-precision quantized representation. This parameter transformation achieves storage compression while the patent demonstrates that accuracy can be maintained through proper quantization strategies and subsequent fine-tuning.
Solution Approach 2:
The patent employs feedback mechanisms during the quantization process, where the quantized network is evaluated on validation data and the quantization parameters are adjusted accordingly. This feedback loop ensures that the quantized model maintains accuracy comparable to the original high-precision model while achieving significant storage reduction.
3Adaptability or versatility
If multiple neural network applications are installed on a device, then the functional versatility is improved, but the device performance deteriorates due to storage constraints
Solution Approach 1:
The patent segments the neural network model into compressed layers, enabling more models to be stored within the same storage capacity. This segmentation and compression approach allows mobile devices to host multiple AI applications simultaneously without compromising overall device performance.
Solution Approach 2:
By changing the precision parameters of neural network weights to lower precision representations, the patent reduces the storage footprint of each AI model. This enables users to install and run multiple neural network-based applications on resource-constrained mobile devices, thereby improving functional versatility without overwhelming device storage and performance capabilities.
Data Source
AI summary
A method and apparatus for compressing a neural network are provided. A specific embodiment of the method includes: acquiring a to-be-compressed trained neural network; selecting at least one layer from layers of the neural network as a to-be-compressed layer; performing the following processing steps sequentially on each of the to-be-compressed layers in descending order of the number of level of the to-be-compressed layer in the neural network: quantifying parameters of the to-be-compressed layer based on a specified number, and training the quantified neural network based on a preset training sample using a machine learning method; and determining the neural network obtained after performing the processing steps on the selected at least one to-be-compressed layer as a compressed neural network, and storing the compressed neural network. This embodiment achieves efficient compression of the neural network.