Neural Network Model Compression via Layer Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network models require high storage and computing capabilities due to millions or billions of parameters, limiting their use on terminal devices, and compression methods often reduce precision and effectiveness.
Innovation Solution
A method to compress a neural network model by obtaining a first low-bit model through training and then compressing it to a second low-bit model, where the operation layers are combined to maintain equivalence and reduce storage requirements without compromising precision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the neural network model is compressed by reducing parameters, then the storage and computing requirements are reduced, but the precision and effectiveness of the model are reduced
Solution Approach 1:
The patent combines multiple adjacent operation layers into a single fused operation layer. Specifically, it merges a first operation layer with a second operation layer to form a third operation layer, reducing the total number of operation layers while preserving the functional capabilities of the original layers. This merging approach reduces model size and computational overhead without sacrificing precision because the fused layer maintains equivalent computational power through careful design of the fusion operations.
2Quantity of substance
If the neural network model is compressed by reducing parameters, then the storage requirements are reduced, but the effectiveness of the model is reduced
Solution Approach 1:
The patent changes the parameters of the operation layers during the fusion process. When merging the first and second operation layers into the third operation layer, it adjusts the parameters (such as weights, biases, and activation functions) to ensure that the fused layer produces results equivalent to the sequential execution of the original layers. This parameter transformation maintains model effectiveness while achieving compression.
3Device complexity
If the operation layers are combined to compress the model, then the number of operation layers is reduced, but the complexity of model construction increases
Solution Approach 1:
The patent performs preliminary actions by pre-defining the fusion rules and parameter transformation formulas before actually merging the operation layers. It establishes the mathematical relationships and computational procedures needed for fusion in advance, which simplifies the actual model construction process. This preliminary preparation makes the complex task of layer fusion more manageable and systematic.
Data Source
AI summary
A neural network model processing method includes obtaining a first low-bit neural network model through training, where the model includes a first operation layer and a second operation layer. Each operation layer includes at least one operation. Values/a value of a parameter and/or data used for the operation are/is represented by using N bits, and N is a positive integer less than 8. The neural network model processing method further includes compressing the model to obtain a second low-bit neural network model, where the compressed model includes a third operation layer. The third operation layer is equivalent to the first operation layer and the second operation layer, and an operation layer other than the third operation layer in the at least one operation layer is the same as an operation layer other than the first operation layer and the second operation layer in the at least two operation layers.