Neural Network Layer Reduction for Mobile Memory Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks require significant memory resources, making them challenging to apply on mobile devices while maintaining a satisfactory recognition rate, as the memory size increases with the number of deeper layers, which complicates their implementation on devices with limited storage.
Innovation Solution
A method to reduce the neural network by adding a reduced layer, computing layer and result losses, and determining parameters based on these losses to minimize network loss, allowing for the removal of layers when the network loss is below a threshold, thereby reducing the memory size while maintaining recognition accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deeper layers are added to the neural network to improve recognition accuracy, then the recognition rate is improved, but the memory size increases
Solution Approach 1:
The patent segments the neural network into multiple layers and applies different reduction strategies to different layers. By dividing the network structure and applying layer-wise reduction with selective removal, the system maintains recognition accuracy while reducing overall memory size through targeted optimization of individual layer contributions.
2Measurement precision
If the number of layers is increased to improve recognition accuracy, then the recognition rate is improved, but the device complexity increases
Solution Approach 1:
The patent implements a dynamic layer removal mechanism where layers are selectively removed based on their contribution to recognition accuracy. The system dynamically evaluates layer importance and removes redundant layers, transforming the static complex structure into a dynamic optimized structure that maintains accuracy while reducing complexity.
3Quantity of substance
If layers are removed to reduce memory size, then the memory size is reduced, but the recognition rate may deteriorate
Solution Approach 1:
The patent employs feedback mechanisms by evaluating the contribution of each layer to the overall recognition accuracy before removal. The system monitors recognition rate changes and uses this feedback to determine whether layer removal is acceptable, ensuring that memory size is reduced while maintaining satisfactory recognition performance through iterative optimization.
Data Source
AI summary
A method to reduce a neural network includes: adding a reduced layer, which is reduced from a layer in the neural network, to the neural network; computing a layer loss and a result loss with respect to the reduced layer based on the layer and the reduced layer; and determining a parameter of the reduced layer based on the layer loss and the result loss.