Neural Network Depth Compression Using Dynamic Programming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks require significant computing resources and time for inference due to their increasing depth and complexity, especially in tasks involving large datasets, such as image classification and regression tasks in industries like semiconductor manufacturing.
Innovation Solution
A dynamic programming-based method that measures importance and inference time for blocks of consecutive linear layers, identifies the optimal location of nonlinear layers to remove, and merges adjacent linear layers using fine-tuning, thereby compressing the neural network depth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the depth of the neural network deepens to solve more difficult tasks and process more data, then the performance and accuracy of the neural network improve, but the computing resources and inference time required increase significantly
Solution Approach 1:
The patent merges consecutive linear layers into single linear layers by combining their weight matrices and bias vectors. This consolidation reduces the total number of layers in the neural network while preserving the computational functionality, thereby decreasing inference time and computational resource requirements while maintaining performance accuracy.
Solution Approach 2:
The patent selectively removes nonlinear layers from the neural network architecture based on their contribution to performance. By identifying and extracting redundant nonlinear layers that provide minimal performance benefit, the network depth is reduced, leading to faster inference while maintaining acceptable accuracy levels.
2Measurement precision
If the depth of the neural network deepens to solve more difficult tasks, then the performance and accuracy of the neural network improve, but the computing resources required increase significantly
Solution Approach 1:
The patent merges consecutive linear layers into single linear layers by combining their weight matrices and bias vectors. This consolidation reduces the total number of layers in the neural network while preserving the computational functionality, thereby decreasing inference time and computational resource requirements while maintaining performance accuracy.
Solution Approach 2:
The patent selectively removes nonlinear layers from the neural network architecture based on their contribution to performance. By identifying and extracting redundant nonlinear layers that provide minimal performance benefit, the network depth is reduced, leading to faster inference while maintaining acceptable accuracy levels.
3Speed
If nonlinear layers are removed to compress neural network depth and accelerate inference, then the inference speed improves, but the performance of the neural network may deteriorate
Solution Approach 1:
The patent selectively removes nonlinear layers from the neural network architecture based on their contribution to performance. By identifying and extracting redundant nonlinear layers that provide minimal performance benefit, the network depth is reduced, leading to faster inference while maintaining acceptable accuracy levels.
Solution Approach 2:
The patent employs a feedback mechanism where the importance of each nonlinear layer is evaluated based on its contribution to overall network performance. This feedback information guides the selective removal process, ensuring that only layers with minimal impact on performance are removed, thus maintaining performance while achieving speed improvements.
Data Source
AI summary
An electronic device includes one or more processors configured to measure importance and inference time for a plurality of blocks in which consecutive linear layers are merged, detect a location of a nonlinear layer maximizing the importance when the inference time is limited, using a dynamic programming algorithm, remove the remaining nonlinear layers except for the nonlinear layer at the detected location, and merge adjacent linear layers by removing the remaining nonlinear layers.