Neural Network Model Compression via Intermediate Teacher Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural network models with large-scale parameters and complex structures are difficult to deploy on mobile devices due to storage constraints and inference delays, necessitating efficient compression methods that minimize translation quality degradation.
Innovation Solution
A neural network model compression method involving multiple intermediate teacher models is introduced, where knowledge from an original teacher model is transferred to candidate student models through multi-path training, allowing for the selection of a high-quality student model with reduced parameters and layers, thereby improving deployment on mobile devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If deep neural network models with large-scale parameters and complex structures are used to obtain optimal output results, then translation quality is improved, but model storage overhead increases and inference delays lengthen
Solution Approach 1:
The patent segments the knowledge transfer process into multiple stages by introducing intermediate teacher models with varying complexities. Instead of directly transferring from a single large teacher model, the system divides the compression path into multiple steps, where each intermediate model serves as a bridge, gradually reducing model complexity while preserving translation quality.
Solution Approach 2:
The patent introduces intermediate teacher models as intermediary components between the original large teacher model and the final student model. These intermediate models act as mediators that facilitate knowledge transfer through multiple paths, enabling gradual complexity reduction while maintaining translation performance.
2Productivity
If model compression techniques are applied to reduce storage overhead and improve inference speed, then deployment on mobile devices becomes feasible, but translation quality deteriorates
Solution Approach 1:
The patent implements feedback mechanisms by using the original teacher model to supervise the training of intermediate teacher models and student models. The knowledge distillation process continuously refines student models by comparing their outputs against the teacher model's predictions, ensuring translation quality is maintained while achieving compression goals.
Solution Approach 2:
The patent creates multiple candidate student models with different compression levels and uses dynamic selection based on performance requirements. The system can adaptively choose the appropriate student model depending on the specific deployment scenario, balancing inference speed and translation quality dynamically.
3Ease of manufacture
If single-path knowledge distillation is used to train student models, then training process is simple, but knowledge transfer effectiveness is limited
Solution Approach 1:
The patent adds an additional dimension to the knowledge transfer process by introducing multiple intermediate teacher models that create multiple training paths. This multi-dimensional approach enriches the knowledge transfer process, allowing student models to learn from diverse knowledge sources and improving transfer effectiveness while maintaining manageable training complexity.
Data Source
AI summary
A method for compressing a neural network model, includes: obtaining a set of training samples including a plurality of pairs of training samples, each pair of the training samples including source data and target data corresponding to the source data; training an original teacher model by using the source data as an input and using the target data as verification data; training intermediate teacher models based on the set of training samples and the original teacher model, one or more intermediate teacher models forming a set of teacher models; training multiple candidate student models based on the set of training samples, the original teacher model, and the set of teacher models, the multiple candidate student models forming a set of student models; and selecting a candidate student model of the multiple candidate student models as a target student model according to training results of the multiple candidate student models.