Model compression method and device

Abstract

The embodiment of the invention discloses a model compression method and device, and the method comprises the steps: carrying out the feature extraction of the training data of a first model through employing a first model, and obtaining feature vectors output by the first model in N feature extraction layers of the first model; and taking the training data and the N first feature vectors output by the N feature extraction layers in the first model as training targets of the N feature extraction layers corresponding to the second model, and training the second model. Compared with the mode in the prior art, the prediction effect of the target model can be improved, and meanwhile the training amount of parameters of the model can be effectively reduced. Besides, since learning is the feature extraction layer, the output types of the first model and the second model can be different, so that the generalization ability of the learned model can be expanded, and the application range of model compression is widened.

Description

technical field [0001] The invention relates to the field of data processing, in particular to a model compression method and device. Background technique [0002] In the field of deep learning technology, users can obtain deep learning network models with better prediction effects through training models. However, deep learning network models with better prediction effects usually have relatively complex network structures, thus occupying a large storage space. Correspondingly, when using the deep learning network model to predict the data to be predicted, due to the complex structure of the deep learning network model, it may cause waste of computing resources, making the prediction efficiency poor. [0003] In order to solve this problem, it is usually possible to compress the deep network model with a relatively complex structure to obtain a deep learning network model with a relatively simple network structure and good predictive performance. Knowledge extraction is a ...

AI Technical Summary

Problems solved by technology

However, in this approach, the guidance process of multiple teacher models to the student model is limited to the training data (i.e. the output of the teacher model)

Therefore, the amount of parameters to be trained is huge, the training efficiency is low, and the output types of the teacher model and the student model are required to be the same, which limits the generalization of the model's learning

Images