The invention relates to the field of
machine learning, in particular to a knowledge
distillation method based on teacher model output reconstruction, which comprises the following steps of: preprocessing a
data set to obtain a preprocessed
data set capable of being directly input into a teacher model, inputting the teacher model and a student model, respectively obtaining classification results corresponding to the teacher model and the student model and outputting
logit, and calculating the output
logit of the teacher model and the student model; obtaining the probability distribution of the output
logit after the teacher model is corrected, the probability distribution of the non-target category, and the probability distribution of the output logit of the student model; and performing knowledge
distillation based on the probability distribution of the output logit corrected by the teacher model, the probability distribution of the non-target category and the probability distribution of the output logit of the student model, and training the student model to obtain the required student model. According to the method, the output probability distribution of the teacher model can be reconstructed, the knowledge quality is improved, the knowledge type is perfected, the student
model learning is more accurate, the
model compression performance is optimized, and the generalization ability of the student model is enhanced.