The invention provides a knowledge
distillation-based
model compression method and
system, and the method comprises the steps: inheriting a soft
label of a maturely trained teacher model, defining a
distillation loss function of a student model through the soft
label, enabling the student model to be compressed, generating a new sample extension
training set through random
sample extraction and mixing, and achieving the compression of the student model. The problem of insufficient data is relieved, meanwhile, the robustness of the model to
noise and distribution offset is improved, through dynamic
weight adjustment, a student model is made to quickly fit teacher model knowledge, autonomous optimization is conducted in the later
training period, and over-fitting of soft labels is avoided. On a
test set in which the proportion of unlabeled data is 30%, on the basis that the accuracy of using a real
label is 78.3%, the accuracy is improved by 11.2% by using a pseudo target, deployment is performed by using a maturely trained student model, middle layer feature alignment is omitted, the occupation of a GPU
video memory is reduced from 3.2 GB to 1.8 GB, and the deployment requirement of edge equipment is met.