The application provides an entropy-aware self-
distillation-based
large model migration quantization
performance recovery method and
system, relating to the technical field of
artificial intelligence. The method aims to solve the pain point of "performance collapse" of model accuracy caused by ultra-low bit quantization due to limited memory capacity and bandwidth during the migration and
adaptation of large models to domestic hardware platforms. The technical path is as follows: first, determine the pre-trained large
language model as the teacher model; obtain the probability distribution of the model for Chinese
text processing samples in the random subset of the corpus according to the entropy-aware mechanism, and calculate the teacher and student weighting coefficients; perform parameter quantization on the pre-trained large
language model to obtain the student model to be trained; input each
text processing sample in the corpus into the above two models respectively to obtain the teacher and student probability distributions respectively, and fuse the two based on the two weighting coefficients to obtain the intermediate probability distribution; minimize the difference between the student model and the intermediate probability distribution through self-
distillation training, continuously repair the quantization accuracy loss and realize
performance recovery in
backpropagation, and obtain the trained student model. Finally, apply it to the
target text processing task to obtain the corresponding result. The method aims to improve the performance of ultra-low bit quantization while realizing efficient deployment and performance reconstruction of large models in the
domestic environment.