The invention discloses a retraining-free
pruning and recombination method for a sparse expert
hybrid large model, and belongs to the technical field of
large model compression and optimization. The method aims at solving the problems that due to the fact that an existing sparse expert
hybrid (SMoE) model needs to load all expert parameters, memory occupation is too high, and deployment is difficult. According to the method, firstly, redundant experts are identified and pruned based on routing activation statistics; then, decomposing the pruned experts into
neuron-level functional fragments, and redistributing the fragments to the reserved experts according to
structural similarity; and finally, original fragments and newly distributed fragments are merged in the reserved experts through a weighted clustering
algorithm, so that compact experts with fewer parameters and stronger expression ability are reconstructed. According to the method, fine-grained operation is carried out at the
neuron level, the inherent representation conflict and
dislocation problems among experts are effectively solved, the performance of the compressed model is remarkably improved, and reliable
technical support is provided for deploying a large-scale SMoE model.