The invention discloses a local disturbance agent-based
hybrid expert model heterogeneous
pruning method, which comprises the following steps of: S1, acquiring hierarchical structure information of a MoE model to be pruned, determining the number of MoE
layers contained in the model, and setting a total expert activation budget after
pruning; s2, evaluating output representation of each MoE layer under different activation expert numbers by using a pre-selected calibration
data set, calculating representation similarity of the output representation relative to high-quality baseline configuration, and obtaining a value function of the layer when an expert is adopted; s3, modeling a hierarchical Top-K activation
pruning problem into a multi-choice
knapsack problem under the constraint of total budget based on a value function, and solving the optimal expert activation number configuration of each MoE layer; s4, according to the optimal expert activation number configuration, the reasoning routing logic of the MoE model is updated; the method is used for solving the problem that in the prior art, a uniform pruning strategy with the low Top-K value is adopted for all
layers in a unified mode, the calculation cost is reduced, and meanwhile the model performance is obviously reduced.