Dynamic efficient routing method and device oriented to hybrid expert large model

Abstract

The invention provides a dynamic efficient routing method and device for a hybrid expert large model, and the method comprises the steps: obtaining input data and a pre-trained hybrid expert large model which comprises a distributor module and a router; dividing the input data into a plurality of lexical elements, and determining the optimal expert number of each lexical element by using a distributor module; and selecting an expert combination for each lexical element according to the optimal expert number by using the router, and routing each lexical element to the corresponding expert combination. According to the method, the number of activated experts can be adaptively adjusted according to the difficulty and importance of each lexical element, the mechanism not only improves the utilization efficiency of computing resources, but also further improves the performance of the model through refined resource allocation.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a dynamic and efficient routing method and device oriented to a hybrid expert large model. Background Art

[0002] As the number of parameters of large language models (LLMs) increases significantly, the computational cost of training and inference also increases significantly. The Mixture-of-Experts (MoE) architecture balances model size and computational cost by introducing expert networks and routing strategies. However, the existing MoE large models mostly use a fixed number of Top-K experts for processing, lacking adaptability to tokens of different complexity, resulting in uneven resource allocation and low efficiency. Summary of the invention

[0003] The present invention provides a dynamic and efficient routing method and device for a large hybrid expert model, which is used to solve the defects of using a fixed number of Top-K experts for processing in the prior art, lacking adaptability to tokens of different complexity, resulting in uneven resource allocation and low efficiency, and optimizes model performance and efficiency. The technical solutions proposed by the present invention are as follows: In a first aspect, the present invention provides a dynamic and efficient routing method for a hybrid expert large model, comprising: Obtain input data and a pre-trained hybrid expert large model, wherein the hybrid expert large model includes a distributor module and a router; Divide the input data into multiple tokens and use the allocator module to determine the optimal number of experts for each token; The router is used to select an expert combination for each word-unit according to the optimal number of experts, and each word-unit is routed to the corresponding expert combination.

[0004] Optionally, before dividing the input data into a plurality of word-grams and determining the optimal number of experts for each word-gram using the allocator module, the method further comprises: The allocator module is trained using a policy gradient algorithm.

[0005] Optionally, the using a policy gradient algorithm to train the allocator module comprises: For each training sample, decompose the training sample into multiple tokens; Use the allocator module to generate a probability distribution of the number of experts for each token; Sampling is performed according to the probability distribution to determine the number of experts to be activated; Use the activated experts to reason about the word-units and calculate the corresponding performance metrics; Inputting the activated expert quantity and performance index into a predefined reward function to calculate a reward value for each training sample; Based on the reward value, the policy parameters of the allocator module are updated using the policy gradient algorithm to maximize the expected return.

[0006] Optionally, the distributor module comprises an input layer, at least one hidden layer and an output layer; The step of dividing the input data into a plurality of word units and determining the optimal number of experts for each word unit by using a distributor module comprises: Divide the input data into multiple tokens and generate a representation vector for each token; The input layer receives the representation vector of each word; At least one hidden layer extracts word-unit features of each representation vector; The output layer outputs the probability distribution of the number of experts corresponding to each word-gram according to the word-gram features of each representation vector, and performs sampling based on the probability distribution of the number of experts to select the optimal number of experts for each word-gram.

[0007] Optionally, the expert combination includes at least one expert model; and the method further includes: Process the word-unit based on each expert model in the expert combination and generate corresponding output results; Aggregate the output results of different expert models to obtain the model output.

[0008] Optionally, before using the policy gradient algorithm to train the allocator module, the method further includes: Each MoE layer in the hybrid expert large model is replaced by a dynamic K routing layer including a distributor module.

[0009] In a second aspect, the present invention further provides a dynamic and efficient routing device for a hybrid expert large model, the device comprising the following modules: An acquisition module, used for acquiring input data and a pre-trained hybrid expert large model, wherein the hybrid expert large model includes a distributor module and a router; an allocator module for dividing the input data into a plurality of word-grams and determining the optimal number of experts for each word-gram using the allocator module; The routing module is used to select an expert combination for each word-unit according to the optimal number of experts using a router, and route each word-unit to a corresponding expert combination.

[0010] In a third aspect, the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the dynamic and efficient routing method for the hybrid expert large model as described in the first aspect above is implemented.

[0011] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the dynamic and efficient routing method for a hybrid expert large model as described in the first aspect above.

[0012] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the dynamic and efficient routing method for the hybrid expert large model as described in the first aspect above.

[0013] Based on the above technical solution, the beneficial effects of the present invention compared with the prior art are as follows: The dynamic and efficient routing method and device for a large hybrid expert model provided by the present invention can adaptively adjust the number of activated experts according to the difficulty and importance of each word-unit by dividing the input data into multiple words and using a distributor module to determine the optimal number of experts for each word-unit. The optimal number of experts for each word-unit is dynamically determined by a lightweight distributor module. This mechanism not only improves the utilization efficiency of computing resources, but also further improves the performance of the model through refined resource allocation.

[0014] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0015] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a flow chart of the dynamic and efficient routing method for the hybrid expert large model provided by the present invention.

[0018] Figure 2a This is the architecture diagram of the traditional Top-K routing mechanism.

[0019] Figure 2b This is an architectural diagram of the dynamic and efficient routing mechanism for the hybrid expert large model provided by the present invention.

[0020] Figure 3 It is a schematic diagram of comparative analysis of accuracy and number of activated experts provided by the present invention.

[0021] Figure 4 It is a schematic diagram comparing activation conditions of different levels of depth and task difficulty provided by the present invention.

[0022] Figure 5 It is a schematic diagram comparing the activation conditions of different parts of speech and before and after fine-tuning provided by the present invention.

[0023] Figure 6 It is a structural schematic diagram of a dynamic and efficient routing device for a hybrid expert large model provided by the present invention.

[0024] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] With the rapid development of large language models (LLMs), we are facing an unprecedented challenge: how to effectively control the computational cost while the scale of model parameters continues to expand. The traditional Mixture-of-Experts (MoE) architecture balances the scalability and computational efficiency of the model to a certain extent by introducing expert networks and routing strategies. However, these models generally adopt a fixed Top-K routing strategy, ignoring the differences in complexity of different tokens, resulting in unsatisfactory resource allocation efficiency.

[0027] In order to solve this problem, the present invention proposes a novel adaptive routing learning mechanism - a dynamic routing strategy based on a low-parameter allocator. The core of this strategy is to introduce a parameterized allocator, which can dynamically adjust the number of activated experts according to the intrinsic difficulty and importance of each token. This mechanism not only improves the utilization efficiency of computing resources, but also further improves the performance of the model through refined resource allocation. The motivation of the present invention is to explore a more efficient routing mechanism to cope with the growing model size and computing cost. Through efficient routing learning based on the allocator, we can not only promote the further improvement of LLMs in performance, but also provide new ideas and methods for resource optimization in large-scale applications. With the continuous advancement of artificial intelligence technology, this adaptive and efficient resource allocation strategy will have important theoretical and practical significance for promoting the widespread application and in-depth research of LLMs.

[0028] Combine the following Figure 1-Figure 6 The present invention describes a dynamic and efficient routing method and device for a large hybrid expert model.

[0029] Before routing, each MoE layer in the original mixed expert model is replaced with a dynamic K routing layer including an allocator module and a router. The dynamic K routing layer is designed to dynamically select the number of experts based on the complexity and importance of the input data. The allocator module is responsible for calculating the probability distribution of each input token assigned to different experts. This is usually achieved through functions such as softmax to ensure the non-negativity and normalization of the probability distribution. The router routes the input token to the corresponding expert for processing based on the probability distribution calculated by the allocator module. The router needs to implement an efficient routing mechanism, such as a dynamic routing strategy, to minimize the number of activated experts while maintaining model performance.

[0030] Therefore, the replaced hybrid expert model includes a distributor module and a router. The present invention introduces a lightweight, pluggable distributor module that works with the original router to dynamically perform expert allocation for each token. The lightweight distributor module, as a pluggable component, works with the router to generate a probability distribution of the number of experts for each token.

[0031] After replacing the large hybrid expert model, the present invention uses the Proximal Policy Optimization (PPO) algorithm to train the allocator module. The goal is to reduce the number of activated experts while maintaining or improving the performance of the model in the benchmark test. During the training process, the parameters of other parts of the model remain unchanged, and only the parameters of the allocator module are updated. The PPO algorithm, with its advantages in stability and efficiency, provides an optimization direction for the allocator's decision-making, so that the model can minimize the number of activated experts while maintaining performance.

[0032] First, define the objective function, which is designed to reduce the number of activated experts while maintaining or improving the performance of the model on the benchmark. This is usually achieved by defining a loss function with two parts: one is the loss of model performance (such as cross entropy loss), and the other is the reward for reducing the number of experts (such as the negative number of experts or the entropy of expert activation probability).

[0033] Next, the PPO algorithm is used to train the allocator module. The policy gradient algorithm updates the policy parameters by estimating the performance gradient of the policy, so that the policy tends to choose actions that can obtain higher rewards, that is, reducing the number of experts while maintaining model performance. In each training iteration, a batch of routing decisions for input data are first generated by the current policy, and the corresponding losses and rewards are calculated. Then, the gradient of the policy is estimated using these losses and rewards, and the parameters of the allocator module are updated. During the training process, except for the parameters of the allocator module, other parts of the model such as the parameters of the router remain unchanged. This helps ensure that changes in model performance mainly come from improvements in the allocator module. The performance of the model on the benchmark is regularly evaluated during the training process to ensure that the performance has not dropped significantly. According to the evaluation results, the objective function, the parameters of the policy gradient algorithm, or the structure of the allocator module are adjusted to further optimize the model performance.

[0034] The dynamic K routing layer of the present invention enables the model to adaptively adjust the number of experts according to the complexity and importance of different input data, thereby enhancing the flexibility and adaptability of the model. By dynamically selecting the number of experts, unnecessary computational overhead is reduced and the computational efficiency of the model is improved. The allocator module is trained by the PPO algorithm to ensure that the performance of the model in the benchmark test is maintained or improved while reducing the number of experts. Therefore, replacing the MoE layer in the hybrid expert model with the dynamic K routing layer and training the allocator module by the policy gradient algorithm is an effective optimization method that can maintain or improve the model performance while improving the computational efficiency.

[0035] The process of training the allocator module using the policy gradient algorithm specifically includes the following steps: S101: For each training sample, decompose the training sample into multiple word units.

[0036] For each training sample, use text preprocessing tools such as a tokenizer to decompose the text of the training sample into a series of tokens. These tokens can be words, punctuation marks, numbers, etc. Each token is an independent unit in the text, which is convenient for subsequent processing.

[0037] S102. Use the allocator module to generate a probability distribution of the number of experts for each word.

[0038] The allocator module is a neural network whose input is the embedding representation of the word unit and whose output is a vector, where each element of the vector corresponds to a possible number of experts. First, the embedding representation of the word unit is transformed through one or more linear layers to generate an unnormalized score vector. The output layer of the allocator module uses the softmax function, which can convert the output of the linear layer into a probability distribution. Then, the softmax function is used to convert this score vector into a probability distribution. The softmax function ensures that the sum of all probability values ​​is 1, forming a valid probability distribution, and each probability value represents the possibility of selecting the corresponding number of experts.

[0039] S103. Sampling is performed according to the probability distribution generated in S102 to determine the number of experts to be activated.

[0040] According to the probability distribution, a number of experts is randomly selected as the number of activated experts for the current word. This process is random but guided by the probability distribution.

[0041] S104: Use the activated experts to reason about the word-units and calculate corresponding performance indicators.

[0042] The above performance indicators can be a comprehensive consideration of multiple indicators such as accuracy, speed, resource utilization, etc. The expert reasoning process may involve complex calculations or model predictions, and the calculation of performance indicators depends on the specific task requirements.

[0043] According to the number of activated experts, select the corresponding number of experts from the pre-trained expert library. Input the word unit into the selected expert combination to perform complex calculations or model predictions. Based on the inference results and the true labels, calculate performance indicators such as accuracy, speed, resource utilization, etc.

[0044] S105: Input the activated expert quantity and performance index into a predefined reward function to calculate the reward value of each training sample.

[0045] First, define a reward function. The design of the reward function should be based on the task objective and the optimization objective. For example, if the goal is to maximize accuracy while minimizing the number of experts, the reward function may need to take these two factors into account. The function accepts the number of activated experts and the performance index as input and outputs a scalar reward value. Then the number of activated experts and the performance index are input into the reward function to calculate the reward value for each training sample. This paper takes accuracy as a performance indicator as an example to illustrate how to construct a reward function: The reward function includes two parts: accuracy reward and expert number penalty. Reward function = accuracy reward - expert number penalty

[0046] About accuracy rewards: Accuracy is the proportion of correct predictions or processing. In a hybrid expert system, it can be defined as the ratio of the number of correctly processed word units to the total number of word units. For each training sample, calculate the accuracy of the training sample after being processed by the expert combination. The accuracy calculation process is as follows: Ensure that the training data has accurate labels or reference outputs for subsequent evaluation of model performance. Use the hybrid expert model to predict the training data and obtain the prediction results for each word unit. Compare the prediction results with the true labels or reference outputs, and count the number of correctly processed word units. Calculate the accuracy based on the number of correctly processed word units and the total number of word units. The accuracy can be used directly as a reward value, or it can be transformed in form such as linear transformation, exponential transformation, etc. to adjust the sensitivity of the reward.

[0047] Regarding the penalty for the number of experts: In order to encourage the model to use as few experts as possible, a penalty term for the number of experts can be added to the reward function. Set a penalty value proportional to the number of experts, which is subtracted from the accuracy reward. The weight of the penalty term can be adjusted according to actual needs to balance the relationship between accuracy and the number of experts.

[0048] The above reward function R can be set as:

[0049] Among them, Accuracy represents the accuracy of the model on the validation set, Average Experts Activated represents the average number of experts activated per token, and α and β are weight coefficients to balance the importance of the two. The weights of accuracy reward and expert number penalty are continuously adjusted based on the results of experiments and performance evaluation.

[0050] S106. Based on the reward value, use the PPO algorithm to update the policy parameters of the allocator module to maximize the expected return.

[0051] First, initialize the model and set the initial parameters of the allocator module and the expert model. Then, for each training sample, process it according to steps S101 to S104. Use a predefined reward function to calculate the reward value of each training sample. Based on the reward value, use the PPO algorithm to update the policy parameters of the allocator module. During the update process, the goal is to maximize the expected return, that is, the cumulative sum of the reward values. Regularly evaluate the performance of the model during the training process, including the accuracy and the number of experts used. Adjust the parameters of the reward function or the configuration of the optimization algorithm based on the evaluation results. Repeat the training process and performance evaluation steps until the model performance reaches a satisfactory level or reaches the preset training rounds. Through the above steps, a reward function that can maximize the accuracy and minimize the number of experts can be designed, and the allocator module in the hybrid expert model can be optimized based on the reward function.

[0052] The PPO algorithm guides the update direction of the parameters by calculating the gradient of the policy parameters. Specifically, the expectation of the reward value is estimated, and the difference between the current policy and the old policy is considered to limit the update amplitude. According to the calculated gradient, the policy parameters of the allocator module are updated along the gradient direction to maximize the expected return. Repeat steps S101 to S106 to continuously improve the performance of the allocator module through iterative optimization. Through the above steps, the allocator module can learn how to assign the optimal number of experts to each word element according to the characteristics of the input data and the task requirements, thereby achieving dynamic and efficient routing. During the training process, the parameters of the original hybrid expert model are kept unchanged, and only the newly introduced allocator module is trained.

[0053] The present invention uses the PPO algorithm to optimize the decision of the allocator to maximize the overall performance while minimizing the number of activated experts. Since the PPO algorithm has strong flexibility and adaptability, the present invention uses the iterative optimization of the PPO algorithm to enable the allocator module to learn a more reasonable expert allocation strategy, automatically adjust the expert allocation strategy according to the needs of different tasks and scenarios, and improve the overall performance of the system. By accurately controlling the number of experts, the allocator module can avoid resource waste and improve resource utilization and computing efficiency.

[0054] Reference Figure 1 As shown, the dynamic and efficient routing method for the hybrid expert large model provides a dynamic K routing strategy for adaptively adjusting the number of activated experts according to the intrinsic difficulty and importance of each token. The method includes the following steps: Step S201, obtain input data and a pre-trained hybrid expert large model.

[0055] The above input data is the raw data processed by the model, which can be in various forms such as text, image, speech, etc. The hybrid expert model consists of a distributor module and a router. The distributor module is responsible for determining how many experts are needed to process each word, while the router is responsible for routing these words to the corresponding expert combination.

[0056] Step S202: divide the input data into multiple word-units, and use the distributor module to determine the optimal number of experts for each word-unit.

[0057] The input data is broken down into multiple tokens so that they can be processed independently. The allocator module determines an optimal number of experts for each token based on the characteristics of the input data and the task requirements. This number varies depending on the complexity of the token and the current state of the model.

[0058] Step S203: Using a router to select an expert combination for each word-unit according to the optimal number of experts, and routing each word-unit to the corresponding expert combination.

[0059] The router selects the most appropriate expert combination from the pre-trained expert library for each word-unit based on the information provided by the distributor module. This selection process may be based on a variety of factors, such as the expert's expertise, current load, historical performance, etc. Once the expert combination is selected, the router will route each word-unit to the corresponding expert combination for processing.

[0060] The present invention can flexibly allocate different numbers of experts to each word unit through the allocator module, so as to adapt to the complexity and processing requirements of different word units. By dynamically selecting the number of experts, the allocator module can reduce unnecessary computing overhead and improve the computing efficiency of the overall model. This process can be easily extended to more complex model structures and larger data sets, making it possible to process larger and more complex NLP tasks. And through the optimization of the policy gradient algorithm (such as PPO), the allocator module can gradually learn a better expert allocation strategy, thereby further improving the performance and accuracy of the model. The method of the present invention can be applied to large models such as large language models (LLMs) and large visual models based on the MoE architecture.

[0061] Reference Figure 2a As shown in Figure 1, in traditional Top-K routing, each token always activates the same number of experts regardless of its complexity. This approach may lead to resource waste when processing simple tokens, and may affect model performance due to insufficient resources when facing complex tokens. Figure 2bAs shown in the figure, the routing strategy proposed in the present invention first dynamically determines the number of experts actually activated for each token through the allocator, and then routes each token through the router under the guidance of this number, thereby achieving more accurate resource allocation. This dynamic adjustment capability enables the model to invest more computing resources in complex tokens and reduce resource consumption for simple tokens, thereby optimizing the efficiency and performance of the model as a whole.

[0062] In an optional embodiment, the expert combination includes at least one expert model; the method further includes: S204: Process the word-unit based on each expert model in the expert combination and generate corresponding output results.

[0063] In the aforementioned step S203, each word-gram has been routed to a specific expert combination. In this step, the word-gram will be sent to each expert model in the combination according to the expert combination to which it is assigned. After receiving the word-gram, each expert model will process it independently. The processing process may include calculation, reasoning, prediction, etc., depending on the type of expert model and task requirements. After the processing is completed, each expert model will generate a corresponding output result. These output results may be in the form of probability distribution, prediction label, vector representation, etc., depending on the output layer design of the model.

[0064] S205: Aggregate the output results of different expert models to obtain model output.

[0065] First, the output results from all expert models in the expert combination are collected. Then, an aggregation strategy is used to integrate these output results. The aggregation strategy can be simple averaging, weighted averaging, voting, attention mechanism, etc., depending on the task requirements and model design. Finally, the comprehensive result obtained according to the aggregation strategy will be used as the final output of the model. This output can be used for subsequent tasks such as classification, regression, generation, etc.

[0066] By combining multiple expert models, the hybrid expert model can leverage the expertise and strengths of different experts, thereby enhancing the processing power and performance of the overall model. Each expert model in the expert combination may be good at processing different types of data or features, so by aggregating their outputs, the model can better adapt to various input situations and improve generalization capabilities. The hybrid expert model can dynamically add or remove expert models as needed to adapt to different tasks and data sets. This flexibility makes the model highly scalable. Since not all expert models need to process every word, the hybrid expert model can use computing resources more efficiently. Only when necessary will the relevant expert model be activated for processing. The performance of the hybrid expert model can be further improved by optimizing the allocation and aggregation strategy of the expert combination. For example, using the attention mechanism to dynamically adjust the weights of different expert models can make the model pay more attention to the expert models that have an important impact on the final output.

[0067] In an optional embodiment, the distributor module includes an input layer, at least one hidden layer and an output layer; The step S202 of dividing the input data into a plurality of word units and using the distributor module to determine the optimal number of experts for each word unit includes: S301, dividing the input data into multiple word units, and generating a representation vector for each word unit.

[0068] First, the model preprocesses the raw input text, including removing useless characters, standardizing text formats, etc. The preprocessed text is divided into smaller units, namely tokens. This process breaks down the text into words, subwords, or characters, etc. Then, a fixed-length representation vector (embedding vector) is generated for each word unit. The representation vector is the representation of the word unit in a continuous vector space, which can capture the semantic and contextual information of the word unit. This is usually achieved through a pre-trained embedding layer or lookup table, where each word unit is mapped to a unique vector.

[0069] The generation process of the representation vector is as follows: The generated tokens are mapped to unique indices or identifiers in the vocabulary. The vocabulary is a list of all possible tokens, each with a corresponding numerical representation. This step converts the tokens into a numerical format that the model can process. In order to capture the semantic relationships and contextual information between tokens, the model usually converts the numerical representation of the tokens into representation vectors. These representation vectors are dense, high-dimensional representations of numbers that contain rich semantic information, allowing the model to understand and generate coherent text.

[0070] In the MoE-based LLM model, once the token representations are generated, they are fed into the MoE layer for further processing. In the MoE layer, each token is dynamically routed to different expert models for computation based on its representation vector. The input layer of the allocator module receives the token representation (i.e., the representation vector mentioned above) as input.

[0071] S302: The input layer receives the representation vector of each token as input.

[0072] The input layer of the allocator module is responsible for receiving the representation vectors from step S301. These representation vectors serve as the initial input data of the model and contain the semantic and contextual information of the word.

[0073] S303. At least one hidden layer extracts word-unit features of each representation vector.

[0074] After the input layer, one or more hidden layers perform nonlinear transformations on the representation vectors to extract higher-level word-unit features. These hidden layers usually contain activation functions such as ReLU, which can introduce nonlinearity and enable the model to learn complex mapping relationships. Information is passed between hidden layers through forward propagation, and each layer performs calculations based on the output of the previous layer, gradually extracting more abstract and useful features.

[0075] S304. The output layer outputs the probability distribution of the number of experts corresponding to each word-gram according to the word-gram features of each representation vector, and performs sampling based on the probability distribution of the number of experts to select the optimal number of experts for each word-gram.

[0076] The output layer receives the feature representations from the last hidden layer and applies the softmax function to convert these features into a probability distribution of the number of experts corresponding to each word. Each probability value represents the possibility of selecting the corresponding number of experts. This probability distribution is used to guide how to route tokens to different numbers of expert models for more efficient and accurate calculations. Based on this probability distribution, a random number generator is used for sampling to select a specific number of experts for each word as the optimal number. This number will be used in subsequent steps to guide the router to route the word to the corresponding expert combination.

[0077] The present invention uses a nonlinear transformation of multiple hidden layers, and the distributor module can extract the deep features of the word unit, which are crucial for the subsequent decision on the number of experts. Due to the use of a probability distribution method, the distributor module can flexibly allocate different numbers of experts to each word unit, thereby adapting to the complexity and processing requirements of different word units. By dynamically selecting the number of experts, the distributor module can reduce unnecessary computing overhead and improve the computing efficiency of the overall model. The process can be easily extended to more complex model structures and larger data sets, providing the possibility of processing larger-scale and more complex tasks.

[0078] In order to evaluate the performance of the efficient routing learning strategy of low parameter allocator on different large language models (LLMs), it is compared with the conventional Top-K routing method. Three popular MoE-based LLMs are selected as baseline models, namely Mixtral-8x7B, DeepSeek-MoE-16B and Qwen1.5-MoE-A2.7B. These models are representative in architecture and cover different numbers of experts and routing strategies. In the experiments, the original structures of these models are kept, and only the routing strategies are adjusted. Through extensive evaluation on three popular MoE-based LLMs, it is demonstrated that the proposed routing method significantly outperforms the conventional Top-K routing. It not only consistently reduces the number of activated experts by 30% to 40%, but also almost maintains the performance on multiple standard LLM benchmarks, and even achieves performance improvement in some cases.

[0079] Performance evaluation mainly focuses on two aspects: model accuracy and the number of activated experts. Accuracy is a key indicator to measure the performance of a model on a specific task, while the number of activated experts is directly related to the computational efficiency of the model. This paper integrates the proposed routing strategy into the above-mentioned baseline model and optimizes the allocator through the PPO algorithm. This strategy can dynamically allocate the most appropriate number of experts to each token in order to reduce the consumption of computing resources while maintaining or improving performance. Experimental results show that this routing strategy achieves significant performance improvement on all baseline models. Specifically, referring to Figure 3 As shown in Figure 2, in terms of reducing the number of activated experts, the routing strategy of the present invention reduces the expert activation by 30% to 40% on average, which directly translates into an increase in inference speed. In terms of accuracy, the routing strategy proposed by the present invention almost maintains the original performance in most cases, and even achieves performance improvement in some tasks.

[0080] In order to deeply understand the impact of the dynamic routing strategy proposed in this paper, we further analyzed the impact of different task difficulties, different level depths, and different parts of speech (Part-of-Speech) on expert activation. Figure 4 As shown, the activation status of different layer depths and task difficulties. Figure 5 As shown, the activations of different parts of speech and before and after fine-tuning are shown. It can be found that when processing complex tasks, intermediate layers, and content words, the routing tends to activate more experts, which shows that the strategy can effectively allocate resources according to the complexity and importance of the input. Figure 4 The Layer Index is the layer index, Activated Experts is the activated experts, and Frequency is the frequency. Figure 5 The Expert Index is the expert index, and the Activated Ratio is the activation ratio.

[0081] Based on the experimental results, the routing strategy of the present invention proves its effectiveness in improving the performance of large language models. Through refined expert activation control, the strategy not only optimizes the computational efficiency of the model, but also maintains the performance of the model on a variety of tasks. These findings provide valuable insights for the design and optimization of future LLMs and demonstrate the potential of the routing strategy in practical applications.

[0082] The following is a description of the dynamic and efficient routing device for the hybrid expert large model provided by the present invention. The dynamic and efficient routing device for the hybrid expert large model described below and the dynamic and efficient routing method for the hybrid expert large model described above can be referenced to each other.

[0083] The dynamic and efficient routing device for the hybrid expert large model provided by the present invention refers to Figure 6 As shown, it includes the following modules: An acquisition module 401 is used to acquire input data and a pre-trained hybrid expert large model, wherein the hybrid expert large model includes a distributor module and a router; an allocator module 402 for dividing the input data into a plurality of word-grams and determining the optimal number of experts for each word-gram using an allocator module; The routing module 403 is used to select an expert combination for each word-unit according to the optimal number of experts using a router, and route each word-unit to a corresponding expert combination.

[0084] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 550, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 550. The processor 510 may call the logic instructions in the memory 530 to execute the dynamic and efficient routing method for the hybrid expert large model.

[0085] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0086] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the dynamic and efficient routing method for hybrid expert large models provided by the above methods.

[0087] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the dynamic and efficient routing method for hybrid expert large models provided by the above methods.

[0088] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0089] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dynamic and efficient routing method for a large hybrid expert model, characterized in that: include: Obtain input data and a pre-trained hybrid expert large model, wherein the hybrid expert large model includes a distributor module and a router; Divide the input data into multiple tokens and use the allocator module to determine the optimal number of experts for each token; The router is used to select an expert combination for each word-unit according to the optimal number of experts, and each word-unit is routed to the corresponding expert combination.

2. The dynamic and efficient routing method for hybrid expert large model according to claim 1 is characterized in that: Before dividing the input data into a plurality of word-grams and determining the optimal number of experts for each word-gram using the allocator module, the method further comprises: The allocator module is trained using a policy gradient algorithm.

3. The dynamic and efficient routing method for hybrid expert large model according to claim 2 is characterized in that: The method of using a policy gradient algorithm to train the allocator module comprises: For each training sample, decompose the training sample into multiple tokens; Use the allocator module to generate a probability distribution of the number of experts for each token; Sampling is performed according to the probability distribution to determine the number of experts to be activated; Use the activated experts to reason about the word-units and calculate the corresponding performance metrics; Inputting the activated expert quantity and performance index into a predefined reward function to calculate a reward value for each training sample; Based on the reward value, the policy parameters of the allocator module are updated using the policy gradient algorithm to maximize the expected return.

4. The dynamic and efficient routing method for hybrid expert large model according to claim 1 is characterized in that: The distributor module includes an input layer, at least one hidden layer, and an output layer; The step of dividing the input data into a plurality of word units and determining the optimal number of experts for each word unit by using a distributor module comprises: Divide the input data into multiple tokens and generate a representation vector for each token; The input layer receives the representation vector of each word; At least one hidden layer extracts word-unit features of each representation vector; The output layer outputs the probability distribution of the number of experts corresponding to each word-gram according to the word-gram features of each representation vector, and performs sampling based on the probability distribution of the number of experts to select the optimal number of experts for each word-gram.

5. The dynamic and efficient routing method for hybrid expert large model according to claim 1 is characterized in that: The expert combination includes at least one expert model; the method further includes: Process the word-unit based on each expert model in the expert combination and generate corresponding output results; Aggregate the output results of different expert models to obtain the model output.

6. The dynamic and efficient routing method for hybrid expert large model according to claim 2 is characterized in that: Before training the allocator module using a policy gradient algorithm, the method further includes: Each MoE layer in the hybrid expert large model is replaced by a dynamic K routing layer including a distributor module.

7. A dynamic and efficient routing device for a large hybrid expert model, characterized in that: include: An acquisition module, used for acquiring input data and a pre-trained hybrid expert large model, wherein the hybrid expert large model includes a distributor module and a router; an allocator module for dividing the input data into a plurality of word-grams and determining the optimal number of experts for each word-gram using the allocator module; The routing module is used to select an expert combination for each word-unit according to the optimal number of experts using a router, and route each word-unit to a corresponding expert combination.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the dynamic and efficient routing method for the hybrid expert large model is implemented as claimed in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the dynamic and efficient routing method for a hybrid expert large model according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the dynamic and efficient routing method for a hybrid expert large model according to any one of claims 1 to 6 is implemented.

Images