Method and device for training expert model, storage medium and electronic device

Abstract

The application discloses a kind of training method and device of expert model, storage medium and electronic equipment, it is related to computer technical field, including determining target expert network group from the multiple expert network groups included in initial expert model according to the estimated resource occupancy of input data;Multiple expert networks in target expert network group are successively used as current expert network, determine individual difference evaluation coefficient matched with current expert network;According to the individual difference evaluation coefficient of multiple expert networks in target expert network group respectively, determine at least one target expert network from multiple expert networks;According to at least one target expert network, input data is processed, and according to the load state description information corresponding to at least one target expert network respectively, model parameters in initial expert model are adjusted, the technical problem that selection of expert network is not accurate in prior art is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular to a method and device for training an expert model, a storage medium and an electronic device. BACKGROUND

[0002] As an innovative architecture for large language model training, Mixture of Experts (MoE) has attracted widespread attention in the field of artificial intelligence in recent years. The core idea of MoE model is to assign input data to different expert sub-models, and then combine the outputs of all sub-models to generate the final result, achieving efficient use of computing resources and improvement of model performance. MoE model has the characteristic of sparse activation, in each forward propagation process, only a small number of experts most relevant to the input data are activated for calculation, while most experts are in an inactive state. This sparsity significantly reduces the resources required for each calculation, and compared with traditional dense models, the computing efficiency is greatly improved.

[0003] However, the existing technology uses fixed expert activation rules, fixed activation expert numbers and fixed selection of expert networks, resulting in a situation where the selection of experts does not match the actual task requirements in actual application, and the computing resources cannot be effectively utilized. That is, there is an inaccurate selection of expert networks in the prior art. SUMMARY

[0004] The present application provides a method and device for training an expert model, a storage medium and an electronic device to at least solve the technical problem of inaccurate selection of expert networks in the prior art.

[0005] The present application provides a method for training an expert model, comprising: determining a target expert network group from a plurality of expert network groups included in an initial expert model according to an estimated resource occupation situation of input data, wherein the expert network group includes a plurality of expert networks for processing input data, and the estimated resource occupation situation is used to indicate the computing resources required by the expert networks to process the input data; sequentially taking the plurality of expert networks in the target expert network group as a current expert network, and determining an individual difference evaluation coefficient matched with the current expert network, wherein the individual difference evaluation coefficient is determined according to the respective network similarity between the current expert network and a plurality of reference expert networks in the target expert network group, and the reference expert network is an expert network in the target expert network group other than the current expert network; determining at least one target expert network from the plurality of expert networks according to the respective individual difference evaluation coefficients of the plurality of expert networks in the target expert network group; processing the input data according to the at least one target expert network, and adjusting the model parameters in the initial expert model according to the respective load state description information of the at least one target expert network.

[0006] This application also provides a training apparatus for an expert model, comprising: a first determining unit, for determining a target expert network group from multiple expert network groups included in an initial expert model based on the estimated resource consumption of the input data, wherein the expert network group includes multiple expert networks for processing the input data, and the estimated resource consumption is used to indicate the computing resources required by the expert networks when processing the input data; a second determining unit, for sequentially using multiple expert networks in the target expert network group as current expert networks, and determining individual difference evaluation coefficients matching the current expert networks, wherein the individual difference evaluation coefficients are determined based on the network similarity between the current expert network and multiple reference expert networks in the target expert network group, and the reference expert networks are expert networks in the target expert network group other than the current expert network; a third determining unit, for determining at least one target expert network from multiple expert networks based on the individual difference evaluation coefficients of each of the multiple expert networks in the target expert network group; and an adjusting training unit, for processing the input data according to the at least one target expert network, and adjusting the model parameters in the initial expert model according to the load state description information corresponding to each of the at least one target expert network.

[0007] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the training method of any of the above-described expert models.

[0008] This application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of the training method for any of the above-described expert models.

[0009] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of training any of the above-described expert models.

[0010] This application identifies a target expert network group from multiple expert network groups included in the initial expert model based on the estimated resource consumption of the input data. This ensures that each data request is assigned to an expert network group capable of effectively processing the resources required for that request, thereby directly locating experts whose processing capabilities match the needs and improving the accuracy of selection. Furthermore, by sequentially using multiple expert networks from the target expert network group as the current expert network and determining the individual difference evaluation coefficient matching the current expert network, this approach considers the structural and parameter differences between expert networks. It quantifies the differences between the current network and other networks within the same group, enhancing the accuracy of expert network selection within the group. This allows the system to handle the same task more efficiently. It tends to select experts who best match the task, rather than simply based on load or random selection; it determines at least one target expert network from multiple expert networks based on the individual difference evaluation coefficients of each expert network in the target expert network group, so that the task can be handled by experts with appropriate resource levels, thereby maximizing processing quality and efficiency; it processes the input data based on at least one target expert network, and adjusts the model parameters in the initial expert model based on the load state description information corresponding to each of the at least one target expert network, ensuring that the selection of expert networks is always based on the latest resource state and requirements, thus solving the technical problem of inaccurate expert network selection in existing technologies. Attached Figure Description

[0011] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A hardware structure block diagram of a server device for an expert model training method provided in an embodiment of this application;

[0013] Figure 2 A flowchart illustrating a training method for an expert model provided in an embodiment of this application;

[0014] Figure 3 A flowchart illustrating another expert model training method provided in this application embodiment;

[0015] Figure 4 This is a schematic diagram of the structure of an expert model training device according to an embodiment of this application;

[0016] Figure 5 This is a schematic diagram of the structure of an electronic device for training an expert model according to an embodiment of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0018] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0019] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] The methods and embodiments provided in this application can be executed on a server device or a similar computing device. Taking running on a server device as an example, Figure 1 This is a hardware structure block diagram of a server device for an expert model training method according to an embodiment of this application. For example... Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The server device may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the server equipment described above. For example, the server equipment may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0021] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the expert model training method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thus implementing the aforementioned method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to server devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0022] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the server device. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0023] The embodiments of this application provide a method for training an expert model, and the method is described in detail below in conjunction with the execution flow of the expert model training method.

[0024] To better understand the technical solutions provided in the embodiments of this application, the key terms involved in the embodiments of this application will be introduced first:

[0025] Mixture of Experts (MoE) model.

[0026] Large Language Models (LLMs).

[0027] Reinforcement Learning (RL).

[0028] Proximal Policy Optimization (PPO).

[0029] Long Short-Term Memory (LSTM) network.

[0030] Temporal Difference Error (TD-error).

[0031] Convolutional Neural Network (CNN).

[0032] Transformer Model (Transformer).

[0033] Bidirectional Encoder Representations from Transformers (BERT).

[0034] Generative Pre-trained Transformer (GPT).

[0035] As an optional implementation method, such as Figure 2 As shown, the training methods for the above expert models include:

[0036] S202, determine the target expert network group from the multiple expert network groups included in the initial expert model based on the estimated resource consumption of the input data. The expert network group includes multiple expert networks for processing the input data, and the estimated resource consumption is used to indicate the computing resources required by the expert network when processing the input data.

[0037] S204, take multiple expert networks in the target expert network group as the current expert network in turn, and determine the individual difference evaluation coefficient that matches the current expert network. The individual difference evaluation coefficient is determined based on the network similarity between the current expert network and multiple reference expert networks in the target expert network group. The reference expert networks are the expert networks in the target expert network group other than the current expert network.

[0038] S206, Based on the individual difference evaluation coefficients of each of the multiple expert networks in the target expert network group, determine at least one target expert network from the multiple expert networks;

[0039] S208, the input data is processed according to at least one target expert network, and the model parameters in the initial expert model are adjusted according to the load state description information corresponding to each of the at least one target expert network.

[0040] As an optional implementation, in step S202, a target expert network group is determined from the multiple expert network groups included in the initial expert model based on the estimated resource consumption of the input data. The expert network group includes multiple expert networks for processing the input data, and the estimated resource consumption is used to indicate the computing resources required by the expert network when processing the input data.

[0041] Optionally, the input data mentioned above includes, but is not limited to, text data, code data, multimodal data, etc., and is not limited to, used for tasks such as text understanding, software development, and media content analysis; the resource usage mentioned above includes, but is not limited to, CPU utilization, memory consumption, and GPU usage; the initial expert model mentioned above includes models with multiple hierarchical expert network groups, each group containing a set of expert networks; according to different processing capabilities or resource consumption, the expert networks are divided into different groups, such as lightweight group, balanced group, and high-performance group; the best expert network group is selected based on the estimated resource requirements of the input data and is used to process the data.

[0042] In step S204 above, multiple expert networks in the target expert network group are sequentially used as the current expert network, and the individual difference evaluation coefficient matching the current expert network is determined. The individual difference evaluation coefficient is determined based on the network similarity between the current expert network and multiple reference expert networks in the target expert network group. The reference expert networks are the expert networks in the target expert network group other than the current expert network.

[0043] Optionally, for each current expert network, the system calculates its similarity to the other networks in the group, for example, by comparing their core parameter matrices. If the similarity between current expert network A and expert networks B, C, and D is 0.8, 0.3, and 0.6, respectively, then A's individual difference evaluation coefficient is calculated, for example, by taking the reciprocal of the average similarity; the higher the coefficient, the greater the difference between the network and the other networks in the group.

[0044] In step S206 above, at least one target expert network is determined from the multiple expert networks based on the individual difference evaluation coefficients of each expert network in the target expert network group.

[0045] Optionally, if the individual difference evaluation coefficients of expert networks A, B, and C are 0.5, 0.7, and 0.4, respectively, expert network B is selected as the "target expert network" because its coefficient is the highest, indicating that B has the highest difference relative to other networks in the group.

[0046] In step S208 above, the input data is processed according to at least one target expert network, and the model parameters in the initial expert model are adjusted according to the load state description information corresponding to each of the at least one target expert network.

[0047] Optionally, the aforementioned load status description information is used to describe the current processing capacity or resource utilization of the expert network, monitor whether the expert network is overloaded or idle, and then update the parameters of the entire expert model based on the load status and processing results of the expert network in order to optimize future selections and processing.

[0048] As an optional implementation, after selecting at least one target expert network, taking expert network B as an example, image data processing is performed through target expert network B. The current load status description information of target expert network B is checked, such as CPU utilization and queue depth. Suppose that during the processing, it is found that the CPU utilization of target expert network B has reached 90% in a short period of time, that is, the effect of target expert network B in processing image data is poor. The system needs to adjust the parameters of the expert model. For example, in future tasks, the selection probability of target expert network B for processing image data can be reduced, or the task can be transferred to an expert network with a lower load when the load of expert network B exceeds a certain threshold. Specifically, the learning rate can be set, the optimizer (such as Adam) can be used to update the model parameters, or only the network parameters of specific parts can be updated to quickly adapt to the current load status. No specific restrictions are made here.

[0049] Through the embodiments described in this application, a target expert network group is determined from multiple expert network groups included in the initial expert model based on the estimated resource consumption of the input data. This ensures that each data request can be assigned to an expert network group capable of effectively processing the resources required for that request, thereby directly locating experts whose processing capabilities match the needs and improving the accuracy of selection. Furthermore, by sequentially using multiple expert networks in the target expert network group as the current expert network and determining the individual difference evaluation coefficient matching the current expert network, the structural and parameter differences between expert networks are considered. This quantifies the differences between the current network and other networks within the same group, enhancing the accuracy of expert network selection within the group and enabling the processing of the same task... When performing tasks, the system tends to select experts that best match the task, rather than simply based on load or random selection. Based on the individual difference evaluation coefficients of multiple expert networks in the target expert network group, at least one target expert network is determined from multiple expert networks, ensuring that the task can be handled by experts with appropriate resource levels, thereby maximizing processing quality and efficiency. The system processes the input data based on at least one target expert network and adjusts the model parameters in the initial expert model based on the load status description information corresponding to each of the at least one target expert network. This ensures that the selection of expert networks is always based on the latest resource status and requirements, solving the technical problem of inaccurate expert network selection in existing technologies.

[0050] In one optional implementation, multiple expert networks in the target expert network group are sequentially used as the current expert network, and individual difference evaluation coefficients matching the current expert network are determined, including:

[0051] S1, calculate the network similarity between the current expert network and multiple reference expert networks respectively;

[0052] S2, determine the individual difference evaluation coefficient that matches the current expert network based on the first average of multiple network similarities, wherein the individual difference evaluation coefficient has a negative correlation with the first average.

[0053] In steps S1-S2 above, optionally, assuming that the target expert network group is a high-performance expert network group containing 4 expert networks: A, B, C, and D, first take network A as the "current expert network" and analyze its uniqueness and differences relative to other networks in the group. Then repeat the process, taking networks B, C, and D as the "current expert networks".

[0054] The network similarity mentioned above is used to measure the degree of similarity between two expert networks in terms of parameters, structure, or function. It is usually obtained by comparing core parameters or using specific similarity metrics. Taking network A as the current expert network, the system will calculate the similarity between A and other expert networks B, C, and D respectively. This can be achieved by extracting the core parameter matrix of each network and then using similarity metrics (such as cosine similarity, Euclidean distance, etc.) to quantify the differences between them. No specific restrictions are imposed here.

[0055] Assume the calculated similarity scores are 0.75 (networks A and B), 0.60 (networks A and C), and 0.85 (networks A and D). Further, the average similarity score between the current expert network and the reference expert network is calculated to assess the degree of differentiation of the current expert network, as shown above: First Average = (0.75 + 0.60 + 0.85) / 3 = 0.73. The Individual Difference Evaluation Coefficient = 1 - First Average. For network A, the Individual Difference Evaluation Coefficient is 1 - 0.73 = 0.27, meaning that if an expert network has a lower First Average, its Individual Difference Evaluation Coefficient will be higher, indicating that it has higher uniqueness within the group.

[0056] Through the above implementation methods, calculating similarity can intuitively reveal the differences between the current expert network and other expert networks. If an expert network is highly similar to most experts in the group, its characteristics may not be prominent enough, making it difficult to meet the processing needs of specific input data. Conversely, expert networks with lower similarity may have more unique processing capabilities and are more suitable for specific types of data. Establishing the relationship between individual difference evaluation coefficients and network similarity will favor expert networks with higher differences when selecting experts, avoiding blind selection of expert networks. The system can more accurately match expert networks and input data, reduce processing errors, and improve the accuracy of expert network selection.

[0057] In one optional implementation, determining at least one target expert network from multiple expert networks based on their respective individual difference evaluation coefficients in the target expert network group includes:

[0058] S1, based on the individual difference evaluation coefficients of each expert network in the target expert network group, the initial expert selection vectors that match them are modified to obtain multiple target expert selection vectors. The initial expert selection vectors are used to determine the degree of matching between the expert networks and the input data.

[0059] S2, based on the activation processing results of multiple target expert selection vectors, determine at least one target expert network from multiple expert networks.

[0060] In step S1 above, the initial expert selection vector represents the preliminary probability vector that the gating network directly assigns to each expert network based on the input data, that is, the probability of each expert being selected before considering the differences between expert networks.

[0061] Optionally, the initial expert selection vectors matched with multiple reference expert networks can be modified according to the individual difference evaluation coefficient to obtain multiple target expert selection vectors. For example, the target expert selection vector z' = z + weight × individual difference evaluation coefficient, where z is the Logits of the original topk expert selection (i.e., the initial expert selection vector). Then, the modified Logits z' can be substituted into the subsequent softmax calculation to perform topk expert selection.

[0062] In step S2 above, the top few expert networks with the highest probabilities are selected for parallel processing when processing the input. For example, a topk strategy can be used to select the top k expert networks with the highest probabilities.

[0063] As an optional implementation, assume that the target expert network group includes expert networks A, B, C, and D. The initial expert selection vector is ([0.2,0.3,0.1,0.4]), and the individual difference evaluation coefficient is ([0.6,0.8,0.5,0.7]).

[0064] Correcting the expert selection vector: First, multiply the initial expert selection vector by the individual difference evaluation coefficient to obtain the corrected vector ([0.12,0.24,0.05,0.28]). Then, normalize it using the softmax function to obtain the target expert selection vector [0.12,0.24,0.05,0.28], denoted as ([p_1,p_2,p_3,p_4]).

[0065] Determine the target expert network: Sampling is performed based on the target expert selection vector, or the top k experts are selected. Assuming a top-2 strategy is used, the two expert networks with the highest probabilities are selected as the "target expert networks". In the example, expert networks B and D become the targets.

[0066] Through the above implementation method, the original expert selection vector reflects the priority of the expert network after considering its uniqueness by integrating individual difference evaluation coefficients. The modified target expert selection vector combines the preliminary decision of the gating network and the differentiated evaluation of the expert network, providing more intelligent guidance for task allocation and improving the accuracy of selecting expert networks.

[0067] In one alternative implementation, after determining the individual difference evaluation coefficients that match the current expert network, the method further includes:

[0068] S1, determine the network similarity between any two expert networks in the target expert network group as the inter-group similarity set;

[0069] S2, determine the second average value based on the network similarity in the inter-group similarity set;

[0070] S3, determine the inter-group difference evaluation coefficient that matches the target expert network group based on the second average value.

[0071] In steps S1-S3 above, optionally, the above inter-group similarity set is the set of similarities between all possible expert network pairs in the target expert network group. Assuming that the target expert network group consists of four expert networks A, B, C, and D, it is necessary to calculate the similarity between six pairs of experts (AB), (AC), (AD), (BC), (BD), and (CD), and collect these similarity values ​​into a set, namely the inter-group similarity set.

[0072] After obtaining the set of inter-group similarities, the average of all similarity values ​​in the set is calculated as the "second average". The second average of the above inter-group similarity set {0.7,0.4,0.6,0.5,0.8,0.3} is (0.7+0.4+0.6+0.5+0.8+0.3) / 6=0.55.

[0073] The inter-group difference evaluation coefficient that matches the target expert network group is determined based on the second average value. For example, the inter-group difference evaluation coefficient = 1 - the second average value.

[0074] An ideal level of difference can be set, say 0.4 (meaning the average similarity between groups is expected to be below 0.4 to improve the difference), and then the difference evaluation coefficient between groups can be calculated based on the difference between the ideal level of difference and the second average. If the second average is higher than the ideal level of difference, it indicates insufficient difference between expert networks; if it is lower, it indicates good difference. For example, the difference evaluation coefficient between groups = 1 - (second average - ideal difference value), without specific restrictions.

[0075] By calculating the second average of the inter-group similarity sets, the network variability within the target expert network group can be assessed. Furthermore, the inter-group variability evaluation coefficient is used to quantify the overall variability level of the expert network group. In practical applications, the inter-group variability evaluation coefficient can be further integrated into the learning objective of the policy network to improve the overall performance of the expert network group, ensuring that tasks are assigned more efficiently and accurately to the most suitable expert network for processing.

[0076] In one alternative implementation, determining at least one target expert network from a plurality of expert networks further includes:

[0077] S1, the first decision factor is obtained by multiplying the individual difference evaluation coefficient matched with the expert network with the first weight value;

[0078] S2, the second decision factor is obtained by multiplying the inter-group difference evaluation coefficient and the second weight value;

[0079] S3. Based on the first decision factor and the second decision factor, the initial expert selection vector that matches the corresponding expert network is corrected to obtain multiple target expert selection vectors. The initial expert selection vector is used to determine the degree of matching between the expert network and the input data.

[0080] S4. Based on the activation processing results of the target expert selection vector, determine at least one target expert network.

[0081] In step S1 above, the first decision factor is obtained by multiplying the individual difference evaluation coefficient matched with the expert network with the corresponding first weight value. Optionally, the first decision factor = , This is the individual difference evaluation coefficient. This is the first weight value.

[0082] In step S2 above, the second decision factor is obtained by multiplying the inter-group difference evaluation coefficient and the corresponding second weight value. Optionally, the second decision factor = , This is the coefficient for evaluating differences between groups. This is the second weight value.

[0083] In steps S3-S4 above, the initial expert selection vector matching the corresponding expert network is corrected according to the first decision factor and the second decision factor to obtain multiple target expert selection vectors. The initial expert selection vector is used to determine the degree of matching of the expert network for processing the input data. At least one target expert network is determined according to the activation processing result of the target expert selection vector.

[0084] Optionally, The aforementioned combined reward terms are injected into the topk routing decision, such as: z' = z + αb, where z is the original Logits of the topk expert selection, and α is the weight parameter. The corrected Logits z' are then substituted into the subsequent softmax calculation for topk expert selection.

[0085] Based on the first decision factor (focusing on the differences among individual experts within a group) and the second decision factor (focusing on the differences between groups), the initial expert selection vector is modified to adjust the probability of each reference expert network being selected to process specific input data. The initial expert selection vector reflects the preliminary matching degree of each expert network based on the input data, while the modified target expert selection vector is a further optimization of these preliminary matching degrees to reflect a more comprehensive and accurate expert network selection tendency, thereby improving the specific matching degree between the expert network and the input data and avoiding the selection of expert networks that are too general and lack outstanding processing capabilities.

[0086] In one optional implementation, the target expert network group is determined from multiple expert network groups included in the initial expert model based on the estimated resource consumption of the input data, and the method further includes:

[0087] S1. Based on the complexity information of the input data, determine the estimated resource consumption of the input data, where the complexity information is used to indicate the difficulty of processing the input data;

[0088] S2, determine the selection probability of matching with multiple expert network groups based on the estimated resource consumption;

[0089] S3, the expert network group corresponding to the selection probability that satisfies the target probability condition is determined as the target expert network group.

[0090] In step S1 above, the estimated resource consumption of the input data is determined based on the complexity information of the input data, wherein the complexity information is used to indicate the difficulty of processing the input data.

[0091] Optionally, the aforementioned complexity information includes features of the input data, such as lexical richness, sentence length, structural complexity, image resolution, pixel density, color complexity, etc., to measure the difficulty of processing the data.

[0092] For example, for text data, the analysis determined the complexity information to be 150 words, and the word frequency information indicated a high proportion of specialized terms and complex grammatical structures. Based on the complexity information, the estimated resource consumption is moderate to high.

[0093] In step S2 above, the selection probability of matching with multiple expert network groups is determined based on the estimated resource consumption. In the MoE system, there are multiple expert network groups, such as a lightweight group (handling simple tasks), a balanced group (handling medium-complexity tasks), and a high-performance group (handling complex tasks). Based on the estimated resource consumption obtained in step S1, the probability of each expert network group being selected is determined. For example, for moderate to high resource consumption, the balanced and high-performance groups may have a higher selection probability, while the lightweight group has a lower selection probability.

[0094] As an optional implementation, the selection probabilities of three expert network groups (lightweight group, balanced group, and high-performance group) are set to (p_L), (p_M), and (p_H), respectively. The estimated resource consumption is medium to high, so (p_L=0.1), (p_M=0.4), and (p_H=0.5) are set. That is, the high-performance group is most likely to be selected to process the data, followed by the balanced group, and the lightweight group has the lowest selection probability.

[0095] In step S3 above, the expert network group corresponding to the selection probability that satisfies the target probability condition is determined as the target expert network group. Optionally, the target probability condition can be setting a probability threshold, or selecting the N groups with the highest probabilities. If the target probability condition is "selecting the group with the highest probability", the high-performance group ((p_H=0.5)) will be determined as the unique "target expert network group". If the target condition is "selecting the group with a probability greater than 0.3", then both the high-performance group and the balanced group will be confirmed as target expert network groups.

[0096] Through the above steps, the MoE system can intelligently determine the target expert network group based on the estimated resource consumption and complexity information of the input data, ensuring that the final selected expert network group not only meets the resource requirements, but also takes into account the current operating status of the system, avoiding overload or resource idleness.

[0097] In one optional implementation, the expert network group corresponding to the selection probability that satisfies the target probability condition is determined as the target expert network group, and further includes at least one of the following:

[0098] Method 1: Determine the maximum selection probability among the selection probabilities that match multiple expert network groups, and determine the expert network group corresponding to the maximum selection probability as the target expert network group;

[0099] As an optional implementation, after calculating the selection probability of each expert network group, the expert network group with the highest probability value is selected as the "target expert network group". For example, the calculated selection probabilities of the expert network groups are 0.2 for the lightweight group, 0.6 for the balanced group, and 0.3 for the high-performance group. The balanced group has the highest selection probability of 0.6 and becomes the target expert network group.

[0100] Method 2: Determine the queue length of the task waiting queue that matches multiple expert network groups respectively, and select the expert network group whose probability is greater than the first probability threshold and whose queue length is less than the second quantity threshold as the target expert network group;

[0101] As an optional implementation, the target expert network group is determined based on the selection probability of the expert network group combined with the current task waiting queue length of the expert network group. For example, the system sets a first probability threshold of 0.4 and a second quantity threshold of 100 requests. Assume the selection probabilities and queue lengths of the three expert network groups are as follows: Lightweight group: selection probability 0.3, queue length 150; Balanced group: selection probability 0.5, queue length 80; High-performance group: selection probability 0.6, queue length 120. According to the set thresholds, the selection probabilities of the Balanced group and the High-performance group are higher than 0.4 (the first probability threshold), and the queue length of the Balanced group (80) is lower than the second quantity threshold (100). Therefore, the Balanced group will be determined as the "target expert network group".

[0102] Method 3: Determine the average resource utilization rate that matches multiple expert network groups respectively, and determine the expert network group that has a selection probability greater than the second probability threshold and an average resource utilization rate greater than the third resource utilization threshold as the target expert network group.

[0103] As an optional implementation, both the selection probability and average resource utilization of the expert network groups are considered to avoid selecting overly busy network groups. For example, a second probability threshold of 0.4 and a third resource utilization threshold of 70% are set. The resource utilization and selection probabilities of the three expert network groups are as follows: Lightweight group: selection probability 0.2, average resource utilization 75%; Balanced group: selection probability 0.5, average resource utilization 65%; High-performance group: selection probability 0.6, average resource utilization 80%.

[0104] Only the high-performance group and the balanced group have a selection probability greater than 0.4. Considering the average resource utilization, the high-performance group has a higher resource utilization than the balanced group, but since the high-performance group's utilization is also greater than 70%, it may mean that it is approaching its processing limit. The balanced group not only meets the requirement in terms of selection probability, but also has a resource utilization below the threshold, indicating that it has the capacity to handle more tasks. Therefore, the balanced group becomes the target expert network group.

[0105] Method 4: Determine the historical processing data types that match multiple expert network groups respectively, and select the expert network group whose probability is greater than the third probability threshold and whose historical processing data type is the same as the input data type as the target expert network group.

[0106] As an optional implementation, assuming the third probability threshold is 0.4, and the current input data is of mathematical modeling type, the expert network groups whose historical processing data types match the input data type are: Lightweight group: selection probability 0.2, historical processing data type is text summarization; Balanced group: selection probability 0.5, historical processing data type is mathematical modeling; High-performance group: selection probability 0.6, historical processing data type is code generation. The selection probabilities of the Balanced group and the High-performance group are higher than 0.4, but only the Balanced group has a historical processing data type that matches the input data type (both are mathematical modeling), therefore the Balanced group is selected as the "target expert network group".

[0107] The implementation methods described in this application introduce multiple decision-making criteria (queue length, resource utilization, data type) to optimize the selection process. This ensures that the system not only selects based on the estimated data demand but also considers the real-time status and professional domain of the expert network group, achieving more efficient and accurate task allocation. By comprehensively considering selection probabilities and various real-time statuses or historical data types, the system can more intelligently determine the target expert network group, effectively avoiding bottlenecks caused by excessive resource concentration in a single group, while ensuring that data processing tasks are efficiently completed by the most suitable expert network group.

[0108] In one optional implementation, before determining the estimated resource consumption of the input data based on the complexity information of the input data, the following steps are included:

[0109] S1, vectorize the input data to obtain a vector sequence corresponding to the input data;

[0110] S2, determine the semantic density information of the input data based on the frequency of each word vector in the vector sequence;

[0111] S3, determine the complexity information of the input data based on semantic density information.

[0112] In step S1 above, the input data is vectorized to obtain a vector sequence corresponding to the input data. Optionally, a pre-trained word embedding model is used to convert each word or sentence fragment in the input data into a word vector. Assuming the input data is a piece of English text, word segmentation is first performed, breaking the text down into a series of words and punctuation marks. Then, a pre-trained embedding model is used to obtain the vector representation of each word. Finally, these word vectors are arranged into a vector sequence according to the order in the original text.

[0113] In step S2 above, the semantic density information of the input data is determined based on the frequency of each word vector appearing in the vector sequence. Optionally, the semantic density of the text can be analyzed by statistically analyzing the number of occurrences or frequencies of each word vector. Semantic density information reflects the density of information carried in the text and is usually measured by metrics such as entropy or TF-IDF (Term Frequency-Inverse Document Frequency). TF-IDF combines information from term frequency and inverse document frequency, giving higher weights to words that frequently appear in a particular document but rarely appear in the entire document set. Assuming that in the vector sequence formed in the aforementioned example, the entropy of all word vectors is calculated to obtain the semantic density information.

[0114] In step S3 above, the complexity information of the input data is determined based on the semantic density information. Optionally, if the information entropy is high, the input data is considered to have high complexity; conversely, if the information entropy is low, the input data is considered to have low complexity.

[0115] Through the above-described embodiments described in this application, the complexity information of the input data can be accurately determined step by step from the original text of the input data through word embedding, information entropy calculation and complexity level mapping. This ensures that data with higher complexity can be allocated to expert network groups that are more suitable for handling complex tasks, thereby improving the utilization efficiency of the entire system resources and the accuracy of task processing.

[0116] In one alternative implementation, determining the complexity information of the input data further includes:

[0117] S1. Determine the nesting depth information of the input data based on the number of nested statements in the input data, where the nested statements contain multiple levels of text structure;

[0118] S2, determine the syntax depth information of the input data based on the number of target statements in the input data, where the text length of the target statements is greater than the target threshold;

[0119] S3, determine the semantic coherence information of the input data based on the number of logical connectors in the input data;

[0120] S4. Complexity information is determined based on semantic density information, nesting depth information, syntactic depth information, and semantic coherence information.

[0121] In step S1 above, the nesting depth information of the input data is determined based on the number of nested statements in the input data, wherein the nested statements contain multiple layers of text structure.

[0122] Optionally, the input text data can be analyzed to count the number of nested statements. For example, if a sentence contains multiple nested parentheses or quotation marks, the maximum nesting level can be counted. If the input data is programming code, the nesting depth of code blocks, such as the number of nested functions, loops, or conditional statements, can be counted.

[0123] In step S2 above, the grammatical depth information of the input data is determined based on the number of target statements in the input data, wherein the text length of the target statements is greater than the target threshold.

[0124] Optionally, all target sentences that meet the criteria can be identified and counted, and then the grammatical depth can be measured based on the number of target sentences. For example, if the threshold is set to 20 words, the number of sentences in the text that exceed 20 words can be counted to determine the grammatical depth information.

[0125] In step S3 above, the semantic coherence information of the input data is determined based on the number of logical connectors in the input data.

[0126] Optionally, semantic coherence information is determined by analyzing the number and distribution of logical connectors (such as and, but, if, because, etc.) in the text. The more logical connectors there are, the stronger the logical connection of the text and the higher the semantic coherence.

[0127] In step S4 above, complexity information is determined based on semantic density information, nesting depth information, syntactic depth information, and semantic coherence information.

[0128] Optionally, a weight can be assigned to each indicator, and then the weighted values ​​can be aggregated to determine the complexity information. Assuming that the weight of semantic density information is 0.4, the weight of nesting depth information is 0.2, the weight of syntactic depth information is 0.2, and the weight of semantic coherence information is 0.2, the resulting values ​​are 0.7 (semantic density), 1 (nesting depth), 0.6 (syntactic depth), and 0.8 (semantic coherence), respectively.

[0129] The complexity information of the input data is calculated as follows: Complexity information = 0.4×0.7 + 0.2×1 + 0.2×0.6 + 0.2×0.8 = 0.28 + 0.2 + 0.12 + 0.16 = 0.76.

[0130] By comprehensively considering information such as semantic density, nesting depth, syntactic depth, and semantic coherence through the above-described embodiments in this application, the complexity of input data can be more comprehensively evaluated, text content can be better understood and processed, and more refined basis can be provided for subsequent resource allocation. This ensures that the system can reasonably allocate resources according to the complexity of the input data in order to achieve the best processing effect.

[0131] In one alternative implementation, before adjusting the model parameters in the initial expert model, the following steps are included:

[0132] S1, determine utilization information for at least one target expert network processing input data, wherein the utilization information is used to indicate the activity level of computing resources when the target expert network processes the input data;

[0133] S2, determine the task backlog of at least one target expert network;

[0134] S3, determine the task drop rate information that matches the target expert network group;

[0135] S4. Determine the load status description information based on the utilization information of at least one target expert network, the task backlog, and the task drop rate information matching the target expert network group.

[0136] In step S1 above, utilization information for at least one target expert network processing input data is determined, wherein the utilization information is used to indicate the activity level of computing resources when the target expert network processes the input data.

[0137] Optionally, the time utilization, computing unit utilization, or memory utilization of the target expert network when processing input data can be monitored to calculate its utilization information. For example, if an expert network uses an average of 70% of its computing resources to process data per unit time, its utilization information is 70%. Assuming that target expert networks A, B, and C are processing input data, monitoring reveals that network A's utilization information is 60%, network B's is 80%, and network C's is 40%.

[0138] In step S2 above, the task backlog of at least one target expert network is determined.

[0139] Optionally, the task backlog can be determined by calculating the ratio of the number of unprocessed tasks in the task queue of each target expert network to its processing capacity. For example, expert network A has 1200 tasks in its task queue and a processing capacity of 200 tasks per minute, resulting in a backlog of 6; expert network B has 800 tasks in its task queue and a processing capacity of 150 tasks per minute, resulting in a backlog of 5.33; and expert network C has 300 tasks in its task queue and a processing capacity of 50 tasks per minute, resulting in a backlog of 6.

[0140] In step S3 above, the task drop rate information matching the target expert network group is determined. This task drop rate information can be the proportion of dropped tasks that each expert network failed to successfully process over a past period.

[0141] In step S4 above, load status description information is determined based on the utilization information of at least one target expert network, the task backlog, and the task drop rate information matching the target expert network group. For example, the load status description information can be determined by concatenating the above information or by weighted summation of the above information to quantify a comprehensive index value; no specific limitation is made here.

[0142] As an optional implementation, the utilization information in step S1 can be, for example, expert utilization rate: quantifying the busyness of expert computing resources and reflecting the bottleneck of expert processing capacity, calculated using the following formula:

[0143] ,in The sampling period (obtained through a fixed window). This represents the level of activity of experts during the sampling period.

[0144] In step S2, the task backlog can be determined based on the queue depth, which reflects the backlog of tasks awaiting processing by experts and predicts potential delays. The calculation formula is as follows:

[0145] ,in This represents the number of requests that arrived within the sampling period. This indicates the number of requests processed within the sampling period.

[0146] In step S3 above, the task drop rate information, or drop rate, measures the severity of the expert group's insufficient service capacity. A higher value indicates that the current expert group is more likely to be overloaded, and the proportion of tasks assigned to it should be reduced. The calculation formula is as follows:

[0147] ,in 'g' represents the target expert group (which can be a lightweight group, a balanced group, or a high-performance group), and 't' is the number of expert groups. The load description information is obtained by combining the above information.

[0148] By analyzing the utilization information, task backlog, and task drop rate of the target expert network, detailed load status information can be obtained, providing intuitive data support for subsequent model parameter adjustments. This enables the system to respond more intelligently to resource allocation and performance optimization, ensuring that the expert network group under the MoE architecture can efficiently and stably process various types of input data.

[0149] In one optional implementation, the model parameters in the initial expert model are adjusted based on the load state description information corresponding to at least one target expert network, including:

[0150] S1, the complexity information, utilization information, task backlog, task drop rate information and inter-group difference evaluation coefficient of the input data are vectorized and concatenated to obtain the state vector. The inter-group difference evaluation coefficient is determined based on the network similarity between any two expert networks in the target expert network group.

[0151] S2 adjusts the model parameters based on the state vector of the decision network in the initial expert model.

[0152] In steps S1-S2 above, assume the complexity of the target expert network A is 0.76, the utilization rate is 0.6, the backlog is 6.0, the dropout rate is 0.05, and the inter-group difference evaluation coefficient is 0.85. Then the state vector (S_A) is (0.76, 0.6, 6.0, 0.05, 0.85).

[0153] Using the state vector (S) as input, the model parameters of the decision network are adjusted by a policy network in reinforcement learning (such as the Actor-Critic architecture). The policy network calculates the optimal decision or parameter adjustment scheme based on the given state vector (S) to optimize the resource allocation and processing efficiency of the expert network.

[0154] Specifically, the state vector generated in step S1 is used as input to the Actor network in the decision network. The Actor network calculates a set of action or parameter adjustment probability distributions based on the state vector and selects the most suitable parameter adjustment strategy. The Critic network estimates the expected value or reward after taking the action based on the state vector (S), which is used to evaluate the rationality of the parameter adjustment scheme. Based on the actions selected by the Actor network and the value estimates provided by the Critic network, the parameters of the decision network are adjusted, optimizing the network's resource allocation strategy.

[0155] The above process is illustrated in one optional implementation: the state vectors of the target expert networks A, B, and C are (0.76, 0.6, 6.0, 0.05, 0.85), (0.68, 0.8, 4.5, 0.03, 0.82), and (0.54, 0.4, 7.2, 0.10, 0.75), respectively. The state vector of each expert network is input into the Actor network, and simultaneously provided to the Critic network as the basis for value estimation.

[0156] Action selection and value estimation: The Actor network calculates the parameters and adjusts the probability distribution based on the state vector. For example, if the processing power of expert network A is increased by 20%, the Critic network estimates the expected value after taking this action as -0.02 (a small negative value indicates a decrease in expected reward); for expert network B, the Actor network suggests that it reduce its processing power by 10%, and the Critic network estimates the expected value as 0.04 (a positive value indicates an increase in expected reward).

[0157] Parameter tuning: Based on value estimation and action selection strategies, adjust the parameters of the decision network. For example, increase the model complexity of expert network A or adjust its network architecture to improve its processing capacity; simultaneously, reduce the model complexity of expert network B to alleviate its overload. Specifically, optimizers (such as Adam) can be used to tune network parameters, making predictions of future actions more accurate, while optimizing the load distribution and resource utilization of the expert network.

[0158] By vectorizing and concatenating the complexity of input data, utilization of expert networks, task backlog, task dropout rate, and inter-group difference evaluation coefficients into a state vector, a comprehensive and intuitive description of the environment state is provided for the decision network. Based on the state vector, the parameters of the expert model can be intelligently adjusted, network resource allocation can be optimized, and the accuracy of expert network selection can be improved by utilizing the policy network and value network of reinforcement learning.

[0159] In one optional implementation, after adjusting the model parameters in the initial expert model based on the load state description information corresponding to at least one target expert network, the process includes:

[0160] S1, Obtain actual evaluation information of at least one expert network processing the input data at the current time step, wherein the evaluation information is used to indicate satisfaction with the performance behavior;

[0161] S2, Based on the reward prediction function, determine the prediction and evaluation information for processing the input data at the next time step;

[0162] S3, calculate the loss value based on the actual evaluation information and the predicted evaluation information;

[0163] S4. Repeat the above loss value calculation process until the minimum loss condition is met.

[0164] In step S1 above, the actual evaluation information can be the actual value in TD-error, reflecting the real reward or satisfaction evaluation of the expert network after processing the input data at the current time step.

[0165] Optionally, at the end of the current time step, an actual reward value is calculated based on the quality, speed, and resource consumption of the expert network in processing the input data. For example, if expert network A processes the data with an accuracy of 95%, a processing time of 2 seconds, and moderate resource consumption, then the actual evaluation information (reward) may be +10.

[0166] In step S2 above, the predicted evaluation information can be the predicted value in TD-error, which is the reward that the agent may obtain in future time steps based on the current state predicted by the Critic network. Optionally, the Critic network predicts the reward that the agent (expert network) may obtain when processing the same type of input data in the next time step based on the current state (including load state description information). For example, the Critic network predicts that the predicted evaluation information (reward) of expert network A when processing similar data in the next time step is +9 based on the current state.

[0167] In step S3 above, the TD-error (Time Difference Error) between the actual evaluation information and the predicted evaluation information can be calculated to measure the accuracy of the prediction. This error is used as the loss value to drive the update of the network parameters. For example, if the actual evaluation information of expert network A is +10 and the predicted evaluation information is +9, then the TD-error is (10-9=+1). The loss value can be calculated as the square of the TD-error to avoid positive and negative errors canceling each other out.

[0168] In step S4 above, the above steps are executed iteratively to update the parameters of the Actor and Critic networks until the TD-error (loss value) reaches the minimum loss condition, indicating that the predicted evaluation information is very close to the actual evaluation information. For example, when the TD-error (loss value) stabilizes at around 0.1 after several consecutive iterations, that is, the minimum loss condition is met, it can be considered that the parameters of the expert model have been adjusted to a relatively ideal configuration.

[0169] By quantifying the deviation between the actual execution results and the predicted results, and using this to drive the parameter tuning process of the expert network, after each time step, by calculating the TD-error between the actual evaluation information and the predicted evaluation information, not only can the accuracy of the prediction be evaluated, but the learning of parameters can also be guided. This enables the expert network to gradually optimize its ability to process specific types of input data until the minimum loss condition is reached, thus achieving efficient, fast and resource-efficient input data processing.

[0170] The following complete embodiment describes this application, such as Figure 3 As shown:

[0171] S302, Input data, including preprocessing and feature extraction; specifically, the user-input raw data (text / code / multimodal) is vectorized by a tokenizer. The semantic density and structural complexity of the vectorized feature vectors are then calculated.

[0172] Semantic density is calculated using information entropy, where information entropy H(x) is used to quantify the information density of a random variable X. ,in for The probability of the input sequence appearing, where n is the size of the vocabulary in the input.

[0173] Structural complexity can be estimated in a lightweight manner based on factors such as the density of logical connectors, the depth of punctuation nesting, and the penalty for long and difficult sentences in the context of the actual scenario. For multimodal image data, occlusion between objects and dependency hierarchy can also be added to represent structural complexity.

[0174] S304, Dynamic Complexity Estimator. Reinforcement learning-driven dynamic layering, specifically including policy network parameter initialization: employing the Proximal Policy Optimization (PPO) algorithm to construct an Actor-Critic dual network structure.

[0175] Actor Network: Input features, output hierarchical classifications of those features. The network structure can be a 3-layer LSTM (Long Short-Term Memory), a special type of recurrent neural network (RNN). The number of units in each LSTM layer can be customized based on the actual task complexity and data size. Hidden layers use Tanh (hyperbolic tangent) as the activation function. The output is the probability distribution of the input corresponding to each expert group. The network structure can be configured according to actual needs. Here, expert groups are divided into S304-1 (lightweight expert group for simple tasks), S304-2 (balanced expert group for medium tasks), and S304-3 (high-performance expert group for complex tasks), each handling tasks of different difficulty.

[0176] Critic Network: Input state vector, output state value estimate. The network structure can be a 3-layer fully connected network. The hidden layer dimension can be customized to adapt to the data scale and task complexity. The hidden layer uses "Rectified Linear Unit (ReLU)" as the activation function.

[0177] Initialize the experience replay buffer and adopt the Prioritized Experience Replay mechanism to adjust the sampling weights based on TD-error (Time Difference Error).

[0178] TD-error (Temporal Difference Error) is a metric used in reinforcement learning to measure the difference between predicted and actual values, and is used to update estimates of the advantage function or action value function. In reinforcement learning, an agent learns how to choose actions to maximize cumulative reward by interacting with the environment. The advantage function estimates the expected cumulative reward the agent can obtain in a given state, and TD-error measures the accuracy of this estimate and updates it accordingly. The interaction results at each time step are recorded and stored as an experience tuple, typically represented as (s, a, R, s'), where the current state (s), the action taken (a), the reward obtained (R), and the next state (s') represent the state of the environment after performing the current action a.

[0179] For example, the advantage function corresponding to (s,a,R,s') above is: A=R+γV(s')-V(s), where V(s) is the initial state value, γV(s') is the final state value evaluated, and R is the cumulative reward observed during the intermediate actions. It can be seen that it only measures the evaluator's error in a given state s. If a higher cumulative reward is obtained (a larger R), the advantage estimate will be positive, making it more likely to take action in that state. Conversely, if a higher reward is expected but the reward obtained is small, the advantage estimate will be negative, reducing the likelihood of taking action in that step.

[0180] After feature extraction from the input data, the features are concatenated with semantic density, structural complexity, and expert load feedback to form a state vector, which is then input into a reinforcement learning policy network (such as an Actor network driven by the PPO algorithm). The output is a probability distribution for token classification. Based on the action probability distribution, the input data is assigned to the target expert group cluster. Changes in expert load are recorded, and immediate rewards are calculated.

[0181] An offline experience replay mechanism is adopted to store state-action-reward samples; the policy network parameters are periodically updated to minimize the prediction error of the value function (Critic network) while maximizing the cumulative reward.

[0182] For example, the Actor loss is first calculated by sampling batch data from the experience replay buffer, using the following formula:

[0183] ,in, , It is the probability distribution of a under state S, used to measure the probability of the updated policy executing the operation relative to the previous policy, that is, the difference in the probability of the policy executing the same action before and after. This represents the probability of the strategy being executed under historical conditions. A is the advantage function, and ε is the clipping range, which can be defined as 0.2. To avoid excessive fluctuations in the strategy, E above represents the mean squared error loss function.

[0184] The formula for calculating Critic loss is as follows: The above E represents the mean squared error loss function.

[0185] The joint optimization of Actor and Critic losses is calculated using the following formula:

[0186] Here, c1 and c2 are hyperparameters that measure the importance of policy evaluation and exploration accuracy, respectively, while H is an entropy function that encourages exploration of the entropy of the policy output distribution. Standard entropy. Finally, the Adam optimizer is used to update the network parameters.

[0187] S308 represents the top K routes within a group. This differentiated routing within an expert group addresses the homogeneity problem in traditional MoE routing by dynamically penalizing the similarity of experts within the group, forcing them to learn differentiated features. Essentially, it injects differentiated reward signals into expert decision-making within the group, putting similar experts at a disadvantage in competition. The input consists of m experts in expert group G and the input features assigned to this group; the output is the selection probability of the top k experts and the similarity matrix of experts within the current group. Different heterogeneous experts can be designed to be used in different groups, including lightweight groups for simple tasks: using lightweight small networks (miniature CNNs or Transformers, etc.), depthwise separable convolutions, etc., to reduce computational costs and achieve high inference speed; balanced expert groups for medium tasks: setting up general networks (CNN+Transformer hybrid structures, such as BERT, etc.), combined with residual junctions, etc., to balance efficiency and expressive power; and high-performance expert groups for complex tasks: using deeper model structures (enhanced Transformers, etc., such as GPT-like models), supporting long contexts, and ensuring inference accuracy.

[0188] The specific steps are as follows: First, calculate the original expert Logits: z, and then construct an m×m expert similarity matrix, as follows:

[0189] ,in, For each expert The core parameter matrix is ​​extracted and concatenated, where i and j represent different expert networks, and the operator in the denominator represents the L2 norm of the vector. The differential reward factor is calculated using this matrix.

[0190] Expert-level reward (penalty for highly similar individuals): .

[0191] Group-level rewards (enhancing overall diversity): .

[0192] Combination Rewards: Where β is the balancing weight parameter, and m is the number of experts in the group. This is an expert similarity metric, and it will be subsequently fed back into the training of reinforcement learning-driven dynamic hierarchical guidance reinforcement learning.

[0193] The above combined reward terms are injected into the topk routing decision: z' = z + αb. Here, z is the original Logits of the topk expert selection, and α is the weight parameter. The corrected Logits z' are then substituted into the subsequent softmax calculation for topk expert selection.

[0194] S310, Expert Computation and Gradient Update. Within the selected expert group, the standard expert computation process for the MOE model is performed: routing decisions are made based on the input data to select the top k experts; the selected top k experts perform parallel computation; the weighted fusion of experts is used to calculate the loss and perform corresponding backward gradient updates.

[0195] The S312 is a load balancer monitor used for load monitoring and dynamic feedback. It senses the load status of the expert group in real time and dynamically adjusts routing strategies to prevent overload crashes.

[0196] Expert-level metrics primarily focus on individual expert utilization and queue depth; group-level metrics primarily focus on expert group dropout rate, etc. This includes calculating the following indicators: Expert Utilization: This quantifies the workload of expert computing resources, reflecting bottlenecks in expert processing capacity. The calculation formula is as follows:

[0197] ,in The sampling period (obtained through a fixed window). This represents the level of activity of experts during the sampling period.

[0198] Queue depth: Reflects the backlog of tasks awaiting processing by experts and predicts potential latency. The calculation formula is as follows:

[0199] ,in This represents the number of requests that arrived within the sampling period. This indicates the number of requests processed within the sampling period.

[0200] Dropout rate: Measures the severity of the expert group's insufficient service capacity. A higher value indicates that the current expert group is more likely to be overloaded, and the proportion of tasks assigned to it should be reduced. The calculation formula is as follows:

[0201] ,in , g represents the target expert group (lightweight group, balanced group, high-performance group), and t is the number of expert groups.

[0202] This allows for the feedback of various indicators of expert workload and the similarity index among experts. After normalization, the data is concatenated to obtain the historical load feedback vector, which is then passed to a reinforcement learning-driven complexity estimator, such as the input features obtained by calculating semantic density and structural complexity. , along with expert load feedback and expert similarity metrics Concatenate them into a state vector to form the final state vector, such as: .

[0203] The above state vector is input into the Actor network, which outputs the action probability distribution π(a|s). After softmax sampling, the final action index or weight distribution is obtained.

[0204] The dynamic complexity estimator distributes the task input data to different levels based on action 'a', and then to different expert groups for processing. If the target expert queue is full, the task is downgraded to the next best expert group. After the task is processed, the execution result is recorded, which is the historical load feedback obtained above. Other metrics, such as accuracy, can also be added as reward and penalty conditions to the reward function to adjust agent behavior.

[0205] S314, reinforcement learning policy update, can use the proximal policy optimization (PPO) algorithm. Further optimization returns to S304, the dynamic complex evaluator.

[0206] Specifically, the Proximal Policy Optimization (PPO) algorithm is a gradient-based algorithm for policy optimization. It consists of an actor and a critic. The actor outputs the probability distribution of the next action at time t, while the critic estimates the expected cumulative reward for that state. The algorithm aims to learn a policy that maximizes the cumulative reward gained based on experience during training.

[0207] It should be noted that during the interaction between the reinforcement learning agent and the environment, the interaction result at each time step is recorded and stored as an experience tuple, usually represented as (s,a,R,s'), which represents the current state (s), the action taken (a), the reward obtained (R), and the next state (s'), that is, the state of the environment after the current action a is performed.

[0208] The reward function calculation is mainly divided into historical load reward items and intra-group difference reward items:

[0209] Historical load items are: .

[0210] The inter-group difference reward items are: .

[0211] We could also consider adding a model performance reward item: , ( These are weighting coefficients used to adjust the proportion of each indicator. (Model accuracy obtained from the cross-entropy loss function in the expert calculation and gradient update steps).

[0212] The final reward function is .

[0213] Dynamic deployment involves deploying the above algorithm to real-world scenarios and monitoring expert load fluctuations for real-time fine-tuning. Fine-tuning strategies can be configured: monitor expert load fluctuations, and trigger online fine-tuning if the variance of queue length within a group exceeds a threshold or if a single expert experiences ≥3 consecutive overloads. This involves freezing the Critic network, updating only the Actor network parameters (reducing the learning rate), and continuously adjusting the reinforcement learning module's hierarchical boundaries for the input data for a given number of training steps.

[0214] In summary, this application proposes a dynamic complexity clustering mechanism for input data: based on reinforcement learning, the input tasks are dynamically hierarchically divided into three levels—simple, medium, and complex—by combining feature indicators and load conditions, and different levels of computing resources are allocated downstream.

[0215] Load-triggered elastic degradation system: Construct a load awareness mechanism to quantify the computational pressure, resource consumption, and task processing efficiency of expert nodes; when the load of the target expert group exceeds the threshold, some tasks will be dynamically degraded to lower-level expert groups.

[0216] Heterogeneous expert group setup: Set up heterogeneous expert groups with different structures to handle tasks of varying difficulty, thereby reducing training costs.

[0217] Multi-objective optimization design: Taking the differentiation of experts within the group as the objective, a combined reward term for the differences of experts within the group is added to the expert group calculation to guide the selection of the top k experts; and combining the input task complexity features and the load of the expert group network (expert utilization, queue depth, dropout rate, etc.) to design a multi-objective reward function for reinforcement learning.

[0218] According to another aspect of the embodiments of this application, a training apparatus for an expert model for implementing the above-described expert model training method is also provided. For example... Figure 4 As shown, the device includes:

[0219] The first determining unit 402 determines the target expert network group from the multiple expert network groups included in the initial expert model based on the estimated resource consumption of the input data. The expert network group includes multiple expert networks for processing the input data, and the estimated resource consumption is used to indicate the computing resources required by the expert network when processing the input data.

[0220] The second determining unit 404 sequentially uses multiple expert networks in the target expert network group as the current expert network and determines the individual difference evaluation coefficient that matches the current expert network. The individual difference evaluation coefficient is determined based on the network similarity between the current expert network and multiple reference expert networks in the target expert network group. The reference expert networks are the expert networks in the target expert network group other than the current expert network.

[0221] The third determining unit 406 determines at least one target expert network from multiple expert networks based on the individual difference evaluation coefficients of each expert network in the target expert network group.

[0222] The training unit 408 is adjusted to process the input data according to at least one target expert network, and the model parameters in the initial expert model are adjusted according to the load state description information corresponding to each of the at least one target expert network.

[0223] Optionally, the second determining unit 404 includes: a similarity calculation module, used to calculate the network similarity between the current expert network and multiple reference expert networks respectively; and to determine the individual difference evaluation coefficient matching the current expert network based on the first average value of the multiple network similarities, wherein the individual difference evaluation coefficient and the first average value have a negative correlation.

[0224] Optionally, the third determining unit 406 is further configured to modify the initial expert selection vector that matches the individual difference evaluation coefficient of each of the multiple expert networks in the target expert network group, thereby obtaining multiple target expert selection vectors, wherein the initial expert selection vector is used to determine the degree of matching of the expert network for processing the input data; and to determine at least one target expert network from the multiple expert networks based on the activation processing results of the multiple target expert selection vectors.

[0225] Optionally, the similarity calculation module described above is further used to determine the network similarity between any two expert networks in the target expert network group as an inter-group similarity set; determine a second average value based on the network similarity in the inter-group similarity set; and determine an inter-group difference evaluation coefficient that matches the target expert network group based on the second average value.

[0226] Optionally, the third determining unit 406 is further configured to obtain a first decision factor by multiplying the individual difference evaluation coefficient matched with the expert network by a first weight value; obtain a second decision factor by multiplying the inter-group difference evaluation coefficient by a second weight value; modify the initial expert selection vector matched with the corresponding expert network according to the first decision factor and the second decision factor to obtain multiple target expert selection vectors, wherein the initial expert selection vector is used to determine the degree of matching of the expert network for processing the input data; and determine at least one target expert network according to the activation processing result of the target expert selection vector.

[0227] Optionally, the third determining unit 406 mentioned above includes a fourth determining module, used to determine the estimated resource consumption of the input data based on the complexity information of the input data, wherein the complexity information is used to indicate the difficulty of processing the input data; determine the selection probability that matches multiple expert network groups respectively based on the estimated resource consumption; and determine the expert network group corresponding to the selection probability that meets the target probability condition as the target expert network group.

[0228] Optionally, the fourth determining module is further configured to: determine the maximum selection probability among the selection probabilities matched with multiple expert network groups, and determine the expert network group corresponding to the maximum selection probability as the target expert network group; determine the queue length of the task waiting queue matched with multiple expert network groups, and determine the expert network group whose selection probability is greater than a first probability threshold and whose queue length is less than a second quantity threshold as the target expert network group; determine the average resource utilization rate matched with multiple expert network groups, and determine the expert network group whose selection probability is greater than a second probability threshold and whose average resource utilization rate is greater than a third resource utilization rate threshold as the target expert network group; and determine the historical processing data type matched with multiple expert network groups, and determine the expert network group whose selection probability is greater than a third probability threshold and whose historical processing data type is the same as the input data type as the target expert network group.

[0229] Optionally, the fourth determining module mentioned above includes a fifth determining module, which is used to perform vectorization processing on the input data to obtain a vector sequence corresponding to the input data; determine the semantic density information of the input data based on the frequency of each word vector appearing in the vector sequence; and determine the complexity information of the input data based on the semantic density information.

[0230] Optionally, the fifth determining module is further configured to determine the nesting depth information of the input data based on the number of nested statements in the input data, wherein the nested statements contain multiple layers of text structure; determine the syntactic depth information of the input data based on the number of target statements in the input data, wherein the text length of the target statements is greater than a target threshold; determine the semantic coherence information of the input data based on the number of logical connectors in the input data; and determine the complexity information based on the semantic density information, nesting depth information, syntactic depth information, and semantic coherence information.

[0231] Optionally, the aforementioned adjustment training unit includes a sixth determining module, used to determine the utilization information of at least one target expert network processing input data, wherein the utilization information is used to indicate the activity level of computing resources when the target expert network processes input data; determine the task backlog of at least one target expert network; determine the task drop rate information matching the target expert network group; and determine load state description information based on the utilization information of at least one target expert network, the task backlog, and the task drop rate information matching the target expert network group.

[0232] Optionally, the sixth determining module is also used to perform vectorized concatenation processing on the complexity information, utilization information, task backlog, task drop rate information and inter-group difference evaluation coefficient of the input data to obtain a state vector, wherein the inter-group difference evaluation coefficient is determined based on the network similarity between any two expert networks in the target expert network group; and to adjust the model parameters based on the state vector of the decision network in the input initial expert model.

[0233] Optionally, the aforementioned training unit 408 is further configured to acquire actual evaluation information of at least one expert network processing the input data at the current time step, wherein the evaluation information is used to indicate satisfaction with the performance behavior; determine the predicted evaluation information of processing the input data at the next time step based on the reward prediction function; calculate the loss value based on the actual evaluation information and the predicted evaluation information; and repeat the above loss value calculation process until the minimum loss condition is met.

[0234] For a description of the features in the embodiment corresponding to the training device of the expert model, please refer to the relevant description of the embodiment corresponding to the training method of the expert model, which will not be repeated here.

[0235] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above-described expert model training method embodiments.

[0236] The electronic device used in this application can be a terminal device or a server. This embodiment uses a mobile phone or computer as an example for illustration. Figure 5 As shown, the electronic device includes a memory 502 and a processor 504. The memory 502 stores a computer program, and the processor 504 is configured to execute the steps of any of the above method embodiments through the computer program.

[0237] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0238] Alternatively, as those skilled in the art will understand, Figure 5 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones (such as Android phones, iOS phones, etc.), tablets, PDAs, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 4 This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 5 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 5The different configurations shown.

[0239] The memory 502 can be used to store software programs and modules, such as the program instructions / modules corresponding to the expert model training method and apparatus in this embodiment. The processor 504 executes various functional applications by running the software programs and modules stored in the memory 502, thereby implementing the aforementioned expert model training method. The memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 502 may further include memory remotely located relative to the processor 504, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 502 may be used, but is not limited to, for storing information such as signals and data. As an example, such as... Figure 5 As shown, the memory 502 may include, but is not limited to, the first determining unit 402, the second determining unit 404, the third determining unit 406, and the training adjustment unit 408 in the expert model training device. Furthermore, it may include, but is not limited to, other module units in the expert model training device, which will not be elaborated upon in this example.

[0240] Optionally, the transmission device 506 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 506 includes a Network Interface Controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 506 is a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0241] In addition, the above-mentioned electronic device also includes: a display 508; and a connection bus 510 for connecting the various module components in the above-mentioned electronic device.

[0242] In other embodiments, the aforementioned terminal device or server can be a node in a distributed system, wherein the distributed system can be a blockchain system, which is a distributed system formed by connecting multiple nodes through network communication. The nodes can form a point-to-point network, and any form of computing device, such as a server, terminal, or other electronic device, can become a node in the blockchain system by joining this point-to-point network.

[0243] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described expert model training method embodiments at runtime.

[0244] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0245] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described expert model training method embodiments.

[0246] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described expert model training method embodiments.

[0247] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0248] The foregoing has provided a detailed description of the expert model training method, apparatus, storage medium, and electronic device provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only intended to aid in understanding the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for training an expert model, characterized in that, include: Based on the estimated resource consumption of the input data, a target expert network group is determined from multiple expert network groups included in the initial expert model. The expert network group includes multiple expert networks for processing the input data, and the estimated resource consumption is used to indicate the computing resources required by the expert network when processing the input data. The multiple expert networks in the target expert network group are sequentially used as the current expert network, and an individual difference evaluation coefficient matching the current expert network is determined. The individual difference evaluation coefficient is determined based on the network similarity between the current expert network and the multiple reference expert networks in the target expert network group. The reference expert networks are the expert networks in the target expert network group other than the current expert network. Based on the individual difference evaluation coefficients of each of the multiple expert networks in the target expert network group, the initial expert selection vectors that match them are modified to obtain multiple target expert selection vectors, wherein the initial expert selection vectors are used to determine the degree of matching of the expert networks in processing the input data; Based on the activation processing results of the plurality of target expert selection vectors, at least one target expert network is determined from the plurality of expert networks; The input data is processed by at least one of the target expert networks, and the model parameters in the initial expert model are adjusted according to the load state description information corresponding to each of the at least one target expert network.

2. The method according to claim 1, characterized in that, The multiple expert networks in the target expert network group are sequentially selected as the current expert network, and the individual difference evaluation coefficients matching the current expert network are determined, including: Calculate the network similarity between the current expert network and each of the multiple reference expert networks; The individual difference evaluation coefficient matching the current expert network is determined based on a first average of multiple network similarities, wherein the individual difference evaluation coefficient has a negative correlation with the first average.

3. The method according to claim 2, characterized in that, After determining the individual difference evaluation coefficients that match the current expert network, the method further includes: The network similarity between any two expert networks in the target expert network group is determined as the inter-group similarity set; A second average value is determined based on the network similarity in the inter-group similarity set; The inter-group difference evaluation coefficient matching the target expert network group is determined based on the second average value.

4. The method according to claim 3, characterized in that, Determining at least one target expert network from multiple expert networks also includes: The first decision factor is obtained by multiplying the individual difference evaluation coefficient matched with the expert network by the first weight value. The second decision factor is obtained by multiplying the inter-group difference evaluation coefficient with the second weight value; Based on the first decision factor and the second decision factor, the initial expert selection vector matching the corresponding expert network is modified to obtain multiple target expert selection vectors, wherein the initial expert selection vector is used to determine the degree of matching of the expert network for processing the input data; Based on the activation processing result of the target expert selection vector, at least one target expert network is determined.

5. The method according to claim 1, characterized in that, Based on the estimated resource consumption of the input data, the target expert network group is determined from the multiple expert network groups included in the initial expert model, which also includes: Based on the complexity information of the input data, the estimated resource consumption of the input data is determined, wherein the complexity information is used to indicate the difficulty of processing the input data; Based on the estimated resource usage, the selection probability for matching with each of the multiple expert network groups is determined. The expert network group corresponding to the selection probability that satisfies the target probability condition is determined as the target expert network group.

6. The method according to claim 5, characterized in that, Determining the expert network group corresponding to the selection probability that satisfies the target probability condition as the target expert network group further includes at least one of the following: Determine the maximum selection probability among the selection probabilities that match the multiple expert network groups respectively, and determine the expert network group corresponding to the maximum selection probability as the target expert network group; Determine the queue length of the task waiting queue that matches the multiple expert network groups respectively, and determine the expert network group whose selection probability is greater than a first probability threshold and whose queue length is less than a second quantity threshold as the target expert network group; Determine the average resource utilization rate that matches each of the multiple expert network groups, and identify the expert network group whose selection probability is greater than a second probability threshold and whose average resource utilization rate is greater than a third resource utilization threshold as the target expert network group. Determine the historical processing data types that match the multiple expert network groups respectively, and identify the expert network groups whose selection probability is greater than a third probability threshold and whose historical processing data type is the same as the data type of the input data as the target expert network group.

7. The method according to claim 5, characterized in that, Before determining the estimated resource consumption of the input data based on the complexity information of the input data, the method further includes: The input data is vectorized to obtain a vector sequence corresponding to the input data; The semantic density information of the input data is determined based on the frequency of each word vector appearing in the vector sequence; The complexity information of the input data is determined based on the semantic density information.

8. The method according to claim 7, characterized in that, Determining the complexity information of the input data further includes: The nesting depth information of the input data is determined based on the number of nested statements in the input data, wherein the nested statements contain multiple layers of text structure; Based on the number of target statements in the input data, the grammatical depth information of the input data is determined, wherein the text length of the target statements is greater than a target threshold; The semantic coherence information of the input data is determined based on the number of logical connectors in the input data; The complexity information is determined based on the semantic density information, the nesting depth information, the syntactic depth information, and the semantic coherence information.

9. The method according to claim 7, characterized in that, Before adjusting the model parameters in the initial expert model, the following steps are included: Determine utilization information for at least one of the target expert networks processing the input data, wherein the utilization information is used to indicate the activity level of the computing resources when the target expert network processes the input data; Determine the task backlog of at least one of the target expert networks; Determine the task drop rate information that matches the target expert network group; The load status description information is determined based on the utilization information of at least one of the target expert networks, the task backlog, and the task drop rate information matching the target expert network group.

10. The method according to claim 9, characterized in that, Adjusting the model parameters in the initial expert model based on the load state description information includes: The complexity information, utilization information, task backlog, task dropout rate information, and inter-group difference evaluation coefficient of the input data are vectorized and concatenated to obtain a state vector. The inter-group difference evaluation coefficient is determined based on the network similarity between any two expert networks in the target expert network group. The model parameters are adjusted based on the state vector of the decision network in the initial expert model.

11. The method according to claim 1, characterized in that, After adjusting the model parameters in the initial expert model based on the load state description information, the following steps are included: Obtain actual evaluation information of at least one of the expert networks processing the input data at the current time step, wherein the evaluation information is used to indicate satisfaction with the performance behavior; The reward prediction function determines the predicted evaluation information for processing the input data at the next time step; The loss value is calculated based on the actual evaluation information and the predicted evaluation information; Repeat the above loss value calculation process until the minimum loss condition is met.

12. A training device for an expert model, characterized in that, include: The first determining unit determines a target expert network group from multiple expert network groups included in the initial expert model based on the estimated resource consumption of the input data. The expert network group includes multiple expert networks for processing the input data, and the estimated resource consumption is used to indicate the computing resources required by the expert network when processing the input data. The second determining unit sequentially uses multiple expert networks in the target expert network group as the current expert network and determines an individual difference evaluation coefficient that matches the current expert network. The individual difference evaluation coefficient is determined based on the network similarity between the current expert network and multiple reference expert networks in the target expert network group. The reference expert networks are the expert networks in the target expert network group other than the current expert network. The third determining unit determines at least one target expert network from the multiple expert networks in the target expert network group based on the individual difference evaluation coefficients of each of the multiple expert networks. The training unit is adjusted to process the input data according to at least one of the target expert networks, and the model parameters in the initial expert model are adjusted according to the load state description information corresponding to each of the at least one target expert network.

13. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the training method for the expert model as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the training method for the expert model as described in any one of claims 1 to 11.

Images