A training method, inference method, apparatus, and equipment for a hybrid expert model.

Abstract

This invention discloses a training method, inference method, apparatus, and device for a hybrid expert model. The method includes: determining the number of expert groups and task positioning of the target hybrid expert model based on the required scenario; determining the expert quantity configuration information of each expert group based on the multidimensional evaluation parameters of each expert group; training the routing module of the target hybrid expert model based on routing training sample data to obtain a trained routing module; the routing module includes expert group routing modules and intra-group expert routing modules for each expert group; and training each expert group based on expert group training sample data until the global loss function of the target hybrid expert model converges. This invention can accurately differentiate and define the functions and boundaries of each expert in the hybrid expert model, clarify the task adaptability and task positioning of each expert, reduce the ineffective consumption of computing resources, and improve the training convergence rate.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a training method, inference method, apparatus, and device for a hybrid expert model. Background Technology

[0002] More and more companies are starting to use hybrid expert models for inference processing such as text generation, machine translation, and question answering. A hybrid expert model typically consists of an expert network and a routing module. The expert network contains multiple experts. Each expert is a sub-neural network used for inference processing. The routing module is a sub-neural network used to dynamically select an expert for inference processing for each piece of information input into the hybrid expert model.

[0003] In related technologies, a common training scheme for hybrid expert models (HEMs) involves technicians setting a fixed number of experts in the HEM based on human experience. The routing module of the HEM is trained using pre-collected sample data, resulting in a routing mechanism that selects an expert for inference processing for each piece of information input into the HEM using an approximately random strategy. The pre-collected sample data is then used to train each expert in the HEM to obtain the experts used for inference processing. However, this training scheme relies on fixed presets based on human experience, lacking standardized quantitative methods for determination. It fails to accurately differentiate and define the functions and boundaries of each expert in the HEM, and cannot clearly define the task adaptability and task positioning of each expert, thus limiting the adaptability of the HEM to different datasets and task scenarios. Furthermore, the approximately random routing mechanism results in a large number of redundant experts in the HEM, not only wasting computational resources but also significantly slowing down the training convergence rate. Summary of the Invention

[0004] This invention provides a training method, inference method, apparatus, and device for a hybrid expert model, in order to solve the problems in related technologies where training schemes for hybrid expert models cannot accurately differentiate and define the functions and boundaries of each expert in the hybrid expert model, cannot clearly define the task adaptability and task positioning of each expert, and are prone to ineffective consumption of computing resources and slow training convergence rate.

[0005] According to one aspect of the present invention, a method for training a hybrid expert model is provided, comprising: Based on the requirements of the target hybrid expert model, the number of expert groups and their task positioning in the target hybrid expert model are determined; wherein, the target hybrid expert model contains multiple expert groups, and each expert group has a different task positioning; Based on the multidimensional evaluation parameters of each expert group, the number of experts in each expert group is determined. The routing module of the target hybrid expert model is trained based on the routing training sample data to obtain the trained routing module; wherein, the routing module includes an expert group routing module and an expert routing module within each expert group; The target hybrid expert model is trained by training each expert group based on the expert group training sample data corresponding to each expert group until the global loss function of the target hybrid expert model converges, thus obtaining the trained target hybrid expert model.

[0006] According to another aspect of the present invention, a reasoning method using a hybrid expert model is provided, comprising: Based on the current inference requirements, determine each current application expert group in the trained target hybrid expert model; wherein, the target hybrid expert model contains multiple expert groups, each with a different task positioning, and each current application expert group is a part of the expert group or the full expert group. Based on the static mapping configuration information, determine the computing node to which each current application expert group belongs, and deploy each current application expert group to the computing node to which it belongs. When an inference request corresponding to the target hybrid expert model is obtained, the inference request is processed by one or more current application expert groups adapted to the inference request, and the inference result is fed back.

[0007] According to another aspect of the present invention, a training apparatus for a hybrid expert model is provided, comprising: The expert group division module is used to determine the number and task positioning of the expert groups in the target hybrid expert model based on the requirement scenario of the target hybrid expert model; wherein, the target hybrid expert model contains multiple expert groups, and each expert group has a different task positioning; The expert number configuration module is used to determine the expert number configuration information for each expert group based on the multi-dimensional evaluation parameters of each expert group. The routing training module is used to train the routing module of the target hybrid expert model based on the routing training sample data to obtain the trained routing module; wherein, the routing module includes an expert group routing module and an expert routing module within each expert group; The expert group training module is used to train each expert group based on the expert group training sample data corresponding to each expert group until the global loss function of the target hybrid expert model converges, thus obtaining the trained target hybrid expert model.

[0008] According to another aspect of the present invention, a hybrid expert model reasoning apparatus is provided, comprising: The expert group determination module is used to determine each current application expert group in the trained target hybrid expert model based on the current inference requirement information; wherein, the target hybrid expert model contains multiple expert groups, each with a different task positioning, and each current application expert group is a part of the expert groups or the full set of expert groups. The expert group deployment module is used to determine the computing node to which each current application expert group belongs based on the static mapping configuration information, and to deploy each current application expert group to the computing node to which it belongs. The expert group inference module is used to process the inference request through one or more currently applied expert groups adapted to the inference request when an inference request corresponding to the target hybrid expert model is obtained, and to provide feedback on the inference result.

[0009] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that is executed by the at least one processor, which enables the at least one processor to execute the training method of the hybrid expert model or the inference method of the hybrid expert model according to any embodiment of the present invention.

[0010] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to execute and implement the training method of the hybrid expert model or the inference method of the hybrid expert model according to any embodiment of the present invention.

[0011] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the training method of the hybrid expert model or the inference method of the hybrid expert model according to any embodiment of the present invention.

[0012] The technical solution of this invention determines the number of expert groups and task positioning of the target hybrid expert model based on the required scenario of the target hybrid expert model. The target hybrid expert model includes multiple expert groups, each with a different task positioning. Then, based on the multi-dimensional evaluation parameters of each expert group, the configuration information of the number of experts in each expert group is determined. The routing module of the target hybrid expert model is trained using routing training sample data to obtain a trained routing module. This routing module includes expert group routing modules and intra-group expert routing modules for each expert group. Each expert group is trained using training sample data corresponding to that expert group until the global loss function of the target hybrid expert model converges, thus obtaining a trained target hybrid expert model. This solves the problem that training schemes for hybrid expert models in related technologies cannot adequately address the needs of the hybrid expert model. Precise differentiation and boundary definition of the functions of each expert in the hybrid expert model cannot clearly define the task adaptability and task positioning of each expert, which can easily lead to the ineffective consumption of computing resources and slow training convergence rate. Based on the demand scenario, the expert network of the hybrid expert model can be divided into multiple expert groups, and the number of experts in each expert group can be dynamically determined according to the multi-dimensional evaluation parameters of each expert group. This allows for the precise classification and hierarchical division of the experts in the hybrid expert model according to their functional attributes or domain affiliation, achieving precise differentiation and boundary definition of the functions of each expert in the hybrid expert model, clarifying the task adaptability and task positioning of each expert. It can train a routing module that uses a two-layer dynamic routing mechanism to accurately select experts for inference processing for each piece of information input into the hybrid expert model, reducing the ineffective consumption of computing resources and improving the training convergence rate.

[0013] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart of a training method for a hybrid expert model provided in Embodiment 1 of the present invention.

[0016] Figure 2 This is a flowchart of a training method for a hybrid expert model provided in Embodiment 2 of the present invention.

[0017] Figure 3 This is a flowchart of a reasoning method using a hybrid expert model provided in Embodiment 3 of the present invention.

[0018] Figure 4 This is a schematic diagram of the reasoning process of a hybrid expert model provided in Embodiment 3 of the present invention.

[0019] Figure 5 This is a schematic diagram of the structure of a training device for a hybrid expert model provided in Embodiment 4 of the present invention.

[0020] Figure 6 This is a schematic diagram of the structure of a hybrid expert model inference device provided in Embodiment 5 of the present invention.

[0021] Figure 7 A schematic diagram of the structure of an electronic device for implementing the training or inference method of the hybrid expert model in this embodiment of the invention. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "target," "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising," "including," and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] Example 1 Figure 1This is a flowchart illustrating a training method for a hybrid expert model according to Embodiment 1 of the present invention. This embodiment is applicable to the training of hybrid expert models. The method can be executed by a hybrid expert model training device, which can be implemented in hardware and / or software and can be configured in an electronic device. This electronic device can be an electronic device installed in an enterprise for training hybrid expert models. Figure 1 As shown, the method includes: Step 101: Based on the requirements of the target hybrid expert model, determine the number of expert groups and task positioning of the target hybrid expert model.

[0025] The target hybrid expert model comprises multiple expert groups, each with a different task positioning.

[0026] Optionally, the target hybrid expert model can be a hybrid expert model that the enterprise needs to use for reasoning processing. The demand scenario for the target hybrid expert model can be text representing the scenario in which the enterprise uses the target hybrid expert model for reasoning processing. For example, the demand scenario for the target hybrid expert model can be a multi-task scenario. When the demand scenario for the target hybrid expert model is a multi-task scenario, it indicates that the enterprise needs to use the target hybrid expert model to handle multiple different types of tasks. The demand scenario for the target hybrid expert model can also be a general-purpose large language model (LLM) and a cross-task related scenario. When the demand scenario for the target hybrid expert model is a general-purpose large language model and a cross-task related scenario, it indicates that the enterprise needs to use the target hybrid expert model to simulate human brain cognitive functions. Human brain cognitive functions include, but are not limited to, language understanding, logical reasoning, memory storage, and creative generation. The demand scenario for the target hybrid expert model can also be a multi-domain scenario. When the demand scenario for the target hybrid expert model is a multi-domain scenario, it indicates that the enterprise needs to use the target hybrid expert model to handle tasks related to vertical industry large models and the knowledge domain in which the vertical industry large models reside. Vertical industry large models include, but are not limited to, medical artificial intelligence (AI) and legal assistants.

[0027] Optionally, an expert group can be a group of experts performing the same type of inference processing. The input to the experts is the information requiring inference processing. This information includes, but is not limited to, text, images, video, and audio. The output of the experts is the inference result corresponding to the input information. The inference result corresponding to the input information can be information obtained after inference processing of the input information. Each expert group contains at least two experts. The number of expert groups in the target hybrid expert model can refer to the total number of expert groups that the target hybrid expert model needs to include. The task localization of the expert groups in the target hybrid expert model can be composed of the task localizations of each expert group that the target hybrid expert model needs to include. The task localization of the expert group can be text used to characterize the type of inference processing performed by each expert in the expert group.

[0028] Optionally, the electronic device stores the number of experts and their task assignments corresponding to each demand scenario. For each demand scenario, the number of experts corresponding to the demand scenario can refer to the total number of expert groups that the hybrid expert model for that demand scenario needs to include. The task assignments corresponding to the demand scenario can refer to the task assignments of each expert group that the hybrid expert model for that demand scenario needs to include. The task assignments of each expert group are different. Determining the number of expert groups and their task assignments for the target hybrid expert model based on the demand scenario of the target hybrid expert model includes: determining the number of experts corresponding to the demand scenario of the target hybrid expert model as the number of expert groups in the target hybrid expert model, and determining the task assignments of the expert groups corresponding to the demand scenario of the target hybrid expert model as the task assignments of the expert groups in the target hybrid expert model, thereby determining the task assignments of each expert group that the target hybrid expert model needs to include. Thus, based on the demand scenario, the expert network of the target hybrid expert model is divided into multiple expert groups.

[0029] Optionally, in a specific instance, the number of experts corresponding to the multi-task scenario is 13. The expert tasks corresponding to the multi-task scenario are defined as text classification, open-ended question answering, copywriting generation, text matching, summary generation, code generation, entity recognition, sentiment analysis, text correction, cross-language translation, reading comprehension, keyword extraction, and text inference. Based on the number of experts corresponding to the multi-task scenario and the expert task definitions, the expert network of the target hybrid expert model can be divided into multiple expert groups according to task type, enabling the target hybrid expert model to adapt to the multi-task integration system and achieve precise binding between tasks and expert groups. For example, the multi-task integration system includes, but is not limited to, a multi-functional AI assistant.

[0030] Optionally, in a specific instance, the number of experts corresponding to the general-purpose large language model and cross-task related scenarios is 4. The expert task assignments corresponding to the general-purpose large language model and cross-task related scenarios are categorized as inductive summarization expert group, knowledge retrieval expert group, logical reasoning expert group, and cognitive language expert group. Based on the number of experts and their task assignments corresponding to the general-purpose large language model and cross-task related scenarios, the expert network of the target hybrid expert model can be divided into multiple expert groups, enabling the target hybrid expert model to possess strong generalization ability, conform to human cognitive logic, and have high reusability.

[0031] Optionally, in a specific instance, the number of experts corresponding to the multi-domain scenario is 5. The expert task positioning corresponding to the multi-domain scenario is legal, medical, financial, programming, and educational. Based on the number of experts corresponding to the multi-domain scenario and the expert task positioning, the expert network of the target hybrid expert model can be divided into multiple expert groups to achieve a targeted correspondence between domains and expert groups.

[0032] Step 102: Determine the number of experts in each expert group based on the multidimensional evaluation parameters of each expert group.

[0033] Optionally, based on the multidimensional evaluation parameters of each expert group, the configuration information of the number of experts in each expert group is determined, including: determining the participation frequency, contribution intensity, and hierarchical synergy of each expert group based on the evaluation sample data of each expert group; determining the configuration information of the number of experts in each expert group based on the participation frequency, contribution intensity, and hierarchical synergy of each expert group; wherein, the configuration information of the number of experts includes the number of senior experts and the number of core experts in the group.

[0034] Optionally, each expert in the expert group can be either a senior expert or a core expert. A senior expert is one who must be activated after the expert group is selected for inference processing. A core expert is one who can choose whether to be activated after the expert group is selected for inference processing. The expert group's number configuration information includes the number of senior experts and the number of core experts. The number of senior experts is the total number of senior experts required in the expert group. The number of core experts is the total number of core experts required in the expert group.

[0035] Optionally, for each expert group required by the target hybrid expert model, the evaluation sample data for the expert group can be relevant data from a pre-collected sample expert group. The sample expert group can be an existing expert group in another hybrid expert model with the same task positioning as the expert group. The sample expert group contains A experts, where A is an integer greater than or equal to 2. The evaluation sample data for the expert group can include the number of activated samples, weights, and expert type for each expert in the sample expert group. The number of activated samples for an expert can be the total number of times the expert performs the reasoning process. The weights of experts can be multiple values ​​used to characterize the magnitude of their contribution to the reasoning process. A larger expert weight indicates a greater contribution to the reasoning process, while a smaller expert weight indicates a smaller contribution. The expert type is either a senior expert or a core expert. The multidimensional evaluation parameters for the expert group can be values ​​determined based on the evaluation sample data, used to evaluate the expert participation level and assist in dynamically adjusting the number of experts in the expert group. The multidimensional evaluation parameters for the expert group can include participation frequency, contribution intensity, and hierarchical synergy.

[0036] Optionally, participation frequency can be a numerical value reflecting the activity level of each expert in the sample expert group. Participation frequency primarily focuses on the probability and frequency with which an expert is selected by the routing mechanism. Participation frequency includes the expert call frequency of each expert in the sample expert group. The expert call frequency of the i-th expert in the sample expert group can be calculated using the following formula: ;in, Let be the expert call frequency of the i-th expert in the sample expert group. Let Z be the number of activated samples for the i-th expert in the sample expert group, and Z be the sum of the number of activated samples for all experts in the sample expert group. The sample expert group contains A experts. i = 1, 2, ..., A.

[0037] Optionally, contribution strength can be a numerical value used to quantify the importance of each expert's participation in the sample expert group. Contribution strength focuses not only on whether an expert participates, but also on the weight of their contribution during participation, avoiding misjudging low-weight experts (selected but with minimal contribution) as high-participation experts. Contribution strength includes the high-weight ratio of each expert in the sample expert group. The high-weight ratio of the i-th expert in the sample expert group can be calculated using the following formula: ;in, Let be the percentage of high-weighted participation of the i-th expert in the sample expert group. The weight of the i-th expert in the sample expert group is greater than Q represents the total number of values ​​in the weights of the i-th expert in the sample expert group. It is a pre-set coefficient. The sample expert group contains A experts. i=1,2,…,A.

[0038] Optionally, hierarchical synergy can be a numerical value used to evaluate the efficiency of expert participation and synergy within a sample expert group. Hierarchical synergy includes the intra-group balance of expert participation within the sample expert group. Intra-group balance can be a numerical value used to measure the balance of participation frequency among experts within the sample expert group. The closer the intra-group balance is to 1, the more fully the group's resources are utilized. The intra-group balance of expert participation can be calculated using the following formula: ;in, S represents the balance of expert participation within the sample expert group, S represents the standard deviation of the expert call frequency of each expert in the sample expert group, and M represents the maximum value of the expert call frequency of each expert in the sample expert group.

[0039] Optionally, based on the evaluation sample data of each expert group, determine the participation frequency, contribution intensity, and hierarchical synergy of each expert group, including: performing the following operations for each expert group: calculating the participation frequency of the expert group based on the number of activated samples of each expert in the sample expert group in the evaluation sample data of the expert group; wherein, the participation frequency includes the expert call frequency of each expert in the sample expert group of the expert group; calculating the contribution intensity of the expert group based on the weight of each expert in the sample expert group in the evaluation sample data of the expert group; wherein, the contribution intensity includes the high-weight participation ratio of each expert in the sample expert group of the expert group; calculating the hierarchical synergy of the expert group based on the participation frequency of the expert group; wherein, the hierarchical synergy includes the balance of expert participation within the sample expert group of the expert group.

[0040] Optionally, based on the participation frequency, contribution intensity, and hierarchical coordination of each expert group, the configuration information of the number of experts in each expert group is determined, including: performing the following operations for each expert group: calculating the average expert call frequency of each expert in the sample expert group of the expert group, and determining the total number of experts in the expert group by multiplying the initial total number of experts by the obtained average; counting the total number of experts in the sample expert group of the expert group whose high-weight participation ratio is greater than or equal to the first value, determining the total number as the number of senior experts in the expert group, and determining the difference between the total number of experts in the expert group and the number of senior experts in the expert group as the number of core experts in the expert group; if the participation balance of experts in the sample expert group of the expert group is less than the second value, then counting the number of experts in the sample expert group of the expert group. The final number of senior experts in an expert group is determined by subtracting the total number of senior experts with low participation rates and high-weight participation percentages from the total number of core experts with low participation rates and high-weight participation percentages. Similarly, the final number of core experts in an expert group is determined by subtracting the total number of senior experts with low participation rates and high-weight participation percentages from the total number of core experts with low participation rates and high-weight participation percentages. If the participation balance of experts within the sample expert group is greater than or equal to a second value, then the final number of senior experts and core experts in the expert group determined based on the high-weight participation percentages will be determined as follows: The first and second values ​​are two pre-set values. For example, the first value is 0.1, and the second value is 0.5. The fact that the balance of expert participation within the expert group is less than the second value indicates an uneven activation among experts. It is necessary to focus on analyzing the expert call frequency and the proportion of high-weight participation indicators for each expert within the group, and to remove experts with low expert call frequency and high-weight participation proportion indicators. Low expert call frequency and high-weight participation proportion indicators can mean that the expert call frequency is lower than the third value and the high-weight participation proportion is lower than the fourth value. The third and fourth values ​​are two pre-set values.

[0041] Step 103: Train the routing module of the target hybrid expert model based on the routing training sample data to obtain the trained routing module.

[0042] The routing module includes an expert group routing module and intra-group expert routing modules for each expert group.

[0043] Optionally, the expert group routing module can be a sub-neural network used to select a suitable expert group for inference processing based on the information input to each hybrid expert model. The input to the expert group routing module is the information to be processed for inference. The output of the expert group routing module is the matching probability between each expert group and the information to be processed for inference. The matching probability between an expert group and the information to be processed for inference can refer to the probability that an expert group is suitable for inference processing of the information. The expert group routing module analyzes and judges the features of the input information to be processed for inference, determines the matching probability between each expert group and the information to be processed for inference, and achieves targeted matching between the input information to be processed for inference and the expert group. The expert group routing module can also filter out core experts to be activated within its expert group, completing the precise scheduling of experts within the group.

[0044] Optionally, for each expert group, the intra-group expert routing module can be a sub-neural network that filters out key experts to be activated within the expert group after the expert group has been selected for inference processing. Key experts to be activated are those that need to be activated and used for inference processing of information. The input to the intra-group expert routing module is the information that the expert group needs to process for inference. The output of the intra-group expert routing module is the activation probability of each key expert in the expert group after the expert group has been selected to process the information. The activation probability of a key expert can refer to the probability that the key expert is a key expert to be activated.

[0045] After deploying expert groups in the target hybrid expert model to the corresponding computing nodes, the expert grouping structure of the target hybrid expert model allows for collaborative grouping processing of the information input to the model. This prioritizes routing the input information to the computing nodes belonging to the corresponding expert groups, reducing cross-node communication overhead. The target expert group is the expert group with the highest matching probability to the input information. The target expert group is therefore suitable for reasoning processing the input information. Each input information is forcibly passed to a senior expert in the target expert group, and then distributed to core experts within the group through the secondary routing strategy of the group's expert routing module. The final output of the target hybrid expert model can be obtained by averaging the outputs of the senior experts and core experts in the target expert group.

[0046] Optionally, the routing module of the target hybrid expert model is trained based on the routing training sample data to obtain the trained routing module, including: training the expert group routing module based on the routing sample data corresponding to the expert group routing module until the loss function of the expert group routing module converges; and for each expert group's intra-group expert routing module, training the intra-group expert routing module based on the routing sample data corresponding to the intra-group expert routing module to obtain the trained intra-group expert routing module.

[0047] Optionally, the routing sample data corresponding to the expert group routing module includes multiple training data sets and multiple test data sets. Each training data set and each test data set consists of a piece of information that needs to be processed for inference and the matching probabilities of each expert group with that information. The loss function of the expert group routing module is a function used to calculate the loss value of the expert group routing module. The loss value of the expert group routing module is a numerical value used to characterize the difference between the matching probabilities of each expert group with the information output by the expert group routing module and the correct matching probabilities of each expert group with the information.

[0048] Optionally, the expert group routing module is trained using routing sample data corresponding to the expert group routing module until its loss function converges. This includes: selecting a sub-neural network from the unselected sub-neural networks in the target hybrid expert model as the expert group routing module; iteratively training the expert group routing module using training data from the routing sample data corresponding to the expert group routing module, and detecting whether the loss function of the expert group routing module has converged based on test data from the routing sample data corresponding to the expert group routing module when the number of iterations is an integer multiple of the target value; if the loss function of the expert group routing module has not converged, iterative training of the expert group routing module continues using the training data from the routing sample data corresponding to the expert group routing module; if the loss function of the expert group routing module converges, the training of the expert group routing module is determined to be complete, and iterative training of the expert group routing module using the training data from the routing sample data corresponding to the expert group routing module is stopped. The target value can be a pre-set value.

[0049] Optionally, the convergence of the loss function of the expert group routing module is detected based on test data in the routing sample data corresponding to the expert group routing module. This includes: obtaining a test data from the routing sample data corresponding to the expert group routing module; inputting the information to be processed in the test data into the expert group routing module to obtain the matching probabilities of each expert group and the information output by the expert group routing module; using the loss function of the expert group routing module, determining the loss value of the expert group routing module based on the matching probabilities of each expert group and the information in the test data and the matching probabilities of each expert group and the information output by the expert group routing module; determining whether the loss value of the expert group routing module is less than a preset loss value threshold; if yes, the loss function of the expert group routing module is determined to be converged; if no, the loss function of the expert group routing module is determined to be unconverged.

[0050] Optionally, the core objective of the expert group routing module is to allocate the input information requiring inference processing to functionally appropriate expert groups. The expert group routing module can use the normalization function softmax to calculate the matching probability between each expert group and the information. The number of expert groups in the target hybrid expert model is G. The matching probability between the expert groups and the information output by the expert group routing module can be expressed as... .in, Let x be the matching probability between the expert group and the information, and let x be the information that needs to be processed through reasoning. It is a preset coefficient matrix. These are preset coefficients. . It is a A real matrix with G rows and G columns. These are the feature dimensions of information that require reasoning and processing.

[0051] Optionally, the loss function of the expert group routing module is used to determine the loss value of the expert group routing module based on the matching probability of each expert group and the information in the test data and the matching probability of each expert group and the information output by the expert group routing module. This includes: using the following loss function of the expert group routing module to determine the loss value of the expert group routing module based on the matching probability of each expert group and the information in the test data and the matching probability of each expert group and the information output by the expert group routing module: Where L is the loss value of the expert group routing module, and G is the number of expert groups in the target hybrid expert model. This represents the matching probability of the i-th expert group among all expert groups output by the expert group routing module, corresponding to the information. Let G be the matching probability of the i-th expert group in each expert group in the test data with the information. i = 1, 2, ..., G.

[0052] The core objective of the loss function in the expert group routing module is to achieve accurate matching between the input information requiring inference processing and the expert groups. Since each expert group is pre-classified and corresponds to a specific task location, a multi-classification loss model is used to model this matching process, and this loss branch is independently optimized and updated. Simultaneously, based on the matching probability between the expert group and the information, the expert group with the highest matching probability corresponding to the input information is determined as the target expert group for inference processing of the input information. The first-layer routing mechanism of the expert group routing module completes the selection of the target expert group.

[0053] Optionally, the core objective of the in-group expert routing module is to select the experts best suited to the input information within the expert group selected by the expert group routing module, which needs to be aligned with the task positioning of the expert group. The in-group expert routing module can use the GELU activation function to calculate the activation probability of each core expert in the expert group. The number of core experts in the expert group is... The activation probability of core experts output by the expert routing module within the group can be expressed as: .in, Let x represent the activation probability of the core expert, and x represent the information that needs to be processed through reasoning. It is a preset coefficient matrix. These are preset coefficients. . It is a OK, A column of real numbers. These are the feature dimensions of information that require reasoning and processing.

[0054] Optionally, after the expert routing module outputs the activation probabilities of each core expert in the expert group, the activation probabilities of each core expert can be sorted, and the core expert with the activation probability ranked Kth can be identified as the core expert to be activated. K is an integer greater than 0 and less than the number of core experts in the expert group.

[0055] Optionally, before ranking the activation probabilities of each core expert, Gaussian noise can be added to the activation probabilities of each core expert to avoid route overfitting. The activation probabilities after adding Gaussian noise can be expressed as: .in, For activation probability, The activation probability after adding Gaussian noise. for A random number within, These are preset coefficients. For example, It is 0.1 or 0.01. It can decay with each training round. For example, in the initial stage... =0.1, later stage =0.01. For each core expert, the activation probability of the core expert after adding Gaussian noise is... Divide by temperature parameter Then, normalization is performed to obtain the final activation probability of the core experts. The activation probabilities of each core expert are ranked, and the core experts with the Kth highest activation probabilities are identified as those to be activated. The core experts with the Kth highest activation probabilities can be represented as: For example, It should be 1.5 or 1.0. For example, in the early stages of training. =1.5, later stage =1.0. During the inference phase, noise is removed. After the activation probability of each core expert in the expert group is output directly in the expert routing module within the group, the activation probabilities of each core expert can be sorted, and the core expert with the activation probability ranked in the Kth place is determined as the core expert to be activated.

[0056] Optionally, the routing sample data corresponding to the expert routing module within the group includes multiple training data sets. Each training data set consists of a piece of information that needs to be processed for inference and the activation probabilities of each core expert in the expert group after the expert group has been selected to process the information.

[0057] Optionally, for each expert group's intra-group expert routing module, the intra-group expert routing module is trained based on the routing sample data corresponding to the intra-group expert routing module to obtain a trained intra-group expert routing module. This includes performing the following operations for each expert group: selecting a sub-neural network from the unselected sub-neural networks in the target hybrid expert model as the expert group's intra-group expert routing module; iteratively training the intra-group expert routing module using the routing sample data corresponding to the intra-group expert routing module, and determining that the training of the intra-group expert routing module is complete when the iterative training reaches a specified value, and stopping the iterative training of the intra-group expert routing module using the routing sample data corresponding to the intra-group expert routing module.

[0058] Step 104: Train each expert group according to the expert group training sample data corresponding to each expert group until the global loss function of the target hybrid expert model converges, and obtain the trained target hybrid expert model.

[0059] Optionally, after the routing module of the target hybrid expert model is trained, the parameters of the routing module are frozen, and each expert group is trained according to the expert group training sample data corresponding to each expert group until the global loss function of the target hybrid expert model converges, thus obtaining the trained target hybrid expert model.

[0060] Optionally, each expert group is trained based on the expert group training sample data corresponding to each expert group until the global loss function of the target hybrid expert model converges. This includes: training the senior experts and core experts of each expert group based on the expert group training sample data corresponding to each expert group until the global loss function of the target hybrid expert model converges; wherein, the global loss function consists of a task core loss function, a hierarchical load balancing loss function, and a hierarchical matching loss function.

[0061] Optionally, for each expert group, the corresponding expert group training sample data includes multiple training data sets. Each training data set consists of a piece of information that needs to be processed for inference and the corresponding inference result.

[0062] Optionally, the senior experts and core experts of each expert group are trained based on the expert group training sample data corresponding to each expert group until the global loss function of the target hybrid expert model converges. This includes: for each expert group, selecting a target number of sub-neural networks from the unselected sub-neural networks in the target hybrid expert model as experts in the expert group; determining the senior experts and core experts in the expert group based on the expert number configuration information of the expert group; and iteratively training each expert in the expert group using the training data from the expert group training sample data corresponding to the expert group; wherein, the target number is equal to the sum of the number of senior experts and the number of core experts in the expert group, and selecting senior experts from the experts in the expert group. The selected experts are designated as senior experts, and the remaining unselected experts are designated as core experts. When the number of training iterations is an integer multiple of the target value, the global loss function of the target hybrid expert model is checked for convergence based on the test sample data of the expert group corresponding to the target hybrid expert model. If the global loss function of the target hybrid expert model has not converged, the training data from the training sample data of each expert group is used to iteratively train each expert in each expert group. If the global loss function of the target hybrid expert model converges, the training of the target hybrid expert model is considered complete, and the iterative training of each expert in each expert group using the training sample data of each expert group is stopped.

[0063] Optionally, the expert group test sample data corresponding to the target hybrid expert model includes multiple test data sets. Each test data set consists of a piece of information that needs to be processed through reasoning and the corresponding reasoning result.

[0064] Optionally, based on the expert group test sample data corresponding to the target hybrid expert model, the convergence of the global loss function of the target hybrid expert model is checked. This includes: for each test data in the expert group test sample data corresponding to the target hybrid expert model, the target expert group and the core experts to be activated in the target expert group are determined through the expert group routing module and the intra-group expert routing module of each expert group; the senior experts in the target expert group and the core experts to be activated perform reasoning on the information; the reasoning results output by the senior experts in the target expert group and the core experts to be activated are fused by averaging to obtain the final reasoning result; and the task core loss of the target hybrid expert model is used. The system uses the following functions to determine the core task loss value of the target hybrid expert model: hierarchical load balancing loss function; hierarchical matching loss function; and global loss function. Based on the core task loss value, hierarchical load balancing loss value, and hierarchical matching loss value, the system determines the global loss value of the target hybrid expert model. It then checks whether the global loss value is less than a preset loss threshold. If it is, the global loss function is considered converged; otherwise, it is considered not converged.

[0065] Optionally, the task core loss function of the target mixed expert model can be a function used to calculate the task core loss value of the target mixed expert model. The task core loss value of the target mixed expert model is a numerical value used to characterize the difference between the inference result of the target mixed expert model and the correct inference result from the perspective of task accuracy. The goal of the task core loss function is to guarantee the task performance of the target mixed expert model. Determining the task core loss value of the target mixed expert model using the task core loss function includes: using the following task core loss function of the target mixed expert model, based on the inference results in each test data and the determined inference results corresponding to the information in each test data, to determine the task core loss value of the target mixed expert model: ;in, Let N be the core loss value of the objective hybrid expert model, and N be the total number of test data in the expert group test sample data corresponding to the objective hybrid expert model. This refers to the numerical value used to characterize the reasoning result of the i-th test data in the expert group's test sample data after normalization. The value is a normalized representation of the reasoning result corresponding to the information that needs to be reasoned in the i-th test data in the expert group's test sample data, where i = 1, 2, ..., N.

[0066] Optionally, the hierarchical load balancing loss function of the target hybrid expert model can be a function used to calculate the hierarchical load balancing loss value of the target hybrid expert model. The hierarchical load balancing loss value of the target hybrid expert model is a numerical value used to characterize the performance of the target hybrid expert model from the perspective of load balancing. The goal of the hierarchical load balancing loss function is to solve the problems of unpopular expert groups, unpopular experts within a group, and hotspot overload, and it is divided into two parts: group-level load balancing and intra-group expert load balancing. The group-level load balancing loss value of the target hybrid expert model can be a numerical value used to characterize the performance of the target hybrid expert model from the perspective of load balancing among various expert groups. The intra-group expert load balancing loss value of the target hybrid expert model can be a numerical value used to characterize the performance of the target hybrid expert model from the perspective of load balancing among various experts within an expert group.

[0067] Optionally, the hierarchical load balancing loss value of the target hybrid expert model is determined using the hierarchical load balancing loss function of the target hybrid expert model, including: calculating the group-level load balancing loss value of the target hybrid expert model using the following formula: ;in, Let G be the group-level load balancing loss value of the objective hybrid expert model, and G be the number of expert groups in the objective hybrid expert model. Let be the call frequency of the g-th expert group in the process of reasoning and processing the information in each test data in the expert group's test sample data. The call frequency of the g-th expert group is equal to the ratio of the total number of times the g-th expert group performs the reasoning process in the process of reasoning and processing the information in each test data to the total number of reasoning processes. The total number of reasoning processes is the sum of the total number of times each expert group performs the reasoning process in the process of reasoning and processing the information in each test data. This represents the ideal call frequency, where, if all expert groups are called evenly, the call frequency of each expert group is exactly equal to the total frequency. , g=1,2,…,G; use the following formula to calculate the in-group expert load balancing loss value of the target hybrid expert model: ;in, Let G be the in-group expert load balancing loss value of the objective hybrid expert model, and G be the number of expert groups in the objective hybrid expert model. Let g be the frequency of inference performed by the e-th expert in the g-th expert group during the reasoning process of processing information from various test data in the expert group's test sample data. g = 1, 2, ..., G. The frequency of inference performed by the e-th expert is equal to the ratio of the total number of inference processes performed by the e-th expert during the reasoning process of processing information from various test data to the total number of inference processes performed within the group. The total number of inference processes performed within the group is the sum of the total number of inference processes performed by all experts in the g-th expert group during the reasoning process of processing information from various test data. Let e ​​be the number of experts in the expert group, where e = 1, 2, ... Using the hierarchical load balancing loss function of the target hybrid expert model as described below, determine the hierarchical load balancing loss value of the target hybrid expert model based on the group-level load balancing loss value and the intra-group expert load balancing loss value: ;in, The hierarchical load balancing loss value of the target hybrid expert model. The group-level load balancing loss value of the target hybrid expert model. Let $\frac{ ... and These are preset coefficients.

[0068] Optionally, the hierarchical matching loss function of the target hybrid expert model can be a function used to calculate the hierarchical matching loss value of the target hybrid expert model. The hierarchical matching loss value of the target hybrid expert model is a numerical value used to characterize the performance of the target hybrid expert model from the perspective of hierarchical synergy. The hierarchical matching loss value is used to ensure the synergy between the first layer (expert group selection) and the second layer (intra-group expert selection), avoiding the situation where an expert group is selected but there are no suitable experts within the group (i.e., the weights of experts within the group are all extremely low), and improving the consistency of routing decisions. The main design logic of the hierarchical matching loss value is to calculate the average weight of the core experts within each selected expert group. If the average weight of the core experts within the group is too low, it indicates that there are no suitable experts within the group, and a penalty is applied. Using the hierarchical matching loss function of the target hybrid expert model, the hierarchical matching loss value of the target hybrid expert model is determined, including: using the following formula to calculate the intra-group average weight of the g-th expert group in each expert group: ;in, Let K be the average weight within the g-th expert group, and K be the sum of the weights of all activated core experts in the g-th expert group. Let G be the sum of the weights of the core experts performing the reasoning process in the g-th expert group, where G is the number of expert groups in the target hybrid expert model, g = 1, 2, ..., G; calculate the expert group-level average weight using the following formula: ;in, The average weight is based on the expert panel level. Let G be the average weight within the g-th expert group, and G be the number of expert groups in the target hybrid expert model, where g = 1, 2, ..., G. Use the following hierarchical matching loss function for the target hybrid expert model to determine the hierarchical matching loss value: ;in, Let be the hierarchical matching loss value of the target hybrid expert model. Let g be the average weight within the g-th expert group. It is a preset threshold coefficient. Indicates to retain only and The positive difference value, g = 1, 2, ..., G. For example, It is 0.5.

[0069] Optionally, the global loss function of the target hybrid expert model can be a function used to calculate the global loss value of the target hybrid expert model. The global loss value of the target hybrid expert model is a numerical value used to comprehensively characterize the performance of the target hybrid expert model from the perspectives of task accuracy, load balancing, and hierarchical synergy. The global loss function consists of a task core loss function, a hierarchical load balancing loss function, and a hierarchical matching loss function, taking into account task accuracy, load balancing, and hierarchical synergy. Using the global loss function of the target hybrid expert model, the global loss value of the target hybrid expert model is determined based on the task core loss value, hierarchical load balancing loss value, and hierarchical matching loss value, including: using the following global loss function of the target hybrid expert model, the global loss value of the target hybrid expert model is determined based on the task core loss value, hierarchical load balancing loss value, and hierarchical matching loss value: ;in, The global loss value of the objective hybrid expert model. The core loss value of the objective hybrid expert model is the target. The hierarchical load balancing loss value of the target hybrid expert model. Let be the hierarchical matching loss value of the target hybrid expert model. , as well as These are preset weighting values. Different values ​​can be set for different tasks. It can be set to 0.5 or higher. and It can be set to a relatively small value.

[0070] The technical solution of this invention determines the number of expert groups and task positioning of the target hybrid expert model based on the required scenario of the target hybrid expert model. The target hybrid expert model includes multiple expert groups, each with a different task positioning. Then, based on the multi-dimensional evaluation parameters of each expert group, the configuration information of the number of experts in each expert group is determined. The routing module of the target hybrid expert model is trained using routing training sample data to obtain a trained routing module. This routing module includes expert group routing modules and intra-group expert routing modules for each expert group. Each expert group is trained using training sample data corresponding to that expert group until the global loss function of the target hybrid expert model converges, thus obtaining a trained target hybrid expert model. This solves the problem that training schemes for hybrid expert models in related technologies cannot adequately address the needs of the hybrid expert model. Precise differentiation and boundary definition of the functions of each expert in the hybrid expert model cannot clearly define the task adaptability and task positioning of each expert, which can easily lead to the ineffective consumption of computing resources and slow training convergence rate. Based on the demand scenario, the expert network of the hybrid expert model can be divided into multiple expert groups, and the number of experts in each expert group can be dynamically determined according to the multi-dimensional evaluation parameters of each expert group. This allows for the precise classification and hierarchical division of the experts in the hybrid expert model according to their functional attributes or domain affiliation, achieving precise differentiation and boundary definition of the functions of each expert in the hybrid expert model, clarifying the task adaptability and task positioning of each expert. It can train a routing module that uses a two-layer dynamic routing mechanism to accurately select experts for inference processing for each piece of information input into the hybrid expert model, reducing the ineffective consumption of computing resources and improving the training convergence rate.

[0071] Example 2 Figure 2 This is a flowchart illustrating a training method for a hybrid expert model according to Embodiment 2 of the present invention. Embodiments of the present invention can be combined with various optional solutions from one or more of the above embodiments. For example... Figure 2 As shown, the method includes: Step 201: Based on the requirements of the target hybrid expert model, determine the number of expert groups and task positioning of the target hybrid expert model.

[0072] The target hybrid expert model comprises multiple expert groups, each with a different task positioning.

[0073] Step 202: Based on the evaluation sample data of each expert group, determine the participation frequency, contribution intensity, and hierarchical coordination of each expert group.

[0074] Step 203: Determine the number of experts in each expert group based on their participation frequency, contribution intensity, and hierarchical coordination.

[0075] The expert number configuration information includes the number of senior experts and the number of core experts within the group.

[0076] Step 204: Train the expert group routing module based on the routing sample data corresponding to the expert group routing module until the loss function of the expert group routing module converges.

[0077] Step 205: For each expert group's intra-group expert routing module, train the intra-group expert routing module based on the routing sample data corresponding to the intra-group expert routing module to obtain the trained intra-group expert routing module.

[0078] Step 206: Train the senior experts and core experts of each expert group according to the expert group training sample data corresponding to each expert group until the global loss function of the target hybrid expert model converges, and obtain the trained target hybrid expert model.

[0079] The global loss function consists of a task core loss function, a hierarchical load balancing loss function, and a hierarchical matching loss function.

[0080] The technical solution of this invention can divide the expert network of a hybrid expert model into multiple expert groups based on the demand scenario. It dynamically determines the number of experts in each expert group according to their participation frequency, contribution intensity, and hierarchical collaboration. This allows for precise classification and hierarchical division of the experts in the hybrid expert model based on their functional attributes or domain affiliation. It achieves precise differentiation and boundary definition of the functions of each expert in the hybrid expert model, clarifies the task adaptability and task positioning of each expert, and can train an expert group routing module and an intra-group expert routing module that use a two-layer dynamic routing mechanism to accurately select experts for inference processing for each piece of information input into the hybrid expert model. This reduces the ineffective consumption of computing resources and improves the training convergence rate.

[0081] Example 3 Figure 3 This is a flowchart illustrating a hybrid expert model inference method according to Embodiment 3 of the present invention. This embodiment is applicable to the deployment of a trained hybrid expert model and its use for inference processing. The method can be executed by an inference device for the hybrid expert model, which can be implemented in hardware and / or software. The training device for the hybrid expert model can be configured in an electronic device. This electronic device can be an electronic device installed in an enterprise for deploying the trained hybrid expert model and its use for inference processing. Figure 3 As shown, the method includes: Step 301: Based on the current inference requirements information, determine the current application expert groups in the trained target hybrid expert model.

[0082] The target hybrid expert model includes multiple expert groups, each with a different task positioning. Each currently applied expert group is either a subset or the full set of expert groups.

[0083] Optionally, the current application expert group can refer to the expert group that the enterprise currently needs to use in the trained target hybrid expert model. The various current application expert groups that the enterprise needs to use can be a subset of the expert groups or the full set of expert groups. The full set of expert groups refers to all expert groups in the trained target hybrid expert model.

[0084] Optionally, the current inference requirement information can be the task positioning of each expert group that the enterprise currently needs to use. Based on the current inference requirement information, determine each current application expert group in the trained target hybrid expert model, including: defining each expert group whose task positioning is included in the current inference requirement information in the trained target hybrid expert model as a current application expert group, thereby determining each current application expert group in the trained target hybrid expert model.

[0085] Step 302: Based on the static mapping configuration information, determine the computing node to which each current application expert group belongs, and deploy each current application expert group to the computing node to which it belongs.

[0086] Optionally, for each current application expert group, the computing node belonging to the current application expert group consists of the computing nodes belonging to each expert group within the current application expert group. The computing node belonging to an expert group is the computing node used to run the expert group. The computing node can be an Artificial Intelligence (AI) accelerator card. The identification information of the expert group can be information used to uniquely identify the expert group. The identification information of the computing node can be information used to uniquely identify the computing node. The static mapping configuration information can consist of the identification information of each expert group within each current application expert group and the identification information of the computing nodes to which each expert group belongs. The computing nodes belonging to expert groups within the same current application expert group are the same computing node or adjacent computing nodes.

[0087] Optionally, based on the static mapping configuration information, determine the computing node to which each current application expert group belongs, and deploy each current application expert group to the computing node to which it belongs. This includes: based on the identification information of each expert group in each current application expert group and the identification information of the computing node to which each expert group belongs in the static mapping configuration information, determine the computing node to which each expert group belongs, and deploy each expert group to the computing node to which it belongs.

[0088] Step 303: When an inference request corresponding to the target hybrid expert model is obtained, the inference request is processed by one or more current application expert groups adapted to the inference request, and the inference result is fed back.

[0089] Optionally, the inference request corresponding to the target hybrid expert model can refer to information that requires inference processing by the target hybrid expert model. The inference request is processed by one or more currently applied expert groups adapted to it, and the inference result is fed back. This includes: determining the target expert group and the core experts to be activated within the target expert group through the expert group routing module and the intra-group expert routing module of each expert group; processing the inference request through senior experts in the target expert group and the core experts to be activated; averaging and fusing the inference results output by senior experts in the target expert group and the core experts to be activated to obtain the final inference result; and feeding back the final inference result to the electronic device or user that sent the inference request.

[0090] The core design principle of hybrid expert models is sparse activation, meaning that only a subset of experts are activated during training iterations and inference. However, in actual inference deployment, the inference schemes of hybrid expert models in related technologies still require pre-loading and deploying all parameters, and then dynamically activating the target expert subset through a routing mechanism. In scenarios with limited computing resources, the storage and loading requirements of the full set of parameters in the inference schemes of hybrid expert models in related technologies can exceed the hardware's carrying capacity, making it difficult to deploy hybrid expert models. In addition, in specific scenarios such as vertical domains where generalization capabilities are not required, the solutions in related technologies only activate a small number of experts after deploying all parameters, resulting in a large number of idle parameters occupying storage and computing resources, leading to serious resource waste.

[0091] The technical solution of this invention determines each current application expert group in the trained target hybrid expert model based on current inference requirement information. The target hybrid expert model contains multiple expert groups, each with a different task positioning. Each current application expert group is either a subset or the full set of expert groups. Then, based on static mapping configuration information, the assigned computing node for each current application expert group is determined, and each current application expert group is deployed to its assigned computing node. When an inference request corresponding to the target hybrid expert model is obtained, the inference request is processed by one or more current application expert groups adapted to the request, and the inference result is fed back. This solves the problem in related technologies where the inference scheme of a hybrid expert model is directly deployed to the full set, leading to difficulties in implementing the hybrid expert model and causing serious resource waste. Instead, after the hybrid expert model is trained, based on current inference requirement information, a subset or full set of expert groups can be selected for deployment, significantly improving deployment flexibility and scenario adaptability, supporting flexible deployment strategies, and avoiding the problems of difficulty in implementing the hybrid expert model and resource waste.

[0092] Optionally, after deploying each current application expert group to the computing node to which each current application expert group belongs, the method further includes: when a fine-tuning request corresponding to the target current application expert group is obtained, fine-tuning the target current application expert group according to the fine-tuning request; wherein, the target current application expert group is a portion of the current application expert groups or all current application expert groups.

[0093] Optionally, the fine-tuning request corresponding to the target current application expert group can be information sent by the user based on the task objective and its own computing resource constraints, instructing the user to fine-tune the target current application expert group. Fine-tuning the target current application expert group means updating the parameter values ​​of the expert group's parameters from their current values ​​to specified values. When a fine-tuning request corresponding to the target current application expert group is received, the target current application expert group can be fine-tuned according to the fine-tuning request.

[0094] Therefore, for the fine-tuning needs of the trained hybrid expert model, the targeted expert group fine-tuning or full expert fine-tuning mode can be flexibly selected to achieve the optimal balance between fine-tuning cost and task effect.

[0095] This invention designs an expert participation evaluation mechanism, which learns to adaptively and dynamically adjust the size of the expert group (not a fixed number) to achieve a scientific configuration of the number of experts, improve the adaptability of the network structure and training data, and thus accelerate the training convergence rate.

[0096] This invention introduces a two-layer dynamic routing mechanism into a hybrid expert model, achieving customized training through structured grouping and hierarchical management of the expert network. This invention combines preset task objectives and scenario requirements to design targeted training paradigms, promoting precise differentiation and boundary definition of expert group functions, clarifying the task adaptability and functional positioning of each expert group, improving the interpretability of the model structure and the traceability of its functions, and addressing the pain points of ambiguous expert functions and insufficient interpretability in traditional hybrid expert models.

[0097] After expert grouping, this invention supports static pre-mapping configuration between experts and computing nodes. By grouping constraints, it reduces cross-node data transmission and expert scheduling overhead during distributed training or inference processes, significantly improving the operating efficiency of distributed systems.

[0098] The hybrid expert model trained in this invention has a clear and interpretable function for expert groups. For users with limited computing resources or those who need fine-tuning in specific vertical fields, the model can selectively choose suitable expert groups without the need for full fine-tuning, thus significantly reducing resource consumption.

[0099] In the inference deployment phase, this invention can flexibly select a portion of the expert group for targeted deployment based on the needs of the target application scenario, without loading all the parameters of the model. This enables the flexible transformation of large models into lightweight deployment forms, making it possible to deploy large models in scenarios where computing resources cannot support the complete model and the complexity of the inference task is low.

[0100] After expert grouping using the solution of this invention, the task computation density of the hybrid expert model is significantly improved, which can fully adapt to the hardware characteristics of computationally intensive accelerator cards, maximize the release of their computing power advantages, and thus optimize the model computation efficiency.

[0101] Optional, exemplary, Figure 4 This is a schematic diagram illustrating the inference process of a hybrid expert model provided in Embodiment 3 of the present invention. The various expert groups required by the enterprise in the target hybrid expert model are deployed on corresponding computing nodes, each with a different task positioning. Each expert group contains multiple senior experts and multiple core experts. The enterprise also deploys multiple globally shared experts. Globally shared experts are experts without clearly defined task positioning. When an inference request corresponding to the target hybrid expert model is received, the target expert group and the core experts to be activated within the target expert group are determined through the expert group routing module and the intra-group expert routing module of each expert group. The inference request is processed by the globally shared experts, the senior experts in the target expert group, and the core experts to be activated. The inference results output by the globally shared experts, the senior experts in the target expert group, and the core experts to be activated are averaged and fused to obtain the final inference result. The final inference result is then fed back to the electronic device or user that sent the inference request.

[0102] Example 4 Figure 5 This is a schematic diagram of a training device for a hybrid expert model provided in Embodiment 4 of the present invention. The device can be configured in an electronic device. Figure 5 As shown, the device includes: an expert group division module 401, an expert number configuration module 402, a route training module 403, and an expert group training module 404.

[0103] The system includes: an expert group partitioning module 401, used to determine the number and task positioning of expert groups in the target hybrid expert model based on the required scenario of the target hybrid expert model; wherein the target hybrid expert model contains multiple expert groups, each with a different task positioning; an expert number configuration module 402, used to determine the expert number configuration information of each expert group based on the multidimensional evaluation parameters of each expert group; a routing training module 403, used to train the routing module of the target hybrid expert model based on routing training sample data to obtain a trained routing module; wherein the routing module includes expert group routing modules and intra-group expert routing modules for each expert group; and an expert group training module 404, used to train each expert group based on expert group training sample data corresponding to each expert group until the global loss function of the target hybrid expert model converges, thereby obtaining a trained target hybrid expert model.

[0104] The technical solution of this invention determines the number of expert groups and task positioning of the target hybrid expert model based on the required scenario of the target hybrid expert model. The target hybrid expert model includes multiple expert groups, each with a different task positioning. Then, based on the multi-dimensional evaluation parameters of each expert group, the configuration information of the number of experts in each expert group is determined. The routing module of the target hybrid expert model is trained using routing training sample data to obtain a trained routing module. This routing module includes expert group routing modules and intra-group expert routing modules for each expert group. Each expert group is trained using training sample data corresponding to that expert group until the global loss function of the target hybrid expert model converges, thus obtaining a trained target hybrid expert model. This solves the problem that training schemes for hybrid expert models in related technologies cannot adequately address the needs of the hybrid expert model. Precise differentiation and boundary definition of the functions of each expert in the hybrid expert model cannot clearly define the task adaptability and task positioning of each expert, which can easily lead to the ineffective consumption of computing resources and slow training convergence rate. Based on the demand scenario, the expert network of the hybrid expert model can be divided into multiple expert groups, and the number of experts in each expert group can be dynamically determined according to the multi-dimensional evaluation parameters of each expert group. This allows for the precise classification and hierarchical division of the experts in the hybrid expert model according to their functional attributes or domain affiliation, achieving precise differentiation and boundary definition of the functions of each expert in the hybrid expert model, clarifying the task adaptability and task positioning of each expert. It can train a routing module that uses a two-layer dynamic routing mechanism to accurately select experts for inference processing for each piece of information input into the hybrid expert model, reducing the ineffective consumption of computing resources and improving the training convergence rate.

[0105] In an optional embodiment of the present invention, the expert number configuration module 402 is specifically used to: determine the participation frequency, contribution intensity, and hierarchical synergy of each expert group based on the evaluation sample data of each expert group; and determine the expert number configuration information of each expert group based on the participation frequency, contribution intensity, and hierarchical synergy of each expert group; wherein the expert number configuration information includes the number of senior experts and the number of core experts in the group.

[0106] In an optional embodiment of the present invention, the routing training module 403 is specifically configured to: train the expert group routing module according to the routing sample data corresponding to the expert group routing module until the loss function of the expert group routing module converges; and for each expert group's intra-group expert routing module, train the intra-group expert routing module according to the routing sample data corresponding to the intra-group expert routing module to obtain the trained intra-group expert routing module.

[0107] In an optional embodiment of the present invention, the expert group training module 404 is specifically used to: train the senior experts and backbone experts of each expert group according to the expert group training sample data corresponding to each expert group, until the global loss function of the target hybrid expert model converges; wherein, the global loss function is composed of a task core loss function, a hierarchical load balancing loss function, and a hierarchical matching loss function.

[0108] The training apparatus for the hybrid expert model provided in the embodiments of the present invention can execute the training method for the hybrid expert model provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0109] Example 5 Figure 6 This is a schematic diagram of the structure of a hybrid expert model inference device provided in Embodiment 5 of the present invention. The device can be configured in an electronic device. Figure 6 As shown, the device includes: an expert group determination module 501, an expert group deployment module 502, and an expert group reasoning module 503.

[0110] The system includes several modules: an expert group determination module 501, which determines each current application expert group in the trained target hybrid expert model based on current inference requirement information; the target hybrid expert model contains multiple expert groups with different task positioning, and each current application expert group is either a subset or the full set of expert groups; an expert group deployment module 502, which determines the assigned computing node for each current application expert group based on static mapping configuration information and deploys each current application expert group to its assigned computing node; and an expert group inference module 503, which, when an inference request corresponding to the target hybrid expert model is received, processes the inference request through one or more current application expert groups adapted to the inference request and provides feedback on the inference result.

[0111] The technical solution of this invention determines each current application expert group in the trained target hybrid expert model based on current inference requirement information. The target hybrid expert model contains multiple expert groups, each with a different task positioning. Each current application expert group is either a subset or the full set of expert groups. Then, based on static mapping configuration information, the assigned computing node for each current application expert group is determined, and each current application expert group is deployed to its assigned computing node. When an inference request corresponding to the target hybrid expert model is obtained, the inference request is processed by one or more current application expert groups adapted to the request, and the inference result is fed back. This solves the problem in related technologies where the inference scheme of a hybrid expert model is directly deployed to the full set, leading to difficulties in implementing the hybrid expert model and causing serious resource waste. Instead, after the hybrid expert model is trained, based on current inference requirement information, a subset or full set of expert groups can be selected for deployment, significantly improving deployment flexibility and scenario adaptability, supporting flexible deployment strategies, and avoiding the problems of difficulty in implementing the hybrid expert model and resource waste.

[0112] In an optional embodiment of the present invention, the inference apparatus of the hybrid expert model may further include: a fine-tuning module, configured to fine-tune the target current application expert group according to the fine-tuning request when a fine-tuning request corresponding to the target current application expert group is obtained; wherein the target current application expert group is a portion of the current application expert groups or the full set of current application expert groups.

[0113] The inference apparatus of the hybrid expert model provided in the embodiments of the present invention can execute the inference method of the hybrid expert model provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0114] Example 6 Figure 7 A schematic diagram of an electronic device 10 is shown, which can be used to implement the training or inference methods of the hybrid expert model in embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, electronic devices, blade electronic devices, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0115] like Figure 7As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory 12 or a random access memory 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the read-only memory 12 or loaded from storage unit 18 into the random access memory 13. The random access memory 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, read-only memory 12, and random access memory 13 are interconnected via a bus 14. An input / output interface 15 is also connected to the bus 14.

[0116] Multiple components in electronic device 10 are connected to input / output interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of monitors, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0117] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, central processing units, graphics processing units, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, digital signal processors, and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as hybrid expert model training or inference methods.

[0118] In some embodiments, the training or inference method of the hybrid expert model can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as a storage unit. In some embodiments, part or all of the computer program can be loaded and / or installed on a heterogeneous hardware accelerator via read-only memory and / or a communication unit. When the computer program is loaded into random access memory and executed by a processor, one or more steps of the training or inference method of the hybrid expert model described above can be performed. Alternatively, in other embodiments, the processor can be configured to execute the training or inference method of the hybrid expert model by any other suitable means (e.g., by means of firmware).

[0119] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays, application-specific integrated circuits (ASICs), application-specific standard products (ASICs), systems-on-a-chip (SoCs), payload programmable logic devices (PLCs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0120] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or electronic device.

[0121] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0122] To provide user interaction, the systems and techniques described herein can be implemented on a heterogeneous hardware accelerator, which includes: a display device (e.g., a cathode ray tube or liquid crystal display monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the heterogeneous hardware accelerator. Other types of devices can also be used to provide user interaction; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or haptic feedback); and input from the user can be received in any form (including sound input, voice input, or haptic input).

[0123] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data electronic devices), or computing systems that include middleware components (e.g., application electronic devices), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0124] A computing system can include clients and electronic devices. Clients and electronic devices are generally geographically separated and typically interact via communication networks. The client-electronic device relationship is created by computer programs running on the respective computers and establishing a client-electronic device relationship between them. Electronic devices can be cloud electronic devices, also known as cloud computing electronic devices or cloud servers, which are host products within the cloud computing service system. These address the shortcomings of traditional physical hosts and virtual private server services, such as high management difficulty and weak business scalability.

[0125] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0126] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A training method for a hybrid expert model, characterized in that, include: Based on the requirements of the target hybrid expert model, the number of expert groups and their task positioning are determined. The target hybrid expert model includes multiple expert groups, each with a different task positioning. The input of the experts in each expert group is the information that needs to be reasoned and processed, which may be text, image, video, or audio. Based on the evaluation sample data of each expert group, the participation frequency, contribution intensity, and hierarchical synergy of each expert group are determined. Specifically, for each expert group, the following operations are performed: The participation frequency of the expert group is calculated based on the number of activated samples of each expert in the sample expert group from the evaluation sample data; the participation frequency includes the expert call frequency of each expert in the sample expert group; the contribution intensity of the expert group is calculated based on the weight of each expert in the sample expert group; the contribution intensity includes the proportion of high-weighted participation indicators of each expert in the sample expert group; and the hierarchical synergy of the expert group is calculated based on the participation frequency; the hierarchical synergy includes the balance of expert participation within the sample expert group. The number of experts in each expert group is determined based on their participation frequency, contribution intensity, and hierarchical coordination. The number of experts in each expert group includes the number of senior experts and the number of core experts within the group. The routing module of the target hybrid expert model is trained based on the routing training sample data to obtain the trained routing module; wherein, the routing module includes an expert group routing module and an expert routing module within each expert group; The target hybrid expert model is trained by training each expert group based on the expert group training sample data corresponding to each expert group until the global loss function of the target hybrid expert model converges, thus obtaining the trained target hybrid expert model.

2. The training method for the hybrid expert model according to claim 1, characterized in that, The routing module of the target hybrid expert model is trained based on the routing training sample data to obtain the trained routing module, which includes: The expert group routing module is trained based on the routing sample data corresponding to the expert group routing module until the loss function of the expert group routing module converges. For each expert group's intra-group expert routing module, the intra-group expert routing module is trained based on the routing sample data corresponding to the intra-group expert routing module, resulting in a trained intra-group expert routing module.

3. The training method for the hybrid expert model according to claim 1, characterized in that, Training is performed on each expert group using the corresponding expert group training sample data until the global loss function of the target hybrid expert model converges, including: The senior experts and core experts of each expert group are trained based on the training sample data corresponding to each expert group until the global loss function of the target hybrid expert model converges; wherein, the global loss function consists of the task core loss function, the hierarchical load balancing loss function, and the hierarchical matching loss function.

4. A reasoning method using a hybrid expert model, characterized in that, include: Based on the current inference requirement information, determine each current application expert group in the trained target hybrid expert model; wherein, the target hybrid expert model is the target hybrid expert model trained in the training method of the hybrid expert model according to any one of claims 1-3, the target hybrid expert model contains multiple expert groups, each expert group has a different task positioning, the input of the experts in the expert group is the information to be processed by inference, the information is text, image, video or audio, and each current application expert group is a part of the expert group or the full expert group in each expert group; Based on the static mapping configuration information, determine the computing node to which each current application expert group belongs, and deploy each current application expert group to the computing node to which it belongs; wherein, based on the identification information of each expert group in each current application expert group and the identification information of the computing node to which each expert group belongs in the static mapping configuration information, determine the computing node to which each expert group in each current application expert group belongs, and deploy each expert group to the computing node to which it belongs. When an inference request corresponding to the target hybrid expert model is obtained, the inference request is processed by one or more current application expert groups adapted to the inference request, and the inference result is fed back.

5. The reasoning method of the hybrid expert model according to claim 4, characterized in that, After deploying each current application expert group to its respective home compute node, the process also includes: When a fine-tuning request corresponding to the target current application expert group is obtained, the target current application expert group is fine-tuned according to the fine-tuning request; wherein, the target current application expert group is a part of the current application expert groups or the full current application expert groups.

6. A training device for a hybrid expert model, characterized in that, include: The expert group division module is used to determine the number and task positioning of the expert groups in the target hybrid expert model according to the needs of the target hybrid expert model. The target hybrid expert model contains multiple expert groups, each with a different task positioning. The input of the experts in the expert group is the information that needs to be reasoned and processed, which can be text, image, video, or audio. The expert quantity configuration module is used to determine the participation frequency, contribution intensity, and hierarchical synergy of each expert group based on the evaluation sample data of each expert group. Specifically, for each expert group, the following operations are performed: The participation frequency of the expert group is calculated based on the number of activated samples of each expert in the sample expert group from the evaluation sample data; the participation frequency includes the expert call frequency of each expert in the sample expert group; the contribution intensity of the expert group is calculated based on the weight of each expert in the sample expert group; the contribution intensity includes the proportion of high-weighted participation indicators of each expert in the sample expert group; the hierarchical synergy of the expert group is calculated based on the participation frequency; the hierarchical synergy includes the balance of expert participation within the sample expert group; and the expert quantity configuration information for each expert group is determined based on the participation frequency, contribution intensity, and hierarchical synergy of each expert group; the expert quantity configuration information includes the number of senior experts and the number of core experts within the group. The routing training module is used to train the routing module of the target hybrid expert model based on the routing training sample data to obtain the trained routing module; wherein, the routing module includes an expert group routing module and an expert routing module within each expert group; The expert group training module is used to train each expert group based on the expert group training sample data corresponding to each expert group until the global loss function of the target hybrid expert model converges, thus obtaining the trained target hybrid expert model.

7. A reasoning device using a hybrid expert model, characterized in that, include: The expert group determination module is used to determine each current application expert group in the trained target hybrid expert model based on the current inference requirement information; wherein, the target hybrid expert model is the target hybrid expert model trained in the training method of the hybrid expert model according to any one of claims 1-3, the target hybrid expert model contains multiple expert groups, each expert group has a different task positioning, the input of the experts in the expert group is the information to be processed by inference, the information is text, image, video or audio, and each current application expert group is a part of the expert group or the full expert group; The expert group deployment module is used to determine the computing node to which each current application expert group belongs based on the static mapping configuration information, and to deploy each current application expert group to the computing node to which it belongs; specifically, based on the identification information of each expert group in each current application expert group and the identification information of the computing node to which each expert group belongs in the static mapping configuration information, the module determines the computing node to which each expert group in each current application expert group belongs, and deploys each expert group to the computing node to which it belongs. The expert group inference module is used to process the inference request through one or more current application expert groups adapted to the inference request when an inference request corresponding to the target hybrid expert model is obtained, and to provide feedback on the inference result.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that is executed by the at least one processor, which enables the at least one processor to perform the training method of the hybrid expert model according to any one of claims 1-3, or the inference method of the hybrid expert model according to any one of claims 4-5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute and implement the training method of the hybrid expert model according to any one of claims 1-3, or the inference method of the hybrid expert model according to any one of claims 4-5.

Images