Method, device, equipment and product for selecting expert module in hybrid expert model

Abstract

This application relates to the field of large language model technology, and provides a method, apparatus, device, and product for selecting expert modules in a hybrid expert model. An embodiment of this application provides a method for selecting expert modules in a hybrid expert model, which involves retaining the expert module for the first slot corresponding to a lexical unit; determining the relationship between the first parameter of the expert module corresponding to one or more subsequent slots and a score threshold; if the first parameter of the expert module corresponding to the j-th slot is greater than the score threshold, then the expert module corresponding to the j-th slot is retained; otherwise, the expert module corresponding to the j-th slot is discarded, where j≥2. This embodiment can save expert invocation resources. Furthermore, this embodiment requires no training and can achieve a plug-and-play effect, only changing the execution method of the router outputting the top-k list during inference, thus it is applicable to any published MoE checkpoint without retraining or fine-tuning.

Description

Technical Field

[0001] This application belongs to the field of large language model technology, and in particular relates to methods, apparatus, equipment and products for selecting expert modules in a hybrid expert model. Background Technology

[0002] Mixture-of-Experts (MoE) architecture has been widely used in large-scale language models. MoE models use a router to select the most relevant expert module for each token, achieving performance comparable to models with a large number of parameters with fewer activation parameters. In released open-source MoE models, the router scores and ranks the expert modules for each token, producing an ordered list of the top k expert modules. However, standard inference interfaces can only use this ranking to select experts, meaning a fixed number (k) of expert modules are always used for computation. This inference approach using a fixed top-k expert modules wastes computational resources. Summary of the Invention

[0003] This application provides a method, apparatus, device, and product for selecting expert modules in a hybrid expert model, which can solve the technical problem of wasted computing resources caused by using a fixed number of expert modules for inference in existing hybrid expert models.

[0004] In a first aspect, embodiments of this application provide a method for selecting expert modules in a hybrid expert model, including: The expert module for the first slot corresponding to the word element is retained; Determine the relationship between the first parameter of the expert module corresponding to one or more subsequent slots and the score threshold. If the first parameter of the expert module corresponding to the j-th slot is greater than the score threshold, then retain the expert module corresponding to the j-th slot; otherwise, discard the expert module corresponding to the j-th slot, where j≥2.

[0005] In one possible implementation of the first aspect, the method further includes: Save the hybrid expert model; The hybrid expert model is calibrated offline to obtain the score threshold.

[0006] In one possible implementation of the first aspect, the offline calibration of the hybrid expert model to obtain the score threshold includes: Collect unlabeled routing trajectories, use unlabeled prediction to perform forward inference on the hybrid expert model, and obtain triple data of terms, layers, and slots, as well as the routing probability of each slot; Based on the triplet data of the term, layer, and slot, and the routing probability of each slot, determine the first parameter of each slot; The score threshold is determined based on the user-specified budget.

[0007] In one possible implementation of the first aspect, the method further includes: The expert modules that retain all lexical units within the same layer are grouped according to the expert's identity; When receiving lexical units within the same layer, perform forward computation of the same group of expert modules in batches.

[0008] Secondly, embodiments of this application provide an apparatus for selecting expert modules in a hybrid expert model, comprising: The first retention module is used to retain the expert module for the first slot corresponding to the word character; The second judgment module is used to determine the relationship between the first parameter of the expert module corresponding to one or more subsequent slots and the score threshold. If the first parameter of the expert module corresponding to the j-th slot is greater than the score threshold, the expert module corresponding to the j-th slot is retained; otherwise, the expert module corresponding to the j-th slot is discarded, where j≥2.

[0009] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the electronic device performs the method as described in any one of the first aspects above.

[0010] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method as described in any one of the first aspects above.

[0011] Fifthly, embodiments of this application provide a computer program product, including a computer program, which, when run, causes the method as described in any one of the first aspects above to be performed.

[0012] The beneficial effects of the first aspect of this application compared with the prior art are: This application provides a method for selecting expert modules in a hybrid expert model. The method retains the expert module for the first slot corresponding to a word element. It determines the relationship between the first parameter of the expert module for one or more subsequent slots and a score threshold. If the first parameter of the expert module for the j-th slot is greater than the score threshold, the expert module for the j-th slot is retained; otherwise, it is discarded, where j ≥ 2. This application can save expert access resources. Furthermore, this application requires no training and can achieve a plug-and-play effect. It does not require modification of the router parameters, expert weights, or parameters of the dense layer or Transformer layer in the hybrid expert model. It only changes the execution method of outputting the top-k list to the router during inference. Therefore, it is applicable to any published MoE checkpoint without retraining or fine-tuning.

[0013] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart illustrating a method for selecting expert modules in a hybrid expert model according to an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a device for selecting expert modules in a hybrid expert model according to an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0016] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0017] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0018] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0019] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0020] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0021] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0022] Mixture-of-Experts (MoE) is a neural network architecture where each Transformer layer contains multiple independent feedforward network sub-modules (called "experts" or "expert modules"). MoE includes a gating network called a "router." The router generates and ranks the scores of each expert for each token, selecting the top k experts for actual computation. Therefore, the router can selectively activate some experts in the MoE network based on the input token, thereby increasing model capacity while controlling the computational cost per token.

[0023] Existing hybrid expert networks have the following technical problems.

[0024] First, the fixed top-k inference method wastes computational resources. The released hybrid expert models perform a fixed number of expert calculations for all tokens (e.g., the first model performs 8 calculations, the second model performs 6 calculations), but different tokens have different requirements for experts—simple tokens may only require 1-2 experts to be fully modeled, and performing a fixed number of k experts leads to a large amount of redundant computation.

[0025] Second, existing dynamic expert activation methods require training and are not applicable to published models. Existing dynamic expert activation methods include, for example, Adaptive MoE (AdaMoE), Dynamic MoE (DynMoE), and Deep-Adaptive MoE (DA-MoE). These methods reduce computation by learning adaptive budgets for tokens during training, inserting zero-computation experts, or setting threshold policies. However, these methods require training routers, modifying expert weights, or performing self-distillation, which is infeasible for published checkpoints—model owners typically cannot retrain for every published version.

[0026] Third, post-training methods cannot utilize the marginal information already present in the frozen routers. Existing work mainly focuses on how to train new computational allocation strategies, but ignores the reusable marginal expert ranking signals already embedded in the frozen top-k routers. Currently, there is a lack of a method to directly extract this signal from the routers of a published model and transform it into deployable computational budget control.

[0027] Fourth, there is a lack of a standardized deployment auditing framework. Existing MoE acceleration methods typically declare expert call reduction, quality preservation, and system throughput as an inseparable whole, but there are important separations among them: the effectiveness of routing allocation signaling, the usefulness of budget selection, and whether system scheduling can translate the saved expert calls into actual acceleration are three independent issues. Existing technologies lack a standardized interface for auditing these three aspects separately.

[0028] This application provides a method for selecting expert modules in a hybrid expert model, which can solve the technical problem of wasted computational resources caused by using a fixed number of expert modules for inference in existing hybrid expert models. More specifically, the technical problem that this application can solve is: how to achieve token-level adaptive inference computation control by freezing the existing ranking information in the router without modifying any parameters of the published MoE model, such as routers, expert weights, and fully connected layers (dense layers), thereby significantly reducing the number of expert calls while maintaining the quality of model output.

[0029] Figure 1 This is a flowchart illustrating a method for selecting expert modules in a hybrid expert model according to an embodiment of this application.

[0030] S11, retain the expert module for the first slot corresponding to the word element.

[0031] Here, a fixed list of the top k (top-k) expert modules corresponding to each term can be saved. When a term is received, the expert module in the first slot is reserved by default, and prefix closure constraint is enforced.

[0032] S12, determine the relationship between the first parameter of the expert module corresponding to one or more subsequent slots and the score threshold. If the first parameter of the expert module corresponding to the j-th slot is greater than the score threshold, then retain the expert module corresponding to the j-th slot; otherwise, discard the expert module corresponding to the j-th slot, where j≥2.

[0033] "Subsequent slots (one or more)" refers to the 2nd slot, 3rd slot, ... up to the kth slot. After reserving the expert module for the 1st slot, the first parameters of the expert modules corresponding to the 2nd and 3rd slots can be determined sequentially according to the order of the expert modules. The first parameter of the expert module for the jth slot can be expressed as... The score threshold can be expressed as If the first parameter of the expert module corresponding to the j-th slot is greater than the score threshold If the j-th slot is selected, the expert module corresponding to the j-th slot is retained; otherwise, the expert module corresponding to the j-th slot is discarded. According to step S12, the set of retained expert modules conforms to the prefix closure rule, that is, if the j-th expert module is retained, then the 1st to (j-1st)th expert modules must also be retained.

[0034] The first parameter includes, but is not limited to, prefix-normalized marginal score, original route weight (also known as original route probability), and entropy-marginal score. Score thresholds include, but are not limited to, globally shared thresholds, slot-specific independent thresholds, and layer-specific independent thresholds. Furthermore, a trained shadow price optimizer can be used instead of an untrained thresholding strategy.

[0035] The method for selecting expert modules in a hybrid expert model provided in this application embodiment can be called a Calibrated Expert Deferral-Mixture-of-Experts (CED-MoE). The output of the CED-MoE model can be represented by the following calculation formula (1).

[0036] (1) in, This represents the output of the CED-MoE model. The execution indicator variable for the j-th slot ( A value of 1 indicates that the expert module is retained. A value of 0 indicates that the expert module is discarded. Indicates the original route weight. This represents the output of the j-th expert module to the hidden state h.

[0037] Here, a key design decision can be made for this embodiment: do not renormalize the route weights. Specifically, the contribution of slots that are delayed (discarded, not executed) is 0, and the route weights of the expert modules can be retained as their original values ​​without renormalization. This ensures that: First, the contribution magnitude of the expert modules is completely consistent with the original inference. Second, the model does not need to adapt to the new route distribution. Third, the output of the original inference is accurately restored when all are retained. This embodiment enforces the prefix closure constraint, that is, the execution of subsequent expert modules cannot be skipped, and the route weights of the experts are retained without renormalization. The design of this embodiment makes the method precisely reversible, achieving the effect that full retention equals the original inference, and avoiding the route distribution offset that may be introduced by renormalization.

[0038] This application provides a method for selecting expert modules in a hybrid expert model. The method retains the expert module for the first slot corresponding to a word element. It determines the relationship between the first parameter of the expert module for one or more subsequent slots and a score threshold. If the first parameter of the expert module for the j-th slot is greater than the score threshold, the expert module for the j-th slot is retained; otherwise, it is discarded, where j ≥ 2. This application can save expert access resources. Furthermore, this application requires no training and can achieve a plug-and-play effect. It does not require modification of the router parameters, expert weights, or parameters of the dense layer or Transformer layer in the hybrid expert model. It only changes the execution method of outputting the top-k list to the router during inference. Therefore, it is applicable to any published MoE checkpoint without retraining or fine-tuning.

[0039] Verification shows that in the mathematical reasoning task of the third model, the method provided in this application embodiment can save 61.2% of expert call resources and improve accuracy from 43.52% to 46.25%. In the language modeling task, it can save 74.7% of expert calls and improve throughput by 2.97 times in the "speed" setting. Furthermore, the method provided in this application embodiment can achieve cross-model and cross-architecture portability. The reasoning and calibration methods provided in this application embodiment have been verified to be effective on three different hybrid expert family models, with Spearman correlation coefficients of 0.862, 0.980, and 0.802 for prefix-normalized marginal scores and local output impairments, respectively.

[0040] In one embodiment, the method for selecting expert modules in a hybrid expert model further includes the following steps S21 and S22.

[0041] S21, save the hybrid expert model.

[0042] Saving the hybrid expert model is equivalent to freezing the ordered expert list of the router in the hybrid expert model, which contains a fixed top-k expert modules displayed in order.

[0043] S22, perform offline calibration of the hybrid expert model to obtain the score threshold.

[0044] Here, by collecting the route trajectory, calculating the first parameter, and calibrating the user budget and score threshold, a score threshold corresponding to the user-specified budget can be obtained.

[0045] This application addresses the technical problem of extracting reusable marginal expert ranking signals from an ordered, fixed list of the top k expert modules of a frozen router. This application discovers and utilizes the marginal ranking signal of a frozen router in a hybrid expert model. This application demonstrates for the first time, through controlled variable experiments (controlling original probabilities, layer positions, slot positions, and random shuffling within tokens), that a reusable local marginal expert ranking signal is encoded in the top-k ordered list of routers in a published MoE model—the routing score of subsequent slots can predict the damage to the layer output caused by delaying that slot. The partial Spearman correlation coefficient is 0.634, while the random shuffling control group is only 0.140.

[0046] In one embodiment, S22 of the above embodiment involves offline calibration of the hybrid expert model to obtain a score threshold, including the following S221 to S223.

[0047] S221, collect unlabeled routing trajectories, use unlabeled prediction to perform forward inference on the hybrid expert model, and obtain triple data of terms, layers, and slots, as well as the routing probability of each slot.

[0048] Without task labels, a small amount of unlabeled corpus can be used to perform a forward inference on the stored target MoE model, recording the router output for each token in each layer—that is, the top-k expert indices and the corresponding router probabilities (p1≥p2≥…≥p_k). The storage cost for each record is O(k), and there is no need to save intermediate activations or gradients.

[0049] S222: Determine the first parameter of each slot based on the triplet data of word, layer, and slot, as well as the routing probability of each slot.

[0050] The first parameter includes, but is not limited to, prefix normalized marginal score, original route weight (which can also be called original route probability), and entropy-marginal score. Here, the prefix normalized marginal score is used as the first parameter for example. The prefix normalized marginal score can be expressed as follows (2).

[0051] (2) in, This represents the prefix-normalized marginal score for each slot j. The denominator represents the original routing probability of the j-th slot. This represents the total routing probability of the top j experts whose tokens have been retained. To prevent division by zero for small constants.

[0052] The physical meaning of the above calculation formula (2) is: given that the first j experts have already been retained, how much more routing quality does the j-th expert contribute relative to the retained prefixes? This design solves the problem of the original probability not being able to cross tokens. That is, for some tokens, the top-1 experts are absolutely dominant, and for some tokens, the expert scores are close to a uniform distribution. If the original probability is used directly... Then it is impossible to distinguish between these two situations.

[0053] This application embodiment can reinterpret the ordered top-k list of the frozen MoE router as an implicit expert prefix cascade, and determine whether each subsequent slot is worth executing by using the marginal score of prefix normalization, thereby achieving variable depth expert execution at the token level.

[0054] S223, determine the score threshold based on the user-specified budget.

[0055] The user-specified budget includes, but is not limited to, the user-specified average number of experts, which can be represented by B. The system can search for a threshold from the calibrated trajectory such that the average number of experts retained under the threshold strategy is exactly close to B, as shown in the following calculation formula (3).

[0056] (3) Where C is the set of calibration tokens, To be at the score threshold The number of experts retained for this token. This search is monotonic; the higher the score threshold, the fewer experts are retained. It can be efficiently completed through sorting or one-dimensional search. The calibration process requires no task labels.

[0057] This application addresses the technical problem of converting marginal expert ranking signals into a user-controllable, task-label-free computational budget interface, achieving a user-controllable continuous computational budget. Users only need to specify an intuitive target average number of experts B (such as preset levels for speed, balance, quality, etc.), and this application can automatically convert it into a router score threshold through label-free calibration. This eliminates the need for users to understand the underlying model details, reducing the user's operating costs. The calibration process requires no task labels or training, only a single forward inference with a small amount of unlabeled corpus, thus implementing a mapping mechanism between unlabeled calibration and user budget.

[0058] Furthermore, embodiments of this application propose a prefix-normalized marginal score mechanism. Embodiments of this application propose using... Instead of the original probability This serves as the basis for reserving expert modules. This design makes scores comparable across different tokens, resolving the incomparability of original route probabilities in the two scenarios of "top-1 dominance" and "nearly uniform distribution of experts," while naturally satisfying the prefix closure constraint without requiring additional processing.

[0059] In one embodiment, the method for selecting expert modules in a hybrid expert model further includes the following steps S31 and S32.

[0060] S31, group the expert modules that retain all lexical units in the same layer according to the expert's identity; S32, when receiving lexical units within the same layer, performs forward computation of the same group of expert modules in batches.

[0061] Expert modules that retain all tokens within the same layer can be grouped according to their expert identities. When a token from the same layer is received, the forward computation of the expert modules in the same group is performed in batches, ensuring that reducing expert calls translates into an actual increase in inference throughput.

[0062] In one embodiment, the method for selecting an expert module in a hybrid expert model further includes the following step S41.

[0063] S41, fully retains native recovery verification (pre-deployment gating).

[0064] Before deployment, it can be verified that when the score threshold is set to... When =0 (i.e., retaining all k slots), the output of the method for selecting expert modules in the hybrid expert model provided in this application embodiment is completely consistent with the native top-k inference output (maximum logit difference is 0, and top-token consistency rate is 1.0). This gating method can ensure that the budget interface has accurate native inference endpoints.

[0065] The embodiments of this application can guarantee native recovery and achieve precise reversibility. When the user selects the full-preservation mode ( When the logit difference is 0, the CED-MoE output is mathematically equivalent to the native top-k inference result—the maximum logit difference is 0, the top-token consistency rate is 1.0, and it provides a path for security degradation in deployment.

[0066] In one embodiment, auditing boils down to three checks. Router rule measurement and oracle prefix capture support allocation, but are limited to the local portion of the routing layer output. Precise prefix and score control with matching depth support budget selection, but not task-level improvements. Guarded subset and grouping model scheduling are supported for deployment only when risk calibration and the service kernel translate saved expert calls into speed.

[0067] This application addresses the technical challenge of establishing a standardized auditing framework that separately evaluates three dimensions: the effectiveness of route allocation, the advantages of budget selection relative to fixed prefixes, and whether system scheduling translates saved expert calls into actual acceleration. This application proposes a three-dimensional separate audit of route allocation signaling (Gate I), budget selection (Gate II), and system scheduling (Gate III), enabling deployment effects to be independently verified and attributed—clearly answering the three independent questions: "Is the signal effective?", "Is the selection useful?", and "Does the system accelerate?".

[0068] Figure 2 This is a schematic diagram of the structure of a device for selecting expert modules in a hybrid expert model according to an embodiment of this application.

[0069] like Figure 2 As shown, the device includes a first retention module 21 and a second judgment module 22.

[0070] The first retention module 21 is used to retain the expert module of the first slot corresponding to the word character; The second judgment module 22 is used to determine the relationship between the first parameter of the expert module corresponding to one or more subsequent slots and the score threshold. If the first parameter of the expert module corresponding to the j-th slot is greater than the score threshold, the expert module corresponding to the j-th slot is retained; otherwise, the expert module corresponding to the j-th slot is discarded, where j≥2.

[0071] Another embodiment of the present invention discloses an apparatus. This embodiment is based on the above... Figure 2 Based on the corresponding embodiments, the device further includes: The third storage module is used to store the hybrid expert model; The fourth module is used to perform offline calibration of the hybrid expert model and obtain the score threshold.

[0072] Another embodiment of the present invention discloses an apparatus. This embodiment is based on the above... Figure 2 Based on the corresponding embodiment, the fourth obtaining module is used for: Collect unlabeled routing trajectories, use unlabeled prediction to perform forward inference on the hybrid expert model, and obtain triple data of terms, layers, and slots, as well as the routing probability of each slot; Based on the triplet data of word, layer, and slot, and the routing probability of each slot, determine the first parameter of each slot; The score threshold is determined based on the user-specified budget.

[0073] Another embodiment of the present invention discloses an apparatus. This embodiment is based on the above... Figure 2 Based on the corresponding embodiments, the device further includes: The fifth grouping module is used to group all expert modules that retain lexical units within the same layer according to the expert's identity; The sixth execution module is used to perform forward computation of the same group of expert modules in batches when receiving lexical units within the same layer.

[0074] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0075] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0076] This application also provides an electronic device. For example... Figure 3 As shown, the electronic device 3 includes: at least one processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the at least one processor 30, wherein the processor 30 executes the computer program 32 to implement the steps in any of the above-described method embodiments.

[0077] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0078] This application provides a computer program product, which includes a computer program that, when run, causes the steps described in the above method embodiments to be performed.

[0079] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographic device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0080] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0081] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0082] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0083] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0084] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for selecting expert modules in a hybrid expert model, characterized in that, include: The expert module for the first slot corresponding to the word element is retained; Determine the relationship between the first parameter of the expert module corresponding to one or more subsequent slots and the score threshold. If the first parameter of the expert module corresponding to the j-th slot is greater than the score threshold, then retain the expert module corresponding to the j-th slot; otherwise, discard the expert module corresponding to the j-th slot, where j≥2.

2. The method as described in claim 1, characterized in that, The method further includes: Save the hybrid expert model; The hybrid expert model is calibrated offline to obtain the score threshold.

3. The method as described in claim 2, characterized in that, The offline calibration of the hybrid expert model to obtain the score threshold includes: Collect unlabeled routing trajectories, use unlabeled prediction to perform forward inference on the hybrid expert model, and obtain triple data of terms, layers, and slots, as well as the routing probability of each slot; Based on the triplet data of the term, layer, and slot, and the routing probability of each slot, determine the first parameter of each slot; The score threshold is determined based on the user-specified budget.

4. The method as described in claim 1, characterized in that, The method further includes: The expert modules that retain all lexical units within the same layer are grouped according to the expert's identity; When receiving lexical units within the same layer, perform forward computation of the same group of expert modules in batches.

5. An apparatus for selecting expert modules in a hybrid expert model, characterized in that, include: The first retention module is used to retain the expert module for the first slot corresponding to the word character; The second judgment module is used to determine the relationship between the first parameter of the expert module corresponding to one or more subsequent slots and the score threshold. If the first parameter of the expert module corresponding to the j-th slot is greater than the score threshold, the expert module corresponding to the j-th slot is retained; otherwise, the expert module corresponding to the j-th slot is discarded, where j≥2.

6. The apparatus as claimed in claim 5, characterized in that, The device further includes: The third storage module is used to store the hybrid expert model; The fourth module is used to perform offline calibration on the hybrid expert model to obtain the score threshold.

7. The apparatus as claimed in claim 6, characterized in that, The fourth obtaining module is used for: Collect unlabeled routing trajectories, use unlabeled prediction to perform forward inference on the hybrid expert model, and obtain triple data of terms, layers, and slots, as well as the routing probability of each slot; Based on the triplet data of the term, layer, and slot, and the routing probability of each slot, determine the first parameter of each slot; The score threshold is determined based on the user-specified budget.

8. The apparatus as claimed in claim 5, characterized in that, The device further includes: The fifth grouping module is used to group all expert modules that retain lexical units within the same layer according to the expert's identity; The sixth execution module is used to perform forward computation of the same group of expert modules in batches when receiving lexical units within the same layer.

9. An electronic device, characterized in that, The device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device performs the method as described in any one of claims 1-4.

10. A computer program product, characterized in that, Includes a computer program, which, when run, causes the method as described in any one of claims 1-4 to be performed.

Images