Model optimization method and apparatus, and information processing method and apparatus

Abstract

Provided are a model optimization method and apparatus, and an information processing method and apparatus. The model optimization method comprises: inputting a plurality of tokens into a mixture-of-experts model, and by means of a router in the mixture-of-experts model, calculating a target entropy value corresponding to each token; on the basis of the target entropy values, dividing the plurality of tokens into global tokens and local tokens; broadcasting the global tokens to all expert sub-models in the mixture-of-experts model for processing, and broadcasting the local tokens to expert sub-models, which match the local tokens, in the mixture-of-experts model for processing; and on the basis of processing results, adjusting parameters of the mixture-of-experts model until a target mixture-of-experts model which meets an optimization stop condition is obtained.

Description

Model optimization methods and apparatus, information processing methods and apparatus

[0001] This application claims priority to Chinese Patent Application No. 202411191438.2, filed on August 28, 2024, entitled “Model Optimization Method and Apparatus, Information Processing Method and Apparatus”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The embodiments in this specification relate to the field of machine learning technology, and in particular to model optimization methods and apparatus, and information processing methods and apparatus. Background Technology

[0003] With the rapid growth of computing power and data volume, large-scale language models (LLMs) have been widely developed and applied. These models have demonstrated outstanding performance in fields such as natural language processing, computer vision, and recommender systems. Their core technologies include the Transformer architecture, attention mechanisms, and large-scale pre-training. However, as the scale of model parameters continues to expand, the computational cost and energy consumption of the models have become increasingly prominent. To address this issue, the Mixture of Experts (MoE) model has been proposed. The MoE model achieves computational sparsity by dynamically selecting and activating a subset of experts. Specifically, the MoE model consists of a router and multiple experts, with each input activating only a small number of experts, significantly reducing computational costs. While existing MoE models excel in reducing computational costs, they suffer from uncertainty in the expert selection process. This uncertainty mainly manifests as some tokens having similar scores when selecting experts, leading to increased randomness in the selection and thus affecting model performance. Therefore, an effective solution is urgently needed to address these issues. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a model optimization method. One or more embodiments of this specification also relate to an information processing method, a model optimization apparatus, an information processing device, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a model optimization method is provided, comprising:

[0006] Multiple word segmentation units are input into a hybrid expert model, and the target entropy value corresponding to each word segmentation unit is calculated through the router in the hybrid expert model.

[0007] Based on the target entropy value, the multiple word segmentation units are divided into global word segmentation units and local word segmentation units;

[0008] The global word segmentation unit is broadcast to all expert sub-models in the hybrid expert model for processing, and the local word segmentation unit is broadcast to the expert sub-models in the hybrid expert model that match the local word segmentation unit for processing;

[0009] Based on the processing results, the parameters of the hybrid expert model are adjusted until a target hybrid expert model that meets the optimization stopping condition is obtained.

[0010] Optionally, the calculation of the target entropy value corresponding to any one of the plurality of word segmentation units includes:

[0011] The router in the hybrid expert model is used to calculate the selection score of each expert sub-model in the hybrid expert model for the word segmentation unit;

[0012] The initial entropy value of the selection score corresponding to each expert sub-model is determined, and the target entropy value corresponding to the word segmentation unit is obtained by normalizing the initial entropy value.

[0013] Optionally, dividing the plurality of word segmentation units into global word segmentation units and local word segmentation units based on the target entropy value includes:

[0014] The target entropy value corresponding to each word segmentation unit is compared with a preset entropy threshold. Based on the comparison result, word segmentation units with an entropy value greater than the preset entropy threshold are selected as global word segmentation units, and word segmentation units with an entropy value less than or equal to the preset entropy threshold are selected as local word segmentation units; or,

[0015] The multiple word segmentation units are sorted according to the target entropy value corresponding to each word segmentation unit to obtain a word segmentation unit sequence. A global word segmentation unit is selected from the word segmentation unit sequence according to the preset proportion information, and the remaining word segmentation units other than the global word segmentation unit are used as local word segmentation units.

[0016] Optionally, broadcasting the local word segmentation unit to the expert sub-model that matches the local word segmentation unit in the hybrid expert model for processing includes:

[0017] The local word segmentation unit selects the target selection score of each expert submodel in the hybrid expert model;

[0018] Based on the target selection score, a local expert sub-model is determined in the hybrid expert model to match the local word unit, and the local word unit is broadcast to the local expert sub-model for processing.

[0019] Optionally, the step of tuning the parameters of the hybrid expert model based on the processing results until a target hybrid expert model that meets the optimization stopping condition is obtained includes:

[0020] Based on the processing results, the parameters of the hybrid expert model are tuned, and it is checked whether the tuned hybrid expert model meets the optimization stopping condition.

[0021] If not, select multiple candidate word segmentation units as the multiple word segmentation units, and perform the step of inputting the multiple word segmentation units into the hybrid expert model, and calculating the target entropy value corresponding to each word segmentation unit through the router in the hybrid expert model;

[0022] If so, use the parameter-tuned hybrid expert model as the target hybrid expert model.

[0023] Optionally, before the step of inputting multiple word segmentation units into the hybrid expert model is executed, the method further includes:

[0024] Obtain the original sample data associated with the hybrid expert model, preprocess the original sample data, and obtain multiple initial word segmentation units;

[0025] The initial word segmentation units are embedded separately to obtain multiple word segmentation units.

[0026] According to a second aspect of the embodiments of this specification, an information processing method is provided, comprising:

[0027] Obtain the question information submitted by the client, construct multiple question word segmentation units based on the question information, and input them into the target hybrid expert model, wherein the target hybrid expert model is obtained through the above method;

[0028] In the target hybrid expert model, a target expert sub-model matching each question segmentation unit is selected to process the multiple question segmentation units;

[0029] The prediction results of the target expert sub-model matched by each question segmentation unit are merged to obtain the answer information, and the answer information is fed back to the client.

[0030] According to a third aspect of the embodiments of this specification, a model optimization apparatus is provided, comprising:

[0031] The calculation module is configured to input multiple word segmentation units into a hybrid expert model, and calculate the target entropy value corresponding to each word segmentation unit through the router in the hybrid expert model;

[0032] The segmentation module is configured to divide the plurality of word segmentation units into global word segmentation units and local word segmentation units based on the target entropy value;

[0033] The processing module is configured to broadcast the global word segmentation unit to all expert sub-models in the hybrid expert model for processing, and to broadcast the local word segmentation unit to the expert sub-models in the hybrid expert model that match the local word segmentation unit for processing.

[0034] The parameter tuning module is configured to tune the parameters of the hybrid expert model based on the processing results until a target hybrid expert model that meets the optimization stopping condition is obtained.

[0035] According to a fourth aspect of the embodiments of this specification, an information processing apparatus is provided, comprising:

[0036] The information acquisition module is configured to acquire question information submitted by the client, construct multiple question segmentation units based on the question information, and input them into the target hybrid expert model, wherein the target hybrid expert model is obtained through the above method;

[0037] The word segmentation processing module is configured to select a target expert sub-model matching each question segmentation unit in the target hybrid expert model to process the multiple question segmentation units;

[0038] The feedback information module is configured to merge the prediction results of the target expert sub-model matched for each question segmentation unit to obtain the answer information, and then feed the answer information back to the client.

[0039] According to a fifth aspect of the embodiments of this specification, a computing device is provided, comprising:

[0040] Memory and processor;

[0041] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the above-described model optimization method or information processing method.

[0042] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the above-described model optimization method or information processing method.

[0043] According to a seventh aspect of the embodiments of this specification, a computer program product is provided, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described model optimization method or information processing method.

[0044] The model optimization method provided in this embodiment aims to reduce router selection uncertainty, improve model robustness, and thus ensure prediction accuracy. During the model optimization phase, after inputting multiple word segmentation units into the hybrid expert model, the routers within the hybrid expert model first calculate the target entropy value corresponding to each word segmentation unit. Based on this, the multiple word segmentation units can be divided into global word segmentation units and local word segmentation units according to the target entropy values. Then, the global word segmentation units can be broadcast to all expert sub-models in the hybrid expert model for processing, while the local word segmentation units are broadcast to the expert sub-models that match the local word segmentation units for processing. This allows uncertain word segmentation units to be broadcast to all expert sub-models for learning during the model optimization phase, while certain word segmentation units are still used for learning by the matching expert sub-models. This enables parameter tuning of the hybrid expert model based on the processing results to obtain a target hybrid expert model that meets the optimization stopping condition. During the fine-tuning of the hybrid expert model, all expert sub-models learn the relevant knowledge of uncertain word segmentation units, ensuring better robustness of the expert sub-models in the application phase. Attached Figure Description

[0045] Figure 1 is a schematic diagram of a model optimization method provided in one embodiment of this specification;

[0046] Figure 2 is a flowchart of a model optimization method provided in one embodiment of this specification;

[0047] Figure 3 is a flowchart of an information processing method provided in one embodiment of this specification;

[0048] Figure 4 is a flowchart of the processing procedure of an information processing method provided in one embodiment of this specification;

[0049] Figure 5 is a schematic diagram of a model optimization device provided in one embodiment of this specification;

[0050] Figure 6 is a schematic diagram of the structure of an information processing device provided in one embodiment of this specification;

[0051] Figure 7 is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0052] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0053] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0054] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0055] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0056] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0057] Mixture of Experts (MoE) model: This is an ensemble learning method that combines multiple specialized sub-models (i.e., "experts") into a single overall model, where each "expert" contributes within its area of ​​expertise.

[0058] LLMs (Large Language Models) refer to large-scale language models, which are an important class of models in the field of Natural Language Processing (NLP). These models are typically based on deep learning techniques, especially the Transformer architecture, such as extensions or variants of models like BERT and GPT. LLMs are capable of processing and understanding complex natural language text, performing generative tasks (such as text generation and dialogue generation) and analytical tasks (such as sentiment analysis and information extraction).

[0059] This specification provides a model optimization method. One or more embodiments of this specification also relate to an information processing method, a model optimization apparatus, an information processing device, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0060] In practical applications, the standard MoE model selects several experts with the highest scores based on router scores for computation. However, in real-world applications, router scores are often close, especially when dealing with complex data. These uncertain tokens having similar scores when selecting experts makes it impossible to determine the optimal expert, potentially leading to the selection of the wrong expert and impacting model performance. Therefore, an effective solution is urgently needed to address these issues.

[0061] Referring to the schematic diagram in Figure 1, the model optimization method provided in this embodiment, in order to reduce router selection uncertainty, improve model robustness, and thus ensure prediction accuracy, can, during the model optimization stage, first calculate the target entropy value corresponding to each word segmentation unit by inputting multiple word segmentation units into the hybrid expert model. Based on this, the multiple word segmentation units can be divided into global word segmentation units and local word segmentation units according to the target entropy value. Then, the global word segmentation units can be broadcast to all expert sub-models in the hybrid expert model for processing, while the local word segmentation units can be broadcast to the expert sub-models that match the local word segmentation units for processing. This allows uncertain word segmentation units to be broadcast to all expert sub-models for learning during the model optimization stage, while certain word segmentation units are still used for learning by the matching expert sub-models. This enables parameter tuning of the hybrid expert model based on the processing results to obtain a target hybrid expert model that meets the optimization stopping condition. During the fine-tuning process of the hybrid expert model, all expert sub-models learn the relevant knowledge of uncertain word segmentation units, ensuring better robustness of the expert sub-models in the application stage.

[0062] Referring to Figure 2, which shows a flowchart of a model optimization method provided according to an embodiment of this specification, the method includes the following steps.

[0063] Step S202: Input multiple word segmentation units into the hybrid expert model, and calculate the target entropy value corresponding to each word segmentation unit through the router in the hybrid expert model.

[0064] The model optimization method provided in this embodiment is applied to the fine-tuning task of a hybrid expert model (MoE model). After optimizing the hybrid expert model, even if the router's calculated scores are close and the optimal expert sub-model cannot be determined, the selected expert sub-model can still provide a more accurate prediction result, thereby effectively improving the model's robustness. The hybrid expert model can be a MoE model applicable to any domain, such as natural language processing (hybrid expert models are used for text classification, text generation, translation, etc.), computer vision (hybrid expert models are used for image classification, object detection, image generation, etc.), and recommendation systems (hybrid expert models are used for personalized recommendations), etc. This embodiment does not impose any limitations. This embodiment uses the use of a hybrid expert model for sentiment analysis of input text as an example to illustrate the model optimization method. Descriptions in other scenarios can refer to the same or corresponding descriptions in this embodiment, which will not be elaborated upon further here.

[0065] Specifically, multiple segmentation units refer to the multiple tokens obtained after segmenting the samples during the fine-tuning stage of the hybrid expert model. These tokens can be character units and / or word units from the samples. Correspondingly, the hybrid expert model refers to a MoE model that includes a router and multiple expert sub-models. The router is used to compute the expert sub-model matched for each segmentation unit, enabling the segmentation unit to be submitted to the matched expert sub-model for processing during the application phase. Accordingly, the target entropy value specifically refers to the result obtained after calculating the entropy value of each word segmentation unit using the router. The target entropy value is used to measure the certainty of the word segmentation unit's selection of the expert sub-model. The higher the entropy value, the worse the certainty. That is, it reflects that after the word segmentation unit selects the expert sub-model, the probability that the expert sub-model is the best matching word segmentation unit in the hybrid expert model is uncertain. It may be the best, or it may not be. Therefore, it is necessary to use the model optimization method provided in this embodiment to broadcast this type of word segmentation unit to all expert sub-models for relevant knowledge learning, thereby eliminating the problem of decreased prediction accuracy caused by uncertainty. This ensures that in the application stage, if an uncertain token is encountered, after assigning an expert sub-model to it, the expert sub-model can still give an accurate prediction result for the uncertain token.

[0066] Based on this, in order to reduce the uncertainty of router selection, improve model robustness, and thus ensure prediction accuracy, during the model optimization stage, after inputting multiple word segmentation units into the hybrid expert model, the target entropy value corresponding to each word segmentation unit can be calculated using the router in the hybrid expert model. The target entropy value is used to measure the certainty of each word segmentation unit in selecting the expert sub-model, which facilitates the subsequent classification of word segmentation unit types and fine-tunes the hybrid expert model into a target hybrid expert model that meets the usage requirements.

[0067] Furthermore, to ensure that the fine-tuned hybrid expert model meets the requirements, the word segmentation units input into the hybrid expert model for learning relevant knowledge can be obtained by processing the original sample data. In this embodiment, the specific implementation is as follows:

[0068] Obtain the original sample data associated with the hybrid expert model, preprocess the original sample data to obtain multiple initial word segmentation units, and embed the multiple initial word segmentation units respectively to obtain multiple word segmentation units.

[0069] Specifically, raw sample data refers to text samples used for fine-tuning the hybrid expert model. Correspondingly, preprocessing refers to the process of segmenting and encoding the raw sample data. And correspondingly, embedding refers to the process of converting the initial segmented units into vector representations.

[0070] Based on this, after obtaining the original sample data of the hybrid expert model, the original sample data can be preprocessed to generate multiple initial word segmentation units based on the preprocessing results. On this basis, in order to enable processing after inputting into the hybrid expert model, the multiple initial word segmentation units can be embedded separately to obtain multiple word segmentation units, so that the hybrid expert model can be fine-tuned by combining multiple word segmentation units in the future.

[0071] In summary, by preprocessing the original sample data, we can ensure that the sample data is converted into multiple word segmentation units that meet the input requirements of the hybrid expert model, thereby achieving the purpose of model fine-tuning.

[0072] Furthermore, in calculating the target entropy value corresponding to a word segmentation unit, to accurately reflect the determinism of the word segmentation unit's selection of each expert sub-model through the target entropy value, it can be achieved in the following way. In this embodiment, the calculation of the target entropy value corresponding to any one of the plurality of word segmentation units includes:

[0073] The router in the hybrid expert model calculates the selection score of each expert sub-model in the hybrid expert model for the word segmentation unit; the initial entropy value of the selection score corresponding to each expert sub-model is determined, and the target entropy value corresponding to the word segmentation unit is obtained by normalizing the initial entropy value.

[0074] Specifically, the selection score refers to the probability score of the word segmentation unit selecting each expert sub-model, calculated using a router. The higher the score, the higher the probability that the expert sub-model is selected. Correspondingly, the initial entropy value refers to the entropy value calculated based on the selection score, used to indicate the certainty of the word segmentation unit selecting each expert sub-model. By normalizing it, the target entropy value, which reflects the certainty of the word segmentation unit selecting the expert sub-model, can be obtained.

[0075] Based on this, for any word segmentation unit, the selection score of each expert sub-model in the hybrid expert model can be calculated first through the router in the hybrid expert model. On this basis, the initial entropy value of the selection score corresponding to each expert sub-model can be determined first. Then, by normalizing the initial entropy value, all the initial entropy values ​​can be merged to obtain the target entropy value that reflects the selection of the expert sub-model by the word segmentation unit, so as to facilitate the subsequent division of multiple word segmentation units.

[0076] In practical applications, after the router calculates the selection score corresponding to the word segmentation unit, it can calculate the initial entropy value using the following formula (1): H(x)=-∑igilog(gi) (1)

[0077] Where x represents a word segmentation unit, gi represents the selection score of the i-th expert sub-model selected by the word segmentation unit, and H(x) represents the initial entropy value corresponding to the selection score.

[0078] After obtaining the initial entropy value, the initial entropy values ​​corresponding to all expert sub-models can be normalized to obtain the normalized entropy value, i.e., the target entropy value, which can be calculated using the following formula (2): H norm (x)=[H(x)] / [log(N)] (2)

[0079] Among them, H norm (x) represents the target entropy value, and N represents the total number of expert sub-models.

[0080] For example, to improve the predictive performance of the MoE model for text sentiment analysis, the MoE model will be fine-tuned. The MoE model contains three expert sub-models: expert sub-model a excels at handling text containing positive words and outputs a positive sentiment prediction; expert sub-model b excels at handling text containing negative words and outputs a negative sentiment prediction; and expert sub-model c excels at handling neutral text and outputs a neutral prediction. Based on this, N tokens are obtained and input into the MoE model. The router in the MoE model calculates the scores of each token when selecting the three expert sub-models. For example, token 1's score for expert sub-model a is Sa1, token 1's score for expert sub-model b is Sb1, and token 1's score for expert sub-model c is Sc1; token 2's score for expert sub-model a is Sa2… token N's score for expert sub-model c is ScN. Then, by combining the above formulas (1) and (2), the normalized entropy value H1 of token1 relative to all expert sub-models and the normalized entropy value HN of tokenN relative to all expert sub-models can be calculated, so as to distinguish different types of tokens by combining the normalized entropy value corresponding to each token, and to complete the fine-tuning of the MoE model.

[0081] In summary, by calculating the entropy value to reflect the determinism of each word segmentation unit in selecting the expert sub-model, it is possible to filter out uncertain tokens. This facilitates the broadcasting of these tokens to all expert sub-models during the fine-tuning stage for relevant knowledge learning, thus making the model more robust.

[0082] Step S204: Divide the multiple word segmentation units into global word segmentation units and local word segmentation units according to the target entropy value.

[0083] Specifically, after obtaining the target entropy value corresponding to each word segmentation unit, considering that the target entropy value is used to reflect the determinism of each word segmentation unit in selecting the expert sub-model, the word segmentation unit type can be divided according to the target entropy value corresponding to each word segmentation unit. This allows for the selection of word segmentation units with high uncertainty as global word segmentation units and word segmentation units with low uncertainty as local word segmentation units, which facilitates subsequent fine-tuning of the hybrid expert model.

[0084] In this context, a global segmentation unit refers to a segmentation unit with greater uncertainty among multiple segmentation units. This means that such a unit exhibits greater uncertainty when the router selects its matching expert sub-model. Conversely, a local segmentation unit refers to a segmentation unit that can accurately select its matching expert sub-model among multiple segmentation units; this type of unit has less uncertainty.

[0085] Furthermore, after obtaining the target entropy value for each word segmentation unit, the division into local and global word segmentation units is actually based on the entropy value, which determines the deterministic and uncertain aspects of the word segmentation units. In practice, this can be achieved in the following two ways:

[0086] (1) Compare the target entropy value corresponding to each word segmentation unit with the preset entropy value threshold. Based on the comparison result, select word segmentation units that are greater than the preset entropy value threshold as global word segmentation units, and select word segmentation units that are less than or equal to the preset entropy value threshold as local word segmentation units.

[0087] (2) Sort the multiple word segmentation units according to the target entropy value corresponding to each word segmentation unit to obtain a word segmentation unit sequence. Select a global word segmentation unit in the word segmentation unit sequence according to the preset proportion information, and take the remaining word segmentation units other than the global word segmentation unit as local word segmentation units.

[0088] Specifically, the preset entropy threshold refers to the threshold for filtering uncertain and certain word segmentation units, which can be set according to actual needs. This embodiment does not impose any limitations on this. Correspondingly, the word segmentation unit sequence refers to the sequence obtained by sorting multiple word segmentation units according to the target entropy value. The preset proportion information refers to the proportion of the number of selected word segmentation units, which can be a specific number or a percentage value. This embodiment does not impose any limitations on this.

[0089] Based on this, after obtaining the target entropy value that can reflect the determinism of the expert sub-model for selecting word segmentation units, on the one hand, the target entropy value corresponding to each word segmentation unit can be compared with the preset entropy threshold. Based on the comparison result, word segmentation units that are greater than the preset entropy threshold are selected as global word segmentation units, and word segmentation units that are less than or equal to the preset entropy threshold are selected as local word segmentation units. This enables the subsequent fine-tuning of the hybrid expert model by combining global word segmentation units and local word segmentation units.

[0090] On the other hand, multiple word segmentation units can be sorted according to the target entropy value corresponding to each word segmentation unit to obtain a word segmentation unit sequence. Based on this, global word segmentation units can be selected from the word segmentation unit sequence according to the preset proportion information. In the selection process, the word segmentation units in the sequence can be arranged in the order of the word segmentation units. Finally, the remaining word segmentation units other than the global word segmentation units can be used as local word segmentation units.

[0091] In practice, when sorting multiple word segmentation units, they can be sorted in descending order of target entropy value to form a word segmentation unit sequence. Based on this, a global word segmentation unit can be selected, in descending order of the word segmentation unit corresponding to the preset proportion information as the global word segmentation unit.

[0092] In practical applications, when dividing word segments according to the target entropy value, the processing method that is appropriate for the current scenario can be selected according to actual needs. This embodiment does not impose any limitations on this.

[0093] Continuing with the previous example, after obtaining the normalized entropy values ​​corresponding to the N tokens, we can further sort the N tokens from highest to lowest according to their normalized entropy values. Then, we select the top 5% of tokens by entropy value as uncertain tokens, including {token1, token2, token4, ...}. Next, we take the remaining N tokens (excluding the uncertain tokens) as certain tokens, including {token3, token5, token6, ...}. We can then combine the uncertain and certain tokens to fine-tune the MoE model.

[0094] In summary, by choosing different methods to divide global and local word segmentation units, it is possible to ensure that the model can select different strategies to optimize different types of word segmentation units during the subsequent model fine-tuning stage, thereby enabling the model to have stronger predictive capabilities.

[0095] Step S206: Broadcast the global word segmentation unit to all expert sub-models in the hybrid expert model for processing, and broadcast the local word segmentation unit to the expert sub-models in the hybrid expert model that match the local word segmentation unit for processing.

[0096] Specifically, after determining the global and local segmentation units, to further address the issue of inaccurate prediction results caused by poor expert sub-model selection when the router chooses segmentation units, the global segmentation unit can be broadcast to all expert sub-models in the hybrid expert model during the model fine-tuning phase. This allows all expert sub-models in the hybrid expert model to learn the relevant knowledge of the global segmentation unit, ensuring that even when encountering segmentation units that cannot accurately match the expert sub-models during the application phase, the selected expert sub-model can still provide accurate prediction results. Simultaneously, the local segmentation units can be broadcast to the expert sub-models in the hybrid expert model that match the local segmentation units for processing, enabling normal fine-tuning of the expert sub-models that match the segmentation units.

[0097] Specifically, the expert sub-model for matching local word units refers to the expert sub-model that can accurately determine the processing of local word units after the router calculates the expert sub-model selection score for the local word units.

[0098] Furthermore, when broadcasting local word segments to their matching expert sub-models, this can be achieved by calculating selection scores. In this embodiment, the specific implementation is as follows:

[0099] The target selection score of each expert submodel in the hybrid expert model is calculated for the local word segmentation unit; the local expert submodel that matches the local word segmentation unit is determined in the hybrid expert model based on the target selection score, and the local word segmentation unit is broadcast to the local expert submodel for processing.

[0100] Specifically, the target selection score refers to the score calculated by the router in the hybrid expert model for selecting each expert sub-model for local word segmentation units. A higher score indicates a higher probability of selection. Correspondingly, the local expert sub-model refers to the expert sub-model that matches the local word segmentation units and is used to process these units.

[0101] Therefore, the standard MoE method can be used to select expert sub-models for local word segmentation units. That is, the target selection score of each expert sub-model in the hybrid expert model for selecting local word segmentation units can be calculated first; then, the local expert sub-model that matches the local word segmentation unit can be determined in the hybrid expert model based on the target selection score, and the local word segmentation unit can be broadcast to the local expert sub-model for processing.

[0102] Continuing with the previous example, after obtaining the uncertain tokens ({token1, token2, token4…}) and the certain tokens ({token3, token5, token6…}), the uncertain tokens can be broadcast to all expert sub-models a, b, and c in the MoE model for learning. For the certain tokens, the standard MoE method is used to select the expert sub-models that match each token for learning, based on the scores calculated above. This facilitates the subsequent integration of the learning results from all expert sub-models to update the model parameters.

[0103] In summary, by broadcasting global word segmentation units to all expert sub-models and local word segmentation units to the matching expert sub-models for processing, all expert sub-models can learn relevant knowledge about uncertainty, thereby enabling the hybrid expert model to have more accurate prediction capabilities.

[0104] Step S208: Adjust the parameters of the hybrid expert model according to the processing results until a target hybrid expert model that meets the optimization stopping condition is obtained.

[0105] Specifically, based on the above-mentioned processing of global word segmentation units using all expert sub-models and processing of local word segmentation units using expert sub-models matching local word segmentation units, the parameters of the hybrid expert model can be tuned according to the processing results to obtain a target hybrid expert model that meets the optimization stopping conditions.

[0106] Furthermore, when tuning the parameters of the hybrid expert model, the target hybrid expert model that meets the requirements can be determined by comparing and optimizing the stopping conditions. In this embodiment, the specific implementation is as follows:

[0107] Based on the processing results, the parameters of the hybrid expert model are tuned, and it is checked whether the tuned hybrid expert model meets the optimization stopping condition. If not, multiple candidate word segmentation units are selected as the multiple word segmentation units, and the steps of inputting multiple word segmentation units into the hybrid expert model and calculating the target entropy value corresponding to each word segmentation unit through the router in the hybrid expert model are executed. If yes, the tuned hybrid expert model is used as the target hybrid expert model.

[0108] Based on this, when tuning the hybrid expert model, it is possible to check whether the hybrid expert model meets the optimization stopping condition at each tuning stage. If not, it means that the model still needs to be trained. Therefore, multiple candidate word segmentation units can be selected as multiple word segmentation units, and the steps of inputting multiple word segmentation units into the hybrid expert model and calculating the target entropy value corresponding to each word segmentation unit through the router in the hybrid expert model can be performed. Until a certain tuning stage, if the tuned model meets the optimization stopping condition, the tuned hybrid expert model can be used as the target hybrid expert model for deployment in business scenarios.

[0109] Following the previous example, after all expert sub-models process uncertain tokens and the expert sub-models that have determined the token's match, the learning results of all expert sub-models a, b, and c can be merged. Based on this, the parameters of the MoE model can be updated. Once the updated MoE model meets the business deployment requirements, fine-tuning can stop, and it can be deployed in the business scenario. When a client submits sentiment text to be analyzed, it can be input into the fine-tuned MoE model for processing, thereby determining the sentiment information corresponding to the sentiment text for downstream business use.

[0110] The model optimization method provided in this embodiment aims to reduce router selection uncertainty, improve model robustness, and thus ensure prediction accuracy. During the model optimization phase, after inputting multiple word segmentation units into the hybrid expert model, the routers within the hybrid expert model first calculate the target entropy value corresponding to each word segmentation unit. Based on this, the multiple word segmentation units can be divided into global word segmentation units and local word segmentation units according to the target entropy values. Then, the global word segmentation units can be broadcast to all expert sub-models in the hybrid expert model for processing, while the local word segmentation units are broadcast to the expert sub-models that match the local word segmentation units for processing. This allows uncertain word segmentation units to be broadcast to all expert sub-models for learning during the model optimization phase, while certain word segmentation units are still used for learning by the matching expert sub-models. This enables parameter tuning of the hybrid expert model based on the processing results to obtain a target hybrid expert model that meets the optimization stopping condition. During the fine-tuning of the hybrid expert model, all expert sub-models learn the relevant knowledge of uncertain word segmentation units, ensuring better robustness of the expert sub-models in the application phase.

[0111] Referring to Figure 3, Figure 3 shows a flowchart of an information processing method provided according to an embodiment of this specification, which specifically includes the following steps.

[0112] Step S302: Obtain the question information submitted by the client, construct multiple question word segmentation units based on the question information and input them into the target hybrid expert model, wherein the target hybrid expert model is obtained through the above method;

[0113] Step S304: Select the target expert sub-model that matches each question segmentation unit in the target hybrid expert model to process the multiple question segmentation units;

[0114] Step S306: Merge the prediction results of the target expert sub-model matched by each question segmentation unit to obtain the answer information, and feed the answer information back to the client.

[0115] The information processing method provided in this embodiment, as well as the training of the target hybrid expert model, can be found in the same or corresponding descriptions in the above embodiments. This embodiment will not elaborate further here.

[0116] The following description, in conjunction with Figure 4, uses the application of the information processing method provided in this specification in a text sentiment analysis scenario as an example to further illustrate the information processing method. Figure 4 shows a flowchart of the processing procedure of an information processing method according to an embodiment of this specification, specifically including the following steps.

[0117] Step S402: Input multiple word segmentation units into the hybrid expert model, and calculate the target entropy value corresponding to each word segmentation unit through the router in the hybrid expert model.

[0118] The calculation of the target entropy value for any one of the multiple word segmentation units includes: calculating the selection score of each expert sub-model in the hybrid expert model by using the router in the hybrid expert model; determining the initial entropy value of the selection score corresponding to each expert sub-model; and obtaining the target entropy value corresponding to the word segmentation unit by normalizing the initial entropy value.

[0119] Step S404: Compare the target entropy value corresponding to each word segmentation unit with the preset entropy threshold, and select the word segmentation unit that is greater than the preset entropy threshold as the global word segmentation unit based on the comparison result.

[0120] Step S406: Select word segmentation units that are less than or equal to the preset entropy threshold as local word segmentation units based on the comparison results.

[0121] Step S408: Broadcast the global word segmentation unit to all expert sub-models in the hybrid expert model for processing.

[0122] Step S410: Calculate the target selection score of each expert submodel in the local word segmentation unit selection hybrid expert model.

[0123] Step S412: Based on the target selection score, determine the local expert sub-model that matches the local word segmentation unit in the hybrid expert model, and broadcast the local word segmentation unit to the local expert sub-model for processing.

[0124] Step S414: Adjust the parameters of the hybrid expert model based on the processing results until a target hybrid expert model that meets the optimization stopping condition is obtained.

[0125] Step S416: Upon receiving the question information submitted by the client, construct multiple question segmentation units based on the question information and input them into the target hybrid expert model.

[0126] Step S418: Select the target expert sub-model that matches each question segmentation unit in the target hybrid expert model to process multiple question segmentation units.

[0127] Step S420: Merge the prediction results of the target expert sub-model matched for each question segmentation unit to obtain the answer information, and then feed the answer information back to the client.

[0128] In summary, to reduce router selection uncertainty, improve model robustness, and thus ensure prediction accuracy, during the model optimization phase, multiple word segmentation units can be input into the hybrid expert model. First, the router within the hybrid expert model can be used to calculate the target entropy value for each word segmentation unit. Based on this, the multiple word segmentation units can be divided into global word segmentation units and local word segmentation units according to the target entropy values. Then, the global word segmentation units can be broadcast to all expert sub-models in the hybrid expert model for processing, while the local word segmentation units can be broadcast to the expert sub-models that match the local word segmentation units for processing. This allows for the broadcasting of uncertain word segmentation units to all expert sub-models for learning during the model optimization phase, while the determined word segmentation units are still used for learning by the matching expert sub-models. This enables parameter tuning of the hybrid expert model based on the processing results to obtain a target hybrid expert model that meets the optimization stopping condition. During the fine-tuning of the hybrid expert model, all expert sub-models learn the relevant knowledge of uncertain word segmentation units, ensuring better robustness of the expert sub-models in the application phase.

[0129] Corresponding to the above method embodiments, this specification also provides a model optimization device embodiment. Figure 5 shows a schematic diagram of the structure of a model optimization device provided in one embodiment of this specification. As shown in Figure 5, the device includes:

[0130] The calculation module 502 is configured to input multiple word segmentation units into a hybrid expert model and calculate the target entropy value corresponding to each word segmentation unit through the router in the hybrid expert model;

[0131] The segmentation module 504 is configured to divide the plurality of word segmentation units into global word segmentation units and local word segmentation units according to the target entropy value;

[0132] The processing module 506 is configured to broadcast the global word segmentation unit to all expert sub-models in the hybrid expert model for processing, and to broadcast the local word segmentation unit to the expert sub-models in the hybrid expert model that match the local word segmentation unit for processing.

[0133] The parameter tuning module 508 is configured to tune the parameters of the hybrid expert model based on the processing results until a target hybrid expert model that meets the optimization stopping condition is obtained.

[0134] In an optional embodiment, the calculation of the target entropy value corresponding to any one of the plurality of word segmentation units includes:

[0135] The router in the hybrid expert model calculates the selection score of each expert sub-model in the hybrid expert model for the word segmentation unit; the initial entropy value of the selection score corresponding to each expert sub-model is determined, and the target entropy value corresponding to the word segmentation unit is obtained by normalizing the initial entropy value.

[0136] In an optional embodiment, the partitioning module 504 is further configured to:

[0137] The target entropy value corresponding to each word segmentation unit is compared with a preset entropy threshold. Based on the comparison result, word segmentation units with an entropy value greater than the preset entropy threshold are selected as global word segmentation units, and word segmentation units with an entropy value less than or equal to the preset entropy threshold are selected as local word segmentation units. Alternatively, the multiple word segmentation units are sorted according to the target entropy value corresponding to each word segmentation unit to obtain a word segmentation unit sequence. Global word segmentation units are selected from the word segmentation unit sequence according to preset proportion information, and the remaining word segmentation units other than the global word segmentation units are selected as local word segmentation units.

[0138] In an optional embodiment, the processing module 506 is further configured to:

[0139] The target selection score of each expert submodel in the hybrid expert model is calculated for the local word segmentation unit; the local expert submodel that matches the local word segmentation unit is determined in the hybrid expert model based on the target selection score, and the local word segmentation unit is broadcast to the local expert submodel for processing.

[0140] In an optional embodiment, the parameter tuning module 508 is further configured to:

[0141] Based on the processing results, the parameters of the hybrid expert model are tuned, and it is checked whether the tuned hybrid expert model meets the optimization stopping condition. If not, multiple candidate word segmentation units are selected as the multiple word segmentation units, and the steps of inputting multiple word segmentation units into the hybrid expert model and calculating the target entropy value corresponding to each word segmentation unit through the router in the hybrid expert model are executed. If yes, the tuned hybrid expert model is used as the target hybrid expert model.

[0142] In an optional embodiment, the apparatus further includes:

[0143] The word segmentation module is configured to acquire the original sample data associated with the hybrid expert model, preprocess the original sample data to obtain multiple initial word segmentation units, and embed the multiple initial word segmentation units to obtain multiple word segmentation units.

[0144] The model optimization device provided in this embodiment, in order to reduce router selection uncertainty, improve model robustness, and thus ensure prediction accuracy, can, during the model optimization stage, input multiple word segmentation units into a hybrid expert model, first use the router in the hybrid expert model to calculate the target entropy value corresponding to each word segmentation unit; based on this, the multiple word segmentation units can be divided into global word segmentation units and local word segmentation units according to the target entropy value; then, the global word segmentation units can be broadcast to all expert sub-models in the hybrid expert model for processing, and the local word segmentation units can be broadcast to the expert sub-models that match the local word segmentation units for processing. This allows uncertain word segmentation units to be broadcast to all expert sub-models for learning during the model optimization stage, while certain word segmentation units are still used for learning by the matching expert sub-models. This enables parameter tuning of the hybrid expert model based on the processing results to obtain a target hybrid expert model that meets the optimization stopping condition. During the fine-tuning process of the hybrid expert model, all expert sub-models learn the relevant knowledge of uncertain word segmentation units, ensuring better robustness of the expert sub-models in the application stage.

[0145] The above is a schematic scheme of a model optimization device according to this embodiment. It should be noted that the technical solution of this model optimization device and the technical solution of the above-described model optimization method belong to the same concept. For details not described in detail in the technical solution of the model optimization device, please refer to the description of the technical solution of the above-described model optimization method.

[0146] Corresponding to the above method embodiments, this specification also provides an information processing apparatus embodiment. Figure 6 shows a schematic diagram of the structure of an information processing apparatus provided in one embodiment of this specification. As shown in Figure 6, the apparatus includes:

[0147] The information acquisition module 602 is configured to acquire question information submitted by the client, construct multiple question segmentation units based on the question information, and input them into the target hybrid expert model, wherein the target hybrid expert model is obtained through the above method;

[0148] The word segmentation module 604 is configured to select a target expert sub-model matching each question segmentation unit in the target hybrid expert model to process the plurality of question segmentation units;

[0149] The feedback information module 606 is configured to merge the prediction results of the target expert sub-model matched by each question segmentation unit to obtain answer information, and then feed the answer information back to the client.

[0150] The above is an illustrative scheme of an information processing device according to this embodiment. It should be noted that the technical solution of this information processing device and the technical solution of the information processing method described above belong to the same concept. For details not described in detail in the technical solution of the information processing device, please refer to the description of the technical solution of the information processing method described above.

[0151] Figure 7 shows a structural block diagram of a computing device 700 according to one embodiment of this specification. The components of the computing device 700 include, but are not limited to, a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and a database 750 is used to store data.

[0152] The computing device 700 also includes an access device 740, which enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 740 may include one or more of any type of wired or wireless network interface (e.g., a network interface controller (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0153] In one embodiment of this specification, the aforementioned components of the computing device 700, as well as other components not shown in FIG. 7, may be interconnected, for example, via a bus. It should be understood that the block diagram of the computing device shown in FIG. 7 is merely illustrative and not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0154] The computing device 700 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 700 can also be a mobile or stationary server.

[0155] The processor 720 is used to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-mentioned model optimization method or information processing method.

[0156] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device belongs to the same concept as the technical solutions of the above-described model optimization method or information processing method. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solutions of the above-described model optimization method or information processing method.

[0157] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described model optimization method or information processing method.

[0158] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solutions of the above-described model optimization method or information processing method. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solutions of the above-described model optimization method or information processing method.

[0159] An embodiment of this specification also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described model optimization method or information processing method.

[0160] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product belongs to the same concept as the technical solution of the above-mentioned model optimization method or information processing method. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the above-mentioned model optimization method or information processing method.

[0161] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0162] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0163] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0164] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0165] The preferred embodiments disclosed above are merely illustrative of this specification. Optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described in this specification. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification.

Claims

1. A model optimization method, characterized in that, include: Multiple word segmentation units are input into a hybrid expert model, and the target entropy value corresponding to each word segmentation unit is calculated through the router in the hybrid expert model. Based on the target entropy value, the multiple word segmentation units are divided into global word segmentation units and local word segmentation units; The global word segmentation unit is broadcast to all expert sub-models in the hybrid expert model for processing, and the local word segmentation unit is broadcast to the expert sub-models in the hybrid expert model that match the local word segmentation unit for processing; Based on the processing results, the parameters of the hybrid expert model are adjusted until a target hybrid expert model that meets the optimization stopping condition is obtained.

2. The model optimization method according to claim 1, characterized in that, The calculation of the target entropy value corresponding to any one of the plurality of word segmentation units includes: The router in the hybrid expert model is used to calculate the selection score of each expert sub-model in the hybrid expert model for the word segmentation unit; The initial entropy value of the selection score corresponding to each expert sub-model is determined, and the target entropy value corresponding to the word segmentation unit is obtained by normalizing the initial entropy value.

3. The model optimization method according to claim 1, characterized in that, The step of dividing the plurality of word segmentation units into global word segmentation units and local word segmentation units according to the target entropy value includes: The target entropy value corresponding to each word segmentation unit is compared with a preset entropy threshold. Based on the comparison result, word segmentation units with an entropy value greater than the preset entropy threshold are selected as global word segmentation units, and word segmentation units with an entropy value less than or equal to the preset entropy threshold are selected as local word segmentation units; or, The multiple word segmentation units are sorted according to the target entropy value corresponding to each word segmentation unit to obtain a word segmentation unit sequence. A global word segmentation unit is selected from the word segmentation unit sequence according to the preset proportion information, and the remaining word segmentation units other than the global word segmentation unit are used as local word segmentation units.

4. The model optimization method according to claim 1, characterized in that, The step of broadcasting the local word segmentation unit to the expert sub-model that matches the local word segmentation unit in the hybrid expert model for processing includes: The local word segmentation unit selects the target selection score of each expert submodel in the hybrid expert model; Based on the target selection score, a local expert sub-model is determined in the hybrid expert model to match the local word unit, and the local word unit is broadcast to the local expert sub-model for processing.

5. The model optimization method according to claim 1, characterized in that, The step of tuning the hybrid expert model based on the processing results until a target hybrid expert model that meets the optimization stopping condition is obtained includes: Based on the processing results, the parameters of the hybrid expert model are tuned, and it is checked whether the tuned hybrid expert model meets the optimization stopping condition. If not, select multiple candidate word segmentation units as the multiple word segmentation units, and perform the step of inputting the multiple word segmentation units into the hybrid expert model, and calculating the target entropy value corresponding to each word segmentation unit through the router in the hybrid expert model; If so, use the parameter-tuned hybrid expert model as the target hybrid expert model.

6. The model optimization method according to claim 1, characterized in that, Before the step of inputting multiple word segmentation units into the hybrid expert model is executed, the following is also included: Obtain the original sample data associated with the hybrid expert model, preprocess the original sample data, and obtain multiple initial word segmentation units; The initial word segmentation units are embedded separately to obtain multiple word segmentation units.

7. An information processing method, characterized in that, include: Obtain the question information submitted by the client, construct multiple question segmentation units based on the question information and input them into the target hybrid expert model, wherein the target hybrid expert model is obtained by the method described in any one of claims 1 to 7; In the target hybrid expert model, a target expert sub-model matching each question segmentation unit is selected to process the multiple question segmentation units; The prediction results of the target expert sub-model matched by each question segmentation unit are merged to obtain the answer information, and the answer information is fed back to the client.

8. A model optimization device, characterized in that, include: The calculation module is configured to input multiple word segmentation units into a hybrid expert model, and calculate the target entropy value corresponding to each word segmentation unit through the router in the hybrid expert model; The segmentation module is configured to divide the plurality of word segmentation units into global word segmentation units and local word segmentation units based on the target entropy value; The processing module is configured to broadcast the global word segmentation unit to all expert sub-models in the hybrid expert model for processing, and to broadcast the local word segmentation unit to the expert sub-models in the hybrid expert model that match the local word segmentation unit for processing. The parameter tuning module is configured to tune the parameters of the hybrid expert model based on the processing results until a target hybrid expert model that meets the optimization stopping condition is obtained.

9. An information processing device, characterized in that, include: The information acquisition module is configured to acquire question information submitted by the client, construct multiple question segmentation units based on the question information, and input them into the target hybrid expert model, wherein the target hybrid expert model is obtained by the method described in any one of claims 1 to 7; The word segmentation processing module is configured to select a target expert sub-model matching each question segmentation unit in the target hybrid expert model to process the multiple question segmentation units; The feedback information module is configured to merge the prediction results of the target expert sub-model matched for each question segmentation unit to obtain the answer information, and then feed the answer information back to the client.

10. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that, It stores computer-executable instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 7.

12. A computer program product, characterized in that, It includes a computer program or instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 7.

Images