Data processing method and device based on hybrid expert model, computer equipment and readable storage medium

Abstract

The invention relates to a data processing method and device based on a hybrid expert model, computer equipment, a computer readable storage medium and a computer program product. The method comprises: based on a target character in input data, determining a target probability corresponding to the target character, and determining a target probability difference threshold corresponding to the target character, the target probability comprising a probability estimation value of each expert network model in a preset hybrid expert model; screening the expert network models based on the probability estimation values of the expert network models and a target probability difference threshold to obtain a target expert network model; and processing the target character based on each target expert network model to obtain a data processing result. By adopting the method, the models contained in the expert model can be dynamically mixed based on the probability difference threshold, the data processing of the character dimension of the input data is realized, and the adaptation degree between each dynamically determined expert network model and the characters in the input data is ensured.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, apparatus, computer equipment, computer-readable storage medium and computer program product based on a hybrid expert model. Background Art

[0002] With the development of technology in the field of artificial intelligence, a mixture of experts (MoE) model has emerged. This model aims to improve the overall prediction performance by integrating the decisions of multiple expert network models. Generally speaking, the expert network model can be a model trained for a specific task or data subset.

[0003] In traditional technology, the expert network model included in the hybrid expert model is selected through a static Top-K selection strategy. The hybrid expert model generally selects the top K expert models with the highest weights for integration. Once K is selected, it cannot change during the entire task. In actual application scenarios, due to large differences between input data or large differences in performance between expert network models, it is impossible to fully utilize the predictive power of each expert network model, resulting in poor data processing capabilities of the hybrid expert model. Summary of the invention

[0004] Based on this, it is necessary to provide a data processing method, device, computer equipment, computer-readable storage medium and computer program product based on a hybrid expert model, which can accurately select the expert network model in the hybrid expert model in order to solve the above technical problems.

[0005] In a first aspect, the present application provides a data processing method based on a hybrid expert model, comprising:

[0006] Based on the target character in the input data, determining a target probability corresponding to the target character and determining a target probability difference threshold corresponding to the target character, wherein the target probability includes a probability estimation value of each expert network model in a preset hybrid expert model;

[0007] Based on the probability estimation value of each expert network model and the target probability difference threshold, each expert network model is screened to obtain a target expert network model;

[0008] The target characters are processed based on each of the target expert network models to obtain a data processing result.

[0009] In one embodiment, the hybrid expert model includes an input layer attention mechanism layer; the method further includes:

[0010] Acquire the input data through the input layer, wherein the input data includes at least one character;

[0011] For each of the characters, the attention mechanism layer performs attention calculation on the character to obtain the attention weight of the character, and determines the target character based on the attention weight and the character.

[0012] In one embodiment, determining the target probability corresponding to the target character based on the target character in the input data includes:

[0013] For each of the target characters, the task type corresponding to the input data, the target character and each expert network model in the hybrid expert model are processed through a preset probability algorithm to obtain a probability estimation value corresponding to each of the expert network models. The probability estimation value represents the accuracy of the expert network model's reasoning ability for the target character.

[0014] In one embodiment, the method further comprises:

[0015] Obtaining a plurality of pre-configured probability difference thresholds;

[0016] For each of the pre-configured probability difference thresholds, in the hybrid expert model, the input data is processed based on the preset probability difference threshold to obtain output data, and an indicator value of a threshold evaluation indicator is obtained based on the output data;

[0017] Screening is performed based on the index values ​​of the threshold evaluation indexes corresponding to each of the pre-configured probability difference thresholds to obtain a target probability difference threshold.

[0018] In one embodiment, the method of screening each of the expert network models based on the probability estimation value of each of the expert network models and the target probability difference threshold to obtain a target expert network model includes:

[0019] Based on the order of probability estimation values ​​from large to small, the expert network models are sorted to obtain a sequence, and the expert network model with the highest probability estimation value is obtained, and the expert network model with the highest probability estimation value is determined as the target expert network model;

[0020] Calculating the probability estimation value difference between the target expert network model and the next expert network model of the target expert network model according to the arrangement order of the sequence;

[0021] If the probability estimate difference is less than the target probability difference threshold, the next expert network model is determined to be the target expert network model, and the step of calculating the probability estimate difference between the target expert network model and the next expert network model of the target expert network model in the order of arrangement of the sequence is re-executed until the current probability estimate difference is greater than or equal to the target probability difference threshold, thereby obtaining at least one target expert network model.

[0022] In one embodiment, the hybrid expert model further includes an output layer; the target characters are processed based on each of the target expert network models to obtain a data processing result, including:

[0023] Processing the target characters through each of the target expert network models to obtain a character processing result;

[0024] Performing weighted calculation based on the probability estimation value of each target expert network model and the character processing results respectively corresponding to each target expert network model to obtain a weighted result;

[0025] The weighted result is processed through the processing operation corresponding to the output layer to obtain the data processing result of the target character.

[0026] In a second aspect, the present application also provides a data processing device based on a hybrid expert model, comprising:

[0027] A first determination module is used to determine a target probability corresponding to a target character in the input data, and to determine a target probability difference threshold corresponding to the target character, wherein the target probability includes a probability estimation value of each expert network model in a preset hybrid expert model;

[0028] A first screening module, configured to screen each of the expert network models based on the probability estimation value of each of the expert network models and the target probability difference threshold, to obtain a target expert network model;

[0029] The first processing module is used to process the target characters based on each target expert network model to obtain a data processing result.

[0030] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0031] Based on the target character in the input data, determining a target probability corresponding to the target character and determining a target probability difference threshold corresponding to the target character, wherein the target probability includes a probability estimation value of each expert network model in a preset hybrid expert model;

[0032] Based on the probability estimation value of each expert network model and the target probability difference threshold, each expert network model is screened to obtain a target expert network model;

[0033] The target characters are processed based on each of the target expert network models to obtain a data processing result.

[0034] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0035] Based on the target character in the input data, determining a target probability corresponding to the target character and determining a target probability difference threshold corresponding to the target character, wherein the target probability includes a probability estimation value of each expert network model in a preset hybrid expert model;

[0036] Based on the probability estimation value of each expert network model and the target probability difference threshold, each expert network model is screened to obtain a target expert network model;

[0037] The target characters are processed based on each of the target expert network models to obtain a data processing result.

[0038] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0039] Based on the target character in the input data, determining a target probability corresponding to the target character and determining a target probability difference threshold corresponding to the target character, wherein the target probability includes a probability estimation value of each expert network model in a preset hybrid expert model;

[0040] Based on the probability estimation value of each expert network model and the target probability difference threshold, each expert network model is screened to obtain a target expert network model;

[0041] The target characters are processed based on each of the target expert network models to obtain a data processing result.

[0042] The above-mentioned data processing method, device, computer equipment, computer-readable storage medium and computer program product based on the hybrid expert model, wherein the method includes: based on the target character in the input data, determining the target probability corresponding to the target character, and determining the target probability difference threshold corresponding to the target character, the target probability includes the probability estimation value of each expert network model in the preset hybrid expert model; based on the probability estimation value of each expert network model and the target probability difference threshold, screening each expert network model to obtain a target expert network model; processing the target character based on each target expert network model to obtain a data processing result.

[0043] By adopting this method, the optimization of the selection of the number of expert network models in the model reasoning process can be achieved, the model reasoning efficiency can be improved, the models included in the expert model can be dynamically mixed based on the probability difference threshold, the data processing of the character dimension of the input data can be realized, the data processing accuracy can be improved, and the degree of adaptation between each dynamically determined expert network model and the characters in the input data can be guaranteed, further expanding the decision-making efficiency of the hybrid expert model and improving the utilization rate of the hybrid expert model. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0045] FIG1 is an application environment diagram of a data processing method based on a hybrid expert model in one embodiment;

[0046] FIG2 is a schematic diagram of a flow chart of a data processing method based on a hybrid expert model in one embodiment;

[0047] FIG3 is a schematic flow chart of a step of obtaining a target character in one embodiment;

[0048] FIG4 is a schematic diagram of a process for obtaining a target probability difference threshold in one embodiment;

[0049] FIG5 is a schematic flow chart of the steps of obtaining data processing results in one embodiment;

[0050] FIG6 is a flow chart of a data processing method based on a hybrid expert model in another embodiment;

[0051] FIG7 is a schematic diagram of a model in one embodiment;

[0052] FIG8 is a structural block diagram of a data processing device based on a hybrid expert model in one embodiment;

[0053] FIG. 9 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] In one embodiment, as shown in FIG1 , a data processing method based on a hybrid expert model is provided. This embodiment uses the method applied to a terminal as an example for illustration. It is understandable that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. The above-mentioned terminal can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. In this embodiment, the data processing method based on the hybrid expert model includes the following steps:

[0056] Step 102, based on the target character in the input data, determining the target probability corresponding to the target character, and determining the target probability difference threshold corresponding to the target character.

[0057] Among them, the target probability includes the probability estimation value of each expert network model in the preset hybrid expert model, and the target character in the input data can be multiple characters obtained after dividing the input data into character dimensions, or can be characters obtained after dividing the input data into character dimensions and performing attention weight processing, etc. The input data can be data to be processed by the hybrid expert model; the preset hybrid expert model can include multiple expert network models, and each expert network model can process the input data separately, that is, can process each target character in the input data separately; the probability estimation value corresponding to each expert network model included in the target probability represents the ability of the expert network model to accurately infer the target character, or can also represent the accuracy of the data processing result obtained by the expert network model processing the target character; the target probability difference threshold can be pre-configured, or it can be a threshold range of the difference between two probability estimation values ​​adapted to the target character determined based on the target character.

[0058] Specifically, the terminal can obtain input data input by the user, and the input data can include multiple characters; the terminal can divide the input data based on the character dimension to obtain multiple characters contained in the input data, and the target character can be each character contained in the input data. For each target character, the terminal can determine the target probability and probability difference threshold corresponding to the target character. In other words, for each target character, the terminal can calculate the ability of each expert network model in the mixed expert model to accurately infer the target character, that is, to obtain the probability estimate value of each expert network model for the target character; and the terminal can also calculate the difference threshold for each target character to obtain the probability difference threshold adapted to each target character, that is, to obtain the target probability difference threshold corresponding to the target character.

[0059] Step 104 , based on the probability estimation value of each expert network model and the target probability difference threshold, each expert network model is screened to obtain a target expert network model.

[0060] Specifically, the terminal can arrange the expert network models in descending order of their probability estimation values ​​to obtain an expert network model sequence, and screen the expert network model sequence based on the target probability difference threshold to extract the target expert network model. That is, the target expert network model corresponding to the target character is extracted.

[0061] Step 106, processing the target characters based on each target expert network model to obtain a data processing result.

[0062] Specifically, the terminal can process the target character based on each selected target expert network model to obtain the output result of each target expert network model for the target character. In this way, the terminal can process based on the output result of the target expert network model to obtain the data processing result corresponding to the target character.

[0063] In one example, the current usage scenario may be a translation scenario; the expert network model may be a text translation model; for a target character, the terminal may obtain a target expert network model corresponding to the target character, and perform text translation processing on the target character through each target expert network model to obtain translation results corresponding to each target expert network model; based on this, the terminal may process the translation results output by each target text translation model to obtain a text translation result corresponding to the target character, that is, a data processing result corresponding to the target character.

[0064] In the above-mentioned data processing method based on the hybrid expert model, based on the target character in the input data, the target probability corresponding to the target character is determined, and the target probability difference threshold corresponding to the target character is determined, and the target probability includes the probability estimation value of each expert network model in the preset hybrid expert model. Based on the probability estimation value of each expert network model and the target probability difference threshold, each expert network model is screened to obtain the target expert network model. The target character is processed based on each target expert network model to obtain the data processing result. By adopting this method, the optimization of the number of expert network models selected in the model reasoning process can be achieved, and the model reasoning efficiency can be improved. The models included in the expert model can be dynamically mixed based on the probability difference threshold to achieve data processing of the character dimension of the input data, improve the data processing accuracy, and ensure the degree of adaptation between each dynamically determined expert network model and the characters in the input data, further expand the decision-making efficiency of the hybrid expert model and improve the utilization rate of the hybrid expert model.

[0065] In one embodiment, the hybrid expert model includes an input layer and an attention mechanism layer. Specifically, the hybrid expert model may include an input layer, an attention mechanism layer, a hybrid expert layer, and an output layer, and the hybrid expert layer includes multiple expert network models; or, the model applied in this embodiment may be a data processing model (data processing system), and the data processing model (data processing system) may include an input layer, an attention mechanism layer, a hybrid expert model, and an output layer, and the hybrid expert model may include multiple expert network models, etc.

[0066] As shown in FIG2 , the data processing method based on the hybrid expert model further includes:

[0067] Step 202, obtaining input data through the input layer.

[0068] The input data includes at least one character.

[0069] Specifically, the terminal may obtain multiple character tokens based on dividing the input data into character dimensions.

[0070] Step 204, for each character, the attention mechanism layer performs attention calculation on the character to obtain the attention weight of the character, and determines the target character based on the attention weight and the character.

[0071] Specifically, the terminal may determine the multiple characters obtained by dividing the character dimension as target characters. Alternatively, the terminal may also perform an attention mechanism calculation on the character to obtain an attention weight corresponding to the character, and add the attention weight to the character to obtain the target character, where the attention weight represents the importance of the character.

[0072] In one example, the terminal can input input data containing multiple characters into the attention mechanism layer, perform attention mechanism calculation on the input data through the attention mechanism layer, obtain the attention weights corresponding to each character contained in the input data, and add each attention weight to the corresponding character to obtain multiple target characters.

[0073] Optionally, the specific process of performing the attention mechanism calculation may be that the terminal, in the attention mechanism layer, may perform the calculation based on the relationship between the states stored locally in the attention mechanism layer to obtain the attention weights corresponding to the characters contained in the terminal, or the attention weights corresponding to the characters. The locally stored states may be the hidden states input from the input layer to the attention mechanism layer, or the output data of the decoder of the previous input data of the input data in the output layer of the model, and the corresponding input data may be sequence data.

[0074] In this embodiment, by calculating the attention weight of the input data, the characters of the more important part of the input data can be clearly represented, so that each expert network in the hybrid expert model / hybrid expert layer can pay more attention to and highlight the important parts of the input data when processing data.

[0075] In an exemplary embodiment, the specific implementation process of the step of “determining the target probability corresponding to the target character based on the target character in the input data” may include:

[0076] For each target character, the task type corresponding to the input data, the target character and each expert network model in the hybrid expert model are processed through a preset probability algorithm to obtain the probability estimation value corresponding to each expert network model.

[0077] The probability estimate value may represent the accuracy of the reasoning ability of the expert network model to reason about the target character. The task type corresponding to the input data may be the task type of the reasoning task required for the current application scenario, such as a text translation type or a data prediction type, etc. The preset probability algorithm may be a probability estimate value allocation algorithm, etc.

[0078] Specifically, the terminal calculates the probability estimation value of each expert network model for each target character respectively. For example, the terminal can determine the task type of the reasoning task corresponding to the input data, and input the task type, the target character and each expert network model into a preset probability estimation value allocation algorithm. The probability estimation value of each expert network model is allocated by the preset probability estimation value allocation algorithm to obtain the output result of the probability estimation value allocation algorithm, that is, the probability estimation value of each expert network model for the target character. In other words, the terminal can determine the accuracy of the reasoning ability of each expert network model when processing the target character.

[0079] In this embodiment, by dynamically determining the probability estimation values ​​of each expert network model corresponding to the target character, the degree of adaptation between the determined probability estimation values ​​and the target character can be improved, so that the probability estimation values ​​can better reflect the accurate reasoning ability for the target character.

[0080] In an exemplary embodiment, as shown in FIG3 , the data processing method based on the hybrid expert model further includes:

[0081] Step 302: Acquire multiple pre-configured probability difference thresholds.

[0082] Specifically, the terminal may configure multiple probability value differences based on actual application scenarios, or obtain multiple probability value differences based on pre-configured data, for example, including a first probability threshold difference and a second probability threshold difference, etc.

[0083] Step 304, for each pre-configured probability difference threshold, in the hybrid expert model, the input data is processed based on the preset probability difference threshold to obtain output data, and the index value of the threshold evaluation index is obtained based on the output data.

[0084] The threshold evaluation index may be an index that can characterize the performance of the model, such as data processing efficiency, data processing accuracy, etc. The hybrid expert model may include an input layer, an attention mechanism layer, a hybrid expert layer, and an output layer, wherein the hybrid expert layer includes multiple expert network models. The input data may be a target character.

[0085] Specifically, for each pre-configured probability difference threshold, the terminal can input input data into the hybrid expert model, screen each expert network model in the hybrid expert model through the pre-configured probability difference threshold, obtain each screened target expert network model, and process the input data based on the target expert network model and the output layer to obtain the output result corresponding to the input data. The output result is evaluated based on the preset threshold evaluation index to obtain the index value of the threshold evaluation index corresponding to the output result, that is, to obtain the index value of the threshold evaluation index corresponding to the pre-configured probability difference threshold. In other words, the terminal can obtain the index value of the threshold evaluation index corresponding to each pre-configured probability difference threshold.

[0086] Optionally, the model used in this embodiment may be a data processing model (data processing system), which may include an input layer, an attention mechanism layer, a hybrid expert model, and an output layer. The hybrid expert model may include multiple expert network models, etc., and the input data may be a target character. For each pre-configured probability difference threshold, the terminal may input the input data into the data processing model (data processing system), and screen the expert network models contained in the hybrid expert layer in the data processing model through the pre-configured probability difference threshold to obtain the screened target expert network models, and process the input data based on the target expert network model and the output layer to obtain the output result corresponding to the input data. The output result is evaluated based on a preset threshold evaluation index to obtain the index value of the threshold evaluation index corresponding to the output result, that is, to obtain the index value of the threshold evaluation index corresponding to the pre-configured probability difference threshold. In other words, the terminal can obtain the index value of the threshold evaluation index corresponding to each pre-configured probability difference threshold.

[0087] Step 306 , screening is performed based on the index values ​​of the threshold evaluation indexes corresponding to the pre-configured probability difference thresholds to obtain a target probability difference threshold.

[0088] Specifically, when the threshold evaluation index is positively correlated with the model performance, the terminal screens the index values ​​of the threshold evaluation index corresponding to each pre-configured probability difference threshold, and can determine the pre-configured probability difference threshold with the highest index value as the target probability difference threshold. For example, the threshold evaluation index can be model processing precision, model processing accuracy, model processing efficiency, model processing speed, etc. In one example, when the threshold evaluation index is negatively correlated with the model performance, the terminal screens the index values ​​of the threshold evaluation index corresponding to each pre-configured probability difference threshold, and can determine the pre-configured probability difference threshold with the lowest index value as the target probability difference threshold. The threshold evaluation index can be the degree of model processing abnormality, the model processing error rate, etc.

[0089] In this embodiment, by screening the model performance among multiple pre-configured probability difference thresholds, a target probability difference threshold that can better improve the processing efficiency and model performance of the hybrid expert model / data processing model can be obtained, and a target probability difference threshold that better matches the target character can be obtained.

[0090] In an exemplary embodiment, as shown in FIG4 , the specific implementation process of the step of “screening each expert network model based on the probability estimation value of each expert network model and the target probability difference threshold to obtain the target expert network model” may include:

[0091] Step 402, based on the order of probability estimation values ​​from large to small, the expert network models are sorted to obtain a sequence, and the expert network model with the highest probability estimation value is obtained, and the expert network model with the highest probability estimation value is determined as the target expert network model.

[0092] Step 404: Calculate the probability estimation value difference between the target expert network model and the next expert network model of the target expert network model according to the arrangement order of the sequence.

[0093] Specifically, the terminal arranges the probability estimation values ​​of the expert network models in descending order to obtain an expert network model sequence; in this way, the terminal can start from the expert network model with the highest probability estimation value and first determine the expert network model as the target expert network model; and determine the next expert network model of the expert network model according to the arrangement order of the expert network model sequence, and calculate the probability estimation value difference between the probability estimation value of the expert network model and the probability estimation value of the next expert network model.

[0094] Step 406, if the probability estimate difference is less than the target probability difference threshold, the next expert network model is determined to be the target expert network model, and the step of calculating the probability estimate difference between the target expert network model and the next expert network model of the target expert network model in the order of the sequence is re-executed until the current probability estimate difference is greater than or equal to the target probability difference threshold, and at least one target expert network model is obtained.

[0095] Specifically, the terminal can compare the calculated probability estimate difference with the target probability difference threshold corresponding to the target character. If the currently calculated probability estimate difference is less than the target probability difference threshold, the terminal can use the current next expert network model as the target expert network model, and calculate the probability estimate difference between the target expert network model and the next expert network model of the target expert network model according to the arrangement order of the expert network model sequence until the calculated difference is greater than or equal to the target probability difference threshold, then stop execution, and obtain the target expert network models corresponding to the target character.

[0096] Optionally, the expert network model sequence obtained by the terminal may include expert network model 1, expert network model 2, expert network model 3 and expert network model 4 arranged in a sequence from large to small according to the probability estimation value. In this way, the terminal determines that the expert network model with the highest probability estimation value is expert network model 1, and determines the target expert network model with the expert network model 1, and calculates the probability estimation value difference between the target expert network model (expert network model 1) and the next expert network model (expert network model 2) of the target expert network model, and compares the probability estimation value difference with the target probability difference threshold. For example, the probability estimation value difference may be less than the target probability difference threshold, and the terminal may determine the current next expert network model as the target expert network model, that is, determine expert network model 2 as the target expert network model. In this way, the terminal may calculate the difference between expert network model 2 and the next expert network model (expert network model 3) based on the current target expert network model, and the difference may be greater than or equal to the target probability difference threshold, and the terminal may stop executing the comparison step, and determine that the target expert network model corresponding to the target character is expert network model 1 and expert network model 2.

[0097] In one example, the process of screening multiple expert network models included in the mixed expert model to obtain one or more target expert network models may include: the terminal arranges the probability estimation values ​​of each expert network model from large to small to obtain an expert network model sequence; in this way, for each expert network model (hereinafter referred to as the first expert network model), the terminal can respectively calculate the difference between the probability estimation value of the first expert network model and the next expert network model adjacent to the first expert network model to obtain the difference corresponding to the first expert network model, so that the terminal can obtain multiple differences. According to the arrangement order of the expert network model sequence, the terminal can sequentially compare each difference with the target probability difference threshold, if the difference is less than the target probability difference threshold, then continue to compare the relationship between the next difference and the target probability difference threshold in sequence, until it is determined that the difference is greater than or equal to the target probability difference threshold, then the first expert network model corresponding to the difference, and the other expert network models before the first expert network model, are all determined as the target expert network model, that is, determined as the target expert network model matching the target character.

[0098] In this embodiment, each expert network model is screened by using the target probability difference threshold corresponding to the target character, so that the expert network models with significant model performance differences and model performance differences can be eliminated, and the expert network model with more accurate reasoning ability for the target character is retained as the target expert network model, thereby improving the processing efficiency of the target characters in the input data, and the selection of the target expert network model for each character is more flexible.

[0099] In an exemplary embodiment, the hybrid expert model further includes an output layer. As shown in FIG5 , the specific processing process of the step “processing the target characters based on each target expert network model to obtain a data processing result” includes:

[0100] Step 502, processing the target characters through each target expert network model to obtain a character processing result.

[0101] Specifically, the terminal can input the target characters into each target expert network model respectively, process the target characters through each target expert network model, obtain the output results of each target expert network model, that is, the character processing results of each target expert network model, and determine the output results of each target expert network model as the character processing results.

[0102] Step 504 , performing weighted calculation based on the probability estimation value of each target expert network model and the character processing results corresponding to each target expert network model, to obtain a weighted result.

[0103] Among them, the probability estimation value of each target expert network model can be a calculation weight representing the target expert network model, that is, the terminal can determine the probability estimation value of each target expert network model as the calculation weight, that is, the calculation weight used during processing.

[0104] Specifically, after obtaining the character processing results of each target expert network model for the target character, the probability estimation values ​​of each target expert network model can be used as calculation weights to perform weighted calculations on the character processing results of each target expert network model to obtain weighted results.

[0105] Step 506, the weighted result is processed by the processing operation corresponding to the output layer to obtain the data processing result of the target character.

[0106] Among them, the processing operation corresponding to the output layer may be a processing operation related to the type of model, for example, it may include a prediction operation, a translation operation, a specific processing operation, etc.

[0107] Specifically, the terminal may perform the processing operation corresponding to the output layer on the weighted result through the processing operation corresponding to the output layer, and obtain the data processing result of the target character.

[0108] In an example, the terminal may also determine the weighted result as the data processing result corresponding to the target character.

[0109] In this embodiment, weighted processing is performed on the character processing results corresponding to each target expert network model, which can be flexibly adjusted according to different inputs and scenarios, thereby improving the application capability of the model in a changing environment.

[0110] In the following, in conjunction with a specific embodiment, the specific implementation process of the above-mentioned data processing method based on the hybrid expert model is described in detail as follows:

[0111] In traditional technology, the Mixture of Experts (MoE) model integrates the decisions of multiple expert networks to improve the overall prediction performance. These "experts" are usually models trained specifically for specific tasks or data subsets, and the "gated network" is responsible for determining the weight distribution of different experts. Generally, the experts (expert network model) who participate in the final decision are selected through a static Top-K strategy. In this strategy, the gating network assigns a weight to each expert, which reflects the relative ability of each expert to process the current input. The model selects the top K experts with the highest weights for integration. Although K can change before model inference, once K is selected, it cannot change during the entire task process.

[0112] However, the static Top-K strategy has several limitations. First, it considers a preset number (K) of experts, regardless of the weights of these experts. The weights represent the performance of the experts in a specific task. If K is set in advance, it may cause inappropriate expert results to be considered, which in turn affects the final result. For example, if K is set to 2, 2 experts will definitely be considered. Although the specific 2 experts involved in the decision-making are variable, 2 is fixed. In some cases, even if the weight of an expert is significantly higher than that of other experts, it may still not be included in the final decision. Secondly, when the weights of two adjacent experts are very different, still selecting a fixed number of K experts may introduce unnecessary noise or reduce the overall prediction accuracy. For example, there are 5 experts with weights of [0.9, 0.08, 0,01, 0,005, 0,005]. If k=2, two experts with weights of 0.9 and 0.08 will definitely be selected, but the expert with weights of 0.08 obviously has poor token processing capabilities and is noise, which will interfere with the overall accuracy. The best choice this time is k=1.

[0113] The data processing method based on the hybrid expert model provided in this embodiment optimizes the number of expert network models and the model selection method in the task reasoning process, and improves the reasoning efficiency of the entire model. Specifically, a dynamic expert number selection mechanism based on a probability difference threshold is used to select different numbers of experts when processing different character tokens in the reasoning process, thereby optimizing the model's computing resource allocation and decision quality, considering the probability distribution between the weights of each expert network model included in the hybrid expert model for each target character in the input data, and avoiding ignoring the opinions output by the key expert network model with significantly high probabilities. In the case where the performance differences between expert network models are significant and the input data distribution is highly dynamic, the model mixing potential can be fully utilized, and different numbers of experts can be determined for different tokens, thereby improving the decision-making efficiency and accuracy of the hybrid expert model.

[0114] The data processing method based on the mixed expert model provided in this embodiment can improve the performance and accuracy of the mixed expert model (Mixture of Experts, MoE for short) when processing different input token data, and optimize the performance and accuracy accurately to the token level. By introducing a probability-based dynamic expert number mechanism and an expert selection mechanism, the most appropriate number of experts and the corresponding experts can be selected according to the actual situation of each input token to ensure that computing resources are used more efficiently. For example, it may include: determining the number of experts and screening experts accurately to the token level: k has different k values ​​when processing different tokens, rather than selecting k values ​​at the task level, and all k values ​​at the task level are unified; reducing unnecessary calculations: the traditional static Top-K selection method may use too many or too few experts in some cases, resulting in waste of resources or insufficient performance. The dynamic selection mechanism can adaptively adjust the number of experts as needed; Improve model adaptability: By adjusting the expert selection criteria in real time, the model can better adapt to data changes, especially when there are significant changes in data distribution, and can dynamically adjust and maintain high accuracy; Enhance model accuracy: Focus on calculating and utilizing those experts who are most knowledgeable about the current input, rather than considering all expert opinions on average, which helps to make more accurate predictions; Improve model efficiency: By reasonably distributing the computing load to the most influential experts, the overall computing efficiency is improved without sacrificing accuracy.

[0115] In summary, the data processing method based on the hybrid expert model provided in this embodiment can make the hybrid expert model more flexible and effective under different tokens by dynamically determining the expert network model corresponding to each character at the token level, real-time probability estimation, and adaptive probability threshold judgment, thereby ensuring the output quality while improving the economy of resource utilization.

[0116] As shown in FIG6 , the specific execution process of the data processing method based on the hybrid expert model provided in this embodiment may be:

[0117] For a specific token, the expert probability is normalized and the expert sequence is obtained. If the probability difference is greater than ΔP, the selection is stopped and the expert outputs are combined. If the probability difference is less than or equal to ΔP, the next expert is selected. Specifically, the gating network calculates and assigns a probability estimate value to each expert model in real time for each input token. This value reflects the ability of each expert to make accurate inferences about the current input token. The terminal can set a probability threshold (ΔP), which provides a judgment standard for expert selection. This threshold can be a fixed value or a dynamic value adjusted according to the input data or model performance. The terminal starts with the expert with the highest estimated probability and compares the probability difference of adjacent experts one by one. When the probability difference between an expert and the next expert is greater than ΔP, the expert selection process is terminated. For example, when ΔP is set to 0.1, the token sequence processed is "I", "love", "you", and there are 5 experts processing this sequence. The normalized probability of processing "I" is [0.5, 0.35, 0.1, 0.01, 0.04]. The number of experts selected at this time is 1, the normalized probability of processing "love" is [0.4, 0.35, 0.2, 0.05, 0], and the number of experts selected at this time is 2. The normalized probability of processing "you" is [0.2, 0.2, 0.2, 0.2, 0.2], and the number of experts selected at this time is 5; the model therefore only combines those experts whose probabilities are higher than the set threshold. This ensures that the model relies more on experts whose suggestions are more likely to be correct during reasoning.

[0118] Based on such a dynamic selection mechanism, the model can adaptively reduce or increase the number of experts used according to the complexity of real-time token data, which not only reduces the amount of calculation in some cases, but also enables the model to adapt to the changing data environment, thereby improving overall efficiency and performance. By introducing adaptive probability thresholds and real-time probability estimates, a new dynamic expert selection mechanism accurate to the token level is implemented. This mechanism provides a more flexible and efficient decision-making method in the hybrid expert model. It can dynamically adjust the selection of experts to ensure that the model focuses on the most influential expert opinions at each reasoning step, thereby improving the accuracy of decision-making and reducing unnecessary resource consumption.

[0119] As shown in Figure 7, the model used in this embodiment may include an input layer, an attention mechanism layer, a hybrid expert model, and an output layer. The input data is obtained through the input layer, and the attention mechanism layer performs attention calculation on the input data to obtain multiple target characters. The hybrid expert model may include an expert network model (expert 1), an expert network model (expert 2), an expert network model (expert 3), ..., an expert network model (expert n), and the weights of each expert corresponding to the target characters are obtained through a gating mechanism, weight w1, weight w2, weight w3, ..., weight wn, so that an aggregated expert output can be obtained. Based on this, the terminal can process the connection information of the expert output and the output of the attention mechanism layer through the input layer to obtain output data, that is, the data processing results corresponding to each target character.

[0120] The data processing method based on the hybrid expert model provided in this embodiment realizes a dynamic expert selection mechanism at the token level, which can adaptively adjust which experts should participate in the analysis and processing of specific input data based on the token, is more efficient, and can focus on the experts who are most critical to the current task. This embodiment can evaluate the contribution probability of each expert to a given token, and dynamically select the most relevant expert combination based on these probability values, thereby achieving an adaptive response to the input data. By giving priority to experts with significantly high probability values, redundant calculations are reduced when processing simple tasks or when expert opinions are highly consistent, thereby saving computing resources. The dynamic selection method means that fewer expert resources may be used in some cases, which is particularly valuable in an environment with limited resources, and can also release resources for other tasks. Based on the contribution of each expert network model to the target character, the expert network model is selected, so that the selected target expert network model may be more able to accurately predict or process specific data, thereby improving the overall output quality of the model.

[0121] The data processing method based on the hybrid expert model provided in this embodiment can also dynamically adapt to different data states (such as different probability distributions) when facing complex data, especially those with dynamically changing characteristics, so as to ensure that the reasoning performance of the model can be maintained or improved in a complex and changing environment. In general, the dynamic selection mechanism in this embodiment enables the model to operate more flexibly and efficiently under various conditions, which can not only improve the prediction accuracy but also save resources, and is particularly suitable for modern computing environments with changing demands and tight resources.

[0122] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0123] Based on the same inventive concept, the embodiment of the present application also provides a data processing device based on a hybrid expert model for implementing the data processing method based on a hybrid expert model involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more embodiments of the data processing device based on the hybrid expert model provided below can refer to the limitations of the data processing method based on the hybrid expert model above, and will not be repeated here.

[0124] In an exemplary embodiment, as shown in FIG8 , a data processing device 800 based on a hybrid expert model is provided, comprising:

[0125] A first determination module 801 is used to determine a target probability corresponding to a target character in the input data, and to determine a target probability difference threshold corresponding to the target character, wherein the target probability includes a probability estimation value of each expert network model in a preset hybrid expert model;

[0126] A first screening module 802 is used to screen each of the expert network models based on the probability estimation value of each of the expert network models and the target probability difference threshold to obtain a target expert network model;

[0127] The first processing module 803 is used to process the target characters based on each target expert network model to obtain a data processing result.

[0128] In one embodiment, the hybrid expert model includes an input layer attention mechanism layer; the device also includes:

[0129] A first acquisition module, configured to acquire the input data through the input layer, wherein the input data includes at least one character;

[0130] The first calculation module is used to perform attention calculation on each of the characters through the attention mechanism layer to obtain the attention weight of the character, and determine the target character based on the attention weight and the character.

[0131] In one embodiment, the first determining module is specifically configured to:

[0132] For each of the target characters, the task type corresponding to the input data, the target character and each expert network model in the hybrid expert model are processed through a preset probability algorithm to obtain a probability estimation value corresponding to each of the expert network models. The probability estimation value represents the accuracy of the expert network model's reasoning ability for the target character.

[0133] In one embodiment, the device also includes: a second acquisition module, used to obtain multiple pre-configured probability difference thresholds; a second calculation module, used to process the input data based on the preset probability difference threshold in the hybrid expert model for each of the pre-configured probability difference thresholds to obtain output data, and obtain an index value of a threshold evaluation index based on the output data; a second screening module, used to screen based on the index value of the threshold evaluation index corresponding to each of the pre-configured probability difference thresholds to obtain a target probability difference threshold.

[0134] In one embodiment, the first screening module is specifically used for:

[0135] Based on the order of probability estimation values ​​from large to small, the expert network models are sorted to obtain a sequence, and the expert network model with the highest probability estimation value is obtained, and the expert network model with the highest probability estimation value is determined as the target expert network model; according to the arrangement order of the sequence, the probability estimation value difference between the target expert network model and the next expert network model of the target expert network model is calculated; if the probability estimation value difference is less than the target probability difference threshold, the next expert network model is determined to be the target expert network model, and the step of calculating the probability estimation value difference between the target expert network model and the next expert network model of the target expert network model according to the arrangement order of the sequence is re-executed until the current probability estimation value difference is greater than or equal to the target probability difference threshold, and at least one target expert network model is obtained.

[0136] In one embodiment, the hybrid expert model also includes an output layer; the first processing module is specifically used to: process the target character through each of the target expert network models to obtain a character processing result; perform weighted calculation based on the probability estimation value of each of the target expert network models and the character processing results corresponding to each of the target expert network models to obtain a weighted result; process the weighted result through the processing operation corresponding to the output layer to obtain the data processing result of the target character.

[0137] Each module in the above-mentioned data processing device based on the hybrid expert model can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0138] In an exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG9. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a data processing method based on a hybrid expert model is implemented. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0139] Those skilled in the art will understand that the structure shown in FIG. 9 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0140] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps described in the above embodiment when executing the computer program.

[0141] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps described in the above embodiment are implemented.

[0142] In one embodiment, a computer program product is provided, including a computer program, which implements the steps described in the above embodiments when executed by a processor.

[0143] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0144] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0145] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0146] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data processing method based on a hybrid expert model, characterized in that: The method comprises: Based on the target character in the input data, determining a target probability corresponding to the target character and determining a target probability difference threshold corresponding to the target character, wherein the target probability includes a probability estimation value of each expert network model in a preset hybrid expert model; Based on the probability estimation value of each expert network model and the target probability difference threshold, each expert network model is screened to obtain a target expert network model; The target characters are processed based on each of the target expert network models to obtain a data processing result.

2. The method according to claim 1, characterized in that The hybrid expert model includes an input layer attention mechanism layer; the method also includes: Acquire the input data through the input layer, wherein the input data includes at least one character; For each of the characters, the attention mechanism layer performs attention calculation on the character to obtain the attention weight of the character, and determines the target character based on the attention weight and the character.

3. The method according to claim 1, characterized in that The step of determining a target probability corresponding to a target character based on the target character in the input data comprises: For each of the target characters, the task type corresponding to the input data, the target character and each expert network model in the hybrid expert model are processed through a preset probability algorithm to obtain a probability estimation value corresponding to each of the expert network models. The probability estimation value represents the accuracy of the expert network model's reasoning ability for the target character.

4. The method according to claim 1, characterized in that The method further comprises: Obtaining a plurality of pre-configured probability difference thresholds; For each of the pre-configured probability difference thresholds, in the hybrid expert model, the input data is processed based on the preset probability difference threshold to obtain output data, and an indicator value of a threshold evaluation indicator is obtained based on the output data; Screening is performed based on the index values ​​of the threshold evaluation indexes corresponding to each of the pre-configured probability difference thresholds to obtain a target probability difference threshold.

5. The method according to any one of claims 1 to 4, characterized in that: The method of screening the expert network models based on the probability estimation values ​​of the expert network models and the target probability difference threshold to obtain the target expert network model includes: Based on the order of probability estimation values ​​from large to small, the expert network models are sorted to obtain a sequence, and the expert network model with the highest probability estimation value is obtained, and the expert network model with the highest probability estimation value is determined as the target expert network model; Calculating the probability estimation value difference between the target expert network model and the next expert network model of the target expert network model according to the arrangement order of the sequence; If the probability estimate difference is less than the target probability difference threshold, the next expert network model is determined to be the target expert network model, and the step of calculating the probability estimate difference between the target expert network model and the next expert network model of the target expert network model in the order of arrangement of the sequence is re-executed until the current probability estimate difference is greater than or equal to the target probability difference threshold, thereby obtaining at least one target expert network model.

6. The method according to claim 1, characterized in that The hybrid expert model also includes an output layer; the target characters are processed based on each of the target expert network models to obtain a data processing result, including: Processing the target characters through each of the target expert network models to obtain a character processing result; Performing weighted calculation based on the probability estimation value of each target expert network model and the character processing results respectively corresponding to each target expert network model to obtain a weighted result; The weighted result is processed through the processing operation corresponding to the output layer to obtain the data processing result of the target character.

7. A data processing device based on a hybrid expert model, characterized in that: The device comprises: A first determination module is used to determine a target probability corresponding to a target character in the input data, and to determine a target probability difference threshold corresponding to the target character, wherein the target probability includes a probability estimation value of each expert network model in a preset hybrid expert model; A first screening module, configured to screen each of the expert network models based on the probability estimation value of each of the expert network models and the target probability difference threshold, to obtain a target expert network model; The first processing module is used to process the target characters based on each target expert network model to obtain a data processing result.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images