Text processing method and device based on hybrid expert model, equipment and medium

Abstract

The invention relates to the artificial intelligence technology, can be applied to service system platforms such as medical health and financial science and technology, and discloses a data processing method, device, equipment and medium based on a hybrid expert model.The method comprises the steps that to-be-processed data is input into the hybrid expert model, routing probability distribution is obtained, the Tsallis entropy of the routing probability distribution is calculated, and the Tsallis entropy of the routing probability distribution is calculated; generating a prediction result according to the Tsallis entropy; evaluating loss based on an auxiliary entropy loss function, constructing a subspace, and adjusting pre-training parameters in the subspace to obtain an optimized hybrid expert model; and obtaining a target prediction result according to the optimized hybrid expert model. According to the dynamic routing mechanism, experts are flexibly selected, an entropy loss function is assisted to optimize a routing decision, uncertainty is reduced, and model convergence is accelerated; and interference of new tasks on old tasks is eliminated through re-parameterization and subspace design, so that the model achieves good balance between stability and plasticity, and generalization and adaptability of model data processing are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a text processing method, device, equipment and medium based on a hybrid expert model. Background Art

[0002] With the development of artificial intelligence (AI) technology, a variety of models have emerged. Big data processing models are complex computational models built using deep neural network technology, capable of handling large amounts of data. These models typically contain millions or even billions of parameters. Through specialized training processes, they are able to efficiently analyze and process massive amounts of data and are widely used in fields such as finance, healthcare, and education. The Mixture of Experts (MoE), a type of big data processing model, significantly improves computational efficiency while maintaining high model capacity by decomposing large models into multiple specialized "expert" sub-networks and combining them with an intelligent gating system to dynamically allocate tasks.

[0003] Various techniques have been proposed for continuous model learning and efficient parameter tuning in mixture of expert models (MEMs). For efficient parameter tuning, methods such as LoRA (Low-Rank Adaptation), Adapter, and Prompt-tuning are widely used. These methods freeze most of the parameters in the pre-trained model and adjust only a small number of introduced parameters to adapt to the downstream task, improving learning efficiency to a certain extent. In the field of continuous learning, various methods based on regularization, memorization, and expansion exist. Regarding the routing mechanism of Mixture of Experts (MoEs), strategies such as Softmax Routing, Top-K Routing, and Top-P Routing are used to select expert networks. However, these techniques still have certain issues: the existing routing mechanism of MoEs is insufficient. Soft Routing activates all expert networks, which can improve prediction accuracy but is computationally expensive. Sparse Routing activates only a subset of experts, which improves efficiency but leads to an unbalanced expert load and performs poorly when handling routing decisions with high uncertainty.

[0004] In the healthcare sector, the AT-MoE (Adaptive Mixed Experts for Task Planning) architecture divides medical experts into three categories: functional experts (e.g., case generation and triage), domain experts (e.g., surgery and radiology), and style experts (e.g., conclusion-oriented and reference-oriented). This architecture enables precision medical decision-making through an innovative hierarchical routing mechanism. This specialized division of labor significantly improves the accuracy of the DeepSeek-R1 medical model in disease risk prediction and image analysis tasks, while reducing data annotation requirements by 30%. The complexity and diversity of medical scenarios have traditionally been a bottleneck for AI implementation. The MoE effectively addresses this challenge by enabling domain experts to focus on optimizing their own performance and then integrating the output through intelligent scheduling. However, existing mixed expert models, in some medical scenarios (e.g., multimodal voice and text recognition in intelligent consultations), activate all expert networks to output consultation information. While this improves prediction accuracy, it incurs significant computational overhead. Activating only a few experts, while improving efficiency, leads to an unbalanced expert load and poor performance when handling routing decisions with high uncertainty.

[0005] In the financial sector, existing financial MoEs can simultaneously run multiple expert modules, such as risk assessment, customer profiling, and real-time transaction analysis. Dynamic routing technology enables millisecond-level response times, improving overall business processing efficiency by over 40%. This parallel processing capability is particularly important for scenarios like high-frequency trading and real-time risk control. Banks implementing this technology have seen a 42% reduction in credit card fraud cases, while computing power consumption has been halved. The high sensitivity of financial data and regulatory requirements have also led to the widespread adoption of data isolation in MoEs. Customer data from different business lines can be processed by dedicated expert modules, preventing cross-information leakage and meeting regulatory requirements. However, existing financial MoEs contain large amounts of real-time data. When optimizing existing MoE models, the existing routing mechanism will increase data processing time if all expert networks are activated. Activating only some expert networks can lead to unbalanced load on the expert networks, resulting in poor performance in the output of financial decisions when handling highly uncertain routing decisions.

[0006] When the existing hybrid expert model is applied in the medical and health fields and the financial fields, the existing routing mechanism based on the hybrid expert model will result in high system overhead if all experts are activated; if only some experts are activated, there will be an unbalanced expert load, which is prone to decision-making errors when dealing with highly uncertain routing decisions. Summary of the Invention

[0007] In view of the above-mentioned deficiencies in the prior art, the present invention provides a text processing method, apparatus, device and medium based on a hybrid expert model, aiming to solve the existing routing mechanism based on the hybrid expert model in the prior art. If all experts are activated, the system overhead will be large; if only some experts are activated, the expert load will be unbalanced, and decision-making errors are likely to occur when processing highly uncertain routing decisions.

[0008] The technical solutions of the present invention are as follows:

[0009] A first embodiment of the present invention provides a text processing method based on a hybrid expert model, the method comprising:

[0010] receiving data to be processed input by a user, inputting the data to be processed into a gating network of a hybrid expert model, and outputting a routing probability distribution based on the gating network, wherein the routing probability distribution is used to indicate the relevance between each expert network in the hybrid expert model and the data to be processed;

[0011] Calculating the Tsallis entropy of the routing probability distribution, and dynamically activating the corresponding expert network according to the Tsallis entropy;

[0012] After inputting the data to be processed into the currently activated expert network for data processing, a corresponding prediction result is generated;

[0013] Evaluate the loss between the prediction result and the true label based on an auxiliary entropy loss function, and calculate the gradient of the loss relative to the pre-trained parameters of the hybrid expert model through a back-propagation algorithm;

[0014] Constructing a subspace, and adjusting the pre-trained parameters of the hybrid expert model based on the gradient in the subspace until the loss between the predicted result and the true label meets a preset condition, thereby obtaining an optimized hybrid expert model;

[0015] The data to be processed is input into the optimized hybrid expert model to obtain a target prediction result.

[0016] Another embodiment of the present invention provides a text processing device based on a hybrid expert model, the device comprising:

[0017] a data receiving module, configured to receive data to be processed input by a user, input the data to be processed into a gating network of a hybrid expert model, and output a routing probability distribution based on the gating network, wherein the routing probability distribution is used to indicate the correlation between each expert network in the hybrid expert model and the data to be processed;

[0018] A dynamic routing module, configured to calculate the Tsallis entropy of the routing probability distribution and dynamically activate the corresponding expert network according to the Tsallis entropy;

[0019] A first data output module is used to input the data to be processed into the currently activated expert network for data processing and generate corresponding prediction results;

[0020] A loss evaluation module is used to evaluate the loss between the prediction result and the true label based on an auxiliary entropy loss function, and calculate the gradient of the loss relative to the pre-trained parameters of the hybrid expert model through a back propagation algorithm;

[0021] A model optimization module is used to construct a subspace and adjust the pre-trained parameters of the hybrid expert model based on the gradient in the subspace until the loss between the predicted result and the true label meets a preset condition, thereby obtaining an optimized hybrid expert model;

[0022] The second data output module is used to input the data to be processed into the optimized hybrid expert model to obtain a target prediction result.

[0023] Another embodiment of the present invention provides a computer device, the computer device comprising at least one processor; and

[0024] a memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the above-mentioned text processing method based on the hybrid expert model.

[0026] Another embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the one or more processors can perform the steps of the above-mentioned text processing method based on the hybrid expert model.

[0027] Beneficial effects: The text processing method, device, equipment and medium based on the hybrid expert model of the embodiment of the present invention can be realized through the data processing method, device, equipment and medium based on the hybrid expert model, including: inputting the data to be processed into the gating network of the hybrid expert model, outputting the routing probability distribution based on the gating network, calculating the Tsallis entropy of the routing probability distribution, and dynamically activating the corresponding expert network according to the Tsallis entropy; generating corresponding prediction results according to the currently activated expert network; evaluating the loss based on the auxiliary entropy loss function, constructing a subspace, and adjusting the pre-trained parameters of the hybrid expert model within the subspace to obtain an optimized hybrid expert model; inputting the data to be processed into the optimized hybrid expert model to obtain the target prediction result. The present invention flexibly selects experts through a dynamic routing mechanism, and uses the auxiliary entropy loss function to optimize routing decisions, reduce uncertainty, and accelerate model convergence; eliminates the interference of new tasks on old tasks through reparameterization and subspace design, so that the model achieves a good balance between stability and plasticity, improves generalization and adaptability, and thus improves the efficiency and accuracy of data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0029] Figure 1 A schematic diagram of an application environment of an embodiment of a text processing method based on a hybrid expert model of the present invention;

[0030] Figure 2 Flowchart of a preferred embodiment of a text processing method based on a hybrid expert model of the present invention;

[0031] Figure 3 for Figure 2 A flowchart of a specific embodiment of step S100;

[0032] Figure 4 for Figure 2 A flowchart of a specific embodiment of step S200;

[0033] Figure 5 for Figure 2 A flowchart of a specific embodiment of step S300;

[0034] Figure 6 for Figure 2 A flowchart of a specific embodiment of step S400;

[0035] Figure 7 for Figure 2 A flowchart of a specific embodiment of step S500;

[0036] Figure 8 for Figure 2 A flowchart of another specific embodiment of step S500;

[0037] Figure 9 Schematic diagram of functional modules of a preferred embodiment of a text processing device based on a hybrid expert model of the present invention;

[0038] Figure 10 A schematic structural diagram of a preferred embodiment of a computer device of the present invention;

[0039] Figure 11 This is another structural diagram of a preferred embodiment of a computer device of the present invention. DETAILED DESCRIPTION

[0040] To make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail below. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0041] The following describes the embodiments of the present invention in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0042] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. Here, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0043] The method provided in this application can be applied to artificial intelligence (AI) scenarios. AI is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is also the study of the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making. Research in the field of artificial intelligence includes robotics, natural speech processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, basic AI theory, etc.

[0044] The text processing method based on the hybrid expert model provided by the embodiment of the present invention can be applied in Figure 1 In an application environment, the client communicates with the server through a network. The client accesses the network or business platform of the server, and the server can receive the data to be processed input by the user, input the data to be processed into the gating network of the hybrid expert model, and output a routing probability distribution based on the gating network, wherein the routing probability distribution is used to indicate the correlation between each expert network in the hybrid expert model and the data to be processed; calculate the Tsallis entropy of the routing probability distribution, and dynamically activate the corresponding expert network according to the Tsallis entropy; input the data to be processed into the currently activated expert network for data processing, and generate a corresponding prediction result; evaluate the loss between the prediction result and the true label based on the auxiliary entropy loss function, and calculate the gradient of the loss relative to the pre-trained parameters of the hybrid expert model through the back-propagation algorithm; construct a subspace, and adjust the pre-trained parameters of the hybrid expert model based on the gradient in the subspace until the loss between the prediction result and the true label meets the preset conditions, thereby obtaining an optimized hybrid expert model; input the data to be processed into the optimized hybrid expert model to obtain the target prediction result. In the present invention, experts are flexibly selected through a dynamic routing mechanism, and the entropy loss function is used to assist in optimizing routing decisions, reducing uncertainty and accelerating model convergence. The interference of new tasks on old tasks is eliminated through reparameterization and subspace design, so that the model can achieve a good balance between stability and plasticity, improve generalization and adaptability, and thus improve the efficiency and accuracy of data processing. Among them, the client can be, but is not limited to, various personal computers, laptops, smart phones, tablets and portable wearable devices. The server can be implemented with an independent server or a server cluster composed of multiple servers. The present invention is described in detail below through specific embodiments.

[0045] To solve the above problems, the present invention provides a method for text processing based on a hybrid expert model. Figure 2 , Figure 2 FIG. 1 is a flow chart of a preferred embodiment of a text processing method based on a hybrid expert model of the present invention. Figure 2 As shown, it includes:

[0046] Step S100: receiving data to be processed input by a user, inputting the data to be processed into a gating network of a hybrid expert model, and outputting a routing probability distribution based on the gating network, wherein the routing probability distribution is used to indicate the correlation between each expert network in the hybrid expert model and the data to be processed.

[0047] The main components of the MoE hybrid expert model include:

[0048] Experts: Each expert in the model is an independent neural network that specializes in processing a specific subset of the input data or a specific task. For example, in natural language processing tasks, one expert may focus on processing content related to language syntax, while another expert may focus on semantic understanding.

[0049] Gating Network: The gating network determines which expert or experts should process each input sample. It calculates the weight or importance of each expert based on the input sample's characteristics and then assigns the input sample to the appropriate expert based on these weights. The gating network is typically a simple neural network whose output is processed by a softmax activation function to ensure that the sum of all expert weights is 1.

[0050] Softmax activation function: To ensure that the sum of the weights of all experts is 1, the output of the router is usually processed by the softmax activation function. The softmax function can convert a real number vector into a probability distribution, so that the weight of each expert is in the range [0,1] and the sum of all weights is 1. In this way, the weight can be interpreted as the probability that the input data belongs to each expert, providing a reasonable basis for the distribution of input data. For example, for a MoE model containing three experts, the weights calculated by the router may be [0.2, 0.5, 0.3], which means that the input data has a 20% probability of being processed by the first expert, a 50% probability of being processed by the second expert, and a 30% probability of being processed by the third expert.

[0051] The expert network in the Mixture of Experts (MoE) model, a core component of the model, is an independent neural network specialized for processing a specific subset of input data or a specific task. Each expert is designed to deeply learn and optimize the data characteristics or task types in a specific domain to achieve efficient processing and accurate prediction of input data.

[0052] In most MoE model applications, the expert network typically adopts a feed-forward neural network (FFN) structure. This structure consists of multiple fully connected layers, each of which is usually followed by a nonlinear activation function, such as ReLU (Rectified Linear Unit) or GELU (Gaussian Error Linear Unit), to increase the model's nonlinear expression capabilities. For example, in natural language processing tasks, the expert network can be composed of two fully connected layers. The first layer maps the input word embedding vector to a high-dimensional space, and the second layer maps the high-dimensional features back to the output space. Through the nonlinear transformation of the ReLU activation function, the model can capture complex language patterns and semantic information.

[0053] Each expert network is designed to focus on a specific task or data feature. Taking image classification tasks in the medical industry as an example, one expert network may be specialized in identifying features of images of human lesions, such as skin color and wound size; another expert may focus on identifying features of lesion locations, such as abdominal contours or leg contours. This task-specific design enables experts to conduct deep learning and optimization in their areas of expertise, thereby improving processing efficiency and prediction accuracy for specific types of data. In natural language processing, some experts may focus on processing grammatical structures, such as the subject-verb-object relationship in a sentence; while other experts may focus on semantic understanding, such as semantic role labeling of words. In this way, the MoE model can comprehensively utilize the expertise of multiple experts to conduct a comprehensive and in-depth analysis of complex input data.

[0054] The data processing method of the present invention can be used in intelligent customer service Q&A in the financial sector. The user input data may be text or voice data. Using a hybrid expert model, text can be processed by an expert network that is more adept at processing text, while voice can be processed by an expert network in the model that is more adept at processing voice.

[0055] In some other embodiments, the data processing method can also be a pre-diagnosis system in the medical field, where the user enters basic information and a hybrid expert model is used. Different expert models can be used according to different departments. Patient data related to ENT is pushed to the ENT expert network, and patient data related to internal medicine is pushed to the internal medicine expert network.

[0056] Among them, Figure 3 As shown, step S100, i.e. receiving data to be processed input by a user, inputting the data to be processed into a gating network of a hybrid expert model, and outputting a routing probability distribution based on the gating network, includes:

[0057] Step S101: receiving data to be processed input by a user, performing feature extraction on the data to be processed, and generating an input vector corresponding to the data to be processed;

[0058] Step S102: inputting the input vector into the gating network of the hybrid expert model;

[0059] Step S103: Outputting a routing probability distribution corresponding to the expert network based on the gating network.

[0060] The network receives data to be processed from the user, including but not limited to voice, text, images, and videos. Feature extraction is performed on the data based on its format to generate an input vector for the gating network. The gating network is often a simple neural network, whose input is either the raw input data or a preprocessed feature vector. The network learns the correlation between the input data and each expert and calculates a weight or importance score for each expert. These weights reflect the degree of fit between the input data and the expert's area of expertise. Higher weights indicate a stronger expert's ability to process the current input data. The gating network then distributes the input data to the corresponding experts according to the calculated weights, completing the data routing task.

[0061] For example, in text classification tasks in the medical field, a gating network might assign text to experts who specialize in processing information related to age, gender, and medical history based on features such as the patient's age, gender, and medical history. For example, if a female patient over 60 years old experiences abdominal pain, the patient might be directed to an expert network specializing in gynecology-related information based on their medical history. The weights of the expert networks for gynecology and internal medicine will be much greater than those for the expert network for pediatrics. If the patient is a child under 14 years old and experiences abdominal pain, the patient will be directed to an expert network specializing in pediatrics-related information based on their medical history. The weight of the expert network for pediatrics will also be higher than that of the expert network for gynecology.

[0062] For another example, in intelligent customer service Q&A in the financial sector, assume that the hybrid expert model has three expert networks: a first expert network that processes text information, a second expert network that processes voice information, and a third expert network that processes video information. When an input vector is fed into the gated network, the gated network recognizes the input vector. If the input vector is text, weights can be assigned by setting the weight of the first expert network to 0.8, and the weights of the second and third expert networks to 0.1. If the input vector is voice information, weights can be assigned by setting the weight of the second expert network to 0.8, and the weights of the first and third expert networks to 0.1. If the input vector is video information, weights can be assigned by setting the weight of the third expert network to 0.8, and the weights of the first and second expert networks to 0.1.

[0063] Step S200: Calculate the Tsallis entropy of the routing probability distribution, and dynamically activate the corresponding expert network according to the Tsallis entropy.

[0064] Tsallis entropy is a type of thermodynamic entropy in thermodynamics, related to Shannon entropy and Renyi entropy, and is a generalization of the traditional concept of entropy. It introduces a non-negative real number parameter, q, that can be adjusted to accommodate different probability distributions and system characteristics, providing a more flexible information measurement when dealing with complex systems.

[0065] For a grayscale image, it can be regarded as a probability distribution function, where each pixel value has a certain probability of occurring. According to the size of the Tsallis entropy, the activation method of the expert network can be dynamically selected.

[0066] Among them, Figure 4 As shown, step S200, i.e., calculating the Tsallis entropy of the routing probability distribution and dynamically activating the corresponding expert network according to the Tsallis entropy, includes:

[0067] Step S201, calculating the Tsallis entropy of the routing probability distribution, and determining whether the Tsallis entropy is greater than a preset threshold, if so, executing step S202, if not, executing step S203;

[0068] Step S202: activating the corresponding expert network based on the weighted probability, wherein the weighted weight is generated based on Tsallis entropy and the expert network score;

[0069] Step S203: activating at least k expert networks based on the dynamic routing mechanism, wherein the cumulative probability of the k expert networks exceeds a preset probability threshold, and k is a positive integer greater than or equal to 1.

[0070] The calculation formula of Tsallis entropy is: Where p is the routing probability distribution, q is the order of Tsallis entropy, and M is the number of experts. By calculating the Tsallis entropy of the routing probability distribution, the routing mechanism is dynamically adjusted. For high entropy tokens (i.e., T q (p) is greater than a certain threshold τ), soft routing is adopted to enable the model to select from a wider range of experts; for low entropy tokens (i.e., T q (p) is less than or equal to the threshold τ), a more deterministic Top-P routing is adopted to focus on more likely experts, while ensuring that at least k experts are activated to prevent overfitting, balancing exploration and utilization, and improving the performance of the model under different conditions.

[0071] Top-P routing is a dynamic routing mechanism and a routing strategy in the Mixture of Experts (MoE) model. It selects experts with higher routing probabilities based on the input token until the cumulative probability exceeds a threshold p. It allows the number of experts to be dynamically selected based on the complexity of the input. The expert selection probability P reflects the model's confidence in the expert's ability to process the input. Entropy minimization goal: By minimizing the entropy of the distribution P, each token is ensured to focus on as few experts as possible, thereby improving computational efficiency. Load balancing loss: A load balancing loss mechanism is added to achieve load balancing between different experts and improve training efficiency.

[0072] Step S300: Input the data to be processed into the currently activated expert network for data processing, and generate corresponding prediction results.

[0073] In the expert network activated by the input of the data to be processed, the expert network processes the data to be processed and generates an intermediate prediction result. Then, according to the output of the gating network, the corresponding expert is selected to process the input data and obtain the corresponding prediction result.

[0074] Among them, such as Figure 5 As shown, step S300, that is, inputting the data to be processed into the currently activated expert network for data processing, generates corresponding prediction results, including:

[0075] Step S301: input the data to be processed into the currently activated expert network to obtain the intermediate prediction result output by the expert network;

[0076] Step S302: Obtain the expert model weight of the gating network configuration;

[0077] Step S303: weightedly combine the intermediate prediction results based on the expert model weights to generate corresponding prediction results.

[0078] In the soft routing stage, the selection of experts is based on weighted probability, and the weight is determined by Tsallis entropy and expert network score. Assume that the score of expert i is si , then its weighted probability w i w i =s i ·(1+T q In the Top-P routing phase, the minimum set of experts whose cumulative probability exceeds P is selected, and the number of experts in the set is not less than k.

[0079] Step S400: Evaluate the loss between the prediction result and the true label based on the auxiliary entropy loss function, and calculate the gradient of the loss relative to the pre-trained parameters of the hybrid expert model through a back propagation algorithm.

[0080] A loss function, or cost function, maps the value of a random event or its associated random variables to a non-negative real number to represent the "risk" or "loss" of the random event. In applications, loss functions are often used as learning criteria in conjunction with optimization problems, where models are solved and evaluated by minimizing the loss function. In this embodiment of the present invention, an auxiliary entropy loss function is introduced for evaluation, reducing router uncertainty.

[0081] Among them, Figure 6 As shown, in step S400, the loss between the prediction result and the true label is evaluated based on the auxiliary entropy loss function, and the gradient of the loss relative to the pre-trained parameters of the hybrid expert model is calculated by the back propagation algorithm, including:

[0082] Step S401: Calculate the entropy loss term based on Tsallis entropy;

[0083] Step S402: Obtain the number of times each expert is activated and the total number of activations, and calculate a load balancing loss item based on the number of times each expert is activated and the total number of activations;

[0084] Step S403: obtaining an auxiliary entropy loss function based on the entropy loss term and the load balancing loss term;

[0085] Step S404: Evaluate the loss between the prediction result and the true label based on the auxiliary entropy loss function;

[0086] Step S405: Calculate the gradient of the loss relative to the pre-trained parameters of the hybrid expert model through a back propagation algorithm.

[0087] An auxiliary loss function based on Tsallis entropy and load balancing is introduced. The entropy loss term is calculated based on Tsallis entropy and is used to reduce routing uncertainty. Its formula is: T q (p), where α is a hyperparameter that controls the weight of the entropy loss term.

[0088] The load balancing loss term draws on existing methods to encourage experts to balance loads. The formula is: Where β is a hyperparameter, n i is the number of times the i-th expert is activated, and N is the total number of activations.

[0089] The total auxiliary loss function is Through this loss function, the uncertainty of routers is effectively reduced, the balanced use of experts is promoted, and the model performance is improved.

[0090] Step S500: construct a subspace, and adjust the pre-training parameters of the hybrid expert model based on the gradient in the subspace until the loss between the prediction result and the true label meets a preset condition, thereby obtaining an optimized hybrid expert model.

[0091] The gradient of the loss with respect to the model parameters is calculated using the backpropagation algorithm. For the expert network, its internal parameters are updated to reduce the loss. The parameters of the gating network also need to be updated. This step is crucial to ensure that the gating network can accurately select the appropriate expert for different inputs. Parameter Update: Optimization algorithms (such as SGD and Adam) are used to adjust all learnable parameters to reduce the overall loss and improve model performance. This is done until the loss between the predicted results and the true labels meets the preset conditions, resulting in the optimized hybrid expert model.

[0092] Among them, such as Figure 7 As shown, in step S500, constructing a subspace includes:

[0093] Step S511: obtaining a first approximate matrix corresponding to an original task gradient of the hybrid expert model, where the original task is a processing task of the to-be-processed data performed by the hybrid expert model;

[0094] Step S512: obtaining a first approximation matrix corresponding to an original task gradient of the hybrid expert model, where the original task is a processing task of the to-be-processed data performed by the hybrid expert model;

[0095] Step S513: performing singular value decomposition on the first approximate matrix and the second approximate matrix;

[0096] Step S514: construct a subspace according to the decomposition result.

[0097] Use DualGPM and other methods to approximate the gradient space of the new and old tasks. Assume that the approximate matrix G of the old task gradient is obtained by DualGPM old and the approximate matrix G of the new task gradient new .

[0098] DualGPM (Dual Gradient Projection Memory) is a technology used to approximate the gradient space of old tasks or its orthogonal complement space, and plays a key role in continuous learning scenarios such as incremental learning.

[0099] DualGPM uses projection to map the new task input matrix into the orthogonal complement of the old task gradient space, thereby determining a basis vector that is orthogonal to the old task gradient and lies within the subspace of the new task gradient. This design allows the model to avoid interfering with the old task when processing the new task, maintaining task independence.

[0100] To G old and G new Perform singular value decomposition, that is and By selecting appropriate singular vectors and singular values and constructing the subspace S, the design of the subspace can not only avoid the interference of new task learning on old tasks, but also allow the model to focus on new tasks, effectively reducing the conflict of parameter updates in the continuous learning process and improving the overall performance of the model.

[0101] Among them, such as Figure 8 As shown, in step S500, the pre-trained parameters of the hybrid expert model are adjusted based on the gradient in the subspace until the loss between the prediction result and the true label meets the preset conditions, thereby obtaining an optimized hybrid expert model, including:

[0102] Step S521: obtaining pre-training weights in the subspace, injecting a low-rank matrix into the pre-training weights, and generating updated pre-training weights;

[0103] Step S522: adjusting the pre-training parameters of the hybrid expert model based on the updated pre-training weights to generate an adjusted hybrid expert model;

[0104] Step S523: input the data to be processed into the adjusted hybrid expert model again, and calculate the loss between the predicted result and the real label output by the adjusted hybrid expert model until the loss between the predicted result and the real label meets the preset conditions, thereby obtaining the optimized hybrid expert model.

[0105] The LoRA interference elimination mechanism is achieved through reparameterization and subspace design.

[0106] Reparameterization: Reparameterize the pre-trained weights by injecting a small amount of parameters. Let the pre-trained weights be W, and the injected low-rank matrices be A and B, then the reparameterized weights are in d is the dimension of the original weight matrix, r is the rank of the low-rank matrix, and r<<d.

[0107] Fine-tuning the parameters is equivalent to fine-tuning the pre-trained weights in a pre-designed subspace, which is designed to be orthogonal to the old task gradient and located in the subspace where the new task gradient is located. Let the old task gradient be g old , the new task gradient is g new , through operations such as singular value decomposition (SVD), find a subspace S such that And S can effectively represent g new , thereby eliminating the interference of new tasks on old tasks and achieving a good balance between stability and plasticity.

[0108] Step S600: input the data to be processed into the optimized hybrid expert model to obtain a target prediction result.

[0109] After parameter self-optimization, the data to be processed is subjected to feature extraction and then input into the optimized hybrid expert model to obtain the final target prediction result. The feature extraction method for specific fields is as follows:

[0110] Text processing field: TF-IDF: Extract term frequency-inverse document frequency features for text classification.

[0111] Word2Vec / GloVe: Map words into low-dimensional vectors to capture semantic information.

[0112] Image processing: SIFT / SURF: Extract local features of images for target detection. HOG: Extract the gradient histogram of images for pedestrian detection.

[0113] Signal Processing: Fourier Transform: Converts signals from the time domain to the frequency domain to extract frequency features. Mel-Frequency Cepstral Coefficients (MFCC): Extracts spectral features of speech signals.

[0114] Compared with the prior art, the text processing method based on the hybrid expert model in the embodiment of the present invention has the following technical advantages:

[0115] This approach combines a dynamic hybrid routing strategy with a low-rank adaptive method for LoRA interference elimination. In terms of routing, a dynamic routing mechanism is designed based on Tsallis entropy calculation, using soft routing and Top-P routing for high-entropy and low-entropy tokens, respectively. An auxiliary loss function based on Tsallis entropy and load balancing is introduced to optimize routing decisions. In terms of parameter fine-tuning, the pre-trained weights are re-parameterized by injecting a small amount of parameters. A subspace orthogonal to the old task gradient and located in the new task gradient subspace is designed to eliminate interference from the new task on the old one, achieving a balance between stability and plasticity, thereby improving model performance and optimizing resource utilization efficiency.

[0116] It should be noted that there is not necessarily a certain order between the above steps. A person skilled in the art can understand, based on the description of the embodiments of the present invention, that in different embodiments, the above steps may have different execution orders, that is, they may be executed in parallel, or may be executed interchangeably, etc.

[0117] Another embodiment of the present invention provides a text processing device based on a hybrid expert model, which corresponds one-to-one to the text processing method based on the hybrid expert model in the above embodiment. Figure 9 As shown, the device 1 includes:

[0118] The data receiving module 100 is configured to receive data to be processed input by a user, input the data to be processed into a gating network of a hybrid expert model, and output a routing probability distribution based on the gating network, wherein the routing probability distribution is used to indicate the correlation between each expert network in the hybrid expert model and the data to be processed;

[0119] A dynamic routing module 200 is configured to calculate the Tsallis entropy of the routing probability distribution and dynamically activate a corresponding expert network based on the Tsallis entropy;

[0120] The first data output module 300 is used to input the data to be processed into the currently activated expert network for data processing and generate corresponding prediction results;

[0121] A loss evaluation module 400 is configured to evaluate the loss between the prediction result and the true label based on an auxiliary entropy loss function, and calculate the gradient of the loss relative to the pre-trained parameters of the hybrid expert model through a back propagation algorithm;

[0122] A model optimization module 500 is configured to construct a subspace and adjust the pre-trained parameters of the hybrid expert model based on the gradient within the subspace until the loss between the predicted result and the true label meets a preset condition, thereby obtaining an optimized hybrid expert model;

[0123] The second data output module 600 is used to input the data to be processed into the optimized hybrid expert model to obtain a target prediction result.

[0124] The specific implementation method is shown in the method embodiment, which will not be repeated here.

[0125] In one embodiment, the data receiving module 100 is specifically configured to:

[0126] Receive data to be processed input by a user, perform feature extraction on the data to be processed, and generate an input vector corresponding to the data to be processed;

[0127] Inputting the input vector into the gating network of the mixture of experts model;

[0128] The routing probability distribution corresponding to the expert network is output based on the gating network.

[0129] The specific implementation method is shown in the method embodiment, which will not be repeated here.

[0130] In one embodiment, the dynamic routing module 200 is specifically configured to:

[0131] Calculating the Tsallis entropy of the routing probability distribution, and determining whether the Tsallis entropy is greater than a preset threshold;

[0132] If the Tsallis entropy is greater than a preset entropy threshold, the corresponding expert network is activated based on the weighted probability, wherein the weighted weight is generated based on the Tsallis entropy and the expert network score;

[0133] If the Tsallis entropy is less than or equal to a preset entropy threshold, at least k expert networks are activated based on a dynamic routing mechanism, wherein the cumulative probability of the k expert networks exceeds a preset probability threshold, and k is a positive integer greater than or equal to 1.

[0134] The specific implementation method is shown in the method embodiment, which will not be repeated here.

[0135] In one embodiment, the first data output module 300 is specifically configured to:

[0136] Inputting the data to be processed into the currently activated expert network to obtain the intermediate prediction result output by the expert network;

[0137] Obtaining expert model weights for the gating network configuration;

[0138] The intermediate prediction results are weighted and combined based on the expert model weights to generate corresponding prediction results.

[0139] The specific implementation method is shown in the method embodiment, which will not be repeated here.

[0140] In one embodiment, the loss assessment module 400 is specifically configured to:

[0141] Calculate the entropy loss term based on Tsallis entropy;

[0142] Obtain the number of times each expert is activated and the total number of activations, and calculate the load balancing loss item based on the number of times each expert is activated and the total number of activations;

[0143] Obtaining an auxiliary entropy loss function based on the entropy loss term and the load balancing loss term;

[0144] Evaluate the loss between the predicted result and the true label based on an auxiliary entropy loss function;

[0145] The gradient of the loss with respect to the pre-trained parameters of the mixture of experts model is calculated by a back-propagation algorithm.

[0146] The specific implementation method is shown in the method embodiment, which will not be repeated here.

[0147] In one embodiment, the model optimization module 500 is specifically configured to:

[0148] Obtaining a first approximate matrix corresponding to an original task gradient of the hybrid expert model, where the original task is a processing task of the to-be-processed data performed by the hybrid expert model;

[0149] Obtaining a second approximate matrix corresponding to a new task gradient of the hybrid expert model, where the new task is a pre-training parameter adjustment task;

[0150] Performing singular value decomposition on the first approximate matrix and the second approximate matrix;

[0151] Construct subspaces based on the decomposition results.

[0152] The specific implementation method is shown in the method embodiment, which will not be repeated here.

[0153] In one embodiment, the model optimization module 500 is further configured to:

[0154] Obtaining pre-trained weights in the subspace, injecting a low-rank matrix into the pre-trained weights, and generating updated pre-trained weights;

[0155] Adjusting the pre-training parameters of the hybrid expert model based on the updated pre-training weights to generate an adjusted hybrid expert model;

[0156] The data to be processed is input into the adjusted hybrid expert model again, and the loss between the predicted result and the real label output by the adjusted hybrid expert model is calculated until the loss between the predicted result and the real label meets the preset conditions, thereby obtaining the optimized hybrid expert model.

[0157] The specific implementation method is shown in the method embodiment, which will not be repeated here.

[0158] The present invention provides a text processing device based on a hybrid expert model. It flexibly selects experts through a dynamic routing mechanism, assists in optimizing routing decisions with an entropy loss function, reduces uncertainty, and accelerates model convergence. Reparameterization and subspace design eliminate the interference of new tasks on old tasks, so that the model achieves a good balance between stability and plasticity, and improves the generalization and adaptability of the model data processing.

[0159] Another embodiment of the present invention provides a computer device, which may be a server, and its internal structure diagram may be as shown in FIG. Figure 10 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the server side of a text processing method based on a hybrid expert model.

[0160] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 11 As shown. The computer device includes a processor, memory, network interface, display screen and input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the client side of a text processing method based on a hybrid expert model.

[0161] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed:

[0162] receiving data to be processed input by a user, inputting the data to be processed into a gating network of a hybrid expert model, and outputting a routing probability distribution based on the gating network, wherein the routing probability distribution is used to indicate the relevance between each expert network in the hybrid expert model and the data to be processed;

[0163] Calculating the Tsallis entropy of the routing probability distribution, and dynamically activating the corresponding expert network according to the Tsallis entropy;

[0164] After inputting the data to be processed into the currently activated expert network for data processing, a corresponding prediction result is generated;

[0165] Evaluate the loss between the prediction result and the true label based on an auxiliary entropy loss function, and calculate the gradient of the loss relative to the pre-trained parameters of the hybrid expert model through a back-propagation algorithm;

[0166] Constructing a subspace, and adjusting the pre-trained parameters of the hybrid expert model based on the gradient in the subspace until the loss between the predicted result and the true label meets a preset condition, thereby obtaining an optimized hybrid expert model;

[0167] The data to be processed is input into the optimized hybrid expert model to obtain a target prediction result.

[0168] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0169] receiving data to be processed input by a user, inputting the data to be processed into a gating network of a hybrid expert model, and outputting a routing probability distribution based on the gating network, wherein the routing probability distribution is used to indicate the relevance between each expert network in the hybrid expert model and the data to be processed;

[0170] Calculating the Tsallis entropy of the routing probability distribution, and dynamically activating the corresponding expert network according to the Tsallis entropy;

[0171] After inputting the data to be processed into the currently activated expert network for data processing, a corresponding prediction result is generated;

[0172] Evaluate the loss between the prediction result and the true label based on an auxiliary entropy loss function, and calculate the gradient of the loss relative to the pre-trained parameters of the hybrid expert model through a back-propagation algorithm;

[0173] Constructing a subspace, and adjusting the pre-trained parameters of the hybrid expert model based on the gradient in the subspace until the loss between the predicted result and the true label meets a preset condition, thereby obtaining an optimized hybrid expert model;

[0174] The data to be processed is input into the optimized hybrid expert model to obtain a target prediction result.

[0175] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can be found in the relevant descriptions of the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0176] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchl ink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0177] The embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected to achieve the objectives of the present embodiments as needed.

[0178] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, or of course, can also be implemented by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the relevant technology can be embodied in the form of a software product. This computer software product can be present in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or certain parts of the embodiment.

[0179] It should be noted that if software tools or components other than those of our company appear in the embodiments of this application, they are only used for illustration and do not represent actual use.

[0180] Conditional language such as "can," "may," or "might," among others, unless specifically stated otherwise or otherwise understood within the context as used, is generally intended to convey that particular embodiments can include, while other embodiments do not, particular features, elements, and / or operations. Thus, such conditional language is also generally intended to imply that features, elements, and / or operations are anyway required for one or more embodiments or that one or more embodiments must include logic for determining, with or without input or prompting, whether such features, elements, and / or operations are included or to be performed in any particular embodiment.

[0181] What has been described herein in this specification and the accompanying drawings includes examples that can provide methods and apparatus for text processing based on hybrid expert models. Of course, it is not possible to describe every conceivable combination of elements and / or methods for the purpose of describing the various features of the present disclosure, but it will be appreciated that many additional combinations and permutations of the disclosed features are possible. Therefore, it will be apparent that various modifications can be made to the present disclosure without departing from the scope or spirit of the present disclosure. In addition, or in the alternative, other embodiments of the present disclosure may be apparent from consideration of the present specification and the accompanying drawings and from the practice of the present disclosure as presented herein. It is intended that the examples set forth in this specification and the accompanying drawings be considered in all respects to be illustrative and not restrictive. Although specific terms are employed herein, they are used in a general and descriptive sense and not for purposes of limitation.

Claims

1. A data processing method based on a hybrid expert model, characterized by , the method comprises: receiving data to be processed input by a user, inputting the data to be processed into a gating network of a hybrid expert model, and outputting a routing probability distribution based on the gating network, wherein the routing probability distribution is used to indicate the relevance between each expert network in the hybrid expert model and the data to be processed; Calculating the Tsallis entropy of the routing probability distribution, and dynamically activating the corresponding expert network according to the Tsallis entropy; After inputting the data to be processed into the currently activated expert network for data processing, a corresponding prediction result is generated; Evaluate the loss between the prediction result and the true label based on an auxiliary entropy loss function, and calculate the gradient of the loss relative to the pre-trained parameters of the hybrid expert model through a back-propagation algorithm; Constructing a subspace, and adjusting the pre-trained parameters of the hybrid expert model based on the gradient in the subspace until the loss between the predicted result and the true label meets a preset condition, thereby obtaining an optimized hybrid expert model; The data to be processed is input into the optimized hybrid expert model to obtain a target prediction result.

2. The data processing method based on the hybrid expert model according to claim 1, characterized in that: The receiving of data to be processed input by a user, inputting the data to be processed into a gating network of a hybrid expert model, and outputting a routing probability distribution based on the gating network includes: Receive data to be processed input by a user, perform feature extraction on the data to be processed, and generate an input vector corresponding to the data to be processed; Inputting the input vector into the gating network of the mixture of experts model; The routing probability distribution corresponding to the expert network is output based on the gating network.

3. The data processing method based on hybrid expert model according to claim 1, characterized in that: The calculating of the Tsallis entropy of the routing probability distribution and dynamically activating the corresponding expert network according to the Tsallis entropy include: Calculating the Tsallis entropy of the routing probability distribution, and determining whether the Tsallis entropy is greater than a preset threshold; If the Tsallis entropy is greater than a preset entropy threshold, the corresponding expert network is activated based on the weighted probability, wherein the weighted weight is generated based on the Tsallis entropy and the expert network score; If the Tsallis entropy is less than or equal to a preset entropy threshold, at least k expert networks are activated based on a dynamic routing mechanism, wherein the cumulative probability of the k expert networks exceeds a preset probability threshold, and k is a positive integer greater than or equal to 1.

4. The data processing method based on hybrid expert model according to claim 1, characterized in that: After the data to be processed is input into the currently activated expert network for data processing, a corresponding prediction result is generated, including: Inputting the data to be processed into the currently activated expert network to obtain the intermediate prediction result output by the expert network; Obtaining expert model weights for the gating network configuration; The intermediate prediction results are weighted and combined based on the expert model weights to generate corresponding prediction results.

5. The data processing method based on hybrid expert model according to claim 1, characterized in that: The step of evaluating the loss between the prediction result and the true label based on the auxiliary entropy loss function, and calculating the gradient of the loss relative to the pre-trained parameters of the hybrid expert model through a back-propagation algorithm, includes: Calculate the entropy loss term based on Tsallis entropy; Obtain the number of times each expert is activated and the total number of activations, and calculate the load balancing loss item based on the number of times each expert is activated and the total number of activations; Obtaining an auxiliary entropy loss function based on the entropy loss term and the load balancing loss term; Evaluate the loss between the predicted result and the true label based on an auxiliary entropy loss function; The gradient of the loss with respect to the pre-trained parameters of the mixture of experts model is calculated by a back-propagation algorithm.

6. The data processing method based on hybrid expert model according to claim 1, characterized in that: The constructing of the subspace includes: Obtaining a first approximate matrix corresponding to an original task gradient of the hybrid expert model, where the original task is a processing task of the to-be-processed data performed by the hybrid expert model; Obtaining a second approximate matrix corresponding to a new task gradient of the hybrid expert model, where the new task is a pre-training parameter adjustment task; Performing singular value decomposition on the first approximate matrix and the second approximate matrix; Construct subspaces based on the decomposition results.

7. The data processing method based on hybrid expert model according to claim 1, characterized in that: The pre-training parameters of the hybrid expert model are adjusted based on the gradient in the subspace until the loss between the prediction result and the true label meets a preset condition, thereby obtaining an optimized hybrid expert model, including: Obtaining pre-trained weights in the subspace, injecting a low-rank matrix into the pre-trained weights, and generating updated pre-trained weights; Adjusting the pre-training parameters of the hybrid expert model based on the updated pre-training weights to generate an adjusted hybrid expert model; The data to be processed is input into the adjusted hybrid expert model again, and the loss between the predicted result and the real label output by the adjusted hybrid expert model is calculated until the loss between the predicted result and the real label meets the preset conditions, thereby obtaining the optimized hybrid expert model.

8. A data processing device based on a hybrid expert model, characterized in that: The device comprises: a data receiving module, configured to receive data to be processed input by a user, input the data to be processed into a gating network of a hybrid expert model, and output a routing probability distribution based on the gating network, wherein the routing probability distribution is used to indicate the correlation between each expert network in the hybrid expert model and the data to be processed; A dynamic routing module, configured to calculate the Tsallis entropy of the routing probability distribution and dynamically activate the corresponding expert network according to the Tsallis entropy; A first data output module is used to input the data to be processed into the currently activated expert network for data processing and generate corresponding prediction results; A loss evaluation module is used to evaluate the loss between the prediction result and the true label based on an auxiliary entropy loss function, and calculate the gradient of the loss relative to the pre-trained parameters of the hybrid expert model through a back propagation algorithm; A model optimization module is used to construct a subspace and adjust the pre-trained parameters of the hybrid expert model based on the gradient in the subspace until the loss between the predicted result and the true label meets a preset condition, thereby obtaining an optimized hybrid expert model; The second data output module is used to input the data to be processed into the optimized hybrid expert model to obtain a target prediction result.

9. A computer device, characterized in that: The computer device includes at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the data processing method based on the hybrid expert model according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the one or more processors can perform the steps of the data processing method based on the hybrid expert model according to any one of claims 1 to 7.

Images