Task processing method and device, computer equipment and storage medium

Abstract

The embodiment of the invention discloses a task processing method and device, computer equipment and a storage medium. The method is applied to a task processing system, a hybrid expert model is deployed in the task processing system, the model comprises a plurality of token-level experts and task-level experts, and the method comprises the following steps: acquiring a task text; determining a plurality of tokens from the task text; determining a previous preset number of target token-level experts with the highest token routing probability, and determining a target task expert with the highest task routing probability; for each token, determining a weight update value according to the token routing probability of each target token-level expert, the first rule output of each target token-level expert for the token, the task routing probability of the target task expert and the second rule output of the target task expert for the token; and processing the target task based on each token and the weight update value corresponding to each token. The method can be applied to business scenes such as financial science and technology, medical health, old-age care and the like, and the accuracy of the model can be improved through the scheme.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to task processing methods, devices, computer equipment, and storage media. Background Art

[0002] In the current field of artificial intelligence, there are some technologies related to expert selection in task processing systems, and task processing is performed based on the selected experts. In existing technologies, task processing systems generally use simple classifiers to assign the current task to a specific expert, or use fixed routing strategies to select specific experts for task processing.

[0003] However, expert selection based on simple classifiers or fixed routing strategies fails to fully consider task details and context. For example, in business scenarios like fintech, healthcare, and senior care, when task processing systems interact with customers, a single customer-triggered task often includes multiple instructions. This makes it difficult for the system to accurately select the right expert for such tasks, reducing model accuracy. Summary of the Invention

[0004] The embodiments of the present application provide a task processing method, apparatus, computer equipment, and storage medium, which can improve the accuracy of expert selection and the accuracy of the model.

[0005] In a first aspect, an embodiment of the present application provides a task processing method, which is applied to a task processing system, wherein a hybrid expert model is deployed in the task processing system, wherein the hybrid expert model includes multiple token-level experts and multiple task-level experts. The method includes:

[0006] Get the task text corresponding to the target task;

[0007] Determining a plurality of tokens from the task text according to a preset token segmentation method;

[0008] For each of the tokens, a preset number of target token-level experts with the highest token routing probability are determined from the plurality of token-level experts based on a preset token router and the token; and a target task expert with the highest task routing probability is determined from the plurality of task-level experts based on a preset task router and the task text;

[0009] For each of the tokens, determining a weight update value according to the token routing probability of each target token-level expert, the first rule output of each target token-level expert for the token, the task routing probability of the target task expert, and the second rule output of the target task expert for the token;

[0010] The target task is processed based on each of the tokens and the weight update value corresponding to each of the tokens.

[0011] In some embodiments, obtaining the task text corresponding to the target task includes:

[0012] Obtaining the task voice corresponding to the target task;

[0013] The task speech is converted into text to obtain the task text.

[0014] In some embodiments, determining a preset number of target token-level experts having the highest token routing probability from a plurality of token-level experts based on a preset token router and the token includes:

[0015] Inputting the token into the token router, and determining the token expert score and the token expert ranking probability set corresponding to each of the token-level experts through the token router;

[0016] Determine the token routing probability of each token-level expert according to each token expert score and each token expert ranking probability set;

[0017] The first preset number of token-level experts with the highest token routing probabilities among the multiple token-level experts are determined as the target token-level experts.

[0018] In some embodiments, determining a target task expert with the highest task routing probability from among the plurality of task-level experts based on a preset task router and the task text includes:

[0019] Clustering the task texts according to a plurality of preset task clustering groups to determine a target task clustering group corresponding to the task texts;

[0020] Inputting the task embedding vector corresponding to the target task cluster group into the task router, and determining the task routing probability corresponding to each of the task-level experts in the plurality of task-level experts through the task router;

[0021] The task-level expert with the highest task routing probability among the multiple task-level experts is determined as the target task expert.

[0022] In some embodiments, determining, by the task router, the task routing probability corresponding to each of the plurality of task-level experts, includes:

[0023] Determining, by the task router, a task expert score and a task expert ranking probability set corresponding to each of the task-level experts;

[0024] The task routing probability of each task-level expert is determined respectively according to the score of each task expert and the ranking probability set of each task expert.

[0025] In some embodiments, determining the weight update value based on the token routing probability of each target token-level expert, the first rule output of each target token-level expert for the token, the task routing probability of the target task expert, and the second rule output of the target task expert for the token includes:

[0026] The weight update value is determined based on a preset weight update formula, which is:

[0027]

[0028] Where x is the current token, ΔW x is the weight update value corresponding to x, N1 is the number of target token-level experts corresponding to x, N2 is the number of target task-level experts corresponding to x, G1(x) i is the token routing probability corresponding to the i-th target token-level expert, G2(e) i is the task routing probability corresponding to the i-th target task expert, E i (x) is the first rule output of the i-th target token expert, E hi (x) is the second rule output of the i-th target task expert.

[0029] In some embodiments, after processing the target task based on each of the tokens and the weight update value corresponding to each of the tokens, the method further includes:

[0030] Obtaining the execution result of the target task;

[0031] The LoRA parameter optimization process is performed on the hybrid expert model according to the execution result.

[0032] In a second aspect, an embodiment of the present application further provides a task processing device, wherein the task processing device is deployed in a task processing system, wherein a hybrid expert model is deployed in the task processing system, wherein the hybrid expert model includes multiple token-level experts and multiple task-level experts, and wherein the task processing device includes:

[0033] A transceiver unit, used to obtain the task text corresponding to the target task;

[0034] A processing unit is used to determine multiple tokens from the task text according to a preset token segmentation method; for each of the tokens, determine a preset number of target token-level experts with the highest token routing probability from the multiple token-level experts according to a preset token router and the token, and determine a target task expert with the highest task routing probability from the multiple task-level experts according to a preset task router and the task text; for each of the tokens, determine a weight update value according to the token routing probability of each target token-level expert, the first rule output of each target token-level expert for the token, the task routing probability of the target task expert, and the second rule output of the target task expert for the token; and process the target task based on each of the tokens and the weight update value corresponding to each of the tokens.

[0035] In a third aspect, an embodiment of the present application further provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above method when executing the computer program.

[0036] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the storage medium stores a computer program, wherein the computer program includes program instructions, and the program instructions can implement the above method when executed by a processor.

[0037] The present application provides a task processing method, apparatus, computer device, and storage medium. The method is applied to a task processing system, wherein a hybrid expert model is deployed in the task processing system, the hybrid expert model including multiple token-level experts and multiple task-level experts, and the method includes: obtaining a task text corresponding to a target task; determining multiple tokens from the task text according to a preset token segmentation method; for each token, determining a preset number of target token-level experts with the highest token routing probability from the multiple token-level experts according to a preset token router and the token, and determining a target task expert with the highest task routing probability from the multiple task-level experts according to a preset task router and the task text; for each token, determining a weight update value according to the token routing probability of each target token-level expert, the first rule output of each target token-level expert for the token, the task routing probability of the target task expert, and the second rule output of the target task expert for the token; and processing the target task based on each token and the weight update value corresponding to each token. The embodiment of the present application uses token-level experts to perform fine processing on the input token, and selects multiple token-level experts to participate in one token, thereby improving the accuracy of expert selection. In addition, this embodiment also introduces task-level experts and selects currently suitable target task experts from the perspective of the entire task. Combined with the task-level expert model, it can better handle the diversity and hierarchical structure of tasks, thereby improving the accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0039] Figure 1 A flowchart of a task processing method provided in an embodiment of the present application;

[0040] Figure 2 A schematic diagram of a sub-process of a task processing method provided in an embodiment of the present application;

[0041] Figure 3 A schematic diagram of a sub-process of a task processing method provided in an embodiment of the present application;

[0042] Figure 4 A schematic block diagram of a task processing device provided in an embodiment of the present application;

[0043] Figure 5 A schematic block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0045] It will be understood that when used in this specification and the appended claims, the terms “comprises” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0046] It should also be understood that the terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0047] It should be further understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0048] The embodiments of the present application provide a task processing method, apparatus, computer equipment, and storage medium.

[0049] The executor of the task processing method may be the task processing device provided in the embodiment of the present application, or a computer device integrating the task processing device, wherein the task processing device may be implemented in hardware or software, and the computer device may be a terminal or a server. Specifically, the terminal may be an intelligent robot, such as a medical assistance robot in the field of medical and health technology, or a financial customer service robot in the financial field.

[0050] Specifically, the computer device is a physical device corresponding to the task processing system, and a hybrid expert model is deployed in the task processing system, and the hybrid expert model includes multiple token-level experts and multiple task-level experts. Among them, the token-level expert is a token-level LoRA expert, and each token-level LoRA expert includes a low-rank matrix introduced by LoRA in addition to the pre-trained weights of the basic model; similarly, the task-level expert is a task-level LoRA expert, and similarly, each task-level LoRA expert includes a low-rank matrix introduced by LoRA in addition to the pre-trained weights of the basic model. The token-level expert provided in this embodiment is used to perform fine processing on the input tokens, and the task-level expert is used to control the task from the perspective of the overall task.

[0051] Figure 1 This is a flowchart of the task processing method provided in the embodiment of the present application. Figure 1 As shown, the method includes the following steps S110-S150.

[0052] S110: Obtain the task text corresponding to the target task.

[0053] In this embodiment, the task processing system can directly obtain the task text corresponding to the target task. The task text is the text input by the user, or the task text obtained by the task processing system through an external voice conversion device. At this time, the external voice conversion device obtains the task voice input by the user, performs text conversion on the task voice, obtains and inputs the task text into the task processing system, so that the task processing system processes the target task based on the obtained task text.

[0054] In addition, the task processing system can also directly obtain the task voice input by the user, and obtain the task text based on the task voice input by the user. In this case, step S110 includes: obtaining the task voice corresponding to the target task; performing text conversion processing on the task voice to obtain the task text.

[0055] For example, in healthcare, a doctor might give a medical assistance robot a task via voice: "Turn on the lights in the first examination room and take the stethoscope from the table to the second examination room." After the robot receives the doctor's voice commands through its microphone, it uses its built-in speech-to-text logic to convert the task voice into text.

[0056] For example, in the financial sector, a customer might ask a financial customer service robot a question and say, "Help me check my account balance and send it to my phone as a text message." After the robot receives the voice message through its microphone, it uses its built-in speech-to-text conversion logic to convert the task text into the task text.

[0057] S120: Determine a plurality of tokens from the task text according to a preset token segmentation method.

[0058] In this embodiment, the token segmentation method may specifically be a segmentation method based on a word segmentor, which performs word segmentation processing on the task text to obtain multiple tokens (word segments).

[0059] For example, “turn on the headlight in the first examination room and take the stethoscope on the table to the second examination room” is split into tokens “turn on”, “first examination room”, “headlight”, “take”, “desktop”, “stethoscope”, “bring to”, and “second examination room”.

[0060] S130. For each of the tokens, determine the first preset number of target token-level experts with the highest token routing probability from the multiple token-level experts based on the preset token router and the token, and determine the target task expert with the highest task routing probability from the multiple task-level experts based on the preset task router and the task text.

[0061] In this embodiment, for each token, a preset number of target token-level experts that are most suitable for the token are selected from a preset plurality of token-level experts, and a target task expert that is most suitable for the token is selected from a preset plurality of task-level experts.

[0062] Among them, the above-mentioned preset number can be 3, or can be set to other values ​​according to actual needs. This embodiment does not limit the specific value of the preset number.

[0063] See also Figure 2 In some embodiments, the step of determining a preset number of target token-level experts having the highest token routing probability from among the plurality of token-level experts based on the preset token router and the token includes:

[0064] S1301a, inputting the token into the token router, and determining the token expert score and token expert ranking probability set corresponding to each token-level expert through the token router;

[0065] S1302a, determining the token routing probability of each token-level expert according to each token expert score and each token expert ranking probability set;

[0066] S1303a: Determine a preset number of token-level experts with the highest token routing probabilities among the multiple token-level experts as the target token-level experts.

[0067] Specifically, the token is input into the token router, and then the token router calculates the fitness of each token router for processing the token, that is, calculates the token expert score corresponding to each token-level expert, and determines the token expert ranking probability set. The token expert ranking probability set includes the probability of the current token-level expert being in each ranking. For example, there are a total of 10 token-level experts, then the token expert ranking probability set of the current token-level expert includes the probability of the current token-level expert ranking first (that is, the probability of the current token-level expert having the highest score), the probability of ranking second, the probability of ranking third, ... and the probability of ranking tenth.

[0068] For example, for the token "First Clinic", the token router selects a token-level expert who is good at processing spatial information (that is, the token-level expert who is good at processing spatial information has a higher score) for the token based on the characteristics of the token "First Clinic". For action tokens such as "open" and "take", the token router selects an expert who processes the corresponding action (that is, the token-level expert who processes the corresponding action has a higher score).

[0069] In step S1302a, the token routing probability can be calculated by the token routing probability calculation formula, where the token routing probability calculation formula is:

[0070]

[0071] Among them, G(x) i When token x is input to the token router, the token routing probability of the i-th token-level expert is calculated, where M is the total number of token-level experts. si represents the token expert score of token-level expert i, and sj represents the probability of the j-highest ranking. This embodiment uses this exponential function-based calculation method to enable the model to dynamically adjust the routing probability of each expert based on the expert's score, giving experts with higher scores a higher weight in the model calculation and making them more likely to be selected for task processing.

[0072] In this embodiment, after obtaining the routing probability of each token, the preset routing logic probability G1(·) is used to determine which token-level experts will be activated, where:

[0073] G1(x)={i|p i (x)≥p k (x)}

[0074] Among them, k is the preset number, p kAfter sorting the routing probabilities of each token from large to small, the routing probability of the kth largest token, p i is the token routing probability corresponding to the i-th token-level expert.

[0075] It can be seen that the model in this embodiment does not need to activate all token-level experts when processing tasks, but only activates experts that are more relevant to the current token, which can reduce the computing resources of the model and improve task processing efficiency.

[0076] See also Figure 3 In some embodiments, determining a target task expert with the highest task routing probability from among the plurality of task-level experts based on a preset task router and the task text includes:

[0077] S1301b, clustering the task text according to a plurality of preset task clustering groups to determine a target task clustering group corresponding to the task text;

[0078] S1302b, inputting the task embedding vector corresponding to the target task cluster group into the task router, and determining the task routing probability corresponding to each of the task-level experts in the plurality of task-level experts through the task router;

[0079] S1303b: Determine the task-level expert with the highest task routing probability among the multiple task-level experts as the target task expert.

[0080] Among them, determining the task routing probability corresponding to each of the multiple task-level experts through the task router includes: determining the task expert score and the task expert ranking probability set corresponding to each of the task-level experts through the task router; and determining the task routing probability of each of the task-level experts according to the task expert score and the task expert ranking probability set.

[0081] Specifically, in this embodiment, the task-level router selects the expert with the highest score from a preset set of multiple task-level experts based on the input task embedding vector e. The task embedding vector e is obtained through clustering and represents the task category of the input task text in the multimodal space. By analyzing the task embedding vector, the task-level router can accurately determine the type and level of the current task and select the most suitable task-level expert. For example, when the model receives the task of "turning on the headlights in the first clinic and taking the stethoscope from the table to the second clinic," the task-level router can quickly select a task-level expert who specializes in tasks related to the clinic environment based on the task embedding vector, ensuring that the model can make accurate decisions for the task.

[0082] Among them, the task routing probability in this embodiment can be specifically calculated through a task routing probability calculation formula. The task routing probability calculation formula is similar to the above-mentioned token routing probability calculation formula, and will not be elaborated here specifically.

[0083] S140. For each of the tokens, determine a weight update value according to the token routing probability of each of the target token-level experts, the first rule output of each of the target token-level experts for the token, the task routing probability of the target task expert, and the second rule output of the target task expert for the token.

[0084] Specifically, the weight update value can be determined according to a preset weight update formula, and the weight update formula is:

[0085]

[0086] where x is the current token, and ΔW x is the weight update value corresponding to x, N1 is the number of target token-level experts corresponding to x, N2 is the number of target task-level experts corresponding to x, G1(x) i is the token routing probability corresponding to the i-th target token-level expert, G2(e) i is the task routing probability corresponding to the i-th target task expert, E i (x) is the first rule output of the i-th target token expert, E hi (x) is the second rule output of the i-th target task expert, represents the contribution of the token-level expert to the weight update, represents the contribution of the task-level expert to the weight update.

[0087] Among them, the token-level expert calculates the rule output (the first rule output) according to the following LoRA update rule, and the corresponding LoRA update rule is:

[0088] E(x) = W1 + ΔW1 = W1 + B1A1;

[0089] where W1 is the weight of the base model of the token-level expert, ΔW1 is the weight update corresponding to the token-level expert, B1 and A1 are the low-rank matrices introduced by the token-level expert LoRA, the dimensions r and d correspond to the size of the pre-trained weight matrix in the large language model, and satisfy r << d. The introduction of the low-rank matrix enables the model to learn task-specific representations more efficiently when updating parameters and reduces the complexity of parameter updates.

[0090] Similar to the LoRA update rule of the token-level expert, the task-level expert calculates the rule output (the second rule output) according to the following LoRA update rule, and the corresponding LoRA update rule is:

[0091] E h (x) = W2 + ΔW2 = W2 + B2A2;

[0092] Where W2 is the weight of the task-level expert's base model, ΔW2 is the corresponding weight update of the task-level expert, B2 and A2 are low-rank matrices introduced by the task-level expert LoRA. The dimensions r and d correspond to the size of the pre-trained weight matrix in the large language model, and r < < d. The introduction of low-rank matrices enables the model to learn task-specific representations more efficiently when updating parameters, reducing the complexity of parameter updates.

[0093] S150: Process the target task based on each of the tokens and the weight update value corresponding to each of the tokens.

[0094] In this embodiment, after obtaining the weight update value corresponding to the target task, the target task is processed based on each of the tokens and the weight update value corresponding to each of the tokens. For example, for token x, the target task is processed based on the following formula:

[0095] y=ΔW x @x;

[0096] Where y is the model’s calculated output for token x, ΔW x Update the weight corresponding to token x.

[0097] In this embodiment, after processing the target task based on each of the tokens and the weight update value corresponding to each of the tokens, the method further includes:

[0098] Obtaining the execution result of the target task; and performing LoRA parameter optimization processing on the hybrid expert model according to the execution result.

[0099] It can be seen that in this embodiment, through data-driven updates after task execution, LoRA can dynamically adapt to new tasks or environmental changes without destroying the general capabilities of the pre-trained model. It only needs to fine-tune the low-rank matrix, which can reduce training overhead.

[0100] In summary, the task processing method provided in this embodiment is applied to a task processing system, in which a hybrid expert model is deployed, and the hybrid expert model includes multiple token-level experts and multiple task-level experts. The method includes: obtaining a task text corresponding to the target task; determining multiple tokens from the task text according to a preset token segmentation method; for each of the tokens, determining the first preset number of target token-level experts with the highest token routing probability from the multiple token-level experts according to a preset token router and the token, and determining the target task expert with the highest task routing probability from the multiple task-level experts according to a preset task router and the task text; for each of the tokens, determining a weight update value according to the token routing probability of each target token-level expert, the first rule output of each target token-level expert for the token, the task routing probability of the target task expert, and the second rule output of the target task expert for the token; processing the target task based on each of the tokens and the weight update values ​​corresponding to each of the tokens. The embodiment of the present application uses token-level experts to perform fine processing on the input token, and selects multiple token-level experts to participate in one token, thereby improving the accuracy of expert selection. In addition, this embodiment also introduces task-level experts and selects currently suitable target task experts from the perspective of the entire task. Combined with the task-level expert model, it can better handle the diversity and hierarchical structure of tasks, thereby improving the accuracy of the model.

[0101] The improved approach can better handle the diversity and hierarchical structure of tasks, enhancing the model's generalization capabilities across new tasks and environments. When faced with unseen task combinations or environmental changes, the model can still rapidly adapt and accurately execute tasks through accurate expert selection and efficient parameter updates, expanding the model's scope of application.

[0102] Figure 4 FIG. 4 is a schematic block diagram of a task processing device 400 provided in an embodiment of the present application. Figure 4 As shown, corresponding to the above task processing method, the present application also provides a task processing device 400. The task processing device 400 includes a unit for executing the above task processing method. The task processing device 400 is deployed in a task processing system. The task processing system is deployed with a hybrid expert model. The hybrid expert model includes multiple token-level experts and multiple task-level experts. Figure 4 The task processing device 400 includes a transceiver unit 401 and a processing unit 402, wherein:

[0103] The transceiver unit 401 is used to obtain the task text corresponding to the target task;

[0104] The processing unit 402 is used to determine multiple tokens from the task text according to a preset token segmentation method; for each token, determine the first preset number of target token-level experts with the highest token routing probability from the multiple token-level experts according to the preset token router and the token, and determine the target task expert with the highest task routing probability from the multiple task-level experts according to the preset task router and the task text; for each token, determine the weight update value according to the token routing probability of each target token-level expert, the first rule output of each target token-level expert for the token, the task routing probability of the target task expert, and the second rule output of the target task expert for the token; and process the target task based on each token and the weight update value corresponding to each token.

[0105] In some embodiments, when executing the step of obtaining the task text corresponding to the target task, the transceiver unit 401 is specifically configured to:

[0106] The task speech corresponding to the target task is obtained; and the task speech is converted into text by the processing unit 402 to obtain the task text.

[0107] In some embodiments, when executing the step of determining a preset number of target token-level experts having the highest token routing probability from among the plurality of token-level experts based on the preset token router and the token, the processing unit 402 is specifically configured to:

[0108] The token is input into the token router, and the token router determines the token expert score and the token expert ranking probability set corresponding to each token-level expert; the token routing probability of each token-level expert is determined according to each token expert score and each token expert ranking probability set; and the first preset number of token-level experts with the highest token routing probability among the multiple token-level experts are determined as the target token-level experts.

[0109] In some embodiments, when executing the step of determining a target task expert with the highest task routing probability from the plurality of task-level experts based on the preset task router and the task text, the processing unit 402 is specifically configured to:

[0110] Clustering the task texts according to a plurality of preset task clustering groups to determine a target task clustering group corresponding to the task texts;

[0111] Inputting the task embedding vector corresponding to the target task cluster group into the task router, and determining the task routing probability corresponding to each of the task-level experts in the plurality of task-level experts through the task router;

[0112] The task-level expert with the highest task routing probability among the multiple task-level experts is determined as the target task expert.

[0113] In some embodiments, when executing the step of determining, by the task router, the task routing probability corresponding to each of the task-level experts in the plurality of task-level experts, the processing unit 402 is specifically configured to:

[0114] Determining, by the task router, a task expert score and a task expert ranking probability set corresponding to each of the task-level experts;

[0115] The task routing probability of each task-level expert is determined respectively according to the score of each task expert and the ranking probability set of each task expert.

[0116] In some embodiments, when executing the step of determining the weight update value based on the token routing probability of each target token-level expert, the first rule output of each target token-level expert for the token, the task routing probability of the target task expert, and the second rule output of the target task expert for the token, the processing unit 402 is specifically configured to:

[0117] The weight update value is determined based on a preset weight update formula, which is:

[0118]

[0119] Where x is the current token, ΔW x is the weight update value corresponding to x, N1 is the number of target token-level experts corresponding to x, N2 is the number of target task-level experts corresponding to x, G1(x) i is the token routing probability corresponding to the i-th target token-level expert, G2(e) i is the task routing probability corresponding to the i-th target task expert, E i (x) is the first rule output of the i-th target token expert, E hi (x) is the second rule output of the i-th target task expert.

[0120] In some embodiments, after executing the step of processing the target task based on each of the tokens and the weight update value corresponding to each of the tokens, the processing unit 402 is further configured to:

[0121] Obtaining the execution result of the target task; and performing LoRA parameter optimization processing on the hybrid expert model according to the execution result.

[0122] To sum up, the embodiment of the present application uses token-level experts to perform fine-grained processing on the input token, and selects multiple token-level experts to participate in one token, thereby improving the accuracy of expert selection. In addition, this embodiment also introduces task-level experts and selects currently suitable target task experts from the perspective of the entire task. Combined with the task-level expert model, it can better handle the diversity and hierarchical structure of tasks, thereby improving the accuracy of the model.

[0123] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned task processing device and each unit can refer to the corresponding description in the aforementioned method embodiment. For the convenience and brevity of description, it will not be repeated here.

[0124] The task processing device can be implemented as a computer program. Figure 5 Runs on the computer equipment shown.

[0125] See also Figure 5 , Figure 5 This is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 can be a terminal or a server. Specifically, the computer device is a physical device corresponding to a task processing system, in which a hybrid expert model is deployed. The hybrid expert model includes multiple token-level experts and multiple task-level experts.

[0126] See Figure 5 The computer device 500 includes a processor 502 , a memory, and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0127] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, which, when executed, can cause the processor 502 to perform a task processing method.

[0128] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500.

[0129] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a task processing method.

[0130] The network interface 505 is used to communicate with other devices through the network. Figure 5The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0131] The processor 502 is configured to execute a computer program 5032 stored in the memory to implement the following steps:

[0132] Get the task text corresponding to the target task;

[0133] Determining a plurality of tokens from the task text according to a preset token segmentation method;

[0134] For each of the tokens, a preset number of target token-level experts with the highest token routing probability are determined from the plurality of token-level experts based on a preset token router and the token; and a target task expert with the highest task routing probability is determined from the plurality of task-level experts based on a preset task router and the task text;

[0135] For each of the tokens, determining a weight update value according to the token routing probability of each target token-level expert, the first rule output of each target token-level expert for the token, the task routing probability of the target task expert, and the second rule output of the target task expert for the token;

[0136] The target task is processed based on each of the tokens and the weight update value corresponding to each of the tokens.

[0137] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0138] Those skilled in the art will appreciate that all or part of the steps in the method of the above-described embodiment can be implemented by instructing the relevant hardware through a computer program. The computer program includes program instructions, which can be stored in a storage medium that is computer-readable. The program instructions are executed by at least one processor in the computer system to implement the steps in the method of the above-described embodiment.

[0139] Therefore, the present application also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor performs the following steps:

[0140] Get the task text corresponding to the target task;

[0141] Determining a plurality of tokens from the task text according to a preset token segmentation method;

[0142] For each of the tokens, a preset number of target token-level experts with the highest token routing probability are determined from the plurality of token-level experts based on a preset token router and the token; and a target task expert with the highest task routing probability is determined from the plurality of task-level experts based on a preset task router and the task text;

[0143] For each of the tokens, determining a weight update value according to the token routing probability of each target token-level expert, the first rule output of each target token-level expert for the token, the task routing probability of the target task expert, and the second rule output of the target task expert for the token;

[0144] The target task is processed based on each of the tokens and the weight update value corresponding to each of the tokens.

[0145] The storage medium may be any computer-readable storage medium that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk.

[0146] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0147] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and other division methods may be used in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not implemented.

[0148] The steps in the method of the embodiment of the present application can be adjusted in order, combined, and deleted according to actual needs. The units in the device of the embodiment of the present application can be combined, divided, and deleted according to actual needs. In addition, the functional units in the various embodiments of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit.

[0149] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0150] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A task processing method, characterized in that: The method is applied to a task processing system, wherein a hybrid expert model is deployed in the task processing system, wherein the hybrid expert model includes multiple token-level experts and multiple task-level experts, and the method includes: Get the task text corresponding to the target task; Determining a plurality of tokens from the task text according to a preset token segmentation method; For each of the tokens, a preset number of target token-level experts with the highest token routing probability are determined from the plurality of token-level experts based on a preset token router and the token; and a target task expert with the highest task routing probability is determined from the plurality of task-level experts based on a preset task router and the task text; For each of the tokens, determining a weight update value according to the token routing probability of each target token-level expert, the first rule output of each target token-level expert for the token, the task routing probability of the target task expert, and the second rule output of the target task expert for the token; The target task is processed based on each of the tokens and the weight update value corresponding to each of the tokens.

2. The method according to claim 1, characterized in that The step of obtaining the task text corresponding to the target task includes: Obtaining the task voice corresponding to the target task; The task speech is converted into text to obtain the task text.

3. The method according to claim 1, characterized in that The method of determining a preset number of target token-level experts having the highest token routing probability from a plurality of token-level experts according to the preset token router and the token includes: Inputting the token into the token router, and determining the token expert score and the token expert ranking probability set corresponding to each of the token-level experts through the token router; Determine the token routing probability of each token-level expert according to each token expert score and each token expert ranking probability set; The first preset number of token-level experts with the highest token routing probabilities among the multiple token-level experts are determined as the target token-level experts.

4. The method according to claim 1, wherein The step of determining a target task expert with the highest task routing probability from a plurality of task-level experts according to a preset task router and the task text includes: Clustering the task texts according to a plurality of preset task clustering groups to determine a target task clustering group corresponding to the task texts; Inputting the task embedding vector corresponding to the target task cluster group into the task router, and determining the task routing probability corresponding to each of the task-level experts in the plurality of task-level experts through the task router; The task-level expert with the highest task routing probability among the multiple task-level experts is determined as the target task expert.

5. The method according to claim 4, characterized in that The determining, by the task router, the task routing probability corresponding to each of the task-level experts in the plurality of task-level experts includes: Determining, by the task router, a task expert score and a task expert ranking probability set corresponding to each of the task-level experts; The task routing probability of each task-level expert is determined respectively according to the score of each task expert and the ranking probability set of each task expert.

6. The method according to claim 1, characterized in that Determining a weight update value according to the token routing probability of each target token-level expert, the first rule output of each target token-level expert for the token, the task routing probability of the target task expert, and the second rule output of the target task expert for the token includes: The weight update value is determined based on a preset weight update formula, which is: Where x is the current token, ΔW x is the weight update value corresponding to x, N1 is the number of target token-level experts corresponding to x, N2 is the number of target task-level experts corresponding to x, G1(x) i is the token routing probability corresponding to the i-th target token-level expert, G2(e) i is the task routing probability corresponding to the i-th target task expert, E i (x) is the first rule output of the i-th target token expert, E hi (x) is the second rule output of the i-th target task expert.

7. The method according to claim 1, characterized in that After processing the target task based on each of the tokens and the weight update value corresponding to each of the tokens, the method further includes: Obtaining the execution result of the target task; The LoRA parameter optimization process is performed on the hybrid expert model according to the execution result.

8. A task processing device, characterized in that: The task processing device is deployed in a task processing system. A hybrid expert model is deployed in the task processing system. The hybrid expert model includes multiple token-level experts and multiple task-level experts. The task processing device includes: The transceiver unit is used to obtain the task text corresponding to the target task; A processing unit is used to determine multiple tokens from the task text according to a preset token segmentation method; for each of the tokens, determine a preset number of target token-level experts with the highest token routing probability from the multiple token-level experts according to a preset token router and the token, and determine a target task expert with the highest task routing probability from the multiple task-level experts according to a preset task router and the task text; for each of the tokens, determine a weight update value according to the token routing probability of each target token-level expert, the first rule output of each target token-level expert for the token, the task routing probability of the target task expert, and the second rule output of the target task expert for the token; and process the target task based on each of the tokens and the weight update value corresponding to each of the tokens.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the task processing method according to any one of claims 1 to 7 is implemented.

10. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the task processing method according to any one of claims 1 to 7.

Images