Artificial intelligence routing method, artificial intelligence routing system and computer equipment

Abstract

The invention provides an artificial intelligence routing method. The method comprises the following steps: S100) an AI routing agent receives and analyzes at least one piece of prompt information; s200) the AI routing agent analyzes at least one request intention of the user according to the prompt information, decomposes and splits the request intention into a plurality of sub-tasks, and plans an interdependence relationship among the sub-tasks to form a task list; s300) the AI routing agent selects an AI model agent interface registered on an AI gateway according to the characteristics of an artificial intelligence model so as to match the requirements of the task list, and the selected AI model matching the requirements of the task list is the selected AI model; s400) calling and executing each selected AI model, and returning a result to the AI routing agent; and S500) integrating the prediction results of all the selected AI models by the AI routing agent, and generating a response.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence routing, and in particular to an artificial intelligence routing method and a system and computer equipment thereof. Background Art

[0002] At present, various types of large-scale artificial intelligence (AI) models have appeared in the market, including but not limited to large models in the fields of vision, language, speech and natural language processing. At the same time, thousands of various applications have also been born, all of which are built based on the above models. Figure 1 However, there are still some contradictions and challenges in the actual application process. For most users, when they complete a specific task, they often need to call multiple different applications or intelligent functions.

[0003] This brings up two major problems: first, how to choose; second, how to use these tools correctly. In addition, since it involves frequent switching between multiple applications, it undoubtedly increases the difficulty of use for users. Summary of the invention

[0004] The embodiment of the present invention provides an artificial intelligence routing method and its system and computer device, which are constructed based on a large-scale artificial intelligence model, and through in-depth analysis of the user's real intention, it is decomposed into several sub-tasks that may be solvable, and then according to the specific situation, the most suitable artificial intelligence model application is selected to call it, and finally a corresponding response result is generated.

[0005] To achieve the above purpose, this application proposes an artificial intelligence routing method, comprising the following steps:

[0006] S100) The AI ​​routing agent receives and parses at least one prompt message;

[0007] S200) the AI ​​routing agent analyzes at least one request intention of the user according to the prompt information, decomposes the request intention into multiple subtasks, and plans the interdependencies between the subtasks to form a task list;

[0008] S300) The AI ​​routing agent selects an AI model proxy interface registered on the AI ​​gateway according to the characteristics of the artificial intelligence model to match the requirements of the task list, wherein the selected AI model that matches the requirements of the task list is the selected AI model;

[0009] S400) calling and executing each selected AI model, and returning the result to the AI ​​routing agent; and

[0010] S500) The AI ​​routing agent integrates the prediction results of all selected AI models and generates a response. In some embodiments, in step S300, the AI ​​routing agent selects an AI model from the model proxy interface registered on the AI ​​gateway to select an AI model suitable for the task list as the selected AI model, wherein the factors for selecting the AI ​​model include the availability, performance, response speed and cost of the model.

[0011] In some embodiments, in step S400, the AI ​​routing agent obtains the response results of the asynchronous execution of each subtask by calling the AI ​​model interface corresponding to the selected AI model and post-processing the relevant prediction results, wherein in step S500, the AI ​​routing agent summarizes the prediction results of all AI models and generates a response to return to the user.

[0012] In some embodiments, step S300 includes the following steps:

[0013] S310) selecting a plurality of candidate models according to the task sequence;

[0014] S320) querying AI indicators from the AI ​​gateway;

[0015] S330) screening models, wherein the models are screened based on factors including performance, speed, and cost; and

[0016] S340) Standardize the model proxy URL and input parameters.

[0017] In some embodiments, step S200 uses cross entropy loss and pair ranking loss to evaluate the accuracy of task planning, and the calculation formula is as follows: L task =L class +λ*L rank (1)

[0018]

[0019]

[0020] In formula (1), λ represents the weight coefficient,

[0021] In formula (2), M is the total number of planning tasks, C is the number of categories, O is the number of sequences, and Y ic represents the symbolic function. If the true category of sample i is equal to c, then Y ic Take 1, otherwise Y ic Take 0, where is the predicted probability that the observed sample i belongs to category c,

[0022] In formula (3), T ij represents the true probability of sample i before sample j, T ij represents the predicted probability of sample i before sample j, and δ represents the Sigmoid function.

[0023] In some embodiments, step S300 comprehensively considers the availability of the AI ​​model A i , performance, response time, and cost to score the AI ​​models, and select the Top-K AI models that are most suitable for the current subtask based on the scoring results, where the i-th AI model score S i , the calculation formula is as follows:

[0024]

[0025] In formula (4), F i represents the performance score, T i Represents the response time, C i represents the call cost, where the weights of performance, response time and call cost are ω respectively f ,ω t and ω c ;

[0026] Among them, the performance score F i The calculation formula is as follows:

[0027]

[0028] In formula (5), P i Represents the accuracy, R i represents the recall rate, and β represents P i With R i Relative importance.

[0029] In some embodiments, step S310 compares the text vector similarity of the subtask and the AI ​​model library, and preliminarily screens out M possible candidate models.

[0030] Step S320 queries the AI ​​indicators of the candidate models corresponding to each subtask from the AI ​​gateway.

[0031] Step S330 selects the top-K AI models according to the AI ​​indicators queried in step S320, where K ranges from [1, M], and the score of the i-th AI model is S i , the calculation formula is as follows:

[0032]

[0033] In formula (4), F i represents the performance score, T iRepresents the response time, C i represents the call cost, where the weights of performance, response time and call cost are ω respectively f ,ω t and ω c ;

[0034] Among them, the performance score F i The calculation formula is as follows:

[0035]

[0036] In formula (5), P i Represents the accuracy, R i represents the recall rate, and β represents P i With R i The relative importance,

[0037] Step S340 plans the AI ​​model proxy URL and its input parameters.

[0038] In some embodiments, step S500 summarizes the asynchronous execution results using a large language model, by merging the inference results of the top-K candidate models, where for the input x and its corresponding subtask t i , and the highest ranked candidate AI models or applications m i , producing an improved output as the final response The calculation formula is as follows:

[0039]

[0040] In formula (6), f sum represents generation fusion, and N represents the number of subtasks.

[0041] According to another aspect of the present invention, the present application also provides an artificial intelligence routing system applied to the above-mentioned artificial intelligence routing method, including an AI router and an AI gateway, wherein the AI ​​router obtains AI indicators, unified API interface, context, certificate and prompt template information from the AI ​​gateway, and comprehensively considers availability, performance, response speed and cost factors to screen out a suitable AI model.

[0042] According to another aspect of the present invention, the present application further provides a computer device, the device comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0043] Memory, used to store computer programs;

[0044] The processor is used to implement the steps of the above-mentioned artificial intelligence routing method when executing the program stored in the memory. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0046] Figure 1 The architecture of existing AI applications is explained.

[0047] Figure 2 It is a flow chart of an artificial intelligence routing method according to a preferred embodiment of the present invention.

[0048] Figure 3 The architecture of an artificial intelligence routing system according to the above preferred embodiment of the present invention is explained.

[0049] Figure 4 The workflow of the artificial intelligence routing system according to the above preferred embodiment of the present invention is explained.

[0050] Figure 5 This is an example of application of the artificial intelligence routing method and artificial intelligence routing system according to the above preferred embodiment of the present invention in multi-task planning and execution.

[0051] Figure 6 The working process of an artificial intelligence gateway of the artificial intelligence routing system according to the above preferred embodiment of the present invention is explained.

[0052] Figure 7 The multimodal input and output in the artificial intelligence routing method according to the above preferred embodiment of the present invention is explained.

[0053] Figure 8 A code example in the application of the artificial intelligence routing method according to the above preferred embodiment of the present invention is explained.

[0054] Fig. 9 It is a code example of AI routing in the application of the artificial intelligence routing method according to the above preferred embodiment of the present invention.

[0055] Fig.10 It is a code example of the AI ​​gateway in the application of the artificial intelligence routing method according to the above preferred embodiment of the present invention. DETAILED DESCRIPTION

[0056] The drawings in the specification Figure 2An artificial intelligence routing method according to a preferred embodiment of the present invention is explained. The artificial intelligence routing method not only unifies the large model application entrance through the AI ​​gateway to lower the usage threshold, but also realizes task planning, model selection, task execution and response summary through AI routing.

[0057] like Figure 2 As shown, the artificial intelligence routing method includes the following steps:

[0058] S100) The AI ​​routing agent receives and parses at least one prompt message;

[0059] S200) the AI ​​routing agent analyzes at least one request intention of the user according to the prompt information, decomposes the request intention into multiple subtasks, and plans the interdependencies between the subtasks to form a task list;

[0060] S300) The AI ​​routing agent selects an AI model proxy interface registered on the AI ​​gateway according to the characteristics of the artificial intelligence model to match the requirements of the task list, wherein the selected AI model that matches the requirements of the task list is the selected AI model;

[0061] S400) calling and executing each selected AI model, and returning the result to the AI ​​routing agent; and

[0062] S500) The AI ​​routing agent integrates the prediction results of all selected AI models and generates a response.

[0063] Specifically, the user collects and organizes multimodal data related to a task, and writes corresponding prompt information for the AI ​​routing agent to parse, wherein the prompt information is received and parsed in step S100, wherein the receiving and parsing of the prompt information is performed by the AI ​​routing agent, wherein at least one user collects and organizes multimodal data related to a task, and writes the prompt information accordingly for the AI ​​routing agent to parse.

[0064] Step S200 analyzes at least one request intention of the user according to the prompt information, decomposes the request intention into multiple subtasks, and plans the interdependencies between the subtasks to form a task list to facilitate subsequent model selection and task execution, wherein the above analysis and splitting steps are performed by the AI ​​routing agent.

[0065] In step S300, the AI ​​routing agent selects an AI model from the model proxy interface registered on the AI ​​gateway to select an AI model suitable for the task list (referred to as the selected AI model), wherein the AI ​​routing agent selects the model based on multiple factors such as the model's availability, performance, response speed, and cost.

[0066] In step S400, the AI ​​routing agent obtains the response results of the asynchronous execution of each subtask by calling the AI ​​model interface corresponding to the selected AI model and post-processing the relevant prediction results.

[0067] In step S500, the AI ​​routing agent summarizes the prediction results of all AI models and generates a response to return to the user.

[0068] The drawings in the specification Figure 3 An artificial intelligence routing system according to the above preferred embodiment of the present invention is explained. The artificial intelligence routing system includes an AI router 10 and an AI gateway 20.

[0069] The AI ​​router 10 combines the framework of two agent workflows, task planning and tool calling, and can solve complex AI task planning and tool calling problems through dynamic models and dynamic templates.

[0070] The AI ​​gateway 20 includes an AI indicator module 201, a prompt template module 202, a certificate module 203, a context management module 204, a unified API interface module 205, an AI agent module 206, a log module 207, a data cleaning module 208, a rate limiting module 209, a cache module 210, a retry module 211 and an AI security module 212.

[0071] The AI ​​gateway 20 is based on the Nginx_Lua module, and adds plug-ins such as AI agent, AI indicator, prompt template, context management, etc. to perform centralized management of AI models or applications to achieve the goals of security, observability and traffic control.

[0072] Take a sanitation operation as an example. The requirement is to detect overflowing or uncovered trash cans and notify the relevant person in charge. Its multi-task planning and execution calls 3 AI models and 1 work interface. The operation process is as follows:

[0073] 1) After obtaining a real-time picture of garbage dump station D in the surveillance video, the user writes a multimodal prompt: "How many garbage bins are overflowing in the following pictures? How many garbage bins are uncovered? If yes, please notify the relevant person in charge.";

[0074] 2) After receiving the multimodal prompt word, the AI ​​router 10 plans multiple subtasks according to the multimodal prompt word, and plans the interdependence between each subtask to form a task list. In this example, after receiving the multimodal prompt word, the AI ​​router 10 plans four subtasks of graphic understanding, target detection, work alarm and response summary according to the multimodal prompt word. According to the task list, the AI ​​router 10 obtains information such as AI indicators, unified API interface, context, certificate and prompt template from the AI ​​gateway 20, and comprehensively considers factors such as availability, performance, response speed and cost to select the most suitable AI model.

[0075] Specifically, the AI ​​router 10 obtains AI indicator information from the AI ​​indicator module 201 of the AI ​​gateway 20. The AI ​​router 10 obtains AI unified API interface information from the unified API interface module 205 of the AI ​​gateway 20. The AI ​​router 10 obtains context information from the context management module 204 of the AI ​​gateway 20. The AI ​​router 10 obtains certificate information from the certificate module 203 of the AI ​​gateway 20. The AI ​​router 10 obtains prompt template information from the prompt template module 202 of the AI ​​gateway 20.

[0076] 3) Asynchronously calling the AI ​​model or application interface through the AI ​​agent, and returning Qwen2-VL, YOLOv8-trash and Rwork-note to the AI ​​router 10. Specifically, the AI ​​agent module 206 asynchronously calls the screened AI model and returns Qwen2-VL, YOLOv8-trash and Rwork-note to the AI ​​router 10.

[0077] 4) Use the large model to summarize the asynchronous execution results and send the final results to the user terminal. In this application example, the GLM-4 large model is used to summarize the asynchronous execution results. The example is as follows:

[0078] “The picture shows 3 trash cans overflowing and 14 uncovered. ***The person in charge of ***the platform has been notified***.

[0079] The workflow is as follows:

[0080] 1. Picture and text comprehension [Qwen2-VL]:...

[0081] 2. Object Detection [YOLOv8-trash]:

[0082] 3.RAG: Retrieve the responsible person from the knowledge base...

[0083] 4. Work-note [Rwork-note]: ...

[0084] 5. Response summary [Qwen2.5]: ..."

[0085] According to the preferred embodiment of the present invention, the functions of the AI ​​router 10 include task planning, model selection, model execution and response summary, in which the text vector model and the large language model are called. Specifically, the AI ​​router 10 includes a task planning module 101, a model selection module 102, a model execution module 103 and a response summary module 104, to respectively perform task planning, model selection, model execution and response summary.

[0086] The drawings in the specification Figure 4 The workflow of the AI ​​routing 10 of the artificial intelligence routing system according to the above preferred embodiment of the present invention is explained as follows:

[0087] 1) Task planning: According to the user's multimodal prompts, the text vector model is used to query relevant task planning examples; then the large language model and the few-sample chain of thought (COT) method are used to perform task planning and obtain a subtask sequence. According to this preferred embodiment of the present invention, the accuracy of task planning is evaluated using cross entropy loss and pair sorting loss.

[0088] 2) Model selection: Comprehensively consider multiple scoring factors of the AI ​​model to score the AI ​​model, and select the top-K AI models that are most suitable for the current subtask based on the scoring results, where the scoring factors of the AI ​​model include availability, i , performance, response time, and cost;

[0089] 3) Model execution; and

[0090] 4) Response summary.

[0091] More specifically, in step S200, the task planning module 101 of the AI ​​router 10 uses a text vector model to query relevant task planning examples based on user multimodal prompts, and then uses a large language model and a few-sample chain of thought (COT) method to perform task planning to obtain a subtask sequence, wherein the accuracy of the task planning is evaluated by cross entropy loss and pairing sorting loss.

[0092] In step S300, the model selection module 102 of the AI ​​router 10 scores the AI ​​model based on multiple scoring factors of the AI ​​model, and selects the top-K AI models that are most suitable for the current subtask according to the scoring results, wherein the scoring factors of the AI ​​model include availability A i , performance, response time, and cost.

[0093] In step S400 , the model execution module 203 of the AI ​​router 10 executes the model.

[0094] In step S500, the response summarizing module 104 of the AI ​​router 10 performs response summarization.

[0095] According to the preferred embodiment of the present invention, step S200 includes the following steps:

[0096] S210) querying a task planning example according to the prompt information; and

[0097] S220) Using the few-sample COT to plan the task sequence.

[0098] Step S300 includes the following steps:

[0099] S310) selecting a plurality of candidate models according to the task sequence, for example, 10 candidate models;

[0100] S320) querying the AI ​​indicator from the AI ​​indicator module 201 of the AI ​​gateway 20;

[0101] S330) screening models, wherein the models are screened based on factors including performance, speed, and cost; and

[0102] S340) Standardize the model proxy URL and input parameters.

[0103] Step S400 includes the following steps:

[0104] S410) obtaining a service ID according to a routing strategy; and

[0105] S420) The service mapping model is executed asynchronously.

[0106] The artificial intelligence routing method according to the above preferred embodiment of the present invention can be used for multi-task planning and execution.

[0107] The drawings in the specification Figure 5 The paper describes an example of the AI ​​routing method for multi-task planning and execution, in which a large language model is used as an intermediate controller to manage all AI models, solving complex AI tasks by "mobilizing and combining the power of each model". Specifically, the workflow of AI routing 10 is as follows:

[0108] 1) Task planning: First, according to the user's multimodal prompt "Look at / images / 000000039769.jpg, can you tell me how many objects in the picture? Give me a picture similar to this one.", use the text vector model to query related task planning examples; then use the large language model and the few-sample thinking chain (COT) method to plan tasks and obtain the subtask sequence "

[0109] "task=Task(task='object-detection',id=0,dep=[-1],args={'image':

[0110] ' / images / 000000039769.jpg'})

[0111] "task=Task(task='visual-question-answering',id=1,dep=[0],args={'image':' / images / f589.png','text':'how many objects in the picture?'})

[0112] "task=Task(task='image-to-image', id=2, dep=[-1], args={'image':' / images / 000000039769.jpg'}".

[0113] Among them, dep = [-1] means that the current task depends on the output of the previous task.

[0114] The accuracy of task planning is evaluated using cross entropy loss and pairwise ranking loss, which are calculated as follows:

[0115] L task =L class +λ*L rank (1)

[0116]

[0117]

[0118] In formula (1), λ represents a weight coefficient.

[0119] In formula (2), M is the total number of planning tasks, C is the number of categories, and O is the number of sequences. ic represents the symbolic function. If the true category of sample i is equal to c, then Y ic Take 1, otherwise Y ic Take 0. is the predicted probability that the observed sample i belongs to category c.

[0120] In formula (3), T ij represents the true probability of sample i before sample j, T ij represents the predicted probability of sample i before sample j, and δ represents the Sigmoid function.

[0121] 2) Model selection: Comprehensively consider the availability of AI models i , performance, response time, cost and other factors to score the AI ​​model, and select the Top-K AI models that are most suitable for the current subtask based on the scoring results. The specific process is as follows:

[0122] a. Compare the text vector similarity between the subtask and the AI ​​model library, and preliminarily screen out 10 possible candidate models;

[0123] b. Query the AI ​​indicators of the candidate models corresponding to each subtask from the AI ​​gateway, such as availability, performance, response time, and cost;

[0124] c. Select the top-K AI models based on factors such as availability, performance, response time, and cost, where K ranges from [1,10]; and

[0125] d. Plan the AI ​​model proxy URL and its input parameters.

[0126] In step c, score S according to the AI ​​model i The AI ​​models are screened, where the i-th AI model score S i , the calculation formula is as follows:

[0127]

[0128] In formula (4), F i represents the performance score, T i Represents the response time, C i Represents the call cost. The weights of the above indicators are ω f ,ω t and ω c ;

[0129] Among them, the performance score F i The calculation formula is as follows:

[0130]

[0131] In formula (5), P i Represents the accuracy, R i represents the recall rate, and β represents P i With R i Relative importance. For example, when β = 2, the recall rate can be considered to be twice as important as the precision rate.

[0132] 3) Model execution: Execute the work path according to the task sequence planned in step 2), first obtain the service ID according to the routing strategy, and then asynchronously execute the model mapped by the service ID. For example, execute the work path "target detection" according to the task sequence "facebook / detr-resnet-50", execute the work path "picture and text question answering" according to the task sequence "Salesforce / blip-vqa-base", and execute the work path "picture generation" according to the task sequence "timbrooks / instruct-pix2pix".

[0133] 4) Response summary: Use a large language model to summarize the asynchronous execution results. By merging the inference results of the top-K candidate models, the problem of fluctuation in the selection quality of the candidate models in step 2) can be alleviated.

[0134] For input x and its corresponding subtask t i , and the highest ranked candidate AI models or applications m i , which produces an improved output as the final response The calculation formula is as follows:

[0135]

[0136] In formula (6), f sum represents generation fusion, and N represents the number of subtasks planned in step 1).

[0137] Reference to the attached drawings of the specification Figure 6 The AI ​​gateway 20 uses two object models, routing and service, to define the required traffic management strategies. Services and routes are configured in a coordinated manner to define the routing paths of requests and responses in the system. A service is an abstraction of an upstream model or application that can be associated with a route and can also store a collection of objects such as plug-in configurations and policies. Because there is a one-to-many relationship between a service and an upstream application, complex traffic management behaviors can be created. A route is a path to an upstream model or application. Routes are added to services to allow access to the underlying model or application. Because routes can define rules for matching requests to related services, a route can reference multiple endpoints but can only reference one service.

[0138] It is worth mentioning that step S210 uses the text vector model to query relevant task planning examples based on the user's multimodal prompt information. Step S220 uses a large language model and a few-sample chain of thought (COT) method to perform task planning and obtain a subtask sequence.

[0139] Step S200 uses cross entropy loss and pair ranking loss to evaluate the accuracy of task planning, and the calculation formulas are as shown in formulas (1), (2) and (3).

[0140] Step S300 comprehensively considers the availability of the AI ​​model A i , performance, response time, cost and other factors to score the AI ​​model, and select the Top-K AI models that are most suitable for the current subtask based on the scoring results, among which the i-th AI model score S i , the calculation formula is as shown in formula (4).

[0141] Specifically, step S310 compares the text vector similarity between the subtask and the AI ​​model library, and preliminarily screens out M possible candidate models.

[0142] Step S320 queries the AI ​​indicators of the candidate models corresponding to each subtask from the AI ​​gateway, such as availability, performance, response time and cost.

[0143] Step S330 selects the top-K AI models based on factors such as availability, performance, response time, and cost, where K ranges from [1, M], and the score of the i-th AI model is S i , the calculation formula is as shown in formula (4).

[0144] Step S340 plans the AI ​​model proxy URL and its input parameters.

[0145] According to the task sequence planned in step S300, the work path is executed, and step S410 obtains the service ID according to the routing strategy. Step S420 asynchronously executes the model mapped by the service ID. For example, according to the task sequence "facebook / detr-resnet-50", the work path "target detection" is executed, according to the task sequence "Salesforce / blip-vqa-base", the work path "picture and text question and answer" is executed, and according to the task sequence "timbrooks / instruct-pix2pix", the work path "picture generation" is executed.

[0146] Step S500 uses a large language model to summarize the asynchronous execution results, wherein the problem of fluctuation in the selection quality of the candidate models in step S300 is alleviated by merging the inference results of the top-K candidate models with the highest rankings.

[0147] For input x and its corresponding subtask t i , and the highest ranked candidate AI models or applications m i , which produces an improved output as the final response The calculation formula is as shown in formula (6).

[0148] The drawings in the specification Figure 6 The working process of the AI ​​gateway 20 is explained. Figure 6As shown, the public endpoint of the AI ​​gateway is http: / / ai.crdigital.com. The route matching rule is that the path starts with glm-4v ("~ / glm-4v"). The AI ​​gateway maps route 1 to the gateway service (glm-4v-svc), and service 1 connects to the upstream multimodal large model (glm-4v). For the model response return, the AI ​​gateway takes the opposite path.

[0149] The drawings in the specification Figure 7 The multimodal input and output in the artificial intelligence routing method according to the above preferred embodiment of the present invention is explained.

[0150] The drawings in the specification Figure 8 An application example of the artificial intelligence routing method according to a preferred embodiment of the present invention is explained, wherein the task is "write a slogan for AI Route and explain it with pictures, wherein the artificial intelligence routing method uses a large language model as an intermediate controller to manage all AI models and solves complex AI tasks by "mobilizing and combining the power of each model".

[0151] The drawings in the specification Fig. 9 It is a code example of AI routing in the artificial intelligence routing method according to the preferred embodiment of the present invention.

[0152] The drawings in the specification Fig.10 It is a code example of an AI gateway in an artificial intelligence routing method according to a preferred embodiment of the present invention, including an AI-Proxy plug-in configuration of the AI ​​gateway and an AI-Proxy unified model interface, such as unified authority management.

[0153] In summary, the preferred embodiment of the present invention uses the large language model as an intermediate controller to manage the AI ​​model, and realizes multimodality by mobilizing and combining the power of each model. The advantages are:

[0154] 1) Multimodality: Different from the implementation through mixture of experts (MOE), this preferred embodiment of the present invention improves the generalization ability of the model through dynamic mixture of experts (DMOE) bridging mode;

[0155] 2) AI routing: Different from selecting by model description, this preferred embodiment of the present invention selects the model by comprehensively considering factors such as the availability, performance, response speed, and cost of the AI ​​model;

[0156] 3) AI Gateway: Different from using fixed model + fixed template + scattered interfaces + scattered formats, this preferred embodiment of the present invention implements dynamic model + dynamic template + unified interface + unified format.

[0157] The artificial intelligence routing method and artificial intelligence routing system according to the preferred embodiment of the present invention unify the large model application entrance with its unique AI gateway, effectively reducing the difficulty of user use. In addition, the artificial intelligence routing system also uses AI routing technology for task planning, model selection, task execution and response summary. Through collaborative work, the advantages of various models are fully utilized and their shortcomings are compensated. The advantages are as follows:

[0158] 1) Improve overall performance: Different models have different performances in specific tasks. Collaborative work helps to complement each other and improve overall performance.

[0159] 2) Enhanced multi-task processing capabilities: A single model performs well in some tasks but not in others. Collaborative work helps build a more comprehensive and powerful system.

[0160] 3) Improve computational efficiency: Some collaborative strategies use small models to assist large models, thereby speeding up reasoning and reducing computational costs;

[0161] 4) Reduce errors and misjudgments: Through the collaborative work of multiple models, the output results can be cross-validated, thereby reducing the errors or misjudgments that may occur in a single model.

Claims

1. An artificial intelligence routing method, characterized in that: The method comprises the following steps: S100) The artificial intelligence routing agent receives and analyzes at least one prompt message; S200) the artificial intelligence routing agent analyzes at least one request intention of the user according to the prompt information, decomposes the request intention into multiple subtasks, and plans the interdependencies between the subtasks to form a task list; S300) The artificial intelligence routing agent selects an artificial intelligence model proxy interface registered on the artificial intelligence gateway according to the characteristics of the artificial intelligence model to match the requirements of the task list, wherein the artificial intelligence model selected to match the requirements of the task list is the selected artificial intelligence model; S400) calling and executing each selected artificial intelligence model, and returning the result to the artificial intelligence routing agent; and S500) The AI ​​routing agent integrates the prediction results of all selected AI models and generates a response.

2. The artificial intelligence routing method according to claim 1, characterized in that: In step S300, the artificial intelligence routing agent selects an artificial intelligence model from the model proxy interface registered on the artificial intelligence gateway to select an artificial intelligence model suitable for the task list as the selected artificial intelligence model, wherein the factors for selecting the artificial intelligence model include the availability, performance, response speed and cost of the model.

3. The artificial intelligence routing method according to claim 2, characterized in that: In step S400, the artificial intelligence routing agent obtains the response results of the asynchronous execution of each subtask by calling the artificial intelligence model interface corresponding to the selected artificial intelligence model and post-processing related prediction results, wherein in step S500, the artificial intelligence routing agent summarizes the prediction results of all artificial intelligence models and generates a response to return to the user.

4. The artificial intelligence routing method according to claim 3, characterized in that: Step S300 includes the following steps: S310) selecting a plurality of candidate models according to the task sequence; S320) querying artificial intelligence indicators from the artificial intelligence gateway; S330) screening models, wherein the models are screened based on factors including performance, speed, and cost; and S340) Standardize the model proxy URL and input parameters.

5. The artificial intelligence routing method according to claim 4, characterized in that: Step S200 uses cross entropy loss and pair ranking loss to evaluate the accuracy of task planning, and the calculation formula is as follows: L task =L class +λ*L rank (1) In formula (1), λ represents the weight coefficient, In formula (2), M is the total number of planning tasks, C is the number of categories, O is the number of sequences, and Y ic represents the symbolic function. If the true category of sample i is equal to c, then Y ic Take 1, otherwise Y ic Take 0, where is the predicted probability that the observed sample i belongs to category c, In formula (3), T ij represents the true probability of sample i before sample j, T ij represents the predicted probability of sample i before sample j, and δ represents the Sigmoid function.

6. The artificial intelligence routing method according to claim 5, characterized in that: Step S300 comprehensively considers the availability of the artificial intelligence model A i , performance, response time, and cost to score the AI ​​models, and select the Top-K AI models that are most suitable for the current subtask based on the scoring results, where the score of the i-th AI model is S i , the calculation formula is as follows: In formula (4), F i represents the performance score, T i Represents the response time, C i represents the call cost, where the weights of performance, response time and call cost are ω respectively f ,ω t and ω c ; Among them, the performance score F i The calculation formula is as follows: In formula (5), P i Represents the accuracy, R i represents the recall rate, and β represents P i With R i Relative importance.

7. The artificial intelligence routing method according to claim 5, characterized in that: Step S310 compares the text vector similarity of the subtask and the artificial intelligence model library, and preliminarily screens out M possible candidates. Select model. Step S320 queries the artificial intelligence indicators of the candidate models corresponding to each subtask from the artificial intelligence gateway. Step S330 selects the top-K artificial intelligence models according to the artificial intelligence indicators queried in step S320, where K ranges from [1, M], and the score of the i-th artificial intelligence model is S i , the calculation formula is as follows: In formula (4), F i represents the performance score, T i Represents the response time, C i represents the call cost, where the weights of performance, response time and call cost are ω respectively f ,ω t and ω c ; Among them, the performance score F i The calculation formula is as follows: In formula (5), P i Represents the accuracy, R i represents the recall rate, and β represents P i With R i The relative importance, Step S340 plans the artificial intelligence model proxy URL and its input parameters.

8. The artificial intelligence routing method according to claim 7, characterized in that: Step S500 summarizes the asynchronous execution results using a large language model, by merging the inference results of the top-K candidate models, where for the input x and its corresponding subtask t i , and the highest ranked candidate AI models or applications m i , producing an improved output as the final response The calculation formula is as follows: In formula (6), f sum represents generation fusion, and N represents the number of subtasks.

9. An artificial intelligence routing system applied to the artificial intelligence routing method according to any one of claims 1 to 8, characterized in that: It includes an artificial intelligence router and an artificial intelligence gateway, wherein the artificial intelligence router obtains artificial intelligence indicators, unified API interface, context, certificate and prompt template information from the artificial intelligence gateway, and screens out a suitable artificial intelligence model by comprehensively considering availability, performance, response speed and cost factors.

10. A computer device, characterized in that: The device includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; A processor, for implementing the steps of the artificial intelligence routing method according to any one of claims 1 to 8 when executing a program stored in a memory.

Images