AI large model intelligent routing and dynamic arrangement method, system, device and medium

Abstract

The invention discloses an AI large model intelligent routing and dynamic arrangement method, system and device and a medium. The method comprises the following steps: in response to an input request, performing multi-dimensional feature extraction on the input request to obtain a request feature vector; obtaining the real-time state and metadata of each AI large model, and inputting the request feature vector and the real-time state and metadata of each AI large model into a multi-target optimization model to obtain a candidate model combination corresponding to the input request; according to the dynamic arrangement template, determining a calling sequence and a data dependency relationship of each AI large model in the candidate model combination, and generating an execution plan of the candidate model combination; and the input request is adapted and sent to the candidate model combination, each AI large model in the candidate model combination is called based on the execution plan, and an execution result is obtained and returned to the request terminal. Through the above mode, the candidate model combination can be intelligently selected, the accuracy and efficiency of model selection are improved, and a plurality of candidate models are flexibly arranged to complete a complex task.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of AI large models, in particular to an AI large model intelligent routing and dynamic arrangement method, system, device and medium. BACKGROUND

[0002] At present, various general and specific field AI large models (such as large language models, image generation models, speech recognition models, etc.) emerge in endlessly, which have differences in capability, performance and cost, etc. Some existing model aggregation platforms only provide a unified API entry, therefore, users or application developers often face the following challenges when selecting and using these models: Difficulty in selection: it is difficult to quickly determine which model is most suitable for the current task requirement in the face of numerous models, especially when considering factors such as cost, response time and specific capability preference; Static integration: the traditional way is to statically select and integrate a certain model or a few models during the application development stage, which lacks flexibility and is difficult to adapt to the scenarios of rapid model iteration and dynamic demand changes; Waste of resources: it may cause resource waste and cost virtualization to select a "one-size-fits-all" super large model to process a simple task, or it may affect the task completion quality due to insufficient model capability; Complex arrangement: for complex tasks that require multiple models to cooperate to complete, how to effectively arrange the calling sequence of the models, process the intermediate results, and manage the overall process requires a high requirement for developers.

[0003] In addition, the existing model aggregation platform still has deficiencies in intelligent selection and dynamic arrangement of models, and lacks the ability to make fine and adaptive routing decisions according to real-time request features, model states and user policies. SUMMARY

[0004] The present application mainly provides an AI large model intelligent routing and dynamic arrangement method, system, device and medium to solve the problems of difficulty in selecting AI large models, inflexible integration, low resource utilization and complex arrangement in the prior art.

[0005] To solve the above technical problems, one technical solution adopted by the present application is to provide an AI large model intelligent routing and dynamic arrangement method, which comprises: In response to an input request, performing multi-dimensional feature extraction on the input request to obtain a request feature vector; Obtaining real-time states and metadata of each AI large model, inputting the request feature vector and the real-time states and metadata of each AI large model into a multi-objective optimization model to obtain a candidate model combination corresponding to the input request; determine a calling sequence and a data dependency relationship of each AI large model in the candidate model combination according to the dynamic orchestration template, and generate an execution plan of the candidate model combination; adapt and send the input request to the candidate model combination, call each AI large model in the candidate model combination based on the execution plan, obtain an execution result, and return the request terminal.

[0006] In an optional implementation of the embodiment of the application, the dimensions of the input request include an intent, domain relevance, complexity, data type, and user preference. In response to the input request, the method further includes: In response to the input request, the method further includes:

[0007] In an optional implementation of the embodiment of the application, the method further includes: The method further includes:

[0008] In an optional implementation of the embodiment of the application, the method further includes: The method further includes: In an optional implementation of the embodiment of the application, the method further includes:

[0009] In an optional implementation of the embodiment of the application, the method further includes: The method further includes:

[0010] In an optional implementation of the embodiment of the application, the metadata of the AI large model at least includes model capability, performance portrait, and cost model. The request feature vector, the real-time state of each AI large model, and the metadata are input into a multi-objective optimization model to obtain a candidate model combination corresponding to the input request. The request feature vector, the real-time state of each AI large model, and the metadata are input into the multi-objective optimization model, and the multi-objective optimization model determines a candidate model combination corresponding to the input request based on basic constraint factors and target optimization factors. The basic constraint factors at least include a matching degree of the request feature vector and a model capability of the AI large model. The target optimization factors at least include a matching degree of the request feature vector and a performance portrait and a cost model of the AI large model.

[0011] In an optional implementation of the embodiment of the application, the system further includes: After obtaining the execution result and returning the request terminal, the system collects an execution result feedback sent by the request terminal and a time consumption and a cost of calling each candidate model; According to the time consumption and the cost of calling each candidate model, the system updates a performance portrait and a cost model of the corresponding candidate model; According to the execution result feedback, the system optimizes parameters of the multi-objective optimization model and an arrangement strategy of the dynamic arrangement template through adaptive learning.

[0012] In an optional implementation of the embodiment of the application, the system further includes: According to the calling sequence, the system determines a first-called candidate model in the candidate model combination, adapts and sends the input request to the first-called candidate model according to an API interface specification of the first-called candidate model, and dynamically adapts the remaining candidate models in the candidate model combination according to the data dependency relationship.

[0013] To solve the above technical problems, another technical solution adopted by the application is to provide an AI large model intelligent routing and dynamic arrangement system, which includes a request analysis module, a model resource library module, an intelligent routing decision engine, a dynamic model arrangement unit, an adaptive execution module, and a feedback learning module. The request analysis module is configured to perform multi-dimensional feature extraction on an input request to obtain a request feature vector and send the request feature vector to the intelligent routing decision engine in response to the input request. The model resource library module is configured to register each AI large model and store metadata of each AI large model, and monitor real-time states of each AI large model. The intelligent routing decision engine is configured to obtain real-time states and metadata of each AI large model from the model repository module, input the request feature vector and the real-time states and metadata of each AI large model into a multi-objective optimization model, obtain a candidate model combination corresponding to the input request, and send the candidate model combination to the dynamic arrangement unit; The dynamic model arrangement unit is configured to determine calling sequences and data dependency relationships of each AI large model in the candidate model combination according to a dynamic arrangement template, generate an execution plan of the candidate model combination, and send the execution plan to the adaptive execution module; The adaptive execution module is configured to adapt and send the input request to the candidate model combination, call each AI large model in the candidate model combination based on the execution plan, obtain an execution result, and return the execution result to the request terminal; The feedback learning module is configured to collect execution result feedback sent by the request terminal and time consumption and cost of calling each candidate model after obtaining the execution result and returning the request terminal, update performance portraits and cost models of the corresponding candidate models in the model repository module according to the time consumption and cost of calling each candidate model, and optimize parameters of the multi-objective optimization model in the intelligent routing decision engine and arrangement strategies of the dynamic arrangement template in the dynamic model arrangement unit according to the execution result feedback.

[0014] To solve the above technical problems, another technical solution adopted by the present application is to provide a computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the steps of the AI large model intelligent routing and dynamic arrangement method.

[0015] To solve the above technical problems, another technical solution adopted by the present application is to provide a computer readable storage medium having a computer program stored thereon, characterized in that the computer program is executed by a processor to implement the steps of the AI large model intelligent routing and dynamic arrangement method.

[0016] The beneficial effects of the present application are: different from the prior art, the present application discloses an AI large model intelligent routing and dynamic arrangement method, system, device and medium. The method obtains a request feature vector by performing multi-dimensional feature extraction on an input request, inputs the request feature vector and the obtained real-time state and metadata of each AI large model into a multi-objective optimization model, obtains a corresponding candidate model combination, generates an execution plan through a dynamic arrangement template, adapts and sends the input request to the candidate model combination according to the execution plan, calls the candidate model based on the execution plan, obtains an execution result and returns to the request terminal. The method can intelligently select the most suitable candidate model combination according to the multi-dimensional features of the input request and the real-time state of the AI large model, improve the accuracy and efficiency of model selection, avoid unnecessary resource waste, balance the cost, response time and result quality on the premise of meeting user demand, maximize the overall benefit, avoid models with faults or performance degradation, and improve the overall stability and usability; multiple candidate models can be flexibly combined through a dynamic arrangement template to complete complex tasks, reducing the use threshold of developers. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor. Figure 1 is a flowchart of an AI large model intelligent routing and dynamic arrangement method embodiment one provided by the present application; Figure 2 is an execution and feedback optimization flowchart of the AI large model intelligent routing and dynamic arrangement method embodiment one provided by the present application; Figure 3 is a structure diagram of an AI large model intelligent routing and dynamic arrangement system embodiment two provided by the present application. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0019] The terms "first", "second", "third" in the embodiments of the present application are only used for descriptive purpose and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second", "third" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "plurality" is at least two, for example, two, three, etc., unless otherwise explicitly and specifically limited. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed or can optionally include other steps or units inherent to the process, method, product or device.

[0020] Reference to "embodiments" herein means that the specific features, structures or characteristics described in connection with the embodiments can be included in at least one embodiment of the present application. The phrase appears at various places in the specification does not necessarily all refer to the same embodiment, nor does it necessarily refer to a separate or alternative embodiment in addition to other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0021] Embodiment one The present application provides an AI large model intelligent routing and dynamic arrangement method, referring to Figure 1 , Figure 1 is a flowchart of an AI large model intelligent routing and dynamic arrangement method embodiment one provided by the present application, and the AI large model intelligent routing and dynamic arrangement method comprises: S10: In response to an input request, multi-dimensional feature extraction is performed on the input request to obtain a request feature vector.

[0022] In the present application, the dimensions of the input request include intent, domain relevance, complexity, data type and user preference; in response to the input request, multi-dimensional feature extraction is performed on the input request to obtain a request feature vector, which comprises: In response to the input request, the intent, domain relevance, complexity, data type and user preference of the input request are respectively extracted by a preset hierarchical fusion strategy, and the obtained feature vectors corresponding to each dimension are fused to obtain the request feature vector.

[0023] In the present application, the input request from the requesting terminal is received, and the operator of the requesting terminal can be a user or an application, which sends the task that needs to be selected to call the AI large model to process as an input request through the interactive mode of the dialogue. Among them, the AI large model in the present application refers to various large artificial intelligence models, such as natural language processing models (such as GPT, BERT), multi-modal models (DALL·E), and code generation models (such as Codex) and other large artificial intelligence models.

[0024] After receiving the input request, the input request can be extracted from multiple dimensions, such as analyzing the intent, domain relevance, complexity, data type and user preference (such as the tendency of cost, speed and accuracy) contained in the input request. When extracting features from multiple dimensions, considering the need to balance real-time, semantic depth and dynamic suitability, a hierarchical fusion strategy can be used for feature extraction, that is, the features of each dimension are extracted hierarchically, and then the extracted features of each dimension are fused to obtain the request feature vector corresponding to the input request.

[0025] Among them, the complexity of the input request includes semantic complexity, context dependence and task complexity, etc. For example, from the semantic complexity, the simple input request with single and clear intent is a low complexity input request, and the input request with implicit multi-level intent or abstract concept is a high complexity input request. The data type of the input request refers to the format or structure type of the input data, such as text, multi-modal data, structured data, code and composite instructions.

[0026] Unlike the prior art, when facing numerous models, especially when considering factors such as cost, response time and specific ability preference, it is difficult to quickly determine which model is most suitable for the current task requirement. The AI large model intelligent routing and dynamic arrangement method provided in the present application can analyze the input request from the dimensions of intent, domain relevance, complexity, data type and user preference through the preset hierarchical fusion strategy, refine the input request and grasp the core demand of the user, and can more accurately find the AI large model that meets the current task requirement for the user.

[0027] In the present application, the intent, domain relevance, complexity, data type and user preference of the input request are extracted through the preset hierarchical fusion strategy, including: The corresponding features of the intent and domain relevance of the input request are extracted through the pre-trained language model, the corresponding features of the complexity and data type of the input request are extracted by combining statistical features and rule engines, and the corresponding features of the user preference of the input request are extracted by keyword matching or slot filling.

[0028] In the present application, the pre-trained language model can be used for feature extraction in the two dimensions of intent and field relevance. By capturing the deep semantics of the input request through the pre-trained language model, the task intent (such as translation or code generation) and the relevant field (such as medical or financial) of the input request are identified. This identification method uses an attention mechanism to automatically learn the inter-word dependencies without manual feature engineering. Compared with simply identifying the task intent of the input request, the present application adds the identification of the relevant field to optimize the accuracy of the vertical scene through field adaptation, which can more accurately capture the task requirements contained in the input request. The pre-trained language model can be a context-aware model based on the Transformer architecture (Bidirectional Encoder Representations from Transformers, BERT), which can significantly improve the performance of natural language processing (NLP) tasks through bidirectional context awareness and large-scale pre-training + fine-tuning paradigm. The pre-trained language model can also be a RoBERTa or ELECTRA model, which is not limited here.

[0029] For feature extraction in the two dimensions of complexity and data type, a statistical feature combined with a rule engine can be used. For complexity, the complexity features can be quantitatively extracted by text length, dependency syntax analysis (calculating the depth of the sentence tree), and named entity recognition (NER) entity density. For data type, data type features can be extracted by regular expression matching (such as detecting code snippets or image URLs) and file format parsing to ensure efficient processing of structured data.

[0030] For feature extraction in the dimension of user preference, an interactive method can be used to directly obtain user preference features through keyword matching or slot filling, such as the user specifying cost priority in the input request.

[0031] Unlike existing technologies, the AI large model intelligent routing and dynamic arrangement method provided by the present application extracts the corresponding features of the intent and field relevance of the input request through the pre-trained language model, optimizes the accuracy of the vertical scene through field adaptation, and can more accurately capture the task requirements contained in the input request. By combining statistical features and rule engines to extract the corresponding features of the complexity and data type of the input request, real-time performance, semantic depth, and dynamic adaptability are considered, and dynamic adaptation of the multi-dimensional features of the input request can efficiently process structured data. By keyword matching or slot filling to extract the corresponding features of the user preference of the input request, the user's tendency in terms of cost, speed, or accuracy can be accurately obtained.

[0032] In the present application, the AI large model intelligent routing and dynamic arrangement method further comprises: The evolution trend of the user preference is modeled based on historical input requests through time series analysis and a neural network; When the corresponding feature of the user preference of the input request is not extracted, the corresponding feature of the current user preference is determined according to the evolution trend of the user preference combined with the preset user preference.

[0033] The above-mentioned way of extracting the user preference from the input request extracts the explicit preference of the user, that is, the user preference that can be directly captured in the input request. In the case where the input request does not contain the user preference related feature, the implicit preference of the user can also be obtained according to the historical input request. The evolution trend of the user preference is modeled through time series analysis (such as sliding window) of the historical input request, combined with a recurrent neural network (RNN) or a long short-term memory network (LSTM), so as to dynamically capture the change of the user preference.

[0034] In addition, the user can also preset a user preference in advance. The preset user preference can be a personalized rule configured by the user in advance, which can provide a long-term and stable priority orientation for subsequent intelligent routing decision (for example, the preset "cost priority"). The preset user preference can be combined with the user preference contained in the real-time input request to form the feature of the user preference dimension; or in the case where the real-time input request does not contain the user preference, the preset user preference can be combined with the implicit preference of the user to form the feature of the user preference dimension.

[0035] Unlike the prior art, the AI large model intelligent routing and dynamic arrangement method provided by the present application models the evolution trend of the user preference based on historical input requests as the implicit preference of the user, which can dynamically capture the change of the user preference and provide support for the case where the user preference is not directly expressed in the input request, facilitating the subsequent selection of the cost, speed and accuracy of the AI large model that is most suitable for the current preference of the user.

[0036] After the above-mentioned process extracts the features of the intent, domain relevance, complexity, data type and user preference of the input request respectively, the heterogeneous features are uniformly represented through feature fusion, and standardized and dimension reduction techniques (such as PCA) are used to output the structured request feature vector, so as to ensure the efficiency of the subsequent intelligent routing decision. The feature fusion method can be a concatenation of embedding vectors combined with a fully connected layer or a gating mechanism, which is not limited here.

[0037] S20: Obtain the real-time state and metadata of each AI large model, input the request feature vector and the real-time state and metadata of each AI large model into a multi-objective optimization model, and obtain a candidate model combination corresponding to the input request.

[0038] In the present application, the AI large model intelligent routing and dynamic arrangement method further comprises: Registering each AI large model and storing the metadata of each AI large model, monitoring the real-time state of each AI large model; the real-time state includes real-time health state, load and API availability.

[0039] In the present application, a model repository module is pre-constructed for registering and managing multiple heterogeneous AI large models, and storing the metadata of each AI large model, including the model identifier of the AI large model, capability description, performance profile, cost model, version information and API interface specification, etc. Among them, the model identifier of the AI large model refers to the name of the AI large model, such as DeepSeek-R1; the capability description includes the task type supported by the AI large model, the format of input and output, etc.; the performance profile includes benchmark test score, average response time and throughput, accuracy index, etc.; the cost model includes unit price and charging mode; the version information includes version naming rule, version difference and life cycle management; the API interface specification includes endpoint, request parameter, response format, authentication and flow limiting and error code. Real-time or quasi-real-time monitoring of the dynamic information of the health state, current load and API availability of each AI large model in the model repository, so as to facilitate subsequent calling.

[0040] Different from the prior art, the AI large model intelligent routing and dynamic arrangement method provided by the present application can effectively avoid selecting an AI large model with faults or performance degradation to provide services for users by real-time monitoring of the dynamic information of the health state, load and API availability of each AI large model, thereby improving stability and availability.

[0041] In the above step S20, the real-time state and metadata of each AI large model are obtained, and the request feature vector obtained in the above step S10 and the real-time state and metadata of each AI large model are input into a multi-objective optimization model to obtain a candidate model combination corresponding to the input request. The multi-objective optimization model is a key algorithm model for comprehensively considering multiple factors to make optimal decisions, which includes a demand and supply matching, multi-objective quantization, algorithm solving and closed-loop optimization intelligent decision model, the core of which is to reduce the candidate pool through dimension-level feature mapping, and then find the optimal solution in a dynamic environment by using a multi-objective algorithm driven by user preferences, the specific base model and algorithm used can be flexibly adjusted according to the above functions, which is not limited here. At the same time, it can also continuously learn user preferences and model state through feedback to ensure the accuracy and adaptability of the routing decision.

[0042] In the present application, the metadata of the AI large model at least includes model capability, performance portrait and cost model; the request feature vector and the real-time state and metadata of each AI large model are input into a multi-objective optimization model to obtain a candidate model combination corresponding to the input request, including: The request feature vector and the real-time state and metadata of each AI large model are input into a multi-objective optimization model, and the multi-objective optimization model determines the candidate model combination corresponding to the input request based on the basic constraint factors and the target optimization factors; Wherein, the basic constraint factors at least include the matching degree of the request feature vector and the model capability of the AI large model; the target optimization factors at least include the matching degree of the request feature vector and the performance portrait and cost model of the AI large model.

[0043] In the present application, when determining the most suitable candidate model combination according to the request feature vector and the real-time state and metadata of each AI large model, the multi-objective optimization model comprehensively considers two types of factors, namely the basic constraint factors and the target optimization factors. Among them, the basic constraint factors, i.e. the capability matching factors, are rigidly considered factors, which are strongly bound to the core demand of the input request, such as the matching degree of the request feature vector and the model capability of the AI large model; the optimization target factors, i.e. the performance, cost and accuracy factors, are flexibly selected factors, which are determined by the explicit expression of user preference combined with the default strategy, and when not mentioned in the input request, the default rule is used for weighted consideration, i.e. according to the user preference in the input request and the preset user preference to determine, such as the expected performance of the AI large model (including the response time and accuracy rate predicted according to the historical data and current state), the expected cost of the AI large model, and the weight of performance and cost (such as cost priority, speed priority, quality priority or custom weight). In addition to the above factors, the current health status and current load of the AI large model also need to be considered to ensure that the selected AI large model can be normally called.

[0044] Unlike the prior art, where users usually select and use AI large models statically during the application development stage, the present application provides an AI large model intelligent routing and dynamic arrangement method that can intelligently select the most suitable AI large model based on the real-time characteristics of the input request and the dynamic state of the AI large model, ensuring that different tasks are allocated appropriate AI capabilities, improving the accuracy and efficiency of model selection, and maximizing resource utilization and reducing overall costs while meeting the desired results. This balances costs, response times, and result quality, avoids unnecessary resource waste, and maximizes overall benefits.

[0045] In step S20, the output of the multi-objective optimization model can be a candidate model combination composed of multiple candidate models, or a single candidate model, depending on the complexity of the task.

[0046] S30: According to the dynamic arrangement template, determine the calling order and data dependency relationship of each AI large model in the candidate model combination, and generate an execution plan for the candidate model combination.

[0047] In this application, if the execution of step S20 results in a candidate model combination containing multiple candidate models, i.e., the multi-objective optimization model determines that a single model cannot meet the complex task contained in the input request, or using a combination model can bring better results, then step S30 is continued.

[0048] The dynamic arrangement template is obtained by learning and optimizing the arrangement logic in the predefined arrangement template. The dynamic arrangement template can be a reusable task blueprint that allows flexible arrangement through task pattern abstraction, dynamic slot mechanism, and flow logic, rather than mechanical sorting. Task pattern abstraction involves breaking down complex tasks into standardized processing stages (Stages), with each stage corresponding to a type of model capability requirement. The dynamic slot mechanism involves filling the placeholders (non-specific models) for each stage (e.g., "semantic understanding LLM") with the optimal model at runtime based on real-time state. The flow control logic includes branching judgments (such as condition C), loop mechanisms (such as iterative optimization), and parallel processing.

[0049] In step S30, the calling sequence and data dependency relationship of each AI large model in the candidate model combination are determined according to the dynamic arrangement template, so as to generate an execution plan containing multiple ordered AI large model calling instructions. The data dependency relationship is to ensure the effective transmission of input and output among AI large models, and is a core constraint condition for correct execution of complex tasks, that is, the constraint relationship of data transmission between upstream and downstream models when multiple AI large models work together, which will directly affect the arrangement sequence of AI large models.

[0050] Unlike the prior art, for complex tasks that require multiple models to work together, how to effectively arrange the calling sequence of the models, process the intermediate results, and manage the overall process is a problem that requires high requirements for developers. The AI large model intelligent routing and dynamic arrangement method provided in the present application can flexibly combine multiple AI large models to complete complex tasks through a dynamic arrangement template, thereby reducing the complexity of multi-model integration.

[0051] In addition, if the execution step S20 obtains a single candidate model, the execution plan is directly output as calling this candidate model.

[0052] S40: adapt and send the input request to the candidate model combination, call each AI large model in the candidate model combination based on the execution plan, obtain the execution result and return it to the request terminal.

[0053] In the present application, adapting and sending the input request to the candidate model combination includes: According to the calling sequence, the first called candidate model in the candidate model combination is determined, the input request is adapted and sent to the first called candidate model according to the API interface specification of the first called candidate model, and the remaining candidate models in the candidate model combination are dynamically adapted according to the data dependency relationship.

[0054] In the present application, after obtaining the execution plan of the candidate model combination from step S30, the input request adaptation is performed on the candidate model combination according to the API interface specification in the model repository and the API of each candidate model. According to the execution plan, there is a calling sequence in the execution process of calling each candidate model, for example, the candidate model combination includes candidate model A and candidate model B, candidate model A is called first, candidate model A receives the input request, and output result A is obtained through candidate model A; then candidate model B is called, and output result A is used as the input of candidate model B, and the final execution result is obtained through candidate model B. As can be seen, the input request is only used as the input of the first called candidate model, and the input of each other model called subsequently is determined according to the output of the previous model, so when performing API interface adaptation, the input request is adapted and sent to the first called candidate model according to the API interface specification of the first called candidate model, and then the remaining candidate models in the candidate model combination are dynamically adapted according to the data dependency relationship in the execution plan, and the inputs corresponding to each candidate model are sent in the execution process. In addition, if a single candidate model is obtained by executing step S20, the input request can be directly adapted and sent to the candidate model.

[0055] Compared with the prior art, the AI large model intelligent routing and dynamic arrangement method provided in the present application improves the execution efficiency and stability of the execution plan by dynamically adapting the API interfaces of each candidate model according to the calling sequence and data dependency relationship.

[0056] After the adaptation is completed, each AI large model in the candidate model combination is called based on the execution plan, and the final execution result is obtained, the execution result is collected and returned to the request terminal.

[0057] In the present application, the AI large model intelligent routing and dynamic arrangement method further comprises: After obtaining the execution result and returning to the request terminal, the execution result feedback sent by the request terminal and the time consumption and cost of calling each candidate model are collected; According to the time consumption and cost of calling each candidate model, the performance portrait and cost model of the corresponding candidate model are updated; According to the execution result feedback, the parameters of the multi-objective optimization model and the arrangement strategy of the dynamic arrangement template are optimized through adaptive learning.

[0058] In the present application, after obtaining the execution result and returning to the request terminal, the actual performance data of calling each candidate model in the execution process is collected, including the time consumption and cost, and according to the time consumption and cost of calling each candidate model, the performance portrait and cost model of the corresponding candidate model are updated.

[0059] After the user of the request terminal receives the execution result, the user can also perform execution result feedback on the execution result for this execution plan. The execution result feedback can be satisfaction feedback or problem feedback, etc. Therefore, after obtaining the execution result and returning to the request terminal, the execution result feedback sent by the request terminal can also be collected, and the parameters of the multi-objective optimization model and the arrangement strategy of the dynamic arrangement template are respectively optimized according to the execution result feedback through adaptive learning.

[0060] Unlike the prior art, the AI large model intelligent routing and dynamic arrangement method provided by the application optimizes and updates the parameters of the multi-objective optimization model and the arrangement strategy of the dynamic arrangement template and the performance portrait and cost model of the candidate model respectively by collecting execution result feedback and calling the time consumption and cost of each candidate model, and realizes self-evolution and optimization through adaptive learning, rather than being fixed after one-time configuration, so as to adapt to model decline, environmental changes and user demand evolution, and maintain high efficiency and precision in the long run.

[0061] The execution and feedback optimization process of the AI large model intelligent routing and dynamic arrangement method provided by the application is described with reference to Figure 2 , Figure 2 is the execution and feedback optimization process schematic diagram of the AI large model intelligent routing and dynamic arrangement method provided by the application. First, the input request from the request terminal is received, the input request is subjected to multi-dimensional feature extraction to obtain a request feature vector, and the real-time state and metadata of each AI large model in the model resource library are obtained; then, the request feature vector and the real-time state and metadata of each AI large model are input into the multi-objective optimization model, the multi-objective optimization model calculates and judges whether the task contained in the input request requires a candidate model combination to be implemented, if yes, the calling order and data dependency relationship of each AI large model in the candidate model combination are determined through the dynamic arrangement template, an execution plan of the candidate model combination is generated, if not, a single most suitable candidate model is selected, and a single model execution plan is generated; then, the input request is adapted and sent to the corresponding candidate model based on the execution plan, each candidate model in the execution plan is called to execute the task, the execution result is collected and returned to the request terminal; finally, the calling time consumption and cost data of the AI large model are collected to update the performance portrait and cost model of the AI large model, and the execution result feedback from the request terminal is collected to optimize the multi-objective optimization model and the dynamic arrangement template.

[0062] Embodiment two The application provides an AI large model intelligent routing and dynamic arrangement system, with reference to Figure 3 , Figure 3is a structural schematic diagram of Embodiment Two of the AI large model intelligent routing and dynamic arrangement system provided in the present application. The AI large model intelligent routing and dynamic arrangement system comprises a request analysis module 10, a model resource library module 20, an intelligent routing decision engine 30, a dynamic model arrangement unit 40, an adaptive execution module 50, and a feedback learning module 60: The request analysis module 10 is configured to, in response to an input request, perform multi-dimensional feature extraction on the input request, obtain a request feature vector, and send the request feature vector to the intelligent routing decision engine 30. The model resource library module 20 is configured to register each AI large model and store metadata of each AI large model, and monitor real-time states of each AI large model. The intelligent routing decision engine 30 is configured to obtain the real-time states and the metadata of each AI large model from the model resource library module 20, input the request feature vector and the real-time states and the metadata of each AI large model into a multi-objective optimization model, obtain a candidate model combination corresponding to the input request, and send the candidate model combination to the dynamic model arrangement unit 40. The dynamic model arrangement unit 40 is configured to determine a calling sequence and a data dependency relationship of each AI large model in the candidate model combination according to a dynamic arrangement template, generate an execution plan of the candidate model combination, and send the execution plan to the adaptive execution module 50. The adaptive execution module 50 is configured to adapt and send the input request to the candidate model combination, call each AI large model in the candidate model combination based on the execution plan, obtain an execution result, and return the execution result to a request terminal. The feedback learning module 60 is configured to, after obtaining the execution result and returning the execution result to the request terminal, collect an execution result feedback sent by the request terminal and a time consumption and a cost of calling each candidate model, update a performance portrait and a cost model of a corresponding candidate model in the model resource library module 20 according to the time consumption and the cost of calling each candidate model, and optimize parameters of the multi-objective optimization model in the intelligent routing decision engine 30 and an arrangement strategy of the dynamic arrangement template in the dynamic model arrangement unit 40 according to the execution result feedback through self-adaptive learning.

[0063] In the AI large model intelligent routing and dynamic arrangement system of Embodiment Two, the interactions between the request analysis module 10, the model resource library module 20, the intelligent routing decision engine 30, the dynamic model arrangement unit 40, the adaptive execution module 50, and the feedback learning module 60 realize the specific process of AI large model intelligent routing and dynamic arrangement. For details, reference can be made to the detailed description of the AI large model intelligent routing and dynamic arrangement method in Embodiment One above. Repetitive descriptions are omitted here.

[0064] Embodiment Three Based on the same inventive concept, the embodiment of the present application also provides a computer device, comprising a memory, a processor and a computer program stored on the memory, wherein the processor executes the computer program to implement the AI large model intelligent routing and dynamic arrangement method as described in the above embodiment one.

[0065] Embodiment four Based on the same inventive concept, the embodiment of the present application also provides a computer readable storage medium having a computer program / instruction stored thereon, which is executed by a processor to implement the AI large model intelligent routing and dynamic arrangement method as described in the above embodiment one. Unlike the prior art, the present application discloses an AI large model intelligent routing and dynamic arrangement method, system, device and medium. The method obtains a request feature vector by performing multi-dimensional feature extraction on an input request, inputs the request feature vector and the real-time state and metadata of each AI large model obtained into a multi-objective optimization model to obtain a corresponding candidate model combination and generates an execution plan through a dynamic arrangement template, adapts and sends the input request to the candidate model combination according to the execution plan, calls the candidate model based on the execution plan, obtains an execution result and returns to the request terminal. The method can intelligently select the most suitable candidate model combination according to the multi-dimensional features of the input request and the real-time state of the AI large model, improve the accuracy and efficiency of model selection, avoid unnecessary resource waste, balance the cost, response time and result quality on the premise of meeting user demand, maximize overall benefit, avoid models with faults or performance degradation, and improve overall stability and usability; can flexibly combine multiple candidate models through a dynamic arrangement template to complete complex tasks, reducing the use threshold of developers; by encapsulating a series of highly complex underlying logic such as model selection, arrangement, adaptation, execution and optimization, the upper-layer application is provided with an extremely simple "submit request-get result" use experience, as simple as using a super AI model, shielding the complexity of underlying multi-model selection and integration for the upper-layer application, so that developers can focus more on business logic.

[0066] The above is only an embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for intelligent routing and dynamic orchestration of large AI models, characterized in that, include: In response to an input request, multi-dimensional feature extraction is performed on the input request to obtain a request feature vector; Obtain the real-time status and metadata of each AI large model, input the request feature vector and the real-time status and metadata of each AI large model into a multi-objective optimization model to obtain the candidate model combination corresponding to the input request; Based on the dynamic orchestration template, the calling order and data dependencies of each AI model in the candidate model combination are determined, and the execution plan of the candidate model combination is generated. The input request is adapted and sent to the candidate model combination. Based on the execution plan, each AI model in the candidate model combination is called to obtain the execution result and return it to the requesting terminal.

2. The AI ​​large-scale model intelligent routing and dynamic orchestration method according to claim 1, characterized in that, The dimensions of the input request include intent, domain relevance, complexity, data type, and user preference; In response to an input request, multi-dimensional feature extraction is performed on the input request to obtain a request feature vector, including: In response to an input request, features are extracted from the intent, domain relevance, complexity, data type, and user preference of the input request using a preset hierarchical fusion strategy. The feature vectors corresponding to each dimension are then fused to obtain the request feature vector.

3. The AI ​​large-scale model intelligent routing and dynamic orchestration method according to claim 2, characterized in that, The step of extracting features from the input request's intent, domain relevance, complexity, data type, and user preferences using a preset hierarchical fusion strategy includes: The intent and domain relevance of the input request are extracted using a pre-trained language model. The complexity and data type of the input request are extracted by combining statistical features with a rule engine. The user preferences of the input request are extracted by keyword matching or slot filling.

4. The AI ​​large-scale model intelligent routing and dynamic orchestration method according to claim 3, characterized in that, Also includes: By using time series analysis and neural networks, we model the evolutionary trend of user preferences based on historical input requests; If no corresponding features of the user preferences in the input request are extracted, the corresponding features of the current user preferences are determined based on the evolution trend of the user preferences and the preset user preferences.

5. The AI ​​large-scale model intelligent routing and dynamic orchestration method according to claim 1, characterized in that, Also includes: Register each of the AI ​​large models and store the metadata of each of the AI ​​large models, and monitor the real-time status of each of the AI ​​large models; the real-time status includes real-time health status, load and API availability.

6. The AI ​​large-scale model intelligent routing and dynamic orchestration method according to claim 1, characterized in that, The metadata of the AI ​​large model includes at least the model's capabilities, performance profile, and cost model; The step of inputting the request feature vector and the real-time state and metadata of each of the AI ​​large models into a multi-objective optimization model to obtain the candidate model combination corresponding to the input request includes: The request feature vector, along with the real-time status and metadata of each AI large model, are input into the multi-objective optimization model. The multi-objective optimization model determines the candidate model combination corresponding to the input request based on the basic constraints and objective optimization factors. The basic constraints include at least the degree of matching between the requested feature vector and the model capabilities of the AI ​​large model; The target optimization factors include at least the degree of matching between the requested feature vector and the performance profile and cost model of the AI ​​large model.

7. The AI ​​large-scale model intelligent routing and dynamic orchestration method according to claim 1, characterized in that, Also includes: After obtaining the execution result and returning it to the requesting terminal, collect the execution result feedback sent by the requesting terminal and the time and cost of calling each of the candidate models; Based on the time and cost of calling each candidate model, update the performance profile and cost model of the corresponding candidate model; Based on the feedback from the execution results, the parameters of the multi-objective optimization model and the orchestration strategy of the dynamic orchestration template are optimized through adaptive learning.

8. The AI ​​large-scale model intelligent routing and dynamic orchestration method according to claim 1, characterized in that, The step of adapting the input request and sending it to the candidate model combination includes: The candidate model to be called first in the candidate model combination is determined according to the calling order. The input request is adapted and sent to the candidate model to be called first according to the API interface specification of the candidate model to be called first. The remaining candidate models in the candidate model combination are dynamically adapted according to the data dependency relationship.

9. An AI large-scale model intelligent routing and dynamic orchestration system, characterized in that, It includes a request analysis module, a model resource library module, an intelligent routing decision engine, a dynamic model orchestration unit, an adaptation execution module, and a feedback learning module. The request analysis module is used to respond to the input request, perform multi-dimensional feature extraction on the input request, obtain the request feature vector, and send it to the intelligent routing decision engine. The model resource library module is used to register each AI large model and store the metadata of each AI large model, and monitor the real-time status of each AI large model; The intelligent routing decision engine is used to obtain the real-time status and metadata of each of the AI ​​large models from the model resource library module, input the request feature vector and the real-time status and metadata of each of the AI ​​large models into the multi-objective optimization model, obtain the candidate model combination corresponding to the input request, and send it to the dynamic orchestration unit. The dynamic model orchestration unit is used to determine the calling order and data dependency of each AI model in the candidate model combination according to the dynamic orchestration template, generate the execution plan of the candidate model combination and send it to the adaptation execution module. The adaptation execution module is used to adapt the input request and send it to the candidate model combination, call each AI big model in the candidate model combination based on the execution plan, obtain the execution result and return it to the request terminal; The feedback learning module is used to collect the execution result feedback sent by the requesting terminal and the time and cost of calling each of the candidate models after obtaining the execution result and returning it to the requesting terminal; update the performance profile and cost model of the corresponding candidate model in the model resource library module according to the time and cost of calling each candidate model; and optimize the parameters of the multi-objective optimization model in the intelligent routing decision engine and the orchestration strategy of the dynamic orchestration template in the dynamic model orchestration unit through adaptive learning according to the execution result feedback.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the AI ​​large-scale model intelligent routing and dynamic orchestration method according to any one of claims 1-8.

11. A storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the AI ​​large-scale model intelligent routing and dynamic orchestration method as described in any one of claims 1-8.

Images