Method, device, equipment and computer-readable medium for intelligent routing selection model

Abstract

The present application relates to a method, device, equipment and computer-readable medium for intelligent routing selection model. The method includes: classifying models based on model parameter scale and model deployment mode; creating an intelligent router, extracting multidimensional key features in user task requests and generating multidimensional key feature vectors; performing multidimensional resource pool evaluation on each large model through the intelligent router, obtaining a first evaluation result and constructing a multidimensional model capability vector; determining a model candidate set based on the multidimensional key feature vector and each multidimensional model capability vector; if there are multiple candidate large models, performing multidimensional element scoring on each candidate large model to obtain respective second evaluation results, and each element is matched with an element weight that is dynamically adjusted based on user needs; constructing an objective function, and selecting a target large model to respond based on the objective function, decision constraints and each second evaluation result. The present application solves the problems of low flexibility in model selection and low data security.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and computer-readable medium for intelligent routing selection model. Background Art

[0002] In recent years, large language model technology represented by GPT (Generative Pre-trained Transformer) has made breakthrough progress and has shown great potential in many fields such as natural language understanding, text generation, dialogue systems, code programming, and multimodal processing. However, with the rapid development of LLM (Large Language Model) technology and the deepening of commercial applications, many different types and sizes of LLMs have appeared on the market. They show significant differences in performance, deployment methods, security, applicable scenarios, and usage costs. Although this diversity provides more choices for enterprise-level users, it also brings a series of technical challenges. In particular, the intelligence and flexibility of large models are low, and it is difficult for users to accurately select the model that best suits their specific needs, which leads to insufficient performance or waste of resources. Secondly, some tasks involve sensitive data, and users may be at risk of data leakage when using public APIs directly, especially in sensitive industries such as finance and healthcare.

[0003] In the prior art, although optimization solutions or simple load balancing systems for specific LLMs have been proposed, there is still a lack of an intelligent LLM routing solution that can comprehensively consider multi-dimensional factors such as task requirements, security requirements, performance, cost, and compliance. It can be seen that in the use scenarios where existing enterprises integrate multiple LLMs, the use of LLMs has the problems of low model selection flexibility and low data security, and no more effective solution has been proposed yet. Summary of the invention

[0004] The present application provides a method, apparatus, device and computer-readable medium for intelligent routing selection model to solve the technical problems of low model selection flexibility and low data security in the use of LLM in the above-mentioned enterprise integration scenario of multiple LLMs.

[0005] According to one aspect of an embodiment of the present application, the present application provides a method for intelligent routing model selection, including: classifying the deployed models based on the model parameter scale and model deployment mode, and determining the model category corresponding to each model; creating an intelligent router, obtaining a user task request through the intelligent router, and extracting multidimensional key features from the user task request to generate a multidimensional key feature vector; performing a multidimensional resource pool evaluation on the large models under each model category through the intelligent router to obtain a first evaluation result of each large model, and constructing a multidimensional model capability vector according to the first evaluation result of each large model; calculating based on the multidimensional key feature vector and the multidimensional model capability vector corresponding to each large model to determine a model candidate set, wherein there is at least one candidate large model in the model candidate set; if there are multiple candidate large models in the model candidate set, then performing a multidimensional element scoring on each of the candidate large models to obtain a second evaluation result of each of the candidate large models, wherein the elements each match an element weight, and the element weight is dynamically adjusted based on user needs; constructing an objective function corresponding to the candidate large model, and selecting a target large model for request response based on the objective function, preset decision constraints and the second evaluation results of each of the candidate large models.

[0006] Optionally, the model classification of each deployed model based on the model parameter scale and the model deployment mode to determine the model category corresponding to each model includes: obtaining the model parameter scale and the model deployment mode of each large model; judging the model parameter scale of each large model based on a preset first scale interval and a second scale interval to obtain a parameter scale judgment result, wherein the data processing complexity of the second scale interval is higher than that of the first scale interval; judging the model deployment mode of each large model based on a preset first deployment mode and a second deployment mode to obtain a deployment mode judgment result, wherein the data sensitivity of the second deployment mode is higher than that of the first deployment mode; determining the model category of each large model according to the parameter scale judgment result and the deployment mode judgment result of each large model, wherein the model category includes a type of model that satisfies both the first scale interval and the first deployment mode, a type of model that satisfies both the first scale interval and the second deployment mode, a type of model that satisfies both the second scale interval and the first deployment mode, and a type of model that satisfies both the second scale interval and the second deployment mode.

[0007] Optionally, obtaining the user task request through the intelligent router, extracting multidimensional key features from the user task request, and generating a multidimensional key feature vector includes: obtaining the user task request through the intelligent router; performing task analysis on the user task request based on a natural language algorithm, extracting the multidimensional key features, and determining the task type, task complexity, task domain, expected output length and response time according to the multidimensional key features; generating the multidimensional key feature vector according to the task type, task complexity, task domain, expected output length and response time.

[0008] Optionally, the intelligent router performs a multi-dimensional resource pool evaluation on the large models under each model category to obtain a first evaluation result of each large model, and constructs a multi-dimensional model capability vector based on the first evaluation result of each large model, including: obtaining resource pool data of each large model through the intelligent router, the resource pool data including performance indicator data, security level data, domain performance capability data, deployment method data, cost factor data and compliance data; performing resource evaluation based on the performance indicator data, security level data, domain performance capability data, deployment method data, cost factor data and compliance data of the large model to obtain the first evaluation result of each large model; and constructing the multi-dimensional model capability vector based on the initial weight corresponding to each resource pool data and the first evaluation result.

[0009] Optionally, the calculation based on the multidimensional key feature vector and the multidimensional model capability vector corresponding to each large model to determine the model candidate set includes: calculating the similarity between the multidimensional key feature vector and each of the multidimensional model capability vectors; when the similarity satisfies a similarity threshold, selecting the large model corresponding to the multidimensional model capability vector whose similarity satisfies the similarity threshold into the model candidate set as the candidate large model, until all large models are screened.

[0010] Optionally, the multi-dimensional element scoring is performed on each of the candidate large models to obtain a second evaluation result of each of the candidate large models, including: dynamically adjusting the weights of the elements of each dimension based on user needs to obtain the element weight of each element, wherein the user needs include data sensitivity; and calculating the second evaluation result of each of the candidate large models according to the adjusted element weights of the candidate large models and the first evaluation result.

[0011] Optionally, constructing an objective function corresponding to the candidate large model, and determining the optimal routing decision based on the objective function, preset decision constraints and the second evaluation results of each candidate large model, includes: constructing the objective function to maximize performance indicators and minimize costs, screening the candidate large models based on the preset decision constraints, the preset decision constraints including at least one of response time constraints, data sensitivity constraints and cost constraints; prioritizing the candidate large models according to the second evaluation results, and selecting the target large model for request response according to the priority sorting results.

[0012] According to another aspect of the embodiment of the present application, the present application provides an apparatus for intelligent routing selection model, including: a model classification module, which is used to classify the deployed models based on the model parameter scale and the model deployment method, and determine the model category corresponding to each model; a feature extraction module, which is used to create an intelligent router, obtain user task requests through the intelligent router, and extract multidimensional key features from the user task requests to generate a multidimensional key feature vector; a first evaluation module, which is used to perform a multidimensional resource pool evaluation on the large models under each model category through the intelligent router, obtain the first evaluation results of the large models, and construct a multidimensional model capability vector according to the first evaluation results of the large models; a calculation module , used to calculate based on the multidimensional key feature vector and the multidimensional model capability vector corresponding to each large model, and determine the model candidate set, wherein there is at least one candidate large model in the model candidate set; a second evaluation module, used to perform multidimensional element scoring on each of the candidate large models if there are multiple candidate large models in the model candidate set, and obtain a second evaluation result of each of the candidate large models, wherein each element matches an element weight, and the element weight is dynamically adjusted based on user needs; a selection module, used to construct an objective function corresponding to the candidate large model, and select a target large model for request response based on the objective function, preset decision constraints and the second evaluation results of each of the candidate large models.

[0013] According to another aspect of an embodiment of the present application, the present application provides an electronic device, including a memory, a processor, a communication interface and a communication bus, wherein the memory stores a computer program that can be run on the processor, the memory and the processor communicate through the communication bus and the communication interface, and the processor implements the steps of the method of the intelligent routing selection model when executing the computer program.

[0014] According to another aspect of the embodiment of the present application, the present application provides a computer-readable medium having a non-volatile program code executable by a processor, wherein the program code enables the processor to execute the steps of the method of the intelligent routing selection model.

[0015] The above technical solution provided by the embodiment of the present application has the following advantages compared with the related art:

[0016] This application is applied to scenarios where enterprise-level systems integrate multiple large models, such as being applied to recommendation technology for model management. This application classifies the various models according to the scale of model parameters and model deployment methods, which is conducive to the intelligent router to accurately and quickly match LLM and improve the model response speed. Different model deployment methods target different data sensitivities, which is more conducive to meeting users' needs for data sensitivity and improving data security. By creating an intelligent router as an intermediate layer between the user end and the various models, a unified access interface is provided for large models of different sources and types, and the model selection and access method are more flexible, which is convenient for users' daily work. Through multi-dimensional evaluation, element weight adjustment, construction of objective function and setting of decision constraints, it is converted to a multi-objective optimization to select the target large model, which can accurately match the needs, improve task execution efficiency and ensure the rational use of resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 A schematic diagram of a hardware environment for an optional intelligent routing selection model method provided according to an embodiment of the present application;

[0020] Figure 2 A schematic diagram of a method flow of an optional intelligent routing selection model provided according to an embodiment of the present application;

[0021] Figure 3 A schematic diagram of large model classification provided according to an embodiment of the present application;

[0022] Figure 4 A schematic diagram of intelligent router interaction provided according to an embodiment of the present application;

[0023] Figure 5 A device block diagram of an optional intelligent routing selection model provided according to an embodiment of the present application;

[0024] Figure 6 A schematic diagram of an optional electronic device structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0026] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application and have no specific meaning. Therefore, "module" and "component" can be used interchangeably.

[0027] In order to solve the problems mentioned in the background technology, according to one aspect of the embodiments of the present application, an embodiment of a method for intelligent routing selection model is provided.

[0028] Optionally, in the embodiment of the present application, the method of the intelligent routing selection model can be applied to Figure 1 In the hardware environment composed of the terminal 101 and the server 103 shown in FIG. Figure 1 As shown, the server 103 is connected to the terminal 101 through a network, and can be used to provide services for the terminal or a client installed on the terminal. A database 105 can be set on the server or independently of the server to provide data storage services for the server 103. The above network includes but is not limited to: a wide area network, a metropolitan area network or a local area network, and the terminal 101 includes but is not limited to a PC, a mobile phone, a tablet computer, etc. The method of the intelligent routing selection model in the embodiment of the present application can be executed by the server 103, and can also be executed by the server 103 and the terminal 101 together.

[0029] like Figure 2 As shown, the method for intelligent routing selection model includes the following steps:

[0030] Step S202: classify the deployed models based on the model parameter scale and model deployment method to determine the model categories corresponding to the models.

[0031] In this embodiment, the method of intelligent routing selection model is applicable to complex application scenarios that require the integration of multiple LLMs. The above-mentioned large model is the LLM model. The LLM model is a machine learning model with large-scale parameters and complex computing structure, which is designed to understand and generate human language. LLM models usually have billions or even hundreds of billions of parameters, which enable the model to handle more complex tasks and data. In addition, it also has a complex computing structure. The LLM model is usually constructed by deep neural networks. These network structures are complex and can capture deep information in the language. The method of intelligent routing selection model provided in this embodiment can be applied to medical, financial and other fields to enable users to access medical systems or financial systems through the user terminal, and select the optimal LLM model to respond according to user needs.

[0032] In this embodiment, after matching different model parameter scales and different model deployment methods, each model can be divided into multiple model categories. Among them, the model parameter scale can refer to the number of parameters possessed by the model, for example: the amount of parameter data is within 1 billion or above 10 billion. The model deployment method can refer to the location of model deployment and the way of providing services. The model deployment method can include shared LLM deployed in SaaS (Software as a Service) and exclusive LLM deployed in privatization. Among them, the shared LLM deployed in SaaS means that the trained LLM model and related API interface are provided to customers through cloud services. Customers do not need to build and maintain the underlying infrastructure of LLM by themselves, and can easily integrate LLM into their own applications through API interfaces. Exclusive LLM deployed in privatization means that LLM is deployed in the internal environment of an enterprise or organization, and is dedicated to providing services for internal users of the enterprise. It can ensure that enterprise data is completely isolated from internal users of the enterprise, avoid data leakage to external entities, and meet the high requirements of enterprises for data security and privacy protection; and for enterprises that need to comply with specific industry regulations or data protection regulations, privatization deployment can better meet compliance requirements.

[0033] Step S204: create an intelligent router, obtain a user task request through the intelligent router, extract multidimensional key features from the user task request, and generate a multidimensional key feature vector.

[0034] In this embodiment, the smart router (LLM-Router), also known as an intelligently managed router, is a network device with an independent operating system. As a core component, the smart router is responsible for intelligent scheduling and management of various LLMs. The smart router is equipped with an independent operating system, and users can perform various settings and management on the router like operating a smartphone or computer. Specifically, the smart router can be developed and created in combination with hardware selection, operating system installation, function configuration, application expansion, testing and optimization, and the smart router can be used as an intermediate layer between the client and various LLMs to realize data interaction and model selection between the user end and various large models in the background.

[0035] More specifically, the main functional modules of the intelligent router include: Task analysis module: responsible for receiving and analyzing user task requests and extracting key features. LLM evaluation engine: maintains and evaluates the LLM resource pool, and updates the multi-dimensional scores of each LLM in real time. Intelligent routing decision maker: uses algorithms to generate optimal routing decisions based on multi-dimensional key feature vectors and LLM evaluation results. Security control center: implements data security measures, manages access control and audit tracking. Task execution monitor: monitors the task execution process, collects performance data, and realizes dynamic optimization based on performance data. Result processor: integrates multiple LLM outputs and applies post-processing technology to improve output quality. Learning optimization unit: collects feedback and continuously optimizes routing algorithms and overall system performance. User interface module: provides a friendly configuration interface that allows users to set preferences and view performance reports.

[0036] Furthermore, when the intelligent router obtains the user task request, it can parse the user task request to obtain multidimensional key features. Among them, the multidimensional key features include but are not limited to the type of task, task complexity, data sensitivity, task field, corresponding time, data privacy requirements, etc. After the multidimensional key features are extracted, they can be encoded using a pre-trained natural language processing (NLP) model to generate a multidimensional key feature vector, wherein the multidimensional key feature vector can be a feature vector extracted from a certain layer in the model after encoding. Among them, the pre-trained NLP model may include BERT, GPT series, RoBERTa, ALBERT, ELECTRA, etc. After the encoding process is completed, you can extract feature vectors from a certain layer of the model. In this embodiment, in order to provide richer semantic information, the extracted multidimensional key feature vector is a high-dimensional feature vector, such as a user task for text classification or a question-and-answer system.

[0037] Step S206, performing a multi-dimensional resource pool evaluation on the large models under each model category through the intelligent router to obtain a first evaluation result of each large model, and constructing a multi-dimensional model capability vector according to the first evaluation result of each large model.

[0038] In this embodiment, in order to accurately match the user's tasks, the intelligent router can be used to perform resource pool evaluations in multiple dimensions on the classified large models. Among them, the resource pool can be understood as a resource collection that supports the operation and optimization of LLM. The resource pool evaluation of LLM can include evaluations of performance indicators, security levels, professional field capabilities, deployment methods, cost factors, compliance, etc. When evaluating, in order to facilitate data processing, each evaluation indicator can be quantified, and each indicator can be mapped to a corresponding indicator level. Different indicator levels correspond to different scores, thereby realizing indicator quantification, wherein the indicator level can be a national standard or an international standard, etc., and can also include a custom standard. After evaluating the resource pool of each large model, the evaluation results of each large model can be obtained. Based on the first evaluation result, a multidimensional model capability vector of each large model can be constructed. The multidimensional model capability vector can be a multidimensional capability matrix covering performance, security, professional fields, costs, etc.

[0039] In some examples, each evaluation indicator in the above-mentioned resource pool can be updated based on a dynamic scoring mechanism constructed based on historical evaluation data and real-time feedback data. The historical evaluation data may include evaluation data before the current iteration cycle, and the real-time feedback data may include evaluation data that has changed during the current iteration cycle and affects the evaluation indicator score.

[0040] Step S208, performing calculation based on the multidimensional key feature vector and the multidimensional model capability vector corresponding to each large model to determine a model candidate set, wherein there is at least one candidate large model in the model candidate set.

[0041] In this embodiment, in order to match the large model corresponding to the multidimensional key feature vector, the similarity between the multidimensional key feature vector and the multidimensional model capability vector of each large model can be calculated, and the candidate large model can be selected based on the degree of similarity, and all the candidate large models can be used as a set to construct a model candidate set. Among them, there may be only one candidate large model in the model candidate set, or there may be multiple candidate large models. When there is only one candidate large model, the candidate large model can be directly used as the target large model to respond to the user task request. Of course, if there are multiple candidate large models, it is necessary to select the target large model from the numerous candidate large models.

[0042] Step S210, if there are multiple candidate large models in the model candidate set, each of the candidate large models is scored with multi-dimensional elements to obtain a second evaluation result of each of the candidate large models, wherein each element matches an element weight, and the element weight is dynamically adjusted based on user needs.

[0043] In this embodiment, the candidate large models can be scored with multi-dimensional elements, and each element is equipped with an initial weight. In order to more accurately select the target large model and better meet user needs, the initial weight can be adjusted according to user needs to form new element weights for each element. When the user has higher requirements for data security, the weight corresponding to data security can be increased, and the weight of the element with smaller requirements can be appropriately reduced. Further, the re-scoring is performed based on the adjusted element weights and the first evaluation results of each element in the resource pool, so as to obtain the second evaluation results of each optimized candidate large model.

[0044] Step S212, constructing an objective function corresponding to the candidate large model, and selecting a target large model to respond to the request based on the objective function, preset decision constraints and the second evaluation results of each candidate large model.

[0045] In this embodiment, the above objective function can be created according to the goal that the user wants to achieve, such as maximizing performance indicators, minimizing costs, etc. For the objective function, decision constraints can also be preset. The decision constraints can screen out LLMs whose indicators are more in line with user needs, including constraining the response time of LLM in terms of performance; in terms of security, constraining the security level of LLM for highly sensitive tasks; in terms of cost, selecting a large model with lower cost while meeting performance constraints. For decision constraints, since the priorities of various constraints are different, the priority of decision constraints can be constructed, and high-level constraints are satisfied first, and then low-level constraints are considered. The higher the level, the more it matches the user's needs.

[0046] Furthermore, according to the objective function and the preset decision constraints, some candidate LLMs that do not meet the conditions can be screened out from the candidate LLMs, and the remaining candidate LLMs can be prioritized according to their respective second evaluation results, and finally the first-ranked candidate LLM can be selected as the target LLM for request response. As an example, multiple candidate LLMs ranked in the top N can also be selected as the target large model to respond to the request in a combined manner, where N is greater than 1. For example, if the user task request is highly complex, the task decomposition strategy is used to split it into multiple subtasks, and the target LLM is matched for each subtask, and the final presentation is a combination of LLMs for request response.

[0047] In an embodiment of the present invention, the application classifies each major model according to the model parameter scale and model deployment method, which is more conducive to more targeted model selection, faster and more accurate response, and different model deployment methods target different data sensitivities, which is also more conducive to meeting users' data sensitivity requirements and improving data security. By creating an intelligent router as an intermediate layer between the user end and each major model, a unified access interface is provided for large models of different sources and types, and the model selection and access method are more flexible, which facilitates the user's daily work. Through multi-dimensional evaluation, element weight adjustment, construction of objective functions and preset decision constraints, the most appropriate target large model is selected for each user task request, which is conducive to improving task execution efficiency and task response accuracy.

[0048] In an optional embodiment, in combination Figure 3 As shown, step S202 specifically includes:

[0049] Step S2021, obtaining the model parameter scale and the model deployment method of each large model;

[0050] Step S2022, judging the model parameter scale of each large model based on a preset first scale interval and a second scale interval to obtain a parameter scale judgment result, wherein the data processing complexity of the second scale interval is higher than that of the first scale interval;

[0051] Step S2023, judging the model deployment mode of each large model based on the preset first deployment mode and the second deployment mode, and obtaining a deployment mode judgment result, wherein the second deployment mode has a higher data sensitivity than the first deployment mode;

[0052] Step S2024, determining the model categories of each major model based on the parameter scale judgment results and the deployment method judgment results of each major model, wherein the model categories include one type of model that satisfies both the first scale interval and the first deployment method, two types of models that satisfies both the first scale interval and the second deployment method, three types of models that satisfies both the second scale interval and the first deployment method, and four types of models that satisfies both the second scale interval and the second deployment method.

[0053] In this embodiment, the model parameter scale and model deployment method of each LLM can be obtained first, and then the model parameter scale is compared with the first scale interval and the second scale interval, and the model deployment method is compared with the first deployment method and the second deployment method, so as to determine the model category of each LLM according to the parameter scale judgment results and the deployment method judgment results.

[0054] Furthermore, the above-mentioned first scale interval may refer to an LLM with a parameter scale within 1 billion, and the second scale interval may refer to an LLM with a parameter scale of more than 10 billion. Of course, it may also be other parameter scales, which can be specifically determined according to the deployed LLM. The above-mentioned first deployment method may refer to a shared LLM deployed in SaaS, and the second deployment method may refer to an exclusive LLM deployed in a private manner. The above-mentioned first type of model is a shared LLM with weak capabilities and SaaS deployment; the above-mentioned second type of model is an exclusive LLM with weak capabilities and privatization deployment; the above-mentioned third type of model is a shared LLM with strong capabilities and SaaS deployment; the above-mentioned fourth type of model is an exclusive LLM with strong capabilities and privatization deployment.

[0055] More specifically, the parameter scale of shared LLM with weak capabilities and SaaS deployment is small, usually <1 billion, and services are provided through cloud API (Application Programming Interface). It is suitable for simple query, basic text generation and other scenarios, and has the advantages of low cost and fast response. The parameter scale of exclusive LLM with weak capabilities and private deployment is small, and it can be deployed on local servers or edge devices. It is suitable for scenarios with simple tasks that require low latency or data localization. It has the advantages of high data security and no need for network connection. The parameter scale of shared LLM with strong capabilities and SaaS deployment is large, usually >10 billion, and services are provided through cloud API. It is suitable for scenarios such as complex question and answer, high-quality content generation, and multimodal tasks. It has the advantages of strong capabilities and can handle various complex tasks. The parameter scale of exclusive LLM with strong capabilities and private deployment is large, and it is deployed on local high-performance server clusters. It is suitable for scenarios that require strong capabilities and extremely high data security requirements. It has the advantages of fully controllable data and can be customized according to needs.

[0056] Combination Figure 3 As shown in the figure, due to different parameter scales and model deployment methods, the data sensitivity and data processing complexity are also different. Among them, the shared LLM with weak capabilities and SaaS deployment has the lowest data sensitivity and data processing complexity; the exclusive LLM with weak capabilities and private deployment has higher data sensitivity than the shared LLM with weak capabilities and SaaS deployment; the shared LLM with weak capabilities and SaaS deployment has higher data processing complexity than the shared LLM with weak capabilities and SaaS deployment; the exclusive LLM with strong capabilities and private deployment has the highest data processing complexity and data sensitivity.

[0057] In this embodiment, the large models are divided into four categories according to their model parameter scales and model deployment methods, which is conducive to selecting the most suitable LLM to respond to different user tasks. SaaS deployment allows organizations to quickly respond to changes and adjustments, with fast response and high flexibility; private deployment supports customized development and expansion, and the exclusive LLM of private deployment has higher security when processing sensitive or confidential data, reducing the risk of data leakage.

[0058] In an optional embodiment, the above step S204 specifically includes:

[0059] Step S2041, obtaining the user task request through the intelligent router;

[0060] Step S2042, performing task analysis on the user task request based on a natural language algorithm, extracting the multidimensional key features, and determining the task type, task complexity, task domain, expected output length, and response time according to the multidimensional key features;

[0061] Step S2043, generating the multi-dimensional key feature vector according to the task type, task complexity, task domain, expected output length and response time.

[0062] In this embodiment, the user can send a user task request through a client such as a computer or a smart phone. When the intelligent router receives the user task request, it can analyze the task through a pre-trained natural language processing model to identify the multi-dimensional key features contained in the task, including the task type, task complexity, task domain, expected output length and response time required in the task request, and can also include data sensitivity, model throughput, model response accuracy, etc. For example, the user task requires the model response time to be less than 5 seconds. Among them, the task types include text generation, question and answer, image analysis, sentiment analysis, etc.

[0063] Furthermore, for different types of key features, the pre-trained natural language processing model can be used for encoding to convert the text data into a format that the model can understand, where the encoding method may include string conversion, forced type conversion, regular expression, or encoding and decoding. During the encoding process, the pre-trained natural language processing model can pass the input key features to the embedding layer of the model, and the embedding layer converts the text into a high-dimensional vector. During the encoding process, an attention mechanism can also be added to allow the model to consider contextual information when processing text, which is conducive to more accurate encoding. After the encoding is completed, the output of the top layer of the model can be extracted and used as the key feature vector. Based on all the key feature vectors, the above-mentioned multi-dimensional key feature vectors are further generated, and these key feature vectors are high-dimensional vectors. The multi-dimensional key feature vector can be expressed as A=[A1,A2,A3,A4,A5].

[0064] In this embodiment, multidimensional key features such as task type, task complexity, task field, expected output length and response time in user task requests are comprehensively considered, and a multidimensional key feature vector is constructed through multidimensional key features. This helps the system to match user requests more accurately, thereby adopting better processing strategies, helping to reduce misunderstandings and error processing, and improving the accuracy of task completion. Based on multidimensional restrictions, the system can more reasonably allocate computing resources and time to ensure overall response efficiency.

[0065] In an optional embodiment, the above step S206 specifically includes:

[0066] Step S2061, obtaining resource pool data of each major model through the intelligent router, wherein the resource pool data includes performance indicator data, security level data, domain performance capability data, deployment method data, cost factor data and compliance data;

[0067] Step S2062, performing resource evaluation based on the performance indicator data, security level data, domain performance capability data, deployment method data, cost factor data, and compliance data of the large model to obtain the first evaluation result of each large model;

[0068] Step S2063: construct the multi-dimensional model capability vector according to the initial weight corresponding to each resource pool data and the first evaluation result.

[0069] In this embodiment, each LLM has a corresponding resource pool, and the resource pool data can be extracted from the resource pool through the intelligent router that manages the LLM. Among them, the extracted resource pool data includes performance indicator data, security level data, domain performance capability data, deployment method data, cost factor data and compliance data. Among them, the performance indicator data may include the response time, throughput, accuracy, etc. of the LLM. Security level data may refer to data encryption capabilities, privacy protection capabilities, etc. Domain performance capability data may refer to performance in specific fields, such as law, medical care, finance, etc. Deployment method data includes cloud API, edge computing, local deployment, etc. Cost factor data includes usage cost, resource consumption cost, etc. Compliance data may refer to whether it complies with regulatory requirements in different regions and industries.

[0070] More specifically, for resource evaluation, each acquired data can be quantified and converted into a corresponding score. Based on the score of each resource pool data of LLM, the first evaluation result of each LLM can be determined. Among them, each resource pool data can be used as a first-level indicator. When the first-level indicator includes multiple second-level indicators, the second-level indicators can also be quantified, and the weights can be matched for the second-level indicators. The score of the first-level indicator is calculated based on the evaluation score and weight of the second-level indicator. For example, the performance indicator data includes three second-level indicators of LLM: response time, throughput and accuracy. The evaluation scores of each are 80, 50, and 90, respectively, and the weights are 30%, 10%, and 60%. The final evaluation score of the performance indicator data is 83 points.

[0071] More specifically, after calculating the first evaluation result of the LLM resource pool data, the final evaluation score can be calculated based on the initial weights of each resource pool data included therein and the first evaluation result, and a multidimensional model capability vector can be constructed based on the final score valuation of each resource pool data of the same LLM. The multidimensional model capability vector can be expressed as B=[B1, B2, B3, B4, B5]. For example: in the first evaluation result of LLM1, the performance index data is 90 points, the security level data is 85 points, the domain performance capability data is 80 points, the cost factor data is 70 points, the deployment method data is 95 points, and the compliance data is 85 points. The corresponding initial weights are 25%, 20%, 20%, 15%, 10%, and 10%, respectively. The final score estimates are 22.5, 21.25, 16, 10.5, 9.5, and 8.5, respectively. The multidimensional model capability vector for constructing LLM1 is B=[22.5,21.25,16,10.5,9.5,8.5], and the comprehensive model evaluation score of LLM1 is 88.25 points.

[0072] In some examples, during resource evaluation, not only static data can be evaluated, such as the inherent capabilities of the model, including deployment methods, reliability, throughput, etc., but also dynamic data can be combined for evaluation, such as real-time load, historical performance, etc. The combination of static and dynamic methods can combine the real-time changes of the model, better match the current status, and make the evaluation results more accurate.

[0073] In this embodiment, by comprehensively considering multiple factors such as performance, security, professional field, cost, deployment method, compliance, etc., and matching weights for each element, the current resources of each LLM can be estimated more accurately, which is conducive to accurate matching with user needs. In addition, a multi-dimensional model capability vector of each LLM is constructed to facilitate subsequent similarity calculation based on the multi-dimensional key feature vector corresponding to user needs, so as to accurately select candidate large models.

[0074] In an optional embodiment, the above step S208 specifically includes:

[0075] Step S2081, calculating the similarity between the multidimensional key feature vector and each of the multidimensional model capability vectors;

[0076] Step S2082, when the similarity satisfies the similarity threshold, the large model corresponding to the multidimensional model capability vector whose similarity satisfies the similarity threshold is selected into the model candidate set as the candidate large model, until all large models are screened.

[0077] In this embodiment, the method for calculating similarity may include cosine similarity, Euclidean distance, Chebyshev distance, Manhattan distance, etc. In this embodiment, the cosine similarity between the multidimensional key feature vector and each multidimensional model capability vector can be calculated based on the cosine similarity formula, and finally multiple similarity results are obtained, which can be expressed as d1, d2, d3, ..., d n A similarity threshold may be preset, and then each calculated similarity is compared with the similarity threshold, and the large model seat candidate models whose similarities are greater than or equal to the multidimensional model capability vector corresponding to the similarity threshold in all comparison results are added to the model candidate set.

[0078] In this embodiment, the method of screening candidate large models through similarity calculation based on intelligent routers not only adapts to the needs of different fields and tasks, but also realizes heterogeneous LLM integration, and provides a unified access interface for LLMs of different sources and types. It can also quantify the similarity between the model and the user's task, so as to select the model that best meets the needs, reduce the workload of subsequent evaluation and optimization, and provide a clear quantitative basis for model selection, making the selection process more transparent and explanatory, which is helpful for subsequent model optimization to have data support.

[0079] In an optional embodiment, the above step S210 specifically includes:

[0080] Step S2101, dynamically adjusting the weight of each dimensional element based on user needs to obtain the element weight of each element, wherein the user needs include data sensitivity;

[0081] Step S2102: Calculate the second evaluation result of each candidate large model according to the adjusted element weights of the candidate large model and the first evaluation result.

[0082] In this embodiment, when there are multiple candidate large models, it is necessary to select a target large model from the multiple candidate large models, and the candidate large models can be scored for multi-dimensional elements. Among them, the way to adjust the weights of the elements of each dimension based on user needs includes but is not limited to user customization, task adaptation or machine learning adjustment. Among them, user customization can refer to allowing weights to be adjusted through the interface to meet specific needs. Task adaptation can refer to automatically fine-tuning weights according to task characteristics, for example: for tasks with high security requirements, the weight of security elements is increased, and if fast response time is required, the weight of performance indicators is increased. In this embodiment, considering the application to scenarios such as finance and medical care, there will be higher requirements for data protection and sensitivity. Therefore, the above-mentioned user needs include data sensitivity. The above-mentioned machine learning adjustment can refer to the use of machine learning algorithms to optimize weight distribution based on historical task execution data. Machine learning algorithms can include linear regression, logistic regression, neural networks, etc.

[0083] Furthermore, by redistributing the weights of the elements in each dimension, the evaluation results of the model will eventually be affected. Therefore, the second evaluation results of the candidate large model can be recalculated by combining the adjusted element weights and the scores of each element in the first evaluation results corresponding to each element weight. The formula for calculating the second evaluation result for each candidate LLM is: second evaluation result = Σ(element score of each dimension × dynamically adjusted element weight). For a clearer explanation, the following example is shown: the first evaluation results of a candidate LLM in each dimension are: performance 90 points, security 85 points, professional ability 80 points, cost 70 points, deployment method 95 points, and compliance 85 points. Based on the above initial weights, the dynamically adjusted element weights correspond to: 30%, 25%, 15%, 10%, 10%, and 10%, respectively, and the second evaluation result is 86.25 points.

[0084] In this embodiment, the weight is dynamically adjusted according to different task requirements, and the second evaluation result is calculated in combination with the first evaluation result, so as to realize dynamic mechanism adjustment, screen out models that meet different user requirements, and improve the accuracy of model selection and model response efficiency. At the same time, the data sensitivity factor is taken into account, the weight of security elements is increased, and strict security control is implemented for sensitive tasks, which effectively reduces the risk of data leakage, protects user privacy, and improves the privacy and security of data.

[0085] In an optional embodiment, in combination Figure 4 As shown, the above step S212 specifically includes:

[0086] Step S2121, constructing the objective function by maximizing the performance index and minimizing the cost, and screening the candidate large models based on the preset decision constraints, wherein the preset decision constraints include at least one of a response time constraint, a data sensitivity constraint, and a cost constraint;

[0087] Step S2122: Prioritize the candidate large models according to the second evaluation results, and select a target large model to respond to the request according to the priority ranking results.

[0088] In this embodiment, the LLM selection problem can be modeled as a multi-objective optimization problem. In order to improve the data processing capability of the model while controlling the cost, the objective function is constructed by maximizing the performance index and minimizing the cost, while satisfying the preset decision constraints. The preset decision constraints may include constraints set based on user needs, and may also include some basic constraints. Among them, the preset decision constraints may include at least one of response time constraints, data sensitivity constraints, and cost constraints. Data sensitivity constraints may refer to security and compliance constraints, among which response time constraints and data sensitivity constraints are constraints set based on user needs, and cost constraints are basic constraints. Of course, other constraints may also be included, such as deployment method constraints, response accuracy constraints, etc.

[0089] Furthermore, for response time constraints, a response time threshold can be set to exclude candidate LLMs whose response time exceeds the response time threshold. When facing highly sensitive tasks, a data sensitivity constraint can set a specific security level threshold to exclude candidate LLMs that do not meet the specific security level threshold. For cost constraints, on the premise of meeting the response time constraint and / or data sensitivity constraint, candidate LLMs with lower costs are given priority. After excluding the candidate LMMs that do not meet the constraints, the remaining candidate LLMs are sorted based on their respective second evaluation results, and a target LLM is selected from them to request a response. When faced with multiple subtasks split from the same task request, multiple target LLMs can be selected to respond to different subtasks in a combined manner, and the most appropriate target LLM is matched for each subtask to improve the data processing capability and efficiency of the model.

[0090] In this embodiment, by modeling the LLM selection problem as a multi-objective optimization problem, constructing objective functions and constraints to screen multiple candidate LLMs, it is possible to accurately match the needs, improve screening efficiency, and ensure the rational use of resources. Converting to a multi-objective optimization problem can comprehensively consider the trade-offs between performance and resource consumption (cost) of different candidate LLMs, thereby selecting the target LLM with the highest cost-effectiveness, and making the decision-making process more scientific and objective. By providing multiple LLMs in the system and using intelligent routers to select target LLMs in combination with user needs, not only can diversified model selection be achieved, but also the model selection flexibility and adaptability are higher.

[0091] Optionally, in this embodiment, the intelligent router may also include a security control center for implementing data security measures, managing access control and audit tracking; a task execution monitor for monitoring the task execution process, collecting performance data, and achieving dynamic optimization; a result processor for integrating multiple LLM outputs and applying post-processing technology to improve output quality; a learning optimization unit for collecting feedback and continuously optimizing routing algorithms and overall system performance; and a user interface module for providing a friendly configuration interface that allows users to set preferences and view performance reports.

[0092] Furthermore, during the user task response and model selection process, the security control center can track data security management throughout the process, and the task execution monitor can also collect performance data in real time to provide data support for subsequent system updates and iterations. The intelligent router can also generate a detailed routing decision report, including the selected target LLM or target LLM combination, scoring basis, expected performance, etc., to provide clear call instructions and parameter configuration for the task execution module. For task execution and dynamic optimization, tasks can be distributed to the selected target LLM or target LLM combination, the task execution process can be monitored in real time, performance indicators and result quality data can be collected, and routing strategies can be dynamically adjusted according to the execution results to achieve adaptive optimization. The result processor can intelligently integrate and optimize the output results of multiple LLMs, and apply post-processing technology to improve output quality, such as error correction, formatting, content filtering, etc. The learning optimization unit can collect user feedback and task completion quality evaluation, use machine learning technology to continuously optimize the routing algorithm and scoring mechanism, regularly update the LLM resource pool, evaluate the performance and applicability of the newly added LLM, and improve long-term performance and reliability.

[0093] Optionally, the intelligent router can also perform load balancing and resource scheduling. By real-time monitoring of the load of each LLM, it can perform intelligent load balancing between multiple LLMs with similar / close scores. In addition, when multiple user access requests are concurrent, the intelligent router can filter time-sensitive tasks from the queue, consider the current queue situation, and give priority to the target LLM with faster response speed to respond, which is conducive to maintaining stability and efficiency when handling urgent tasks.

[0094] Optionally, in this embodiment, for highly sensitive tasks, user requirements include data sensitivity requirements, and strict security control measures can be implemented for highly sensitive tasks. For highly sensitive tasks, the intelligent router can use regular expressions for matching and identification, which is fast and efficient. For example: use predefined regular expression patterns to identify common sensitive information formats, including ID numbers, credit card numbers, telephone numbers, etc. Of course, the intelligent router can also match and identify through keyword and phrase detection algorithms, for example: maintain a sensitive vocabulary library, including industry-specific terms, confidential project codes, etc., and use efficient string matching algorithms for rapid retrieval, such as the Aho-Corasick algorithm. Highly sensitive tasks can also be identified through machine learning models, and sensitive content can be identified by training a special text classification model. Deep learning models can be used for more complex semantic understanding and sensitive information identification, such as BERT, XLNet, etc. In addition, methods for identifying highly sensitive tasks may also include but are not limited to named entity recognition (NER), context analysis, metadata analysis, rule engines, sensitivity scoring mechanisms, etc., which are not listed one by one here. Among them, the definition of sensitive words and the degree of sensitivity can be defined in normative documents in combination with the industry in which they are located.

[0095] Optionally, in order to improve the recognition accuracy of sensitive words, recognition can be done not only based on artificial intelligence, but also with human participation. The human participation process can include defining sensitive information through domain experts, setting initial rules and thresholds for sensitive words; requiring human judgment and decision-making for boundary situations that the system cannot determine; constantly adjusting and optimizing recognition rules and algorithms based on actual usage, and timely updating of newly emerging sensitive words; in highly regulated industries, human confirmation is required to determine whether the processing of sensitive information complies with regulatory requirements, etc. There are many other processes of human participation, which are not listed here one by one.

[0096] According to another aspect of the embodiment of the present application, Figure 5 As shown, a device for intelligent routing selection model is provided, including:

[0097] The model classification module 501 is used to classify the deployed models based on the model parameter scale and model deployment mode, and determine the model category corresponding to each model;

[0098] A feature extraction module 503 is used to create an intelligent router, obtain a user task request through the intelligent router, and extract multidimensional key features from the user task request to generate a multidimensional key feature vector;

[0099] A first evaluation module 505 is used to perform a multi-dimensional resource pool evaluation on the large models under each model category through the intelligent router to obtain a first evaluation result of each large model, and construct a multi-dimensional model capability vector according to the first evaluation result of each large model;

[0100] A calculation module 507, configured to calculate based on the multidimensional key feature vector and the multidimensional model capability vector corresponding to each large model to determine a model candidate set, wherein the model candidate set includes at least one candidate large model;

[0101] The second evaluation module 509 is used for, if there are multiple candidate large models in the model candidate set, to perform multi-dimensional element scoring on each of the candidate large models to obtain a second evaluation result of each of the candidate large models, wherein each element is matched with an element weight, and the element weight is dynamically adjusted based on user needs;

[0102] The selection module 511 is used to construct an objective function corresponding to the candidate large model, and select a target large model to respond to the request based on the objective function, preset decision constraints and the second evaluation results of each candidate large model.

[0103] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiments. It should be noted that the above modules as part of the device can be run in Figure 1 In the hardware environment shown, it can be implemented by software or by hardware.

[0104] Optionally, the model classification module is specifically used to: obtain the model parameter scale and the model deployment method of each large model; judge the model parameter scale of each large model based on a preset first scale interval and a second scale interval to obtain a parameter scale judgment result; judge the model deployment method of each large model based on a preset first deployment method and a second deployment method to obtain a deployment method judgment result; determine the model category of each large model according to the parameter scale judgment results and the deployment method judgment results of each large model.

[0105] Optionally, the feature extraction module is specifically used to: obtain the user task request through the intelligent router; perform task analysis on the user task request based on a natural language algorithm, extract the multidimensional key features, and determine the task type, task complexity, task domain, expected output length and response time according to the multidimensional key features; generate the multidimensional key feature vector according to the task type, task complexity, task domain, expected output length and response time.

[0106] Optionally, the first evaluation module is specifically used to: obtain resource pool data of each large model through the intelligent router, the resource pool data including performance indicator data, security level data, domain performance capability data, deployment method data, cost factor data and compliance data; perform resource evaluation based on the performance indicator data, security level data, domain performance capability data, deployment method data, cost factor data and compliance data of the large model to obtain the first evaluation result of each large model; construct the multidimensional model capability vector according to the initial weight corresponding to each resource pool data and the first evaluation result.

[0107] Optionally, the calculation module is also specifically used to: calculate the similarity between the multidimensional key feature vector and each of the multidimensional model capability vectors; when the similarity satisfies a similarity threshold, selecting the large model corresponding to the multidimensional model capability vector whose similarity satisfies the similarity threshold into the model candidate set as the candidate large model, until all large models are screened.

[0108] Optionally, the second evaluation module is specifically used to: dynamically adjust the weights of the elements of each dimension based on user needs to obtain the element weight of each element, and the user needs include data sensitivity; calculate the second evaluation results of each candidate large model according to the adjusted element weights of the candidate large model and the first evaluation results.

[0109] Optionally, the selection module is specifically used to: construct the objective function by maximizing performance indicators and minimizing costs, screen the candidate large models based on the preset decision constraints, and the preset decision constraints include at least one of response time constraints, data sensitivity constraints and cost constraints; prioritize the candidate large models according to the second evaluation results, and select the target large model for request response according to the priority ranking results.

[0110] According to another aspect of the embodiments of the present application, the present application provides an electronic device, such as Figure 6 As shown, it includes a memory 601, a processor 603, a communication interface 605 and a communication bus 607. The memory 601 stores a computer program that can be run on the processor 603. The memory 601 and the processor 603 communicate through the communication interface 605 and the communication bus 607. When the processor 603 executes the computer program, the steps of the above method are implemented.

[0111] The memory and processor in the above electronic device communicate via a communication bus and a communication interface. The communication bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus may be divided into an address bus, a data bus, a control bus, etc.

[0112] The memory may include random access memory, non-volatile memory, storage device located away from the aforementioned processor, etc. The processor may be a general-purpose processor, including a central processing unit, a network processor, etc.; it may also be a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.

[0113] According to another aspect of the embodiment of the present application, a computer program product or computer program is also provided, the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any of the above embodiments. Optionally, in an embodiment of the present application, the computer-readable medium is configured to store program codes for steps S202 to S212 in the method for executing the intelligent routing selection model by the processor.

[0114] Optionally, the specific examples in this embodiment can refer to the examples described in the above embodiments, and this embodiment will not be repeated here. And when the embodiments of this application are specifically implemented, they can refer to the above embodiments and have corresponding technical effects.

[0115] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits, digital signal processors, digital signal processing devices, programmable logic devices, field programmable gate arrays, general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or a combination thereof. For software implementation, the techniques described herein may be implemented by units that perform the functions described herein.

[0116] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0117] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0118] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0119] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) perform all or part of the steps of the method described in each embodiment of the present application. It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish an entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only include those elements, but also include other elements that are not clearly listed.

[0120] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A method for selecting an intelligent routing model, characterized in that: The method comprises the steps of: Classify the deployed models based on the model parameter scale and model deployment method, and determine the model categories corresponding to each model; Creating an intelligent router, obtaining a user task request through the intelligent router, and extracting multidimensional key features from the user task request to generate a multidimensional key feature vector; Performing a multi-dimensional resource pool evaluation on the large models under each model category through the intelligent router to obtain a first evaluation result of each large model, and constructing a multi-dimensional model capability vector according to the first evaluation result of each large model; Calculating based on the multidimensional key feature vector and the multidimensional model capability vector corresponding to each large model, determining a model candidate set, wherein the model candidate set includes at least one candidate large model; If there are multiple candidate large models in the model candidate set, multi-dimensional element scoring is performed on each of the candidate large models to obtain a second evaluation result of each of the candidate large models, wherein each element is matched with an element weight, and the element weight is dynamically adjusted based on user needs; Constructing an objective function corresponding to the candidate large model, and selecting a target large model to respond to the request based on the objective function, the preset decision constraints and the second evaluation results of each of the candidate large models; The step of obtaining a user task request through the intelligent router and extracting multidimensional key features from the user task request to generate a multidimensional key feature vector includes: obtaining the user task request through the intelligent router; performing task analysis on the user task request based on a natural language algorithm to extract the multidimensional key features, and determining a task type, task complexity, task domain, expected output length, and response time according to the multidimensional key features; and generating the multidimensional key feature vector according to the task type, task complexity, task domain, expected output length, and response time; The intelligent router is used to perform a multi-dimensional resource pool evaluation on the large models under each model category to obtain a first evaluation result of each large model, and a multi-dimensional model capability vector is constructed according to the first evaluation result of each large model, including: obtaining resource pool data of each large model through the intelligent router, the resource pool data including performance indicator data, security level data, domain performance capability data, deployment method data, cost factor data and compliance data; performing resource evaluation based on the performance indicator data, security level data, domain performance capability data, deployment method data, cost factor data and compliance data of the large model to obtain the first evaluation result of each large model; and constructing the multi-dimensional model capability vector according to the initial weight corresponding to each resource pool data and the first evaluation result.

2. The method according to claim 1, characterized in that The model classification of the deployed models based on the model parameter scale and model deployment mode to determine the model categories corresponding to the models includes: Obtain the model parameter scale and the model deployment method of each large model; The model parameter scale of each large model is judged based on a preset first scale interval and a second scale interval to obtain a parameter scale judgment result, wherein the data processing complexity of the second scale interval is higher than that of the first scale interval; The model deployment mode of each large model is judged based on the preset first deployment mode and the second deployment mode to obtain a deployment mode judgment result, wherein the second deployment mode has a higher data sensitivity than the first deployment mode; The model categories of each major model are determined according to the parameter scale judgment results and the deployment method judgment results of each major model, wherein the model categories include one type of model that satisfies both the first scale interval and the first deployment method, two types of models that satisfies both the first scale interval and the second deployment method, three types of models that satisfies both the second scale interval and the first deployment method, and four types of models that satisfies both the second scale interval and the second deployment method.

3. The method according to claim 1, characterized in that: The step of calculating based on the multidimensional key feature vector and the multidimensional model capability vector corresponding to each large model to determine a model candidate set includes: Calculating the similarity between the multidimensional key feature vector and each of the multidimensional model capability vectors; When the similarity satisfies a similarity threshold, the large model corresponding to the multidimensional model capability vector whose similarity satisfies the similarity threshold is selected into the model candidate set as the candidate large model until all large models are screened.

4. The method according to claim 1, characterized in that The step of performing multi-dimensional element scoring on each of the candidate large models to obtain a second evaluation result of each of the candidate large models includes: Dynamically adjust the weight of each dimension element based on user needs to obtain the element weight of each element, wherein the user needs include data sensitivity; A second evaluation result of each of the candidate large models is calculated according to the adjusted element weights of the candidate large models and the first evaluation result.

5. The method according to any one of claims 1 to 4, characterized in that The constructing of the objective function corresponding to the candidate large model, and selecting the target large model to respond to the request based on the objective function, the preset decision constraint condition and the second evaluation result of each candidate large model, includes: The objective function is constructed by maximizing the performance index and minimizing the cost, and the candidate large model is screened based on the preset decision constraint condition, wherein the preset decision constraint condition includes at least one of a response time constraint, a data sensitivity constraint, and a cost constraint; Priority sorting is performed according to the second evaluation results of the candidate large models, and a target large model is selected to respond to the request according to the priority sorting results.

6. A device for intelligent routing selection model, characterized in that: include: The model classification module is used to classify the deployed models based on the model parameter scale and model deployment method, and determine the model category corresponding to each model; A feature extraction module is used to create an intelligent router, obtain a user task request through the intelligent router, and extract multidimensional key features from the user task request to generate a multidimensional key feature vector; A first evaluation module, configured to perform a multi-dimensional resource pool evaluation on the large models under each model category through the intelligent router, obtain a first evaluation result of each large model, and construct a multi-dimensional model capability vector according to the first evaluation result of each large model; A calculation module, configured to calculate based on the multidimensional key feature vector and the multidimensional model capability vector corresponding to each large model to determine a model candidate set, wherein the model candidate set includes at least one candidate large model; A second evaluation module is used for, if there are multiple candidate large models in the model candidate set, to perform multi-dimensional element scoring on each of the candidate large models to obtain a second evaluation result of each of the candidate large models, wherein each element is matched with an element weight, and the element weight is dynamically adjusted based on user needs; A selection module, used to construct an objective function corresponding to the candidate large model, and select a target large model to respond to the request based on the objective function, preset decision constraints and the second evaluation results of each candidate large model; The feature extraction module is specifically used to: obtain the user task request through the intelligent router; perform task analysis on the user task request based on a natural language algorithm, extract the multidimensional key features, and determine the task type, task complexity, task domain, expected output length and response time according to the multidimensional key features; generate the multidimensional key feature vector according to the task type, task complexity, task domain, expected output length and response time; The first evaluation module is specifically used to: obtain resource pool data of each large model through the intelligent router, the resource pool data including performance indicator data, security level data, domain performance capability data, deployment method data, cost factor data and compliance data; perform resource evaluation based on the performance indicator data, security level data, domain performance capability data, deployment method data, cost factor data and compliance data of the large model to obtain the first evaluation result of each large model; construct the multidimensional model capability vector according to the initial weight corresponding to each resource pool data and the first evaluation result.

7. An electronic device, comprising a memory, a processor, a communication interface and a communication bus, wherein the memory stores a computer program that can be run on the processor, and the memory and the processor communicate through the communication bus and the communication interface, characterized in that: When the processor executes the computer program, the method of the intelligent routing selection model described in any one of claims 1 to 5 is implemented.

8. A computer-readable medium having a non-volatile program code executable by a processor, characterized in that: The program code enables the processor to execute the method of the intelligent routing selection model described in any one of claims 1 to 5.

Images