Large language model dynamic routing method and device based on context learning model representation, and readable storage medium

Abstract

The invention relates to a large language model dynamic routing method and device based on context learning model characterization and a readable storage medium. Query is embedded and mapped to a language model input space by using a projection model, semantic alignment is realized, a representative evaluation set covering multi-dimensional capability is automatically screened from a benchmark question bank, and the representative evaluation set is used for evaluating the multi-dimensional capability. Performance characteristics of the model on an evaluation set are efficiently obtained at a time, and high-quality context model capability representation is formed; then real-time query embedding and context model capability representation are combined, a lightweight routing language model is used for supervised learning, so that the fine-grained model distinguishing capability is achieved, an increment embedding updating mechanism is designed, and when a new model is accessed or an old model is upgraded, cold start can be rapidly completed only through a very small number of fixed questions, so that the efficiency is improved. The calculation and maintenance cost is greatly reduced, and the accuracy, real-time performance and flexible expansibility of model routing are effectively improved.

Description

Technical Field

[0001] The present invention relates to the fields of computer technology and natural language processing technology, and in particular to a large language model dynamic routing method and device based on context learning model representation, and a readable storage medium. Background Art

[0002] With the rapid expansion of the large model ecosystem, the collaborative reasoning paradigm of "model pool + router" has become a key technical approach for reducing inference costs and improving answer quality. The system first constructs a capability representation for each candidate model, and the router then instantly selects the best model based on the user query. Current large language model routing methods are centered around a three-step process: "query → router → optimal model." By constructing capability representations for the model pool and training (or designing) a query-aware selector, they significantly improve overall performance while keeping the cost of a single inference under control.

[0003] Among the existing works, the LLM-Blender integration framework combines pairwise comparison scoring with Top-k fusion, setting a precedent for promoting online routing with offline capability evaluation. However, the pre-computation cost increases quadratically with the size of the model pool because of the need to compare all models pairwise for each sample. The ZOOTER framework obtains labels that approximate human preferences through reward model distillation, and then trains lightweight routers to achieve millisecond-level decision-making, significantly reducing the inference latency. However, its upstream reward model still relies on a large amount of high-quality preference data, which is prone to performance degradation during domain migration. The RouterDC routing framework further introduces "double contrastive learning" and aligns the "sample-model" and "sample-sample" representations at the same time. It maintains a more stable discrimination on off-line data, but this positive-negative pair construction also requires a large number of offline inference results, and the cold start pressure remains; the EmbedLLM framework compresses each model into a compact vector and concatenates it with the query vector to predict the accuracy. It has fewer parameters and the vectors can be directly stored incrementally, but the benchmark question set must be run for the full pool model in advance to obtain high-quality embeddings; the ModelSAT framework attempts to get rid of pure vector representation. It first selects a batch of representative questions to obtain model output, and then automatically generates natural language "ability instructions" covering multi-dimensional capabilities for micro-large language model routing retrieval, which improves interpretability. However, it is still difficult to achieve accurate differentiation in segmented skills due to the limitation of the instruction template constructed by the fixed data set.

[0004] In summary, current routing systems typically rely solely on embedding vectors generated by user queries to represent models. However, queries often lack a comprehensive description of the model's output behavior, resulting in model embeddings that are high-dimensional but information-sparse, making it difficult to reflect performance differences between models. Therefore, model selection can only be performed at a coarse-grained level. Furthermore, when new models are added or existing model versions are iterated, the industry generally uses re-inference on large, complete query sets to generate embeddings. Since query sets often range from tens of thousands to hundreds of thousands of items, this process not only consumes a large amount of computing resources but also significantly prolongs cold start times and iteration cycles, hindering the flexible expansion of the model pool and increasing operation and maintenance costs. Summary of the Invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a large language model dynamic routing method and device based on context learning model representation, and a readable storage medium, so as to solve or partially solve the problem of low efficiency and large computing resource occupation in routing user queries to adapted candidate models in scenarios where multiple candidate models are deployed.

[0006] The purpose of the present invention can be achieved by the following technical solutions: One aspect of the present invention provides a large language model dynamic routing method based on context learning model representation, comprising the following steps: Query reconstruction pre-training: Use the projection model to map the input query embedding vector to a high-dimensional semantic space, input the mapped embedding vector into the model router, and train the projection model and the model router with the goal of restoring the user query embedding vector. Model representation routing training based on contextual learning: Based on a pre-built representative evaluation set with multiple difficulty levels, the trained projection model is used to inject query vectors from the representative evaluation set into multiple candidate models. For each candidate model, the true and false labels under each query vector are aggregated to obtain a contextual model capability representation. This contextual model capability representation and user queries are used as input, and the true and false labels corresponding to the queries are used as supervision signals to train a model router based on a lightweight language model. Online inference phase: The user query and the context model capability representation of each candidate model are used as input to the trained model router to obtain the adaptation score of each candidate model. Based on the adaptation score, dynamic routing of the large language model is implemented.

[0007] As a preferred technical solution, the process of constructing the representative evaluation set includes the following steps: According to the accuracy rate of the questions, we divide the difficulty range into several equal width intervals from extremely difficult to highly divergent, and select an equal number of questions from each interval to construct a representative assessment set covering different difficulty levels.

[0008] As a preferred technical solution, it also includes: Incremental update process: In response to the access of new candidate models or the upgrade of existing candidate models, the new candidate models and / or upgraded candidate models are reasoned on the representative evaluation set to obtain the corresponding context model capability representation.

[0009] As a preferred technical solution, based on average pooling, maximum pooling, weighted pooling or attention mechanism, the correct and incorrect labels and computational cost penalty items under each query are aggregated to obtain the context model capability representation.

[0010] As a preferred technical solution, in the process of constructing the representative evaluation set, questions are screened based on the difficulty and information entropy of the questions, as well as the difference indicators between candidate models, so as to achieve the characterization of the model capabilities by the questions.

[0011] As a preferred technical solution, the projection model realizes semantic alignment of query embedding and model input space through a linear or nonlinear neural network, and the model router realizes training based on contrastive learning or reinforcement learning.

[0012] As an optimal technical solution, during the model representation routing training process, the projection model and the model router are optimized synchronously by minimizing the reconstruction loss, thereby achieving semantic alignment between the projection model and the model router.

[0013] As a preferred technical solution, the high-dimensional semantic space is the same as the native input of the candidate model.

[0014] Another aspect of the present invention provides an electronic device comprising: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the aforementioned large language model dynamic routing method based on context learning model representation.

[0015] Another aspect of the present invention provides a computer-readable storage medium comprising one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the aforementioned large language model dynamic routing method based on context learning model representation.

[0016] Compared with the prior art, the present invention has at least one of the following beneficial effects: (1) Achieve efficient routing of user queries under multiple candidate models: The present invention uses a projection model to map query embeddings to a semantic space consistent with the language model input; then, by allowing the language model to reconstruct the query embedding, the query semantics and the model representation space are aligned. Subsequently, a small number of difficult questions covering multi-dimensional capabilities are automatically screened from the benchmark question bank, and the answering performance of each candidate model is collected at one time on a representative evaluation set. The performance of the model on the representative evaluation set is compressed into a context model capability representation, and supervised learning is performed with the corresponding user query input lightweight routing language model, so that the router can simultaneously grasp two types of information: query semantics and model performance, thereby obtaining fine-grained differentiation capabilities. During the reasoning process, only one query encoding and one small language model inference are required to complete model selection, ensuring a fast response. The present invention maintains real-time performance while improving routing accuracy, shortening the deployment cycle, and can still stably differentiate model capabilities in cross-domain scenarios, providing a scalable, low-overhead dynamic routing solution for large-scale model pools.

[0017] (2) Low cold start and iteration costs: When a new model is connected or an old model is upgraded, when the model pool is expanded or when the old model is iterated, the present invention only needs to perform one inference on a small number of evaluation questions of a fixed scale to quickly generate a new capability representation. There is no need to re-perform full-scale inference and retrain the model router, which shortens the model cold start time and maintenance iteration cycle, and significantly reduces operation and maintenance and deployment costs.

[0018] (3) Low cost for constructing model capability representations: This invention automatically selects a small number of difficult questions from a benchmark question bank that cover different difficulty levels and fully reflect differences in model performance, forming an efficient evaluation set. This significantly reduces the cost of acquiring capability representations and the cost of constructing model capability representations. Compared to existing methods that require offline reasoning or pairwise comparisons on a large number of benchmark queries, this invention only requires a small number of representative questions to complete the generation of model capability representations, effectively reducing computing resource requirements and time consumption.

[0019] (4) Improving the fine-grained recognition capability of routing decisions: The present invention uses a projection model to map the user query embedding vector to the input semantic space of the language model, and reconstructs the query embedding through the language model to achieve precise alignment of the two in the semantic space. This avoids the problem of high query embedding dimension but sparse information in the existing technology, making the model capability representation more accurate and compact, thereby significantly improving the fine-grained recognition capability of routing decisions.

[0020] (5) Strong robustness in domain transfer: The present invention compresses the performance of the candidate model on the representative evaluation set into a contextual capability representation, and inputs it together with the query vector into a lightweight routing language model for supervised learning, thereby achieving fine-grained differentiation of model performance and improving the robustness of domain transfer. It can more stably maintain the effect of distinguishing model performance differences when applied across scenarios, and enhance the adaptability of the system in actual complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 Flowchart of a large language model dynamic routing method based on contextual learning model representation in an embodiment; Figure 2 A schematic diagram of the query reconstruction pre-training stage in an embodiment; Figure 3 Schematic diagram of a model representation routing training phase based on context learning in an embodiment; Figure 4 Schematic diagram of an electronic device in an embodiment. DETAILED DESCRIPTION

[0022] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0023] Example 1 To address the shortcomings of existing large-scale language model routing methods in terms of fine-grained recognition, cold start time, and model pool expansion costs, this embodiment provides a large language model dynamic routing method based on contextual learning model representation. By utilizing a projection model to map query embeddings to the language model input space and achieve semantic alignment, it then automatically selects representative evaluation sets covering multiple dimensions of capabilities from a benchmark question bank, efficiently obtaining the model's performance characteristics on the evaluation sets in one go, and forming a high-quality contextual model capability representation. Subsequently, it combines real-time user queries with the contextual model capability representation and uses a lightweight routing language model for supervised learning, thereby achieving fine-grained model differentiation capabilities. An incremental embedding update mechanism is designed, allowing rapid cold start with only a minimal number of fixed questions when a new model is connected or an old model is upgraded, significantly reducing computational and maintenance costs and effectively improving the accuracy, real-time performance, and flexible scalability of model routing. This method can be extended to other fields that require dynamic selection of optimal models or knowledge sources, such as recommendation systems, search engines, or knowledge question-answering systems.

[0024] See also Figure 1 ,The method mainly includes the following three stages: S1, query reconstruction pre-training stage: Use the projection model to map the input query embedding vector to a high-dimensional semantic space, input the mapped embedding vector into the model router, and train the projection model and model router with the goal of restoring the user query embedding vector.

[0025] For details, see Figure 2 , the query reconstruction pre-training stage includes S101-S103: S101: Call the query embedding model to generate a vector representation for the user query. The query embedding model is implemented using an existing model.

[0026] S102: Considering that the user query vector is in a universal encoding space, it is difficult to retain sufficient semantic information if it is directly processed by the language model. This step sets up a projection model to map the query vector into the same high-dimensional semantic space as the native input of the candidate model, thereby obtaining a query embedding vector.

[0027] Among them, the projection model can use linear projection or nonlinear neural network, such as multi-layer perceptron, attention mechanism or recurrent neural network, to achieve semantic alignment of query embedding and model input space.

[0028] S103: The mapped query embedding vector is fed into the model router along with the original user query. To ensure the language model truly understands the semantics implied by the mapped vector, the model router is required to "restore" the original user query vector at its output. During training, the projection model and the model router are simultaneously optimized to minimize reconstruction loss, ensuring close semantic alignment. The model router is essentially a language model.

[0029] In the query reconstruction pre-training phase, given a query set , first the pending query Sequential Input Query Embedding Model With two-layer fully connected projection model , get the router located at the model Vector Representation of Semantic Space . Then, Only As a condition, generate the original query The reconstructed text and minimize the loss in the following formula ,in represents the training sample number, Represents the model router at a given vector Generate the original query The conditional probability of 、 express 、 The trainable parameters of It means taking the average of all training samples.

[0030]

[0031] By optimizing this negative log-likelihood objective (i.e., reconstruction loss), the projection model is constrained to establish a correspondence between the embedding space and the semantic subspace of the Large Language Model (LLM). The LLM, in turn, learns the semantic information carried by the parsed vectors during backpropagation. This allows any query to be compressed into a high-density vector and infused with hints, significantly reducing the context length and laying the foundation for a unified representation in the subsequent routing stage.

[0032] After completing the query reconstruction pre-training phase, a projection model is obtained that can map the query vector to the language model representation space, laying the foundation for subsequent training.

[0033] S2, model representation routing training phase based on contextual learning: Based on a pre-constructed representative evaluation set with multiple difficulty levels, the query vectors in the representative evaluation set are injected into multiple candidate models using a trained projection model. For each candidate model, the true and false labels under each query vector are aggregated to obtain a contextual model capability representation. The contextual model capability representation and the query vector are used as input, and the true and false labels corresponding to the user query are used as supervision signals to train a model router based on a lightweight language model.

[0034] For details, see Figure 3 ,The model representation routing training phase based on context learning includes steps S201-S203: In step S201, we first divide the problem into several equally wide difficulty intervals based on the accuracy rate, ranging from the extremely difficult end (where only one model correctly answers the question) to the high divergence end (where approximately half the models correctly answer the question, resulting in the highest information entropy). We then draw an equal number of questions from each interval to construct a representative evaluation set covering different difficulty levels. Then, using the query and model performance for each question as input, we use the query embedding model to generate the corresponding query embedding vector.

[0035] During the construction of representative evaluation sets, dynamic screening is performed based on difficulty and information entropy, and questions are selected based on difference indicators between models (such as mutual information, variance or entropy) to ensure that representative questions represent model capabilities more accurately.

[0036] S202, using the query reconstruction pre-training phase to reconstruct the projection model, inject the query vector of each difficult question into all candidate models in the model pool, and record whether each model answered "correctly" or "wrongly" on the question. For the same candidate model, the performance results on all difficult questions are aggregated and spliced ​​into a vector sequence, which is defined as the context model capability representation. Specifically, the context model capability representation Characterized by:

[0037] Where, For the original query The vector representation of For the A representation of whether the answer of a candidate model is correct.

[0038] Contextual capability representations are aggregated using sequence concatenation, pooling (such as average pooling, max pooling, or weighted pooling), or attention mechanisms. A cost penalty term is introduced into the contextual capability representation to achieve a balance between cost and performance. Specifically, the computational cost of each model (such as inference time, resource usage, or energy consumption) is quantified as a penalty term and incorporated into the model's contextual capability representation or the optimization objective of the model router. This allows routing decisions to not only consider performance differences but also resource consumption.

[0039] S203 trains a model router consisting of a lightweight language model, using the contextual model capability representation and the user query vector as input, and the correctness or incorrectness labels of the candidate models for the user query as supervisory signals. Because the training samples contain both user query semantics and model performance information, the model router can distinguish the strengths and weaknesses of multiple models at a fine-grained capability level after convergence and output the most appropriate routing option. For the training samples, the sample data is the contextual model capability representation and the user query vector, and the label is the candidate model to be assigned. Model performance is embedded in the contextual capability representation.

[0040] Among them, the training strategy of the lightweight language model adopts supervised learning, contrastive learning or reinforcement learning.

[0041] During the routing training phase, the correct and incorrect records of each candidate model on a small batch of representative user queries are encoded into a contextual model capability representation, which is then input into the model router together with the new user query, using binary cross entropy loss. To fit the model Whether the user query can be answered correctly By minimizing the loss ,The model router learns to establish fine-grained mappings between user query features and model capabilities.

[0042]

[0043] in, Indicates a query for all users and candidate models Take the average of the above, Representation Model In user query Performance label on The probability of correct prediction for the model router.

[0044] The model representation routing training phase based on contextual learning aims to integrate the capability characteristics of each candidate model into the vector representation and train the model router.

[0045] S3, online reasoning stage: The user query and the context model capability representation of each candidate model are used as the input of the trained model router to obtain the adaptation score of each candidate model, and dynamic routing of the large language model is implemented based on the adaptation score.

[0046] Specifically, the online reasoning stage includes steps S301-S302: S301: Call the query embedding model to encode the questions in the wrong question set into vector representations.

[0047] S302: The user query and the contextual capability representation of each candidate model are handed over to the model router for processing in turn, an adaptation score is generated for each candidate model, and then the model with the highest score is selected as the final executor, which forwards the user question and returns the answer.

[0048] Since the inference process only involves one query encoding and one small language model inference, and the context length is strictly controlled, the overall response delay is maintained at an extremely low level, which can meet the requirements of real-time interaction in high-concurrency scenarios.

[0049] Furthermore, to support the continuous expansion and version updates of the model pool, this embodiment provides an incremental update mechanism. When a new model is added or an old model is upgraded, only one inference is required on a fixed-size representative evaluation set to obtain the model's performance vectors on each problem and aggregate them into a new contextual model capability representation. The routing language model can then immediately reference this, without reprocessing the entire benchmark dataset or fine-tuning the model router. Because the amount of offline computation is proportional to the size of the problem set, the entire cold start process can be completed in a short time, significantly reducing computing power consumption and operation and maintenance costs.

[0050] The dynamic routing method provided in this embodiment improves routing accuracy while maintaining real-time performance, shortens the deployment cycle, and can still stably distinguish model capabilities in cross-domain scenarios, providing a scalable, low-overhead dynamic routing solution for large-scale model pools.

[0051] Example 2 Based on Example 1, this embodiment provides an electronic device, including: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the large language model dynamic routing method based on context learning model representation as described in Example 1.

[0052] like Figure 4 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Of course, in addition to software implementation, the present invention does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0053] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0054] Example 3 Based on the above embodiments, this embodiment provides a computer-readable storage medium, including one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the large language model dynamic routing method based on context learning model representation as described in Example 1.

[0055] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0056] The present invention maps query embeddings to a semantic space consistent with the language model input through a projection model, and reconstructs the mapped query embeddings with the help of the language model to achieve alignment between the query semantics and the model representation space. At the same time, a representative evaluation set that can fully reflect the performance differences between models is automatically screened out from the benchmark question bank. Only one reasoning is required for each candidate model on this small-scale problem set to generate a high-quality context model capability representation. In the online routing stage, supervised learning is performed by inputting real-time queries and context model representations into a lightweight routing language model, thereby capturing query semantics and model output features at the same time, and achieving accurate distinction of fine-grained model capabilities. When a new model is connected or a model version is iterated, there is no need to re-reason the complete query set or retrain the model router. Only one reasoning is required on a fixed representative evaluation set to quickly update the context model capability representation, effectively reducing cold start and operation and maintenance costs, and achieving a balance between real-time performance, accuracy, flexible scalability and low-cost maintenance.

[0057] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A large language model dynamic routing method based on context learning model representation, characterized in that: The steps include: Query reconstruction pre-training: Use the projection model to map the input query embedding vector to a high-dimensional semantic space, input the mapped embedding vector into the model router, and train the projection model and the model router with the goal of restoring the user query embedding vector. Model representation routing training based on contextual learning: Based on a pre-built representative evaluation set with multiple difficulty levels, the trained projection model is used to inject query vectors from the representative evaluation set into multiple candidate models. For each candidate model, the true and false labels under each query vector are aggregated to obtain a contextual model capability representation. This contextual model capability representation and user queries are used as input, and the true and false labels corresponding to the queries are used as supervision signals to train a model router based on a lightweight language model. Online inference phase: The user query and the context model capability representation of each candidate model are used as input to the trained model router to obtain the adaptation score of each candidate model. Based on the adaptation score, dynamic routing of the large language model is implemented.

2. A large language model dynamic routing method based on context learning model representation according to claim 1, characterized in that: The process of constructing the representative evaluation set includes the following steps: According to the accuracy rate of the questions, we divide the difficulty range into several equal width intervals from extremely difficult to highly divergent, and select an equal number of questions from each interval to construct a representative assessment set covering different difficulty levels.

3. A large language model dynamic routing method based on context learning model representation according to claim 1, characterized in that: Also includes: Incremental update process: In response to the access of new candidate models or the upgrade of existing candidate models, the new candidate models and / or upgraded candidate models are reasoned on the representative evaluation set to obtain the corresponding context model capability representation.

4. The large language model dynamic routing method based on context learning model representation according to claim 1 is characterized in that: Based on average pooling, maximum pooling, weighted pooling or attention mechanism, the correct and incorrect labels of each query and the computational cost penalty are aggregated to obtain the context model capability representation.

5. The large language model dynamic routing method based on context learning model representation according to claim 1 is characterized in that: In the process of constructing the representative evaluation set, questions are screened based on the difficulty and information entropy of the questions, as well as the difference indicators between candidate models, so that the questions can represent the model capabilities.

6. A large language model dynamic routing method based on contextual learning model representation according to claim 1, characterized in that: The projection model realizes semantic alignment between query embedding and model input space through linear or nonlinear neural network, and the model router realizes training based on contrastive learning or reinforcement learning.

7. The large language model dynamic routing method based on context learning model representation according to claim 1 is characterized in that: During the model representation routing training process, the projection model and the model router are optimized synchronously by minimizing the reconstruction loss, thus achieving semantic alignment between the projection model and the model router.

8. The large language model dynamic routing method based on context learning model representation according to claim 1 is characterized in that: The high-dimensional semantic space is the same as the native input of the candidate model.

9. An electronic device, characterized in that: include: One or more processors and a memory, wherein the memory stores one or more programs, wherein the one or more programs include instructions for executing the large language model dynamic routing method based on context learning model representation as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that It includes one or more programs for execution by one or more processors of an electronic device, and the one or more programs include instructions for executing the large language model dynamic routing method based on context learning model representation as described in any one of claims 1-8.

Images