Large language model intelligent scheduling system and method supporting multi-model dynamic adaptation

Abstract

The invention discloses a large language model intelligent scheduling system and method supporting multi-model dynamic adaptation, and relates to the technical field of artificial intelligence, and the system comprises a space-time routing expert module which carries out the difficulty evaluation of a user problem based on an embedded RWKV regression model, and generates a difficulty score; the model space-time capability evaluation module is used for calculating the space-time capability score of each candidate large language model in advance and storing the space-time capability score in the form of a model capability hierarchy table; the space-time expert application and integration module is used for generating a final calling instruction containing a selected model list and a resource allocation scheme in combination with the difficulty score, the space-time capability score and a business rule; the multi-model cluster is deployed in a CPU or a GPU in a three-level heterogeneous manner according to light weight, medium weight and heavy weight, responds to an instruction, performs parallel reasoning, and outputs candidate answers and SST scores of the candidate answers through double channels; and a super-long context trunk model, constructing a super-long context input block, executing deep integration, and generating a unique expert-level answer.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically to a large language model intelligent scheduling system and method that supports dynamic adaptation of multiple models. Background Technology

[0002] In recent years, with the widespread application of large language models in natural language understanding, generation, and reasoning tasks, a large number of open-source or closed-source models (such as the LLaMA series, Qwen, RWKV, Yi, etc.) with varying parameter sizes, training data, and inference capabilities have emerged in the industry. These models exhibit significant differences in performance, latency, and resource consumption. However, current large language model service systems generally employ fixed models or simple routing strategies to respond to diverse user requests. Nevertheless, these methods have the following key limitations: The problem difficulty assessment is rough and lacks a fine-grained quantification mechanism: existing systems mostly rely on keyword matching or static classification rules to judge task complexity, which cannot accurately reflect the true semantic difficulty of natural language problems, resulting in a mismatch between model selection and task requirements; The model's capability representation is singular, and a unified spatiotemporal capability evaluation system has not been established: different models differ significantly in inference speed (time dimension) and knowledge capacity / parameter scale (spatial dimension), but the existing scheduling mechanism lacks quantitative indicators for the comprehensive capabilities of the two, making it difficult to achieve precise alignment of "capability-task".

[0003] Therefore, there is an urgent need for an intelligent scheduling architecture that can dynamically perceive the difficulty of a problem, quantify the spatiotemporal capabilities of a model, and flexibly schedule heterogeneous resources. Summary of the Invention

[0004] In view of the above problems, the present invention is proposed to provide an intelligent scheduling system and method for large language models that supports dynamic adaptation of multiple models, thereby overcoming or at least partially solving the above problems.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, embodiments of the present invention provide a large language model intelligent scheduling system that supports dynamic adaptation of multiple models, comprising: The spatiotemporal routing expert module is used to receive natural language question text input by the user, and to evaluate the difficulty of the question text based on the embedded RWKV architecture regression model, and convert the question text into a difficulty score. The model spatiotemporal capability evaluation module is used to pre-calculate and maintain the spatiotemporal capability scores of each candidate large language model in the multi-model cluster. The spatiotemporal capability scores comprehensively reflect the model's capabilities in the time and space dimensions and are organized and stored in the form of a model capability hierarchy table. The spatiotemporal expert application and integration module is used to receive the difficulty score, spatiotemporal capability score and model capability level table, and generate the final calling instruction in combination with preset business rules; The multi-model cluster contains multiple heterogeneously deployed candidate large language models, which are divided into lightweight, medium, and heavy models according to their capabilities. The lightweight candidate large language models are deployed on CPU resources, while the medium and heavy candidate large language models are deployed on GPU resources. It is used to call the corresponding candidate large language models based on the final call instruction and perform parallel inference, and output their respective candidate answers and their corresponding SST scores through dual channels. The ultra-long context backbone model is used to perform multi-source semantic fusion and logical consistency verification based on all candidate answers and corresponding SST scores of the multi-model cluster, and to combine them with the question text to form an ultra-long context input block, and to perform deep integration to finally generate a unique expert-level answer.

[0006] Preferably, the RWKV architecture regression model is trained using a pre-constructed problem difficulty evaluation dataset, which contains multiple problem samples and their corresponding continuous difficulty labels.

[0007] Preferably, the formula for calculating the spatiotemporal capability score is:

[0008]

[0009]

[0010] in, Rate the spatiotemporal capabilities. For calibrable coefficients, Activation parameters for each model, For total parameters, For the first j Basic test score, For the corresponding weights, As a quantification factor, This is the reasoning depth factor.

[0011] Preferably, the RWKV architecture regression model is an RWKV-MoE routing model, comprising: A gated network is configured to perform a linear transformation on the question text and process it with a Softmax activation function to generate dynamic weight vectors for multiple expert sub-modules. The hybrid expert module includes parallel-deployed spatial experts, temporal experts, and at least one auxiliary routing unit. Each of the aforementioned expert sub-modules independently extracts features from the problem text to generate corresponding spatial complexity feature vectors, temporal complexity feature vectors, and auxiliary routing feature vectors. The output of the hybrid expert module is a set of feature vectors from each expert sub-module, which is provided as input to the state convergence layer for weighted fusion based on the dynamic weight vector. The state aggregation layer is configured to perform a weighted summation operation on the feature vector set to generate a unified hidden state; The regression head, including layer normalization units and linear regression layers, is used to map the uniform hidden state to the difficulty score.

[0012] Preferably, it further includes: The FAISS expert capability index pool, as an independent storage unit, is used to load and maintain the capability feature vectors of all candidate large language models in the model capability hierarchy table, and to construct a high-dimensional vector index. The routing policy engine, coupled to the spatiotemporal routing expert module and the FAISS expert capability index pool, is configured as follows: Expand the difficulty score into a query vector with the same dimension as the ability feature vector; In the FAISS expert capability index pool, k-nearest neighbor retrieval based on L2 distance is performed, and Top-N candidate large language models that meet the capability requirements are recalled according to the screening condition that the spatiotemporal capability score is greater than or equal to the difficulty score. The output contains the initial invocation instructions containing the identifiers of the Top-N candidate large language models.

[0013] Preferably, the spatiotemporal expert application and integration module is configured as follows: Receive the initial invocation instruction, the question text, and the difficulty score. The initial invocation instruction includes a list of Top-N candidate large language models recalled by the routing strategy engine based on the FAISS index. The question text is split into tokens to obtain the original token sequence. The semantic feature vector of each token is extracted, and the corresponding complexity label is generated based on the difficulty score mapping. At the same time, the domain identifier is marked. The original token sequence, semantic feature vector, complexity label, and domain identifier are combined to generate a standardized JSON format input data block; The tasks are classified according to the difficulty score: if the difficulty score is ≤30, it is judged as a simple task; if the difficulty score is 30 < difficulty score ≤60, it is judged as a medium task; if the difficulty score is >60, it is judged as a complex task, and the task classification result is generated. Within the Top-N candidate large language model list, combined with the model capability hierarchy table and real-time resource monitoring data, target models that meet the requirements of spatiotemporal capability score ≥ difficulty score and current resource load ≤ preset threshold are selected to obtain the selection results. Based on the task classification results, the task timeout time and resource allocation ratio are determined, and a unique task ID is assigned to the current task. Based on the task classification and filtering results, a final calling instruction is generated, which includes a list of selected target models, task ID, timeout, resource allocation ratio, and a standardized JSON format input data block.

[0014] Preferably, the multi-model cluster is configured as follows: In response to the final invocation instruction, the invocation method is selected according to the task complexity: if it is a simple task, a single optimal matching model is invoked; if it is a medium or complex task, multiple adapted models are invoked in parallel; and the standardized JSON format input data block is passed to each invoked model. During the inference process, each called model monitors its own resource consumption status in real time, including CPU utilization, GPU utilization, and video memory usage; Based on the current resource consumption status and preset benchmark performance, the SST score corresponding to this inference is dynamically calculated to characterize the real-time effectiveness of this inference. After reasoning is completed, the results are output through two channels: the first channel outputs the generated candidate answers, and the second channel outputs the SST score of the solution itself.

[0015] Preferably, it further includes: The problem difficulty evaluation dataset is used to input a pre-constructed dataset labeled with continuous difficulty labels N into the spatiotemporal routing expert module as a training and reference benchmark, providing a quantitative basis for difficulty assessment.

[0016] Secondly, embodiments of the present invention provide an intelligent scheduling method for large language models that supports dynamic adaptation of multiple models, including: Receive natural language question text input from the user; The difficulty of the question text is assessed based on the embedded RWKV architecture regression model, and a difficulty score is generated. The spatiotemporal capability scores of each candidate large language model in the multi-model cluster are pre-calculated and maintained. The spatiotemporal capability scores comprehensively reflect the model’s capabilities in the time and space dimensions and are organized and stored in the form of a model capability hierarchy table. Based on the difficulty score, spatiotemporal capability score, and model capability hierarchy table, and combined with preset business rules, the final invocation instruction is generated. In response to the final invocation instruction, multiple candidate large language models heterogeneously deployed in the multi-model cluster are invoked. These candidate large language models are divided into lightweight, medium, and heavy-duty types according to their capabilities. The lightweight candidate large language models are deployed on CPU resources, while the medium and heavy-duty candidate large language models are deployed on GPU resources. The invoked candidate large language models are controlled to perform inference in parallel and output their respective candidate answers and their corresponding SST scores through dual channels. The system receives all candidate answers and their corresponding SST scores, performs multi-source semantic fusion and logical consistency verification, combines the question text to form an ultra-long context input block, performs deep integration based on the ultra-long context input block, and finally generates a unique expert-level answer.

[0017] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a large language model intelligent scheduling system and method that supports dynamic adaptation of multiple models, and has the following effects: 1) An embedded RWKV architecture regression model is introduced to perform end-to-end modeling of natural language question text input by users. Compared with traditional keyword matching or rule classification, it can dynamically, continuously and semantically perceive the true difficulty of any natural language question, providing an accurate basis for subsequent model selection.

[0018] 2) Design a spatiotemporal capability score as a unified capability metric, which realizes the standardization, comparability, and quantification of the capabilities of heterogeneous large models, making capability matching tasks possible.

[0019] 3) Driven by the spatiotemporal expert application and integration module, and combined with business rules, real-time load and capacity matching, fine-grained scheduling is performed.

[0020] 4) The multi-model cluster of the present invention achieves precise matching between task difficulty and model capability through capability layering and heterogeneous deployment, which significantly improves resource utilization efficiency and response quality. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0022] Figure 1 This is a block diagram of the intelligent scheduling system for large language models that supports dynamic adaptation of multiple models, provided in an embodiment of the present invention. Figure 2 This is a structural diagram of the RWKV architecture regression model provided in this embodiment of the invention; Figure 3 This is a schematic diagram of the routing policy engine provided in an embodiment of the present invention; Figure 4 This is a flowchart of a large language model intelligent scheduling method that supports dynamic adaptation of multiple models, provided in an embodiment of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] This invention discloses an intelligent scheduling system for large language models that supports dynamic adaptation of multiple models, such as... Figure 1 As shown, it includes: The Spatiotemporal Routing Expert Module is used to receive natural language question text input by users and evaluate the difficulty of the question text based on the embedded RWKV architecture regression model, converting the question text into a difficulty score. The model spatiotemporal capability evaluation module is used to pre-calculate and maintain the spatiotemporal capability scores of each candidate large language model in the multi-model cluster. The spatiotemporal capability scores comprehensively reflect the model's capabilities in the time and space dimensions and are organized and stored in the form of a model capability hierarchy table. The spatiotemporal expert application and integration module is used to receive difficulty scores, spatiotemporal capability scores, and model capability level tables, and generate final call instructions in combination with preset business rules; The multi-model cluster contains multiple heterogeneously deployed candidate large language models, which are divided into lightweight, medium, and heavy models according to their capabilities. The lightweight candidate large language models are deployed on CPU resources, while the medium and heavy candidate large language models are deployed on GPU resources. It is used to call the corresponding candidate large language models based on the final call instruction and perform parallel inference, and output their respective candidate answers and their corresponding SST scores through dual channels. The ultra-long context backbone model is used to perform multi-source semantic fusion and logical consistency verification based on all candidate answers and corresponding SST scores of the multi-model cluster, and to combine them with the question text to form an ultra-long context input block, and to perform deep integration, and finally generate a unique expert-level answer, thereby completing the entire end-to-end closed-loop processing.

[0025] Furthermore, it also includes a problem difficulty evaluation dataset, which is used to input the pre-built dataset labeled with continuous difficulty labels N into the spatiotemporal routing expert module as a training and reference benchmark, providing a quantitative basis for difficulty assessment.

[0026] In this embodiment, the core technology of the spatiotemporal routing expert module lies in decoupling the two stages of "spatiotemporal complexity assessment" and "model resource decision-making" of the problem. By combining a lightweight dedicated assessment model with an efficient indexing decision engine, dynamic, accurate and scalable routing functions can be achieved.

[0027] This invention employs the RWKV architecture, leveraging its linear complexity and extremely low latency when used as an RNN to ensure the high efficiency of the "evaluation" phase and prevent the routing module from becoming a system performance bottleneck. Secondly, it introduces the Hybrid Expert (MoE) principle, using a gated network to dynamically activate spatial and temporal experts based on input features, achieving multi-dimensional and accurate quantification of the problem's "spatiotemporal complexity."

[0028] like Figure 2 As shown, the RWKV architecture regression model is the RWKV-MoE routing model, which includes: The gating network is configured to perform a linear transformation on the question text and process it with the Softmax activation function to generate dynamic weight vectors for multiple expert sub-modules. The hybrid expert module includes parallel-deployed spatial experts, temporal experts, and at least one auxiliary routing unit (Router 1, Router 2). Each of the above expert sub-modules independently extracts features from the problem text to generate corresponding spatial complexity feature vectors, temporal complexity feature vectors, and auxiliary routing feature vectors. The output of the hybrid expert module is the feature vector set of each expert sub-module, which is provided as input to the state aggregation layer for weighted fusion based on dynamic weight vectors. The state aggregation layer is configured to perform a weighted summation operation on the feature vector set to generate a unified hidden state; The regression head, consisting of layer normalization and linear regression layers, is used to map uniform hidden states to difficulty scores.

[0029] In this embodiment, the FAISS index is used for decision-making, abandoning hard-coded or complex reinforcement learning routes. By leveraging its high-speed similarity search capability, near real-time model matching is achieved, and the system scalability problem is fundamentally solved—adding a new model only requires updating the index, without retraining the routes.

[0030] like Figure 3 As shown, it includes: The FAISS expert capability index pool, as an independent storage unit, is used to load and maintain the capability feature vectors of all candidate large language models in the model capability hierarchy table, and to construct a high-dimensional vector index, which specifies the problem difficulty threshold that each model can handle. The routing policy engine, coupled to the spatiotemporal routing expert module and the FAISS expert capability index pool, is configured as follows: Expand the difficulty score into a 7-dimensional query vector with the same dimensions as the ability feature vector; Perform k-NN Search based on L2 distance in the FAISS expert capability index pool, and recall the Top-N candidate large language models that meet the capability requirements according to the screening condition that the spatiotemporal capability score is greater than or equal to the difficulty score. The output contains the initial invocation instructions containing the identifiers of the Top-N candidate large language models.

[0031] This process ensures that the single-round decision delay is controlled within 10-20ms through FAISS's optimized retrieval capabilities, avoids the scalability problem of traditional hard-coded routing by using dynamic vector retrieval, and prevents resource mismatch through the "capability compliance" screening logic. It is a key hub connecting problem difficulty assessment and model scheduling execution.

[0032] In this invention, coupling refers to the direct or indirect data communication or function call relationship between two modules, including but not limited to information transfer achieved through APIs, message queues, shared storage, or function calls.

[0033] In this embodiment, the formula for calculating the spatiotemporal capability score is:

[0034]

[0035]

[0036] in, Rate the spatiotemporal capabilities. For calibrable coefficients, Activation parameters for each model, For total parameters, For the first j The basic test score directly reflects the task adaptability. The corresponding weights are determined by preferences for the importance of tasks. This is a quantization factor that corrects for the loss of capability in low-precision quantization. This is a reasoning depth factor used to reflect the enhanced capabilities of strategies such as Chain-of-Thought and multi-round verification.

[0037] The specific implementation process begins with data collection and standardization, extracting activation parameters for each model through the model framework interface. Total parameters The unit is unified as billion (B), and the MoE model needs to additionally record "number of experts activated per token × number of parameters per expert" to determine the value. Then, run the public test set to obtain the benchmark scores for each model. Normalization was performed on different benchmark scores; simultaneously, quantized and non-quantized models were run on the same test set to calculate the quantization factor. = Quantized baseline score / Unquantized baseline score, and set the inference depth factor according to the inference strategy. The basic reasoning was set to 1.0, Chain-of-Thought to 1.2, and multi-round validation to 1.5. Next, calibrable coefficients were optimized. A validation set was constructed covering 10 typical tasks, including commonsense question answering, code generation, and logical reasoning, ranging from simple to extremely complex difficulty. After initializing the calibrable coefficients c and γ, a grid search was used to traverse the range of c∈[0.1,0.5] and γ∈[0.5,1.2] to calculate the Pearson correlation coefficient between the spatiotemporal ability score and the model's fit to the actual task under different coefficient combinations. The coefficient corresponding to the maximum correlation coefficient was selected as the final value. Subsequently, the spatiotemporal ability score was automatically calculated. A calculation script was written to execute the calculation step by step according to the formula, first based on... , Theoretical ability to calculate coefficient c Then, based on the respective benchmark weights With normalized baseline score ( ) Calculate the experience performance item Finally, the time-space ability score is substituted and combined with... , The spatiotemporal capability scores of all models are calculated in batches using the coefficient γ, and the model is divided into hierarchical intervals ([0,30) for light, [30,60) for medium, and [60,100] for heavy) based on the spatiotemporal capability scores, generating a model spatiotemporal capability hierarchy table.

[0038] Finally, a dynamic update mechanism was established, with a fixed monthly update cycle. Adding a new model or upgrading a model automatically triggers the physical parameter collection, benchmark testing, and spatiotemporal capability score recalculation process. If the public benchmark is updated, the benchmark scores and empirical performance items of all models are recalculated synchronously. When the hardware environment for model deployment changes, GFLOPs and response latency are re-measured to correct for errors. Factors such as these are used to ensure the accuracy of spatiotemporal capability scoring.

[0039] The Spatiotemporal Capability Score is a comprehensive indicator that integrates a model's physical capabilities with its measured performance. Its core purpose is to quantify the balance between a model's ability to answer complex questions and its required computational / spatial resources. A higher score indicates a stronger overall performance in terms of spatiotemporal efficiency and task adaptability. It can be directly applied to routing strategies (selecting the model that best meets the requirements) or to comparing the capabilities of different models. Its core design considers both the model's inherent hardware attributes and publicly available benchmark performance, achieving standardized quantification through a calibrable formula to avoid the bias of a single dimension (such as looking only at parameter size or leaderboard scores).

[0040] In this embodiment, the spatiotemporal expert application and integration module is configured as follows: Receive the initial call instruction, the question text, and the difficulty score. The initial call instruction contains a list of Top-N candidate large language models recalled by the routing strategy engine based on the FAISS index. The question text is split into tokens to obtain the original token sequence. The semantic feature vector of each token is extracted and the corresponding complexity label is generated based on the difficulty score mapping. At the same time, the domain identifier is labeled. Combine the original token sequence, semantic feature vector, complexity label and domain identifier to generate a standardized JSON format input data block; Tasks are classified according to their difficulty score: if the difficulty score is ≤30, it is classified as a simple task; if the difficulty score is 30 < ≤60, it is classified as a medium task; if the difficulty score is >60, it is classified as a complex task, and the task classification results are generated. Within the Top-N candidate large language model list, combined with the model capability hierarchy table and real-time resource monitoring data, target models that meet the requirements of spatiotemporal capability score ≥ difficulty score and current resource load ≤ preset threshold are selected to obtain the selection results. Based on the task classification results, determine the task timeout and resource allocation ratio, and assign a unique task ID to the current task. Based on the task classification and filtering results, the final calling instruction is generated, which includes a list of selected target models, task ID, timeout, resource allocation ratio, and standardized JSON format input data blocks.

[0041] In this embodiment, the multi-model cluster, acting as the system's execution layer, receives Top-N model invocation instructions and corresponding resource allocation schemes distributed by the dynamic resource scheduling submodule as input. This drives models of varying sizes, from Model-XS to Model-XL, within the cluster to perform parallel inference in their respective deployed hardware environments. Each model, while calculating based on the input question token sequence, monitors its own resource consumption status in real time. The final output includes candidate answer text generated by the model and its own SST score (1), representing the real-time capability and efficiency of the inference. The subsequent answer aggregation module receives multiple token-level answer sequences output in parallel from the multi-model cluster as input. It performs token-level result aggregation operations on the discrete output streams from different models, preventing the loss of key information during parallel processing and ensuring the integrity of all candidate answers, thereby outputting structured and complete answer aggregation data (2). The confidence weighting module receives the SST scores of the multi-model cluster outputs and the aggregated answer data processed as described above as inputs. Based on the SST scores as a confidence index, it performs dynamic weight adjustments on the output results of different models, prioritizing the increase of the weight of the output content of models with high SST scores and reducing the influence of models with low fit. Finally, it outputs a preprocessed result set with weighted optimization, which is passed as context input with high confidence logical characteristics to the subsequent ultra-long context backbone model for deep integration.

[0042] The multi-model cluster is the core resource support for the spatiotemporal expert application and integration module. In essence, it integrates a collection of open-source models with different parameter scales and capability levels, namely different candidate large language models (Model-XS, Model-S, Model-M, Model-L, Model-XL). It is specifically designed to receive decision instructions from the spatiotemporal expert Router and provide adaptive model support for different types of token-level tasks. The core logic is to accurately match task requirements with model capabilities, ensuring processing effectiveness while avoiding resource waste.

[0043] From a core principle perspective, the design of the multi-model cluster revolves around "gradient adaptation." The five types of models in the cluster are hierarchically structured based on their capabilities and resource consumption: Model-XS, as a lightweight model, is suitable for simple tasks requiring fast response; Model-S and Model-M, as medium-weight models, balance processing power and resource consumption, adapting to moderately complex tasks; and Model-L and Model-XL, as heavyweight models, with their larger parameter scale, are specifically designed for complex tasks. The core of this hierarchical division is to ensure that tasks of varying difficulty can find models with corresponding capabilities, avoiding the use of complex models for simple tasks and the constraint of using simple models to handle complex tasks. At the implementation level, the construction of the multi-model cluster first completes the model selection and basic configuration. When selecting models, mainstream open source models are chosen to cover the full range of lightweight, medium and heavy-duty models to ensure that they can be adapted to different tasks. At the same time, SST scores are pre-calculated for each model, and clear score threshold ranges are defined and stored for quick query and comparison when the Router makes decisions.

[0044] In terms of resource management, the cluster allocates deployment environments according to model type. Lightweight Model-XS and Model-S are deployed on CPUs to meet the needs of high concurrency and fast response; medium-to-heavyweight Model-M, Model-L, and Model-XL are deployed on GPUs to ensure the processing power for complex tasks. A resource monitoring module is also included to track the running status of each model in real time, including CPU and GPU usage and response speed, avoiding resource conflicts when multiple models are called simultaneously. The scheduling and execution process fully follows the decision results of the spatiotemporal expert Router. After the Router determines the target model, the cluster selects the calling method based on the task: simple tasks call a single optimal model, while complex tasks call multiple adapted models in parallel. During scheduling, resource allocation is adjusted based on resource monitoring data to ensure smooth model operation, such as allocating more computing resources to models handling complex tasks and prioritizing resources for tasks requiring fast response. The cluster also includes fault tolerance and optimization mechanisms. If the selected model experiences a timeout or outputs abnormal results, it automatically switches to a backup model recommended by the Router, ensuring uninterrupted task execution. Simultaneously, it feeds back any abnormal information during model execution to the spatiotemporal expert Router, helping the Router adjust its subsequent decision-making logic. Furthermore, the cluster supports dynamic updates, allowing the addition of higher-performance models or the removal of inefficient or outdated models, ensuring that the cluster's model resources always adapt to changing task requirements. Through a well-structured hierarchical structure, resource management, and scheduling mechanism, the entire multi-model cluster becomes the actual execution platform for the spatiotemporal expert Router's decisions, providing stable and diverse model outputs for subsequent result integration, supporting the entire system in completing tasks efficiently and accurately.

[0045] The specific implementation process is as follows: The multi-model cluster is configured as follows: In response to the final invocation command, the invocation method is selected according to the task complexity: for simple tasks, a single optimal matching model is invoked; for medium or complex tasks, multiple adapted models are invoked in parallel; and standardized JSON format input data blocks are passed to each invoked model. During the inference process, each called model monitors its own resource consumption status in real time, including CPU utilization, GPU utilization, and video memory usage; Based on the current resource consumption status and preset benchmark performance, the SST score corresponding to this inference is dynamically calculated to characterize the real-time effectiveness of this inference. After inference is complete, the system outputs results through two channels: the first channel outputs the generated candidate answers, and the second channel outputs its own SST score. If any called model times out or outputs an anomaly, the system automatically switches to the alternative model specified in the final calling instruction and feeds back the anomaly information to the spatiotemporal expert application and integration module to optimize subsequent scheduling decisions.

[0046] In this embodiment, the ultra-long context backbone model constitutes the final integration and refinement layer in the system process. It is used to generate fused text through multi-source semantic fusion based on candidate answers output by a multi-model cluster and the corresponding SST scores of each candidate answer, constructing an ultra-long context input block, and then performing deep integration with the ultra-long context backbone model to generate the final answer. Multi-source semantic fusion specifically includes: sorting candidate answers according to model priority, performing token-level alignment and deduplication, retaining the source model identifier and confidence information of each token, and performing confidence-weighted fusion of each token based on SST score to generate weighted fused text; The weighted fused text is then subjected to logical consistency verification. Factual conflicts and logical contradictions are detected by combining a preset rule base and a lightweight inference model. When conflicts exist, the output of the high SST model is retained first to complete the correction. After completing multi-source semantic fusion and consistency verification, the weighted fused text, the original question text, and the SST scores of each model are encapsulated into an ultra-long context input block. Based on the ultra-long context input block, deep integration reasoning is performed to finally generate a unique expert-level answer.

[0047] This answer integrates the intelligence of multi-model collaboration and solves the problem of information fragmentation, ultimately serving as the system's sole response to the user.

[0048] Based on the same inventive concept, embodiments of the present invention also provide an intelligent scheduling method for large language models that supports dynamic adaptation of multiple models, such as... Figure 4 As shown, it includes: Receive natural language question text input from the user; The difficulty of the question text is assessed based on the embedded RWKV architecture regression model, and a difficulty score is generated. The spatiotemporal capability scores of each candidate large language model in the multi-model cluster are pre-calculated and maintained. The spatiotemporal capability scores comprehensively reflect the model’s capabilities in the time and space dimensions and are organized and stored in the form of a model capability hierarchy table. Based on the difficulty score, spatiotemporal capability score, and model capability hierarchy table, combined with preset business rules, the final invocation instruction is generated. In response to the final invocation command, multiple candidate large language models heterogeneously deployed in the multi-model cluster are invoked. The candidate large language models are divided into lightweight, medium and heavy according to their capabilities. The lightweight candidate large language models are deployed on CPU resources, while the medium and heavy candidate large language models are deployed on GPU resources. The invoked candidate large language models are controlled to perform inference in parallel and output their respective candidate answers and their corresponding SST scores through dual channels. It receives all candidate answers and their corresponding SST scores, performs multi-source semantic fusion and logical consistency verification, combines the question text to form an ultra-long context input block, performs deep integration based on the ultra-long context input block, and finally generates a unique expert-level answer.

[0049] Since the principle behind this method is similar to that of the aforementioned system, the specific implementation can be found in the implementation of the aforementioned system, and the repetitive parts will not be repeated.

[0050] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0051] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A large language model intelligent scheduling system supporting dynamic adaptation of multiple models, characterized in that, include: The spatiotemporal routing expert module is used to receive natural language question text input by the user, and to evaluate the difficulty of the question text based on the embedded RWKV architecture regression model, and convert the question text into a difficulty score. The model spatiotemporal capability evaluation module is used to pre-calculate and maintain the spatiotemporal capability scores of each candidate large language model in the multi-model cluster. The spatiotemporal capability scores comprehensively reflect the model's capabilities in the time and space dimensions and are organized and stored in the form of a model capability hierarchy table. The spatiotemporal expert application and integration module is used to receive the difficulty score, spatiotemporal capability score and model capability level table, and generate the final calling instruction in combination with preset business rules; The multi-model cluster contains multiple heterogeneously deployed candidate large language models, which are divided into lightweight, medium, and heavy models according to their capabilities. The lightweight candidate large language models are deployed on CPU resources, while the medium and heavy candidate large language models are deployed on GPU resources. It is used to call the corresponding candidate large language models based on the final call instruction and perform parallel inference, and output their respective candidate answers and their corresponding SST scores through dual channels. The ultra-long context backbone model is used to perform multi-source semantic fusion and logical consistency verification based on all candidate answers and corresponding SST scores of the multi-model cluster, and to combine them with the question text to form an ultra-long context input block for deep integration, ultimately generating a unique expert-level answer.

2. The system as described in claim 1, characterized in that, in, The RWKV architecture regression model uses a pre-constructed problem difficulty evaluation dataset for supervised learning during the training phase. The problem difficulty evaluation dataset contains multiple problem samples and their corresponding continuous difficulty labels.

3. The system as described in claim 1, characterized in that, The formula for calculating the spatiotemporal capability score is: in, Rate the spatiotemporal capabilities. For calibrable coefficients, Activation parameters for each model, For total parameters, For the first j Basic test score, For the corresponding weights, As a quantification factor, This is the reasoning depth factor.

4. The system as described in claim 3, characterized in that, The RWKV architecture regression model is an RWKV-MoE routing model, including: A gated network is configured to perform a linear transformation on the question text and process it with a Softmax activation function to generate dynamic weight vectors for multiple expert sub-modules. The hybrid expert module includes parallel-deployed spatial experts, temporal experts, and at least one auxiliary routing unit. Each of the aforementioned expert sub-modules independently extracts features from the problem text to generate corresponding spatial complexity feature vectors, temporal complexity feature vectors, and auxiliary routing feature vectors. The output of the hybrid expert module is a set of feature vectors from each expert sub-module, which is provided as input to the state convergence layer for weighted fusion based on the dynamic weight vector. The state aggregation layer is configured to perform a weighted summation operation on the feature vector set to generate a unified hidden state; The regression head, including layer normalization and linear regression layers, is used to map the uniform hidden state to the difficulty score.

5. The system as described in claim 1 or 4, characterized in that, Also includes: The FAISS expert capability index pool, as an independent storage unit, is used to load and maintain the capability feature vectors of all candidate large language models in the model capability hierarchy table, and to construct a high-dimensional vector index. The routing policy engine, coupled to the spatiotemporal routing expert module and the FAISS expert capability index pool, is configured as follows: Expand the difficulty score into a query vector with the same dimension as the ability feature vector; In the FAISS expert capability index pool, k-nearest neighbor retrieval based on L2 distance is performed, and Top-N candidate large language models that meet the capability requirements are recalled according to the screening condition that the spatiotemporal capability score is greater than or equal to the difficulty score. The output contains the initial invocation instructions containing the identifiers of the Top-N candidate large language models.

6. The system as described in claim 5, characterized in that, The spatiotemporal expert application and integration module is configured as follows: Receive the initial invocation instruction, the question text, and the difficulty score. The initial invocation instruction includes a list of Top-N candidate large language models recalled by the routing strategy engine based on the FAISS index. The question text is split into tokens to obtain the original token sequence. The semantic feature vector of each token is extracted, and the corresponding complexity label is generated based on the difficulty score mapping. At the same time, the domain identifier is marked. The original token sequence, semantic feature vector, complexity label, and domain identifier are combined to generate a standardized JSON format input data block; The tasks are classified according to the difficulty score: if the difficulty score is ≤30, it is judged as a simple task; if the difficulty score is 30 < difficulty score ≤60, it is judged as a medium task; if the difficulty score is >60, it is judged as a complex task, and the task classification result is generated. Within the Top-N candidate large language model list, combined with the model capability hierarchy table and real-time resource monitoring data, target models that meet the requirements of spatiotemporal capability score ≥ difficulty score and current resource load ≤ preset threshold are selected to obtain the selection results. Based on the task classification results, the task timeout time and resource allocation ratio are determined, and a unique task ID is assigned to the current task. Based on the task classification and filtering results, a final calling instruction is generated, which includes a list of selected target models, task ID, timeout, resource allocation ratio, and a standardized JSON format input data block.

7. The system as described in claim 6, characterized in that, The multi-model cluster is configured as follows: In response to the final invocation instruction, the invocation method is selected according to the task complexity: if it is a simple task, a single optimal matching model is invoked; if it is a medium or complex task, multiple adapted models are invoked in parallel; and the standardized JSON format input data block is passed to each invoked model. During the inference process, each called model monitors its own resource consumption status in real time, including CPU utilization, GPU utilization, and video memory usage; Based on the current resource consumption status and preset benchmark performance, the SST score corresponding to this inference is dynamically calculated to characterize the real-time effectiveness of this inference. After reasoning is completed, the results are output through two channels: the first channel outputs the generated candidate answers, and the second channel outputs the SST score of the solution itself.

8. The system as described in claim 1, characterized in that, Also includes: The problem difficulty evaluation dataset is used to input a pre-constructed dataset labeled with continuous difficulty labels N into the spatiotemporal routing expert module as a training and reference benchmark, providing a quantitative basis for difficulty assessment.

9. A method for intelligent scheduling of large language models supporting dynamic adaptation of multiple models, wherein the method is implemented based on the system described in any one of claims 1-8, characterized in that, include: Receive natural language question text input from the user; The difficulty of the question text is assessed based on the embedded RWKV architecture regression model, and a difficulty score is generated. The spatiotemporal capability scores of each candidate large language model in the multi-model cluster are pre-calculated and maintained. The spatiotemporal capability scores comprehensively reflect the model’s capabilities in the time and space dimensions and are organized and stored in the form of a model capability hierarchy table. Based on the difficulty score, spatiotemporal capability score, and model capability hierarchy table, and combined with preset business rules, the final invocation instruction is generated. In response to the final invocation instruction, multiple candidate large language models heterogeneously deployed in the multi-model cluster are invoked. These candidate large language models are divided into lightweight, medium, and heavy-duty types according to their capabilities. The lightweight candidate large language models are deployed on CPU resources, while the medium and heavy-duty candidate large language models are deployed on GPU resources. The invoked candidate large language models are controlled to perform inference in parallel and output their respective candidate answers and their corresponding SST scores through dual channels. The system receives all candidate answers and their corresponding SST scores, performs multi-source semantic fusion and logical consistency verification, combines the question text to form an ultra-long context input block, and performs deep integration based on the ultra-long context input block to finally generate a unique expert-level answer.

Images