Large language model dynamic routing method and related device
By evaluating the mastery scoring mechanism of large language models and dynamically selecting lightweight and high-performance models, the problems of misjudgment and illusion in dynamic routing of large language models are solved, realizing low-cost and high-precision dynamic routing, which is suitable for commercial services.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGDONG CHICO ELECTRONIC INC
- Filing Date
- 2026-01-16
- Publication Date
- 2026-05-29
AI Technical Summary
Existing large language models suffer from misjudgment and illusion phenomena in dynamic routing, making it difficult to accurately assess the model's mastery of specific knowledge. Furthermore, the underlying probability data of commercial services is difficult to utilize, causing traditional detection methods to fail.
By assessing the model's mastery of specific knowledge, the first model generates candidate and distractor answers, calculates the mastery score, and calls the second model for processing when the score is below the confidence threshold. The reasoning ability of the second model is higher than that of the first model.
It enables accurate assessment of a model's mastery of specific knowledge without relying on the underlying probability data interface, reduces API call costs, prevents the output of illusory answers, and improves the accuracy and efficiency of dynamic routing.
Smart Images
Figure CN122114144A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence, and more particularly to a dynamic routing method and related apparatus for large language models. Background Technology
[0002] With the rapid development of deep learning technology, large language models have been widely used in tasks such as question answering, summarizing, and logical reasoning. In practical industrial applications, there are usually several types of models to choose from: one is a lightweight model with a small number of parameters and low inference cost but limited capabilities (such as 7B / 8B parameter models); the other is a large-scale model with a huge number of parameters, high inference cost but excellent capabilities (such as the GPT series, gemini series, etc.).
[0003] To balance cost and effectiveness, dynamic routing strategies are typically employed. Traditional routing methods mainly rely on two approaches: semantic complexity-based classification and perplexity-based thresholding.
[0004] For semantic complexity-based classification, classifiers like BERT are used to determine the domain of the user's question, assigning simpler questions to smaller models and more complex ones to larger models. This method cannot determine whether smaller models truly "know" the answer, easily leading to misclassification. For perplexity-based thresholding, the perplexity or average log probability of the model's generated answer is calculated. However, large models commonly exhibit hallucination, meaning that when generating incorrect facts, they often display extremely high confidence (very low perplexity). This renders self-probability-based evaluation methods ineffective, failing to identify semantically reasonable but factually inaccurate cases.
[0005] Furthermore, most commercial large-scale model services are provided via APIs and often do not return the underlying log probability data, making it difficult to apply traditional probability distribution-based detection methods to these commercial large-scale models. Summary of the Invention
[0006] The following is an overview of the topics described in detail in this article.
[0007] The purpose of this application is to at least partially solve one of the technical problems existing in the related technologies. The embodiments of this application provide a dynamic routing method and related apparatus for large language models, which realizes low-cost and high-precision dynamic model routing by evaluating the model's mastery of specific knowledge.
[0008] An embodiment of the first aspect of this application provides a dynamic routing method for a large language model, comprising: Receive query requests for factual knowledge; The first model is invoked to generate candidate and distractor answers based on the query request; The first model is invoked to perform reasoning based on the reasoning request corresponding to the query request to obtain the reasoning result, and the first frequency of the candidate answer appearing in the reasoning result and the second frequency of the interference answer appearing in the reasoning result are determined; Based on the ratio of the first frequency to the second frequency, calculate the mastery score of the first model for the knowledge point corresponding to the query request; When the mastery score is higher than the confidence threshold, the candidate answer is output; when the mastery score is lower than the confidence threshold, the second model is invoked to output the target answer according to the query request. The reasoning ability of the second model is higher than that of the first model.
[0009] According to certain embodiments of the first aspect of this application, the interfering answer is generated by the first model based on a query request, guided by a prompt word instruction, and the prompt word instruction is generated based on a preset prompt word template.
[0010] According to certain embodiments of the first aspect of this application, the interfering answer is in a structured data format.
[0011] According to certain embodiments of the first aspect of this application, the mastery score is expressed as follows: ;in, In order to understand the degree of scoring, For the first frequency, For the second frequency, This is a smoothing factor.
[0012] According to certain embodiments of the first aspect of this application The value range is from 0.01 to 0.5.
[0013] According to certain embodiments of the first aspect of this application, during the inference process, the number of inferences is set to be greater than or equal to 3, and the sampling temperature parameter used to stimulate diversity of inference results is set to be greater than 0.5.
[0014] According to certain embodiments of the first aspect of this application, the number of parameters of the second model is greater than the number of parameters of the first model, and the inference cost of the second model is higher than the inference cost of the first model.
[0015] A second aspect of this application provides a routing system, comprising: The query receiving module is configured to receive query requests for factual knowledge. The negative sample construction module is configured to call the first model to generate candidate answers and distractor answers based on the query request; The multi-sampling module is configured to call the first model to perform reasoning based on the reasoning request corresponding to the query request to obtain a reasoning result, and determine the first frequency of the candidate answer appearing in the reasoning result and the second frequency of the interference answer appearing in the reasoning result; The scoring calculation module is configured to calculate the mastery score of the first model for the knowledge point corresponding to the query request based on the ratio of the first frequency to the second frequency. The threshold determination module is configured to output the candidate answer when the mastery score is higher than the confidence threshold, and to call the second model to output the target answer according to the query request when the mastery score is lower than the confidence threshold. The reasoning ability of the second model is higher than that of the first model.
[0016] According to a third aspect of this application, an electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the large language model dynamic routing method as described in the first aspect of this application.
[0017] According to a fourth aspect of this application, a computer storage medium stores computer-executable instructions for performing the large language model dynamic routing method as described in an embodiment of the first aspect of this application.
[0018] The above solution has at least the following beneficial effects: it can accurately assess the lightweight model's mastery of specific knowledge without relying on the underlying probability data interface, thereby achieving accurate dynamic routing and preventing illusory answer output while reducing API call costs. Attached Figure Description
[0019] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.
[0020] Figure 1 This is a flowchart illustrating the steps of the dynamic routing method for large language models. Figure 2 This is a flowchart of the dynamic routing method for large language models; Figure 3 This is an architecture diagram of a routing system. Detailed Implementation
[0021] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0022] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, or the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.
[0023] Embodiments of this application provide a dynamic routing method and related apparatus for large language models. Low-cost, high-precision dynamic model routing is achieved by evaluating the model's mastery of specific knowledge.
[0024] The embodiments of this application will be further described below with reference to the accompanying drawings.
[0025] Reference Figure 1 The dynamic routing method for large language models includes the following steps: Step S100: Receive a query request for factual knowledge; Step S200: Invoke the first model to generate candidate answers and distractor answers based on the query request; Step S300: Call the first model to perform reasoning based on the reasoning request of the corresponding query request to obtain the reasoning result, and determine the first frequency of the candidate answer appearing in the reasoning result and the second frequency of the interference answer appearing in the reasoning result; Step S400: Calculate the mastery score of the first model for the knowledge points corresponding to the query request based on the ratio of the first frequency to the second frequency. In step S500, when the mastery score is higher than the confidence threshold, candidate answers are output; when the mastery score is lower than the confidence threshold, the second model is invoked to output the target answer according to the query request. The reasoning ability of the second model is higher than that of the first model.
[0026] It should be noted that the distractor answer is generated by the first model based on the query request, guided by the prompt word instruction, which is generated based on the preset prompt word template.
[0027] The prompt word instruction contains instruction text, which requires the first model to generate a preset number of semantically reasonable but factually incorrect alternatives as distractors while outputting the correct answer, and return the distractors in a structured data format.
[0028] The mastery score is expressed as follows: ;in, In order to understand the degree of scoring, For the first frequency, For the second frequency, This is a smoothing factor used to prevent the denominator from being zero and to adjust sensitivity. The value range is from 0.01 to 0.5.
[0029] Mastery rating along with It increases with the increase of, and with It decreases as it increases.
[0030] During the inference process, the number of inferences was set to be greater than or equal to 3. The sampling temperature parameter was set to be greater than 0.5 to stimulate diversity in inference results, thereby detecting the degree to which the model oscillates between candidate and distractor answers.
[0031] Forwarding a query request to the second model for processing includes the following steps: sending the original text of the query request to the API interface of the second model; receiving the response text generated by the second model and returning it to the user as the final response.
[0032] The second model has more parameters than the first model, and the inference cost of the second model is higher than that of the first model.
[0033] Embodiments of this application provide a routing system.
[0034] Reference Figure 3 The routing system includes: a query receiving module, a negative sample construction module, a multisampling module, a scoring calculation module, and a threshold determination module.
[0035] The query receiving module is configured to receive query requests for factual knowledge; the negative sample construction module is configured to call the first model to generate candidate answers and distractor answers based on the query request; the multisampling module is configured to call the first model to perform inference based on the inference request of the corresponding query request to obtain the inference result, and determine the first frequency of the candidate answer and the second frequency of the distractor answer in the inference result; the scoring calculation module is configured to calculate the mastery score of the first model for the knowledge point corresponding to the query request based on the ratio of the first frequency to the second frequency; the threshold determination module is configured to output the candidate answer when the mastery score is higher than the confidence threshold, and to call the second model to output the target answer based on the query request when the mastery score is lower than the confidence threshold. The reasoning ability of the second model is higher than that of the first model.
[0036] It is understandable that the routing system adopts the large language model dynamic routing method described above. Each module of the routing system corresponds one-to-one with each step of the large language model dynamic routing method. The routing system and the large language model dynamic routing method use the same technical solution, solve the same technical problems, and achieve the same technical results.
[0037] The architecture of the routing system is described below.
[0038] ReferenceFigure 3 From the perspective of static connection relationships, the routing system has an API gateway module and a routing controller.
[0039] The API gateway module serves as the system's entry point, establishing a connection with user terminals via the HTTPS protocol to receive users' natural language query requests.
[0040] As the core processing unit of the system, the routing controller establishes electrical or data connections with the API gateway, the first model interface, and the second model interface, respectively.
[0041] The routing controller further includes the following logically coupled sub-modules: negative sample construction module, multisampling module, scoring calculation module, and threshold determination module.
[0042] The negative sample construction module assembles instructions based on preset prompt word templates. The multisampling module is configured to send multiple concurrent inference requests to the model and collect the returned text sequences. The scoring calculation module has built-in mathematical operation logic to execute a scoring formula based on negative sample comparisons. The threshold determination module stores preset routing thresholds and outputs either a high level (direct output signal) or a low level (forwarding trigger signal).
[0043] In the model interface layer, the first model (Model A) is a lightweight large language model deployed locally or accessed via a low-cost API (such as Llama-3-8B-Instruct), characterized by fast inference speed and low calling cost, but with lower accuracy for long-tail knowledge. The second model (Model B) is an ultra-large-scale language model accessed via a high-bandwidth API (such as GPT-4o), characterized by a huge number of parameters and strong logical reasoning capabilities, but with high calling cost.
[0044] From the perspective of dynamic working relationships, when the API gateway receives a request, the routing controller first activates the first model to perform "exploratory reasoning" and "self-checking". Based on the mastery score output by the scoring calculation module, it dynamically decides whether the data flow should flow directly to the user terminal or be redirected to the second model for secondary processing.
[0045] The implementation process of the dynamic routing method is explained below.
[0046] Reference Figure 2 The system receives a query request (Query) from the user, which contains the original question (Q).
[0047] The routing controller calls the first model, but instead of forwarding the input directly, it constructs a prompt word instruction with the following structure through the negative sample construction module: "For the user's question, please provide: one most accurate short answer; two seemingly reasonable but incorrect distractors. Return in JSON format." The first model performs one inference based on this instruction and outputs: candidate answer (Cand); set of distractors (Neg).
[0048] This step utilizes the model's own associative capabilities to uncover "easily confused areas" around the knowledge point, providing targets for subsequent verification.
[0049] The multisampling module initiates N independent verification requests for the first model.
[0050] The multiple sampling module is configured with the following parameters: sampling number N=5, temperature parameter Temperature=1.0. Higher temperatures can stimulate model diversity; however, if the model is uncertain about the knowledge points, high temperatures can lead to divergent answers.
[0051] The input to the first model from the multiple sampling module is only the original question Q (excluding instructions to generate interference terms). The resulting sampling is as follows: .
[0052] The multiple sampling module counts the frequency of each answer in the sequence S, and counts the number of times the candidate answer Cand appears in S, denoted as . The total number of times any element in the set of distractors Neg appears in S is denoted as . The first frequency of a candidate answer appearing in the reasoning result is calculated as follows: The second frequency of distractor answers appearing in the reasoning results is calculated as follows: .
[0053] The scoring calculation module calculates the mastery score T: ;in, This is a smoothing factor with a value of 0.1, used to prevent the denominator from being zero and to adjust the sensitivity to interference terms.
[0054] The threshold determination module compares T with the preset threshold. (Set to 1.5) Compare: like If the first model is deemed to have mastered the knowledge, the candidate answer Cand is directly output. like If the first model is deemed to have a risk of hallucination, the original query request Q is forwarded to the second model, and the second model's answer is output.
[0055] The following is a detailed description using Example 3.
[0056] Two sets of typical data ("simple facts" and "illusionary details") are selected for comparison and illustration.
[0057] The experimental environment was set up as follows: The first model was Meta-Llama-3-8B-Instruct (called via OpenRouter); the second model was GPT-4o; and the smoothing factor was... Set to 0.1; set the number of samples N to 5.
[0058] Data Set 1: High Confidence Scenarios (Simple Facts).
[0059] calculate: ; Substitute into the formula: .
[0060] Data set 2: Low confidence scenarios (details prone to hallucinations).
[0061] calculate: ; It should be noted that the high frequency of distractors significantly lowered the score. Substituting into the formula: .
[0062] Judgment: -0.22 < 1.5 (threshold).
[0063] Result: Allowed (triggered routing). The system automatically invoked GPT-4.
[0064] Having successfully intercepted a potentially incorrect response, GPT-4 subsequently provided a more rigorous answer (such as pointing out that the history was unclear).
[0065] In this embodiment, compared to existing technologies that typically rely solely on the perplexity or logarithmic probability mean of the model output to judge the quality of an answer, which often leads to the model generating fluent but factually contradictory "illusionary" content and misjudging it as a high-quality answer due to its low perplexity, this method introduces "negative sample comparison." This not only examines whether the model is "confident in the correct answer" but also forces an examination of whether the model "can clearly eliminate erroneous interference." If the model is ambiguous about the knowledge points, it is prone to drifting to interference items in multiple samplings, thus being intercepted; thereby achieving resistance to illusionary interference.
[0066] According to the scoring formula Even the probability of candidate answers A higher value (e.g., 0.9) is acceptable as long as the model fails to completely eliminate interference terms during sampling (i.e., ... (Significantly greater than 0), increasing the denominator will cause the final score T to drop sharply.
[0067] In a test set containing 200 easily confused factual questions, the existing technology (based solely on thresholds) achieved a hallucination interception rate of only 45% (allowing a large number of incorrect answers); however, using this method, the hallucination interception rate increased to 87% at the same threshold, effectively identifying the critical state of the model's "seeming understanding but not quite understanding".
[0068] This approach achieves an optimal balance between computational cost and service quality. In tests simulating real user request distribution (80% common general questions, 20% long-tail difficult questions), for 80% of simple queries (such as common sense questions), the average mastery score determined by this method was above 2.1. The system successfully locked these queries onto the low-cost first model (such as Llama-3-8B) for processing, with a single interaction cost approximately 1 / 30th of that at the GPT-4 level. Only about 22% of complex or ambiguous requests triggered the low-score threshold (score <1.5) and were routed to the second model. Compared to solutions that call the entire large model, this method reduces the overall API call cost (Token Consumption Cost) by approximately 65% while maintaining roughly the same accuracy (error range ±2%).
[0069] By using sampling frequency instead of the underlying layer, this method can be applied to any commercial closed-source model that does not disclose its underlying parameters (such as various models connected to OpenRouter). It does not depend on access permissions for any underlying model parameters, giving the system strong versatility and portability. It can seamlessly integrate with any third-party service that only provides text output interfaces, such as OpenRouter, ChatGPT, and Claude, thus improving its applicability. It can accurately identify the true knowledge boundaries of small models, forwarding only the truly difficult "long-tail problems" to expensive large models, significantly reducing overall system operating costs and maximizing cost-effectiveness.
[0070] This method achieves a qualitative leap in user experience by reducing latency to the millisecond level. Although it introduces negative sample generation and multiple sampling steps, the inference speed of the first model (small model) is extremely fast (usually >100 tokens / s), and the sampling process can be executed asynchronously.
[0071] Embodiments of this application provide an electronic device. The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the large language model dynamic routing method as described above.
[0072] This electronic device can be any smart terminal, including computers.
[0073] In general, for the hardware structure of electronic devices, the processor can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, to execute relevant programs and implement the technical solutions provided in the embodiments of this application.
[0074] The memory can be implemented in the form of read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory and is called and executed by the processor.
[0075] Input / output interfaces are used to implement information input and output.
[0076] The communication interface is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).
[0077] The bus transmits information between various components of a device, such as the processor, memory, input / output interfaces, and communication interfaces. The processor, memory, input / output interfaces, and communication interfaces communicate with each other within the device via the bus.
[0078] Embodiments of this application provide a computer storage medium. The computer storage medium stores computer-executable instructions for executing the large language model dynamic routing method described above.
[0079] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium. In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0080] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.
[0081] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0082] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0083] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0084] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms. Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.
[0085] The above is a detailed description of the preferred embodiments of this application, but this application is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.
Claims
1. A dynamic routing method for a large language model, characterized in that, include: Receive query requests for factual knowledge; The first model is invoked to generate candidate and distractor answers based on the query request; The first model is invoked to perform reasoning based on the reasoning request corresponding to the query request to obtain the reasoning result, and the first frequency of the candidate answer appearing in the reasoning result and the second frequency of the interference answer appearing in the reasoning result are determined; Based on the ratio of the first frequency to the second frequency, calculate the mastery score of the first model for the knowledge point corresponding to the query request; When the mastery score is higher than the confidence threshold, the candidate answer is output; when the mastery score is lower than the confidence threshold, the second model is invoked to output the target answer according to the query request. The reasoning ability of the second model is higher than that of the first model.
2. The dynamic routing method for large language models according to claim 1, characterized in that, The distractor answer is generated by the first model based on the query request, guided by the prompt word instruction, and the prompt word instruction is generated based on a preset prompt word template.
3. The dynamic routing method for large language models according to claim 1, characterized in that, The interference answers are in a structured data format.
4. The dynamic routing method for large language models according to claim 1, characterized in that, The mastery score is expressed as follows: ;in, In order to understand the degree of scoring, For the first frequency, For the second frequency, This is a smoothing factor.
5. The dynamic routing method for large language models according to claim 4, characterized in that, The value range is from 0.01 to 0.
5.
6. The dynamic routing method for large language models according to claim 1, characterized in that, During the inference process, the number of inferences was set to be greater than or equal to 3, and the sampling temperature parameter, which is used to stimulate the diversity of inference results, was set to be greater than 0.
5.
7. The dynamic routing method for large language models according to claim 1, characterized in that, The second model has more parameters than the first model, and the inference cost of the second model is higher than that of the first model.
8. A routing system, characterized in that, include: The query receiving module is configured to receive query requests for factual knowledge. The negative sample construction module is configured to call the first model to generate candidate answers and distractor answers based on the query request; The multi-sampling module is configured to call the first model to perform reasoning based on the reasoning request corresponding to the query request to obtain a reasoning result, and determine the first frequency of the candidate answer appearing in the reasoning result and the second frequency of the interference answer appearing in the reasoning result; The scoring calculation module is configured to calculate the mastery score of the first model for the knowledge point corresponding to the query request based on the ratio of the first frequency to the second frequency. The threshold determination module is configured to output the candidate answer when the mastery score is higher than the confidence threshold, and to call the second model to output the target answer according to the query request when the mastery score is lower than the confidence threshold. The reasoning ability of the second model is higher than that of the first model.
9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the dynamic routing method for a large language model as described in any one of claims 1 to 7.
10. A computer storage medium, characterized in that, The system stores computer-executable instructions for performing the large language model dynamic routing method as described in any one of claims 1 to 7.