A method and system for handling degradation of large language models based on pre-built knowledge base rollback and multi-provider switching
By prioritizing and queuing large language model providers and switching to a pre-built knowledge base in case of failure, the problem of AI function interruption caused by single point of failure and high costs from multiple providers is solved, achieving cost-effectiveness and service continuity.
Patent Information
- Application Number
- CN202611340370.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-01
- Publication Date
- 2026-11-17
AI Technical Summary
A single point of failure can cause the entire AI function to stop, and parallel calls from multiple providers can double the cost.
During system initialization, interface keys for multiple large language model providers are configured, and they are called in a priority queue. In case of failure, the system switches to a pre-built knowledge base for keyword matching. When the provider interface is disconnected, the pre-built knowledge base in memory is used for degradation processing.
Under normal circumstances, the cost is close to that of a single provider, and professional feedback can still be provided during failures. Degradation query speed is fast, it does not rely on external IO, and it ensures uninterrupted service.
Smart Images

Figure CN123114869A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence service reliability technology, specifically relating to a method and system for degrading large language models based on pre-built knowledge base rollback and multi-provider switching. Background Technology
[0002] In application systems based on large language models, model inference is an indispensable link in the entire business chain, and this happens to be tied to someone else's cloud API. Service provider maintenance, API quota exhaustion, or link failures—any one of these can cause the inference service to stop abruptly. What does it look like for the user? A researcher submits a complex problem they've spent ages pondering, only to receive a "service unavailable" message on the screen, rendering all their previous efforts futile. There are generally two solutions at present. One approach is to only connect to one service provider. If the service is interrupted, the system can only send the user an error page or a message. The AI function will come to a complete standstill, and there is no way out—after all, there is no backup provider. Another approach is to send the same request to several service providers simultaneously, using whichever one responds first. While this does improve availability, the cost doubles: each user's question consumes API quotas from several providers, each with different billing standards, making cost control impossible at the end of the month. Summary of the Invention
[0003] To address the problem that a single point of failure can cripple the entire AI function and that parallel calls from multiple providers can double the cost, this invention provides a method and system for handling the degradation of a large language model based on a pre-built knowledge base rollback and multi-provider switching, with the goal of ensuring uninterrupted service even in extreme situations. The technical solution of this invention is roughly as follows: When the system is initialized, the interface key configurations of various large language model providers are first determined, and a list of available providers is set up according to priority; when a user queries, the system tries each provider one by one starting from the provider with the highest priority, and uses the result of whichever provider succeeds; when none of them succeed, the system returns to the pre-set knowledge base in memory to perform keyword matching and submits the pre-written professional proposition text. Compared with the prior art, the present invention has the following advantages. Let's start with the cost. The providers aren't called in parallel; instead, they're prioritized and tested one by one. On normal days, the system only interacts with the first provider in line, incurring no additional API fees. Only when the first provider fails will the next one take over. In other words, most of the time, the cost of this solution is no different from using only one provider; the extra cost is the backup call during a failure. Now let's consider the worst-case scenario. Even if all interfaces are unavailable, the network is completely down, or all quotas are used up at once, the system still has a trump card—a pre-built knowledge base in memory. This base contains complete answers to several frequently asked questions in the application domain (such as lung cancer biomedical research), written by domain experts. It includes complete assumptions, reasoning, mechanism explanations, and confidence assessments—its quality is no less than, and may even be better than, content written directly by the model. Users can't tell whether this information is pre-built or generated on-the-spot; they'll still receive the professional feedback they need. There's no need to worry about speed either. The entire knowledge base is loaded into memory all at once during system initialization. Simply put, it's a mapping table from keywords to pre-defined proposition objects. Degradation queries don't touch files, connect to databases, or perform any external I / O. Matching and response are completed in the blink of an eye. Even in extreme situations—when all external services are paralyzed and all requests flood the degradation path—the response speed of this path will not slow down. Attached Figure Description Figure 1 This is a flowchart of the LLM downgrade processing method based on pre-set knowledge base rollback and multi-provider switching in an embodiment of the present invention.
[0004] Figure 2 This is a block diagram of the module structure of the LLM degradation processing system based on pre-set knowledge base rollback and multi-provider switching in an embodiment of the present invention. Figure 1 The meanings of the various symbols in the text are as follows: 101 - Detect multiple LLM provider interface key configurations during initialization; 102 - Sort providers with non-empty keys into an available list by priority; 103 - Receive user queries and send requests to use the default provider; 104 - Check if the default provider successfully determines the node; 105 - Return the inference result; 106 - Check if there is another provider determination node in the available list; 107 - Try the next priority provider in sequence; 108 - Search for a matching preset proposition in the preset knowledge base; 109 - Check if a preset proposition determination node is matched; 110 - Return the preset proposition text; 111 - Return the bootstrap configuration prompt text. Figure 2 The meanings of the various symbols in the text are as follows: 201 - Initialization module, 202 - Inference call module, 203 - Degradation processing module. Detailed Implementation The following is combined Figure 1 and Figure 2 Go through the entire downgrade process from beginning to end. This embodiment assigns two candidate large language model service providers to the system: the first provider is a domestic large language model service, with its interface key stored in the DEEPSEEK_API_KEY field of the configuration file; the second provider is an international large language model service, with its key stored in the OPENAI_API_KEY field. During initialization... Figure 1 As shown in marks 101 to 102, check these two fields one by one and arrange the providers whose keys are not empty into an available list according to priority. During system startup, initialization is performed first, reading two configuration fields one by one. If DEEPSEEK_API_KEY is not empty, the first provider is added to the available list and assigned the role of default provider; if OPENAI_API_KEY is also not empty, the second provider is then added to the queue. Conversely, if DEEPSEEK_API_KEY is empty, the first provider is skipped, and the second provider moves to the front of the queue and becomes the default provider. What if both are empty? The available provider list becomes an empty table. The pre-built knowledge base needs to be clearly distinguished here: it is hard-coded into the source code of the global singleton class as a Python dictionary structure, and is loaded into memory once during system initialization. It is not temporarily assembled, nor is it read from a file during runtime. The library stores two sets of core mappings. One set is under the keyword "FBXO45", with a pre-built proposition object bearing the title "FBXO45 affects the stemness of lung cancer stem cells by regulating the Wnt / β-catenin pathway through the PUM1 / PUM2 axis", and the body text is a complete scientific hypothesis text of about 1,500 characters; the other set is under "EGFR", with the title "Bypass activation of the PI3K / AKT / mTOR pathway leads to TKI resistance in EGFR-mutant NSCLC", and the body text is about 1,200 characters. A user typed "What is the regulatory mechanism of FBXO45 in lung cancer?" in the chat interface and pressed enter. The response was as follows: Figure 1 The path marked 103 to 107 indicates that the request is first handed over to the interface client of the default provider—that is, the first provider—for inference. If you are lucky, it will return a 200 status code and a response body containing the inference text within the preset timeout period. This text is then returned to the user as is, and the process ends here. Nothing more needs to be done. However, if no response is received within the preset timeout period—either the network is down or the other party sends back a non-200 status code—the system will then switch to calling the interface of the second provider in the available list. If the second provider can handle the call, the result will still be delivered to the user; if not, it will proceed to the next hop. This downgrade path is as follows Figure 1As shown in marks 108 to 111: When both providers fail (all APIs are unreachable), the system immediately plunges into a pre-built knowledge base degradation path. The user's query text is first converted to lowercase "What is the regulatory mechanism of fbxo45 in lung cancer?", and then compared group by group along the mapping table. Is the first keyword "fbxo45" a substring of the query string? If yes, it's a match, and the complete scientific hypothesis text from the first group of pre-built proposition objects is immediately sent out, and the traversal stops there. There's also an even more extreme usage: no API key is configured in the environment at all—both fields are empty during initialization, and the available list is empty—the first call skips even the provider step and directly falls into the pre-built knowledge base degradation path. If the user happens to ask a question completely unrelated to biomedicine, and none of the keywords in the database match, the system will simply reply with a guiding text: "The current AI engine is not connected. Please set the API key in the configuration file and restart the service." Combination Figure 2 Looking at the system's framework, three modules support the entire degradation system. The initialization module (marked 201) checks the interface key configuration status of each candidate provider during system startup, prioritizing available ones. The actual work is done by the inference and invocation module (marked 202): it receives user query requests, first infers from the highest-ranked provider's interface; if the top provider fails, it moves down the priority list to find the next one. The degradation processing module (marked 203) stands at the final checkpoint—when all available interfaces fail, it automatically switches to a pre-defined knowledge base fallback path, retrieving a pre-defined proposition from the keyword mapping that matches the query text and submitting it.
Claims
1. A method for degrading large language models based on pre-built knowledge base rollback and multi-provider switching, characterized in that, include: During the system initialization phase, the interface key configuration fields of at least two candidate large language model service providers are checked sequentially to see if they contain non-empty values. The providers with non-empty interface keys are arranged in descending order of their preset priority to form a list of available providers. The provider ranked first in the list of available providers is taken as the default provider, and the preset knowledge base is loaded into the system memory all at once. Receive the user's query text and send an inference request using the interface of the default provider; If the inference request of the default provider fails, the next provider in the list of available providers after the default provider will be tried in turn until the inference request of a provider succeeds or all providers in the list have been tried. The conditions for the inference request to fail include at least one of the following: the interface key is not configured, the network connection times out, the interface returns a non-success status code, and the interface returns a response body containing error information. If all inference requests from all providers fail, keyword matching is performed directly on the pre-set knowledge base in the memory. The keyword matching is specifically as follows: the user query text is converted to a lowercase string, the keyword strings of each mapping in the pre-set knowledge base are traversed, and each keyword string is converted to lowercase and then it is determined whether it is a substring of the user query lowercase string. The traversal stops when the first match is found. If a match is found, the matched pre-set proposition object is returned as the inference result. If no match is found, a prompt text for configuring the key is returned. The pre-built knowledge base is stored in the system memory, storing multiple sets of mapping relationships between keywords and pre-built proposition objects. Each mapping consists of at least one keyword string and one pre-built proposition object. The pre-built proposition object includes a hypothesis title field, a complete reasoning text field, and a confidence score field, and its content is a complete scientific proposition response written in advance by humans. The pre-built knowledge base is loaded into memory from code constants once during the system initialization phase, and does not involve file reading or database access during subsequent downgrade queries.
2. The method according to claim 1, characterized in that, The candidate large language model service providers include at least a first provider and a second provider, with the first provider having a higher preset priority than the second provider; the first provider's interface key is stored in the configuration field corresponding to the first provider, and the second provider's interface key is stored in the configuration field corresponding to the second provider.
3. The method according to claim 1, characterized in that, The conditions under which the inference request returns failure include any one of the following: the interface key is not configured, the network connection times out, the interface returns a non-success status code, or the interface returns a response body containing error information.
4. The method according to claim 1, characterized in that, The complete reasoning text of the pre-set proposition object includes the hypothesis title, reasoning basis, mechanism explanation, and confidence assessment.
5. A large language model degradation processing system based on pre-built knowledge base rollback and multi-provider switching, characterized in that, include: The initialization module is used to detect the interface key configuration status of candidate large language model service providers, build a list of available providers arranged by priority, and load the pre-set knowledge base into memory at once; the pre-set knowledge base is a mapping table in memory from keywords to pre-set proposition objects, and the pre-set proposition objects include hypothesis title fields, complete inference text fields, and confidence score fields; The inference call module is used to receive the user's query text and attempt to call the interfaces of each provider in order of priority from the list of available providers; The degradation processing module is used to directly perform keyword substring matching on the preset knowledge base in memory and return the matched preset proposition text when all inference requests from all providers return failures; wherein, the keyword substring matching is performed by converting the user query text and each keyword string to lowercase before judging the substring, and stopping the traversal when the first match is found.
6. The system according to claim 5, characterized in that, The degradation processing module is also used to: when there is no pre-set proposition text matching the query text in the pre-set knowledge base, return a guiding text prompting the user to configure the interface key.
7. A computer device comprising a processor and a memory, characterized in that, The memory stores a computer program, and when the processor executes the computer program, it implements the method according to any one of claims 1 to 4.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 4.
