Plug-in calling system based on large language model

Abstract

The invention discloses a plug-in calling system based on a large language model, and relates to the field of artificial intelligence. The plug-in calling system based on the large language model comprises a server and the large language model, the server receives a question prompt word input by a user and sends the question prompt word to the big language model, and the big language model carries out plug-in calling analysis according to the question prompt word and sends an analysis result to the server; the server calls a corresponding plug-in according to an analysis result, builds a reply cue word according to a calling result of the plug-in, and sends the reply cue word to a large language model, and the large language model generates a final reply based on the reply cue word and the question cue word; and the server outputs the final reply to the user. By means of the mode, plug-in calling of the large language model does not depend on API interface support of the large language model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a plug-in calling system based on a large language model. Background Art

[0002] Existing techniques typically use context-based learning and fine-tuning when calling plugins. This approach guides large language models to call plugins by displaying API specification examples (ICLs) or fine-tuning tool usage data. This approach suffers from high parameter parsing errors. Existing models are limited by the API interface of the large language model itself, often leading to call failures due to parameter format mismatches or semantic understanding errors, requiring post-processing functions for correction. Furthermore, this approach's static toolchain struggles with complex problem scenarios involving multiple tools, resulting in limited scalability. Summary of the Invention

[0003] This application mainly provides a plug-in calling system based on a large language model to solve the problem that plug-in calling is limited by the API interface of the large language model itself.

[0004] To solve the above technical problems, a technical solution adopted in this application is: providing a plug-in calling system based on a large language model, including a server and a large language model; the server receives a question prompt word input by a user and sends it to the large language model, the large language model performs a plug-in call analysis based on the question prompt word, and sends the analysis result to the server; the server calls the corresponding plug-in based on the analysis result, constructs a reply prompt word based on the call result of the plug-in, and sends the reply prompt word to the large language model, the large language model generates a final reply based on the reply prompt word and the question prompt word; the server outputs the final reply to the user.

[0005] In some embodiments, the large language model further analyzes whether a plug-in needs to be called based on the question prompt, selects the name of the plug-in to be called, and analyzes the parameters required for the plug-in to be parsed by the user questioner.

[0006] In some embodiments, when a plug-in needs to be called, the large language model sends an analysis result including plug-in usage, a name of the required plug-in, and a parsed result of plug-in parameters to the server.

[0007] In some embodiments, the server further parses the analysis result to obtain the name of the required plug-in and the parsed result of the required plug-in parameters.

[0008] In some embodiments, the server queries the plug-in corresponding to the name of the required plug-in and the plug-in function of the plug-in in the plug-in list, fills the parsed result of the required plug-in parameters into the plug-in through the plug-in function, executes the call of the plug-in through the plug-in function, and obtains the call result sent by the plug-in.

[0009] In some embodiments, the server further matches the registration information of the plug-in in the plug-in list according to the name of the required plug-in.

[0010] In some embodiments, the registration information of the plug-in includes the plug-in name, the function entry corresponding to the plug-in, and the parameter template of the plug-in.

[0011] In some embodiments, the large language model further determines whether a plug-in needs to be called, and when a plug-in needs to be called, determines whether there is a plug-in that meets the requirements, and when there is a plug-in that meets the requirements, determines whether the question prompt word contains parameters, and when the question prompt word contains parameters, determines whether the calling conditions of the plug-in are currently met, and when the calling conditions of the plug-in are met, generates the analysis result of calling the plug-in, otherwise generates the analysis result of not calling the plug-in.

[0012] In some embodiments, the server generates a text reply to the user based on the final reply.

[0013] In some embodiments, the server extracts the question prompt words from the text input by the user.

[0014] The beneficial effect of the present application is that, different from the existing technology, the present application discloses a plug-in calling system based on a large language model. The question prompt words input by the user are received by the server and sent to the large language model. The large language model performs plug-in call analysis based on the question prompt words and sends the analysis results to the server; the server then calls the corresponding plug-in based on the analysis results, constructs the reply prompt words based on the call results of the plug-in, and sends the reply prompt words to the large language model. The large language model generates a final reply based on the reply prompt words and the question prompt words; the server then outputs the final reply to the user, but by reconstructing the prompt words, it guides the large language model to actively understand and think, and make judgments on the selection and calling of various plug-ins in the plug-in list, and no longer relies on the API interface of the large language model itself to support plug-ins. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive work, among which:

[0016] Figure 1 It is a structural diagram of an embodiment of a plug-in calling system based on a large language model provided by this application. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0018] The terms "first", "second" and "third" in the embodiments of the present application are only used for descriptive purposes and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second" and "third" may explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally also include steps or units that are not listed, or may optionally also include other steps or units inherent to these processes, methods, products or devices.

[0019] References to "embodiments" herein mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0020] See Figure 1 , Figure 1 This is a structural diagram of an embodiment of a plug-in calling system based on a large language model provided by the present application. The plug-in calling system includes a server and a large language model.

[0021] The Large Language Model (LLM) is a natural language processing model based on deep learning technology and trained using massive amounts of text data. It can generate text that conforms to human language habits or understand semantics. Its core concept is to learn language statistics and contextual associations through large-scale unsupervised training, simulating the human language cognition and generation process.

[0022] The server receives the question prompt words entered by the user and sends them to the large language model. The large language model performs plug-in call analysis based on the question prompt words and sends the analysis results to the server.

[0023] The server calls the corresponding plug-in based on the analysis results, constructs a reply prompt word based on the plug-in call result, and sends the reply prompt word to the large language model. The large language model generates the final reply based on the reply prompt word and the question prompt word.

[0024] The server outputs the final response to the user.

[0025] The large language model uses natural language understanding technology to classify the intent of user-entered question prompts. Combined with search-enhanced generation technology, the user's question is vectorized and then matched against the plugin library metadata for similarity, screening suitable plugins. Using dynamic template generation, the large language model is fed meta-prompts containing structured information such as task type, keyword slots, and business context to assist it in making plugin invocation decisions.

[0026] The server calls the corresponding plug-in (such as data query API, multimodal processing module) based on the analysis results of the large language model, and pre-processes the original data returned by the plug-in (such as order information, medical image features), extracts key fields in the plug-in output (such as amount value, timestamp) through slot filling technology, and converts them into JSON or natural language fragments that can be understood by the large language model.

[0027] Combine the plug-in return data with the preset prompt project template, generate a text description through multimodal feature extraction, and construct a new prompt word as the final reply.

[0028] The large language model generates a coherent response based on the prompt and the original question. A self-attention mechanism ensures logical consistency, for example, maintaining the relevance of information like order numbers and logistics status in e-commerce scenarios. The server filters sensitive words and verifies the facts of the final response, for example, using a pre-trained risk detection model to intercept unethical generated content.

[0029] The large language model further analyzes whether a plug-in needs to be called based on the question prompt words, selects the name of the plug-in to be called, and parses the parameters required for the plug-in when the user asks the question.

[0030] Constructing prompt words requires the large language model to actively consider three questions when answering user questions: whether a plug-in needs to be called, which plug-in should be called, and what parameters the plug-in requires, and parse them from the question.

[0031] The large language model uses pre-trained semantic understanding capabilities combined with dynamic templates to classify the intent of user questions.

[0032] For example, when a user asks "What's the weather like in Shanghai today?", the model identifies this as a "real-time information query" request, triggering the plugin's decision logic. If the question involves real-time data, specialized computations, or requires multimodal processing, the plugin is deemed necessary.

[0033] Each plugin in the plugin library has metadata tags defined. The large language model vectorizes the user's question and calculates similarity with the plugin metadata to screen candidate plugins that meet the preset criteria.

[0034] Optionally, the large language model adjusts the priority in combination with business rules. For example, in an e-commerce scenario, the "logistics query" plug-in is called first rather than the "product recommendation" plug-in.

[0035] The large language model converts natural language parameters into the format required by the plugin interface (such as JSON or API request body). For example, "price is less than 500 yuan" is converted to {"price":{"max":500}}. If the user uses a mixture of Chinese and English (such as "Query CPU temperature" and "check CPU temperature"), the model standardizes them into the keywords defined by the interface.

[0036] Optionally, when a plug-in needs to be called, the large language model sends an analysis result including plug-in usage, the name of the required plug-in, and parsing results of plug-in parameters to the server.

[0037] Set fixed return rules. When a plug-in needs to be called, the large language model should return to the server the judgment result of whether to use the plug-in, the selection structure of the plug-in name used, and the parsing result of the plug-in parameters, forming an analysis result that includes the plug-in usage, the name of the required plug-in, and the parsing result of the plug-in parameters.

[0038] Among them, the plug-in name is strictly consistent with the registered name of the plug-in library and needs to be screened from the plug-in metadata through semantic matching; the parameter list: passed in the form of key-value pairs, for example, it must comply with the parameter type and format constraints defined by the plug-in interface; the execution strategy: includes the handling method when parameters are missing and the retry mechanism when the plug-in timeout occurs.

[0039] The server further parses the analysis result to obtain the name of the required plug-in and the parsed result of the required plug-in parameters.

[0040] After the large language model returns as required, the server parses the return result of the large language model and obtains the key information of the calling plug-in, including the plug-in name and required parameters.

[0041] The server searches the plug-in list for the plug-in corresponding to the name of the required plug-in and the plug-in function of the plug-in, fills the parsed results of the required plug-in parameters into the plug-in through the plug-in function, executes the call of the plug-in through the plug-in function, and obtains the call result sent by the plug-in.

[0042] The server retrieves the corresponding plug-in function from the plug-in library according to the plug-in name and converts the parameters parsed by the natural language into the structured data required by the plug-in interface.

[0043] Real-time plug-ins use synchronous invocation, while complex computation plug-ins use asynchronous queue processing. The server converts the raw data returned by the plug-in into text snippets that can be parsed by the large language model.

[0044] The returned results are used to form new prompt words and passed to the large language model, which is required to answer the user's question based on its understanding of the original question and the return results of the plug-in call.

[0045] The server further matches the registration information of the plug-in in the plug-in list according to the name of the required plug-in.

[0046] The registration information of the plug-in includes the plug-in name, the function entry corresponding to the plug-in, and the parameter template of the plug-in.

[0047] The server quickly locates the registration information corresponding to the plug-in name through a pre-built hash table or inverted index, and checks whether the plug-in registration information contains the required fields: plug-in name, function entry address, and parameter template.

[0048] Among them, the plug-in function entry usually exists in the form of a dynamic link library or an independent service endpoint. The server loads the function entry address through the reflection mechanism or the plug-in manager to ensure the runtime compatibility of the plug-in and the main program.

[0049] Specifically, the large language model further determines whether a plug-in needs to be called. If a plug-in needs to be called, it determines whether there is a plug-in that meets the requirements. If there is a plug-in that meets the requirements, it determines whether the question prompt contains parameters. If the question prompt contains parameters, it determines whether the calling conditions of the plug-in are currently met. If the calling conditions of the plug-in are met, it generates an analysis result of calling the plug-in; otherwise, it generates an analysis result of not calling the plug-in.

[0050] When a user request involves real-time data, specialized domain calculations, or external system interactions, the LLM triggers a plug-in call tag. The large language model then calculates the similarity between the user request and the plug-in library metadata to screen candidate plug-ins. Key parameters are extracted using a rule engine or context recognition technology.

[0051] Verify whether the user permissions meet the plug-in requirements. If all conditions are met, generate a structured call instruction; if any condition is not met, return a prompt and attach the reason.

[0052] Optionally, the server generates a text replying to the user based on the final reply.

[0053] Receives plugin call results or error codes and extracts key fields as the basis for text generation. Results are classified into types such as successful response, missing parameters, permission denied, and service exceptions, triggering corresponding generation strategies. Dynamically adjusts response coherence based on conversation history.

[0054] Optionally, the server further extracts question prompt words from the text input by the user.

[0055] Different from the existing technology, this application reconstructs the prompt words through the server side, allowing the large language model to actively understand and think about the plug-in call information. All implementation procedures are set up on the service side of the docking customer. When the user asks a question, the plug-in call can be made as usual, and no changes are required to achieve user-side non-perception of the plug-in call. Through structured prompt words, the model is forced to judge the necessity, name and parameters of the plug-in call step by step, combined with dynamic parsing on the server side, to achieve the unity of flexibility and precision; directly parse the parameters through the large language model and return the structured results, reducing the intermediate correction steps, and the server-side plug-in matching and parameter filling process is standardized to reduce the error rate; no additional fine-tuning or complex training process is required, only the prompt word generation rules and plug-in interface need to be designed, and the deployment is lighter; the plug-in list is dynamically updated, and new tools only need to register the interface description, without adjusting the core logic, and the scalability is stronger.

[0056] The above description is merely an embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A plug-in calling system based on a large language model, characterized in that: Includes servers and large language models; The server receives the question prompt word input by the user and sends it to the large language model. The large language model performs plug-in call analysis based on the question prompt word and sends the analysis result to the server. The server calls the corresponding plug-in according to the analysis result, constructs a reply prompt word according to the plug-in call result, sends the reply prompt word to the large language model, and the large language model generates a final reply based on the reply prompt word and the question prompt word; The server outputs the final reply to the user.

2. The plug-in calling system according to claim 1, characterized in that: The large language model further analyzes whether a plug-in needs to be called according to the question prompt word, selects the name of the plug-in to be called, and analyzes the parameters required for the plug-in to be parsed by the user questioner.

3. The plug-in calling system according to claim 1 or 2, characterized in that: When a plug-in needs to be called, the large language model sends an analysis result including plug-in usage, the name of the required plug-in, and the parsing result of the plug-in parameters to the server.

4. The plug-in calling system according to claim 3, characterized in that: The server further parses the analysis result to obtain the name of the required plug-in and the parsed result of the required plug-in parameters.

5. The plug-in calling system according to claim 4, characterized in that: The server searches the plug-in list for the plug-in corresponding to the name of the required plug-in and the plug-in function of the plug-in, fills the parsed result of the required plug-in parameters into the plug-in through the plug-in function, executes the call of the plug-in through the plug-in function, and obtains the call result sent by the plug-in.

6. The plug-in calling system according to claim 5, characterized in that: The server further matches the registration information of the plug-in in the plug-in list according to the name of the required plug-in.

7. The plug-in calling system according to claim 6, characterized in that: The registration information of the plug-in includes the plug-in name, the function entry corresponding to the plug-in, and the parameter template of the plug-in.

8. The plug-in calling system according to claim 2, characterized in that: The large language model further determines whether a plug-in needs to be called. If a plug-in needs to be called, it determines whether there is a plug-in that meets the requirements. If there is a plug-in that meets the requirements, it determines whether the question prompt word contains parameters. If the question prompt word contains parameters, it determines whether the calling conditions of the plug-in are currently met. If the calling conditions of the plug-in are met, it generates the analysis result of calling the plug-in; otherwise, it generates the analysis result of not calling the plug-in.

9. The plug-in calling system according to claim 1, characterized in that: The server generates a text for replying to the user according to the final reply.

10. The plug-in calling system according to claim 1 or 9, characterized in that: The server extracts the question prompt words from the text input by the user.

Images