Intelligent tool calling method based on knowledge base matching

Abstract

The invention discloses an intelligent tool calling method based on knowledge base matching, and the method comprises the following steps: S1, carrying out the blocking processing of prompt words of an MCP tool set, marking a feature tag for each tool, and forming a tool prompt word knowledge base; s2, obtaining a question text input by a user, carrying out semantic matching on the question text of the user and the feature tag of the tool cue word knowledge base, calculating the similarity and obtaining a corresponding matching result; s3, according to the matching result, entering the tool cue word knowledge base to recognize and automatically output a matched tool name; s4, if the user confirms that the tool name is accurate, tool execution is started; if the user neglects the tool name, displaying a tool list arranged according to a similarity descending order to the user for manual selection; s5, calling a tool confirmed or selected by the user and obtaining an execution result; and S6, summarizing the execution result through a large language model, and returning the summarized execution result to the user. The problem that a user cannot accurately call the optimal MCP tool is solved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically to a method for invoking intelligent tools based on knowledge base matching. Background Technology

[0002] In current application scenarios, when users initiate question-and-answer sessions with large language models, the types of questions they ask are diverse and complex, covering various fields and scenarios. To improve the accuracy of the answers, using more specialized tools to answer different types of questions is an effective strategy. For example, Chinese invention patent, patent number CN 119621882 A, discloses a "Large Language Model Inference System and Method for Adaptive Input Window in RAG Scene". The system is characterized by including a user agent part (1), a vector database VDB part (2), and a large language model part (3). The user agent part (1) is used to obtain the user's query request. The vector database VDB part (2) is connected to the user agent part (1) and the large language model part (3). The vector database VDB part (2) is used to obtain the large language model input window parameters according to the large language model part (3), receive the user's query request, and query the candidate text block group from the local knowledge base. Then, the user's query request and the candidate text block group are filled into the prompt template to obtain a prompt template containing context. The large language model part (3) is used to perform content inference on the prompt template containing context using the corresponding large language model and return the inference result to the user agent part (1). This patent was significant for improvement when RAG was first proposed, primarily because the input window of large language models at that time was very limited to only 2K tokens. RAG typically relied on vector databases (VDBs) to return the most similar text blocks as candidates, failing to fully leverage the rapid advancements in long-context understanding made by LLMs. Therefore, with the significant expansion of input windows today, the traditional RAG mechanism appears rigid and inflexible. VDBs only handle the one-time recall of similar text blocks, while in MCP scenarios, the model often needs to dynamically decide whether to call the search engine again, which tool to call, and in what order during the dialogue. Existing technologies lack a closed loop where the model first judges, then selects the tool, and then determines the parameters, leading to the inability to correctly call tools and support complex tasks involving multiple turns, multiple tools, and branching.

[0003] However, for enterprises and institutions deploying large language models locally, a wealth of application tools built around LLMs are emerging, such as weather forecasting tools, travel planning tools, financial tools for financial statement analysis, policy interpretation assistants, real-time online search modules, and text discrepancy comparison tools. These diverse tools can be collectively referred to as the "MCP toolkit" (Model Context Protocol Toolkit). MCP is an open standard protocol designed to solve the integration problem between large language models (LLMs) and external data sources and tools. It provides a standardized way for AI systems to seamlessly access external data sources, tools, and services.

[0004] Each tool in the MCP toolset is accompanied by prompts, which are crucial for the correct use of the tool. The applicant's research revealed that this patent is not applicable to the MCP toolset. When the large language model receives a user's question, its processing flow is typically: the large language model first relies on its own understanding of the question, combined with its understanding of all the tool prompts in the MCP toolset, to select the appropriate tool to answer the question. However, this approach has revealed some problems in practical applications:

[0005] Firstly, when there are many tools, the prompts in the MCP toolkit are quite lengthy. During model processing, passing all the prompts as context into the large model may result in truncation of the prompt text, which in turn affects the model's complete understanding of the prompts and interferes with the accuracy of tool selection.

[0006] Secondly, the prompts for some tools have certain similarities in content, which causes the large language model to output an inaccurate matching result when distinguishing and selecting, which is not the tool that the user needs to choose, increasing the risk of selecting the wrong tool.

[0007] These problems are intertwined, and together they cause large language models to not necessarily be able to accurately select the most suitable MCP tool in practical applications, thus failing to provide highly accurate answers to users' questions. Summary of the Invention

[0008] To overcome the above-mentioned technical defects, the present invention provides an intelligent tool invocation method based on knowledge base matching.

[0009] To solve the above problems, the present invention is implemented according to the following technical solution:

[0010] The present invention discloses an intelligent tool invocation method based on knowledge base matching, comprising the following steps: S1: Segmenting the prompt words of the MCP tool set, labeling each tool with feature tags to form a tool prompt word knowledge base; S2: Obtaining the user's input question text, semantically matching the user's question text with the feature tags of the tool prompt word knowledge base, calculating the similarity and obtaining the corresponding matching results; S3: Based on the matching results, identifying and automatically outputting the matched tool name in the tool prompt word knowledge base; S4: If the user confirms that the tool name is accurate, starting the tool execution; if the user rejects the tool name, displaying a tool list sorted in descending order of similarity for manual selection; S5: Invoking the tool confirmed or selected by the user and obtaining the execution result; S6: Summarizing the execution result using a large language model and returning it to the user.

[0011] Preferably, the feature tags include, but are not limited to, at least one functional identifier among weather query, traffic route planning, text translation, and image processing.

[0012] Preferably, the semantic matching in step S2 is a vector similarity algorithm, including cosine similarity calculation.

[0013] Preferably, when displaying the tool list in step S4, the percentage of each tool's match with the user's question is also displayed.

[0014] Preferably, the process of forming the tool hint knowledge base in step S1 further includes: periodically expanding the knowledge base content based on a strategy for updating newly added tools or hints.

[0015] Preferably, the tool for invoking user confirmation or selection in step S5 includes: transmitting the user's original question text or structured parameters preprocessed by a large language model through an API interface.

[0016] Preferably, summarizing the execution results in step S6 includes: filtering redundant information, extracting key data, and recombining the output in natural language format.

[0017] Compared with the prior art, the beneficial effects of the present invention are:

[0018] By segmenting, tagging, and inputting toolkit prompts into a knowledge base, the system addresses the issues of lengthy prompts and, to some extent, similarity between prompts. Through RAG orchestration and human intervention, it resolves problems caused by prompt similarity and errors in large-scale model understanding. No client-side code needs to be written for each tool; prompts and feature labels are automatically recognized by the system simply by being entered into the knowledge base. Leveraging RAG's dense vector retrieval capabilities, fine-grained semantic alignment between user questions and prompt blocks is achieved, resulting in a higher Top-1 hit rate than traditional keyword matching. Each tool is tagged differently from others, enabling rapid differentiation even in semantically similar scenarios, significantly improving recall accuracy. When automatic matching fails, the system provides a candidate list ranked by similarity for users to choose from, avoiding the embarrassment of "failure on the first rejection" and balancing the efficiency of automation and human intervention. The large language model provides a structured summary of the raw JSON / log / table multimodal outputs returned by the tools, providing users with clear, intuitive conclusions without requiring extensive technical details. Prompt words and feature labels can be incrementally maintained with each tool version update; the RAG vector library supports minute-level incremental indexing, ensuring that new tools can be retrieved and used immediately after deployment, meeting the needs of rapid iteration. This successfully solves the problem of accurately calling the optimal tool during user question-and-answer processes. Attached Figure Description

[0019] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, wherein:

[0020] Figure 1 This invention provides a flowchart of an intelligent tool invocation method based on knowledge base matching. Figure 1 ;

[0021] Figure 2 This invention provides a flowchart of an intelligent tool invocation method based on knowledge base matching. Figure 2 Detailed Implementation

[0022] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0023] In current application scenarios, when users initiate question-and-answer sessions with large language models, the types of questions they ask are diverse and complex, covering various fields and scenarios. To improve the accuracy of answers, using more specialized tools to answer different types of questions is an effective strategy. However, for enterprises and institutions that deploy large language models locally, they often develop numerous applications or tools around the large language model. For example, financial statement analysis tools for financial data analysis, policy assistant tools to assist in the interpretation of regulations, online search tools for real-time information acquisition, and text comparison tools for conducting text difference comparisons, etc. This collection of tools with different functions and distinctive features can be uniformly named the MCP toolset. It is worth noting that each tool in the MCP toolset is equipped with its own prompt words, which play a key role in the accurate invocation and operation of the tools. The applicant's research found that this patent is not applicable to the MCP toolset. When the large language model receives a question from a user, its processing flow is usually: the large language model first relies on its own understanding of the question, combined with its understanding of all the tool prompt words in the MCP toolset, to select the appropriate tool to answer the question. However, this approach has revealed some problems in practical applications:

[0024] Firstly, since the complete tooltips in the MCP toolkit are quite lengthy, the text of the tooltips may be truncated during model processing, which may affect the model's complete understanding of the tooltips and interfere with the accuracy of tool selection.

[0025] Secondly, the suggestion words of some tools have certain similarities in content. This causes the large language model to output an imprecise matching result when distinguishing and selecting, which may not be the tool the user needs, increasing the risk of selecting the wrong tool. These problems are intertwined, and together they cause the large language model to not necessarily be able to accurately select the most suitable tool in practical applications, thus failing to provide highly accurate answers to the user's questions.

[0026] Example 1: Some tooltip words are lengthy, or even have undone special formatting, such as:

[0027] {"toolId":"JW7y1Nwk6aPdQc0d","description" is "Cycling route planning is used to plan cycling commuting routes. The planning process considers factors such as overpasses, one-way streets, and road closures. It supports a maximum cycling route planning of 500km","parameters":{"type":"object","properties":{"origin":{"type":"string","descript ion":"Starting point latitude and longitude, coordinate format: longitude, latitude","enum":[]},"destination":{"type":"string","description":"Destination latitude and longitude, coordinate format: longitude, latitude","enum":[]}},"required":["origin","destination"]}}.

[0028] Example 2: High similarity of prompts between tools:

[0029] Tool 1 prompt: Provides weather forecast service. Based on the location name provided by the user, it can return detailed information such as real-time weather, weather forecast for the next few days, and temperature changes for that location.

[0030] Tool 2 prompts: Provides travel guide services, which can return information such as popular tourist attractions, food recommendations, transportation guides, and weather features for a location provided by the user.

[0031] Both tools use prompts related to "location name" and "weather," exhibiting a degree of similarity. For example, when a user enters the text "Jiuzhaigou weather," their true intention is to obtain weather forecast information for Jiuzhaigou. However, the Large Language Model (LLM) interprets user questions based on its own knowledge base and algorithmic logic. Due to the similarity of the tool prompts, the LLM might incorrectly associate the user's question with travel guide services, thus selecting tool 2 and ultimately returning travel guide information for Jiuzhaigou instead of the weather forecast information the user expects.

[0032] Example 3: Comprehension bias in large language models

[0033] Tool 1 prompt: The driving route planning API can plan commuting routes for passenger cars and sedans based on the latitude and longitude coordinates of the origin and destination provided by the user, and return commuting route data.

[0034] Tool 2 prompts: It supports converting detailed structured addresses into latitude and longitude coordinates, and can resolve landmark scenic spots and building names into latitude and longitude coordinates.

[0035] When a user inputs the text "Guangzhou Tower, Sun Yat-sen Memorial Hall," their intention is to obtain information about these two attractions. However, the large LLM model might misinterpret the user's intent, interpreting it as a request for public transportation routes to these locations, and thus outputting public transportation information for both places, which does not match the user's actual needs.

[0036] like Figures 1-2 As shown, the intelligent tool invocation method based on knowledge base matching according to the present invention includes the following steps:

[0037] S1: Divide the prompt words of the MCP toolset into blocks, label each tool with feature tags, and form a tool prompt word knowledge base;

[0038] S2: Obtain the question text input by the user, perform semantic matching between the user question text and the feature tags of the tooltip word knowledge base, calculate the similarity and obtain the corresponding matching results;

[0039] S3: Based on the matching results, enter the tool suggestion word knowledge base to identify and automatically output the matching tool name;

[0040] S4: If the user confirms that the tool name is accurate, then the tool will be started; if the user rejects the tool name, then a list of tools sorted in descending order of similarity will be displayed to the user for manual selection.

[0041] S5: Invoke the tool confirmed or selected by the user and obtain the execution result;

[0042] S6: Summarize the execution results using a large language model and return them to the user.

[0043] Understandably, a tooltip knowledge base is first constructed, and tooltips are segmented. The complete tooltip for each tool (including function description, parameter format, example calls, etc.) is divided into several "tooltip blocks" based on semantic integrity, with each block controlled to 256-512 tokens to avoid information overload. Significant feature tag extraction: Keyword extraction (KeyBERT) + large model summarization is used to generate 3-5 highly distinctive feature tags for each tool (e.g., "weather - real-time - city level", "travel - guide - attractions"). Vectorization and storage: The same embedding model (egbge-large-zh-v1.5) is used to generate 1024-dimensional vectors for tooltip blocks, feature tags, and tool names respectively; a three-index system is constructed: the primary index is the tooltip vector stored in the Milvus library; indexes two and three are the feature tag vector and tool name vector, respectively, both stored in the FAISS library. Metadata association: The knowledge base stores the correspondence between "tool ID - tool name - tooltip block - feature tag - version number" for easy tracing and hot updates. Examples are shown in Table 1.

[0044] Table 1

[0045]

[0046]

[0047] User Question Understanding and Retrieval: The text is preprocessed to remove stop words, standardize capitalization, and correct spelling errors. The user questions are also vectorized, and the similarity scores S_name (with the tool name), S_tag (with the feature tags), and S_chunk (with the prompt chunks) are calculated.

[0048] Perform similarity reordering: Rearrange name similarity S_name, feature tag similarity S_tag, and cue word similarity.

[0049] Weighted fusion is performed using S_chunk: S_final = α·S_name + β·S_tag + γ·S_chunk, where α + β + γ = 1, with default values ​​of α = 0.3, β = 0.3, and γ = 0.4; the tool with the highest S_final value is selected as the "optimal candidate". Examples are shown in Table 2.

[0050] Table 2

[0051] Tool ID S_final 001 0.88 002 0.32

[0052] The system displays the tool name, a one-sentence summary of its function, parameter examples, and a similarity percentage (S_final × 100) to the user. In one embodiment, if the user clicks "Confirm," the tool is launched; if the user clicks "Not this one," a drawer-style list is displayed, showing ≤5 candidate tools in descending order of S_final; if no tool is found that the user prefers, the user clicks "More," and all candidate tools are then displayed in descending order of S_final.

[0053] LLM is used to extract slots for questions and automatically fill in the parameters required by the tool; the tool is called via the MCP protocol; if an HTTP 5xx error is returned, it automatically retryes once, with a timeout threshold of 10 seconds. The result is cached in Redis as a triple {tool ID, parameter MD5, result} with a TTL of 1 hour to reduce duplicate calls. Sensitive information is filtered, and encoding is standardized. The original result and the user question are concatenated into a prompt: "The user question is: {user question}, and the result obtained by the tool is: {original result}. Please summarize in Chinese to provide feedback to the user, retaining key figures and conclusions."

[0054] The front-end renders the summary text; it provides a "Follow-up Questions" button, allowing users to ask further questions based on the same context without repeatedly calling the tool.

[0055] In one embodiment, optional closed-loop optimization is available, recording a full-link log from "user question - recommended tool - whether to adopt - final selection".

[0056] Furthermore, the feature tags include, but are not limited to, at least one functional identifier among weather queries, route planning, text translation, and image processing. In one embodiment, for each new tool, at least one functional tag needs to be automatically generated or manually specified during the registration phase; if existing tags cannot accurately describe the tool, new tags are added according to the "domain-action-qualifier" rule, and the knowledge base and secondary index are updated simultaneously.

[0057] Furthermore, the semantic matching in step S2 is a vector similarity algorithm, including cosine similarity calculation.

[0058] In one embodiment, the cosine similarity calculation process is as follows: Given two vectors A and B with the same dimension, the cosine similarity between A and B is defined as:

[0059]

[0060] Where A·B represents the dot product of vectors A and B, and ||A|| and ||B|| represent the magnitudes (i.e., the lengths) of vectors A and B, respectively. The closer the cosine value is to 1, the closer the directions of the two vectors are; the closer the cosine value is to 0, the more perpendicular the two vectors are, i.e., the lower the similarity. The tool with the highest similarity is selected as the "optimal match" output.

[0061] After the similarity calculation is complete, the tool with the highest similarity is displayed to the user, awaiting their confirmation. If the user chooses the tool, it is used directly to complete the subsequent steps. Otherwise, the user proceeds to manually select a tool.

[0062] Furthermore, in step S4, when displaying the list of tools, the similarity percentage between each tool and the user's question is also displayed.

[0063] Understandably, mapping abstract similarity to intuitive percentages ranging from 0% to 100% allows users to quickly determine the suitability of various tools for their needs without needing to understand the concept of vector space, significantly reducing manual selection time. The percentages increase in 1% increments, and in one implementation, a color gradient can be used to differentiate between "high-match recommendations" and "low-match candidates" at a glance, reducing the probability of incorrect selection.

[0064] Furthermore, the process of forming the tooltip knowledge base in step S1 includes: periodically expanding the knowledge base content based on a strategy that updates newly added tools or tooltips.

[0065] The system performs periodic scans, automatically scanning the official MCP repository every 24 hours to detect new tools or changes in the version number of prompt words; tool authors can also report updates in real time, triggering incremental addition to the repository within 10 minutes.

[0066] In one embodiment, the dynamic expansion process is as follows:

[0067] Chunking and vectorization: For newly added or revised prompt words, the "chunking-embedding" process is performed in the same way as the initial construction, ensuring the continuity of the semantic space;

[0068] Tag alignment: If the new tool has functions that are not covered, generate new tags according to the established naming conventions and update the tag dictionary synchronously.

[0069] Hot-plugging of indexes allows for the addition of new vectors and the overwriting of old vectors without system downtime, utilizing the online insertion interface of Milvus / FAISS.

[0070] Business systems can detect new tools without restarting, reducing operational costs; a unified embedding model and block strategy ensure comparability between new and old vectors, avoiding retrieval drift; as the tool ecosystem expands, the knowledge base expands in real time, ensuring users always receive the latest and most accurate function matching results.

[0071] Furthermore, the tool for invoking user confirmation or selection in step S5 includes: transmitting the user's original question text or structured parameters preprocessed by a large language model through an API interface.

[0072] No need for tool developers to standardize parameter formats; for tools that support raw text, LLM preprocessing is omitted, saving an average of 150-300ms and reducing latency; for tools that heavily rely on field validation, structured parameters reduce parsing error rates by more than 90%; both raw text and structured parameters are recorded in the interface logs, facilitating subsequent error tracing and model fine-tuning.

[0073] Furthermore, summarizing the execution results in step S6 includes: filtering redundant information, extracting key data, and recombining the output in natural language format.

[0074] The results summary in step S6 includes the following:

[0075] Redundant information is filtered out based on fields such as internal trace-id and debugging logs, eliminating content with no business value; information density is determined by a lightweight SLM, and low-value paragraphs such as duplicate descriptions and advertising disclaimers are deleted entirely.

[0076] Perform key data extraction, generate "key field prompts" through LLM, and accurately extract core entities such as values, time, and status codes from the JSON, HTML, or plain text returned by the tool; perform numerical range and format validation on the extracted results, and trigger secondary confirmation or marking for abnormal values.

[0077] The natural language reconstruction output populates the extracted key data into a preset natural language template, such as "We have checked the weather in Shanghai for tomorrow: cloudy, high of 32℃, low of 26℃, probability of precipitation 20%".

[0078] The sentence structure is dynamically adjusted based on the user's historical preferences (formal / spoken / list-style) to make the reading experience more closely match personal habits.

[0079] Other structures for the knowledge base matching-based intelligent tool invocation method described in this embodiment can be found in existing technologies.

[0080] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Therefore, any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for invoking intelligent tools based on knowledge base matching, characterized in that, Includes the following steps: S1: Divide the prompt words of the MCP toolset into blocks, label each tool with feature tags, and form a tool prompt word knowledge base; S2: Obtain the question text input by the user, perform semantic matching between the user question text and the feature tags of the tooltip word knowledge base, calculate the similarity and obtain the corresponding matching results; S3: Based on the matching results, enter the tool suggestion word knowledge base to identify and automatically output the matching tool name; S4: If the user confirms that the tool name is accurate, then the tool will be started; if the user rejects the tool name, then a list of tools sorted in descending order of similarity will be displayed to the user for manual selection. S5: Invoke the tool confirmed or selected by the user and obtain the execution result; S6: Summarize the execution results using a large language model and return them to the user.

2. The intelligent tool invocation method based on knowledge base matching according to claim 1, characterized in that, The feature tags include, but are not limited to, at least one functional identifier from weather query, traffic route planning, text translation, and image processing.

3. The intelligent tool invocation method based on knowledge base matching according to claim 1, characterized in that, The semantic matching described in step S2 is a vector similarity algorithm, including cosine similarity calculation.

4. The intelligent tool invocation method based on knowledge base matching according to claim 1, characterized in that, When displaying the tool list in step S4, the percentage of each tool's match with the user's question is also shown.

5. The intelligent tool invocation method based on knowledge base matching according to claim 1, characterized in that, Step S1, which involves forming the tooltip knowledge base, also includes periodically expanding the knowledge base content based on a strategy that updates new tools or tooltips.

6. The intelligent tool invocation method based on knowledge base matching according to claim 1, characterized in that, The tools for invoking user confirmation or selection in step S5 include: transmitting the user's original question text or structured parameters preprocessed by a large language model through an API interface.

7. The intelligent tool invocation method based on knowledge base matching according to claim 1, characterized in that, The summary of the execution results in step S6 includes: filtering redundant information, extracting key data, and recombining the output in natural language format.

Images