Question answering method and system based on tool calling, electronic equipment and storage medium

Abstract

The invention discloses a question and answer method and system based on tool calling, electronic equipment and a storage medium. The method comprises the steps that a user question and answer interface obtains question information of a user and sends the question information to a service layer; the tool retrieval module screens out IDs of candidate tools from existing tools according to the question information to obtain a candidate tool list; the tool calling module calls a large language model based on the candidate tool list and the question information; the big language model screens out an ID of a target tool associated with the question information from the candidate tool list, and generates a tool calling instruction based on the ID of the target tool; the tool calling module calls a target tool according to the tool calling instruction and obtains output parameters of the target tool; and the tool calling module calls a large language model to generate reply content based on the output parameters and the question information. The candidate tools are screened out through the tool retrieval model, then the large language model is called based on the ID of the candidate tools and the question information to determine the target tool, and the accuracy and comprehensiveness of reply content can be improved.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to a question-answering method, system, electronic device and storage medium based on tool calling. Background Art

[0002] With the rapid development of artificial intelligence technology, large language models have demonstrated powerful text generation capabilities in the field of natural language processing. In layman's terms, they can provide answers to questions or queries. However, when faced with complex or field-specific queries, a single large language model often finds it difficult to provide accurate and comprehensive answers. For this reason, introducing external knowledge bases and external tool calling mechanisms has become a key way to improve model performance.

[0003] In the prior art, the autonomous selection of external tools is mainly achieved by providing prompt words with tool description information to a large language model. However, this solution exposes significant limitations when calling tools in open scenarios. Specifically, on the one hand, the number of optional tools involved in such scenarios is often huge, and due to the limitation of the token length when processing with a large model, it becomes impractical to integrate the detailed information of all tools into the prompt words. On the other hand, if a large language model is used to screen tools layer by layer, the process is not only cumbersome, but also inefficient. It is difficult to ensure efficient and accurate calling of a variety of external tools in complex and changing scenarios, that is, it is difficult to ensure the accuracy and comprehensiveness of the response content. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a question-answering method, system, electronic device and storage medium based on tool calling.

[0005] In a first aspect, the present invention provides a question-answering method based on tool calling, which is applied to a question-answering system, wherein the question-answering system includes an application layer, a service layer and a large language model, wherein the application layer includes a plurality of tools and a user question-answering interface, the tool includes a question-answering model, and the service layer includes a tool retrieval module and a tool calling module; the question-answering method based on tool calling includes:

[0006] The user question and answer interface obtains the user's question information and sends it to the service layer;

[0007] The tool search module selects the IDs of candidate tools from the existing tools according to the question information to obtain a candidate tool list;

[0008] The tool calling module calls the large language model based on the candidate tool list and the question information;

[0009] The large language model selects the ID of the target tool associated with the question information from the candidate tool list, and generates a tool calling instruction based on the ID of the target tool;

[0010] The tool calling module calls the target tool according to the tool calling instruction and obtains the output parameters of the target tool;

[0011] The tool calling module calls the large language model based on the output parameter and the question information;

[0012] The large language model generates reply content and feeds it back to the user question and answer interface.

[0013] In a second aspect, the present invention provides a question-answering system, comprising an application layer, a service layer and a large language model, wherein the application layer comprises a plurality of tools and a user question-answering interface, the tool comprises a question-answering model, and the service layer comprises a tool retrieval module and a tool calling module;

[0014] The user question and answer interface is used to obtain the user's question information and send it to the service layer;

[0015] The tool search module is used to filter out the IDs of candidate tools from the existing tools according to the question information to obtain a list of candidate tools;

[0016] The tool calling module is used to call the large language model based on the candidate tool list and the question information;

[0017] The large language model is used to filter out the ID of the target tool associated with the question information from the candidate tool list, and generate a tool calling instruction based on the ID of the target tool;

[0018] The tool calling module is used to call the target tool according to the tool calling instruction and obtain the output parameters of the target tool;

[0019] The tool calling module is used to call the large language model based on the output parameter and the question information;

[0020] The large language model is used to generate reply content and feed it back to the user question and answer interface.

[0021] In a third aspect, the present invention provides an electronic device, the electronic device comprising:

[0022] at least one processor; and

[0023] a memory communicatively connected to the at least one processor; wherein,

[0024] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the question-and-answer method based on tool calling described in the first aspect of the present invention.

[0025] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a processor to implement the tool-call-based question-and-answer method described in the first aspect of the present invention when executed.

[0026] The embodiment of the present invention provides a question-and-answer method based on tool calling, which is applied to a question-and-answer system. The question-and-answer system includes an application layer, a service layer and a large language model. The application layer includes multiple tools and a user question-and-answer interface. The tool includes a question-and-answer model. The service layer includes a tool retrieval module and a tool calling module. The user question-and-answer interface obtains the user's question information and sends it to the service layer. The tool retrieval module filters out the IDs of candidate tools from existing tools according to the question information to obtain a candidate tool list. The tool calling module calls the large language model based on the candidate tool list and the question information. The large language model filters out the ID of a target tool associated with the question information from the candidate tool list, and generates a tool calling instruction based on the ID of the target tool. The tool calling module calls the target tool according to the tool calling instruction to obtain the output parameters of the target tool. The tool calling module calls the large language model based on the output parameters and the question information. The large language model generates a reply content and feeds it back to the user question-and-answer interface.

[0027] The advantage of this question-answering method is that there are a large number of tools. The tool retrieval model is first used to screen out candidate tools associated with the question information from a large number of existing tools, and then the large language model is called based on the ID of the candidate tool and the question information to determine the target tool. This can not only ensure the applicability of the obtained target tool to the question information, but also reduce the amount of input data of the large language model, reduce the task of tool screening for the large language model, and improve data processing efficiency. The large language model is not limited by the token length of the input information volume, and can also efficiently and accurately call external tools in different scenarios. On this basis, the question information is answered in combination with the target tool and the large language model, which can improve the accuracy and comprehensiveness of the reply content.

[0028] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 is a structural diagram of a question-answering system provided by an embodiment of the present invention;

[0031] Figure 2 is a flow chart of a question-answering method based on tool calling provided by an embodiment of the present invention;

[0032] Figure 3 is a flowchart of a question-and-answer method provided by an embodiment of the present invention;

[0033] Figure 4 A schematic diagram of the participation of content in tool information at different stages of a system provided by an embodiment of the present invention;

[0034] Figure 5 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only embodiments of a part of the present invention, not all embodiments. According to the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0036] In one embodiment of the present invention, a question-answering method based on tool calling is provided. This embodiment is applicable to situations where a large language model is used to perform tool calling to answer user questions. The method can be executed by a question-answering system. The question-answering system can be implemented in the form of hardware and / or software. The question-answering system can be configured in an electronic device. Figure 1 A structural diagram of a question-answering system provided in Embodiment 1 of the present invention is shown in FIG. Figure 1 As shown, the question-answering system includes: an application layer 100, a service layer 200 and a large language model 300, the application layer 100 includes multiple tools 101 and a user question-answering interface 102, the tool 101 includes a question-answering model, and the service layer 200 includes a tool retrieval module 201 and a tool calling module 202.

[0037] Among them, the essence of tool 101 is a plug-in that includes a question-answering model. There are a large number of tools, and the tool types can cover database queries, API calls, professional software interfaces and other types to meet the query needs of different fields. Different tools can be used to process question information in different scenarios. For example, tool 1 is used to process weather-related question information, tool 2 is used to process traffic-related question information, and tool 3 is used to process food-related question information. It can be seen that there are many types of tools, and the applicable question scenarios are also different.

[0038] Figure 2 A flowchart of a question-answering method based on tool calling provided by an embodiment of the present invention, such as Figure 2 As shown, the question-answering method based on tool invocation includes:

[0039] S201. The user question and answer interface obtains the user's question information and sends it to the service layer.

[0040] The user question and answer interface is equivalent to an interface for users to input information. The user question and answer interface can be a device with a user interaction interface and an input device. The input device can be a keyboard, mouse, recorder, etc. The user can ask questions by clicking the keyboard, mouse or voice input, and obtain feedback reply information through the user interaction interface.

[0041] S202: The tool search module selects IDs of candidate tools from existing tools according to the question information to obtain a list of candidate tools.

[0042] After obtaining the question information, the user question and answer interface transmits it to the service layer. There are many types of tools. The tool retrieval module in the service layer selects the IDs of candidate tools from the existing tools to form a list of candidate tools.

[0043] In an optional embodiment, if Figure 1 As shown, the service layer 200 includes a knowledge base 203 , and the knowledge base 203 stores tool information of tools.

[0044] When in the offline stage, the knowledge base of the tool is constructed, including the construction of the standard vector associated with each tool, specifically obtaining the vocabulary related to the tool and extracting the semantic feature vector of the vocabulary as the standard vector of the tool. By storing the tool information in vector form, the retrieval efficiency and accuracy of the tool information are improved.

[0045] When in the online stage, the tool retrieval module performs tool retrieval based on RAG (Retrieval-Augmented Generation) technology and tool information in the knowledge base. RAG is a technology that enhances the performance of large language models (LLMs) by integrating external knowledge bases. RAG improves the accuracy of LLMs in processing domain-specific or highly specialized queries by combining information retrieval mechanisms and contextual learning. Its workflow mainly includes three key steps: dividing the corpus into discrete blocks, building a vector index, identifying and retrieving blocks based on vector similarity with the query and index blocks, and finally synthesizing a response.

[0046] Optionally, the tool information includes a tool description and an input parameter name. Based on the tool information of the tool stored in the knowledge base 203, the tool retrieval module selects the ID of the candidate tool from the existing tools according to the question information, including: the tool retrieval module extracts a feature vector from the question information; extracts the tool description and the input parameter name in the tool information as a standard vector, and calculates the similarity between the feature vector and the standard vector; and uses the ID of the tool associated with the standard vector whose similarity is greater than a preset similarity threshold as the ID of the candidate tool.

[0047] The standard vector can be a characteristic representation of the applicable scenario of the tool. By determining the candidate tools through the similarity of the vectors, the tool that matches the applicable scenario of the current question information can be obtained. Specifically, the similarity threshold can be set according to actual needs, for example, the similarity threshold can be 85%. The number of candidate tools is usually multiple.

[0048] S203: The tool calling module calls the large language model based on the candidate tool list and the question information.

[0049] A large language model refers to a deep learning model trained with a large amount of text data, which can generate natural language text or understand the meaning of language text. Large language models can handle a variety of natural language tasks, such as text classification, question answering, and dialogue, and are an important path to artificial intelligence. Currently, large language models use a Transformer architecture and pre-training objectives (such as Language Modeling) similar to small models. The difference from small models is the increase in model size, training data, and computing resources.

[0050] After determining the candidate tool list, since there are still many candidate tools, a large language model is needed to screen out at least one target tool that matches the application scenario of the question information from the candidate tools. Therefore, the tool calling module calls the large language model based on the candidate tool list and the question information, that is, generates input information of the large language model according to the candidate tool list and the question information and sends it to the large language model.

[0051] In an optional embodiment, the service layer includes a knowledge base, which stores tool information of the tool, including tool description, input parameter name and instruction template. The tool calling module calls the large language model based on the candidate tool list and question information, including:

[0052] The tool calling module obtains the tool information corresponding to the tool ID in the candidate tool list from the knowledge base to obtain the candidate tool information, which includes the tool description, input parameter name and instruction template of the candidate tool; generates the first prompt word according to the preset first prompt word template, the candidate tool list, the candidate tool information and the question information; and calls the large language model based on the first prompt word.

[0053] Optionally, the ID in the candidate tool list and the corresponding candidate tool information and question information are filled into a preset prompt word template to obtain a first prompt word. The function of the first prompt word is to "prompt" the large language model to determine the target tool with the highest degree of adaptability to the question information from the candidate tools based on the tool information.

[0054] S204: The large language model selects the ID of the target tool associated with the question information from the candidate tool list, and generates a tool calling instruction based on the ID of the target tool.

[0055] After the large language model is called, the large language model selects the ID of the target tool associated with the question information from the candidate tool list according to the input data, and generates a tool calling instruction based on the ID of the target tool.

[0056] In an optional embodiment, the large language model selects the ID of the target tool associated with the question information from the candidate tool list, including:

[0057] The large language model determines the applicable scenarios of the candidate tools based on the tool description in the candidate tool information; if the number of applicable scenarios is 1, the ID of a target tool associated with the question information is screened out from the candidate tool list; if the number of applicable scenarios is N, for each applicable scenario, the ID of a target tool associated with the question information is screened out from the candidate tool list, where N is a positive integer greater than 1. The tool description can usually express the applicable scenarios of the tool. When there are multiple types of each applicable scenario, the target tool finally determined in the applicable scenario is the tool with the highest correlation with the question information and the strongest scenario applicability. In a special case, if the number of applicable scenario types is 0, the ID of the target tool is determined to be a null value, and the output parameter of the target tool is also a null value. In the subsequent steps, the large language model answers questions based on the acquired data.

[0058] For example, if the question information involves both weather and food, then the candidate tools used to answer the question information must also involve weather and food. Therefore, in order to fully and accurately answer the question information, the number of target tools cannot be limited to only one, but must match the number of types of scenarios associated with the question information. Therefore, when the number of types of applicable scenarios for the required tools is multiple, the number of target tools is also multiple, which can ensure the pertinence and practicality of the generated responses.

[0059] In an optional embodiment, based on obtaining the ID of the target tool, the large language model generates a tool calling instruction based on the ID of the target tool, including: the large language model obtains the input parameter type and instruction template corresponding to the ID of the target tool based on the candidate tool information; extracts the parameter value corresponding to the input parameter type from the question information; pairs the input parameter type and the parameter value corresponding to the input parameter type and fills them into the instruction template to obtain the tool calling instruction.

[0060] Exemplarily, the format of the tool call instruction is as follows:

[0061] Tool1 name: {parameter1 name: parameter1 value, parameter2 name: parameter2 value};

[0062] Tool 2 name: {parameter 1 name: parameter 1 value, parameter 2 name: parameter 2 value}.

[0063] The parameter name is the input parameter type.

[0064] Based on the above embodiment, in another optional embodiment, after extracting the parameter value corresponding to the input parameter type from the question information, it also includes: the large language model determines whether the parameter value corresponding to the input parameter type is missing for each input parameter type; if so, sends follow-up information to the user question and answer interface according to the input parameter type, and receives supplementary information of the input parameter type from the user question and answer interface; adds the supplementary information to the existing question information to obtain updated question information, and returns to execute the step of extracting the parameter value corresponding to the input parameter type from the question information.

[0065] That is, the supplementary information is used as the context information of the original question information, and the original question information and the corresponding context information are used together as the updated question information.

[0066] Specifically, in the process of extracting the parameter value corresponding to the input parameter type, if the input parameter extracted by the large model is missing, the parameter value in the generated tool call instruction is filled with a question mark, and the system will interact with the user to ask for the missing parameter information. The user's supplementary answer will be added to the context information of the original question information, and the large model will re-extract the parameters until the parameters are complete.

[0067] Additional answer acquisition example:

[0068] Initial question information: How to get to the train station?

[0069] When the large language model extracts parameter values ​​from the question information based on the input parameter type, it recognizes: tran:{start_station:?, end_station:train station};

[0070] It can be seen that the input parameter type "start_station" lacks the corresponding parameter value.

[0071] The large language model then generates a follow-up question: Where do we start from?

[0072] The user adds the answer: "Start from Tianhe Square". The updated question information is: "How to get to the train station? Start from Tianhe Square."

[0073] The large language model extracts the parameter values ​​of the question information and obtains: tran:{start_station: Tianhe Square, end_station: railway station}.

[0074] S205: The tool calling module calls the target tool according to the tool calling instruction and obtains output parameters of the target tool.

[0075] The tool call instruction includes the ID of the target tool and question information. The question information is input into the target tool corresponding to the ID. The target tool processes the question information and outputs answer information, which is the output parameter of the target tool.

[0076] S206: The tool calling module calls the large language model based on the output parameters and the question information.

[0077] Specifically, the tool calling module calls the large language model based on the output parameters and the question information, including: the tool calling module obtains the output parameter type of the target tool, extracts the target parameters from the output parameters according to the output parameter type of the target tool, generates a second prompt word according to a preset second prompt word template, the question information and the target parameter; and calls the large language model based on the second prompt word.

[0078] Optionally, the question information and target parameters are filled into a preset second prompt word template to obtain a second prompt word. The function of the second prompt word is to "prompt" the large language model to integrate or adjust the target parameters according to the question information to obtain the reply content.

[0079] The output parameters of the target tool may also include some invalid redundant data. In order to prevent the redundant data from affecting the answer of the large language model, the output parameter type is used to extract the content of the output parameters of the target tool to obtain more accurate target parameters. The target parameters are used as the input of the large language model to improve the accuracy of the answer of the large language model.

[0080] Exemplarily, when the question information is a question about a weather scenario, the output parameter types of the target tool corresponding to the question information may be: temperature, humidity, wind speed, weather type, etc.

[0081] In an optional embodiment, the prompt words generated by the tool calling module can also be obtained through model training, which mainly includes three stages:

[0082] 1. During the prompt word development and optimization stage, in the prompt word arrangement module, through the integrated Notebook interface, users can not only freely write prompt words, but also use the "single case debugging preview" function to quickly test and preview each prompt word for a single question, ensuring that every subtle adjustment can provide instant feedback on its effect, laying a solid foundation for subsequent testing tasks.

[0083] 2. In the test task management stage, first prepare the question-answer list data for the test offline, then create the test task on the system, and choose to upload the test data file and prompt words. Through one-click operation, users can create and start test tasks. At the same time, the system provides a real-time task status monitoring interface, allowing users to grasp the test progress at any time, including key information such as task queue, execution status, and estimated completion time, to ensure the efficiency and transparency of the test process.

[0084] 3. In the result confirmation stage, after the test task is completed, the "test result dashboard" will intuitively display the evaluation score of each answer to the question, the total evaluation score of the entire test set, and detailed evaluation indicator analysis in various forms such as charts and lists. In addition, the system also provides a manual confirmation function, allowing users to review the evaluation results to ensure the accuracy of the evaluation results. Through the in-depth analysis and confirmation at this stage, users can fully grasp the performance of the AI ​​agent, providing strong support for subsequent prompt word optimization and model adjustment.

[0085] By introducing an automated batch processing mode, the engine is able to perform parallel testing on a large number of prompt words, greatly shortening the testing cycle, ensuring the objectivity and accuracy of the evaluation results, and improving the ability of the large language model to answer question information.

[0086] In another optional embodiment, the tool calling module includes a prompt word generation model, which is trained based on different application scenarios. This scenario-differentiated evaluation strategy can make the evaluation results of the large language model closer to the actual application scenarios, and meet the strict requirements on the answer quality of the large language model in different application scenarios.

[0087] S207: The large language model generates reply content and feeds it back to the user question and answer interface.

[0088] The large language model itself is also a model with question-answering capabilities. When finally generating the response content, the large language model has three functions. The first function is to integrate the content of the output parameters, the second function is to adjust or supplement the output content; the third function is to "polish" the output content.

[0089] Exemplary:

[0090] The question is: What’s the weather like today?

[0091] The output parameters of the tool are: weather 25-30℃, sunny;

[0092] The response generated by the large language model is: The weather is sunny, the temperature is 25-30℃; the recommended clothing type is short-sleeved, suitable for outdoor activities.

[0093] The embodiment of the present invention provides a question-answering method based on tool calling, specifically a technical solution that utilizes a retrieval-augmented generation (RAG) mechanism and combines a tool calling system to optimize the quality and efficiency of a large language model reply. The method is applied to a question-answering system, wherein the question-answering system comprises an application layer, a service layer and a large language model. The application layer comprises a plurality of tools and a user question-answering interface, the tool comprises a question-answering model, and the service layer comprises a tool retrieval module and a tool calling module. The user question-answering interface obtains the user's question information and sends it to the service layer. The tool retrieval module selects the ID of a candidate tool from existing tools according to the question information to obtain a candidate tool list. The tool calling module calls the large language model based on the candidate tool list and the question information. The large language model selects the ID of a target tool associated with the question information from the candidate tool list, and generates a tool calling instruction based on the ID of the target tool. The tool calling module calls the target tool according to the tool calling instruction to obtain the output parameters of the target tool. The tool calling module calls the large language model based on the output parameters and the question information. The large language model generates a reply content and feeds it back to the user question-answering interface.

[0094] The advantage of this question-answering method is that there are a large number of tools. The tool retrieval model is first used to screen out candidate tools associated with the question information from a large number of existing tools, and then the large language model is called based on the ID of the candidate tool and the question information to determine the target tool. This can not only ensure the applicability of the obtained target tool to the question information, but also reduce the amount of input data of the large language model, reduce the task of tool screening for the large language model, and improve data processing efficiency. The large language model is not limited by the token length of the input information volume, and can also efficiently and accurately call external tools in different scenarios. On this basis, the question information is answered in combination with the target tool and the large language model, which can improve the accuracy and comprehensiveness of the reply content.

[0095] Optionally, when constructing the knowledge base of the tool in the offline stage, the token length of the standard vector can also be restricted to reduce the token length of the tool information of each tool. On the one hand, on the basis of fixing the number of selected candidate tools, the amount of information of the tool information of the candidate tools can be reduced, that is, the amount of input data for the large language model in subsequent steps is reduced, which is conducive to reducing the data processing burden of the large language model and improving data processing efficiency. On the other hand, on the basis of limiting the maximum value of the input token length of the large language model, the number of candidate tools can be increased to make the candidate tools more comprehensive, which is conducive to improving the accuracy and comprehensiveness of the answers.

[0096] Optional, such as Figure 1 As shown, the service layer 200 also includes a knowledge base 204, which includes tool information of the tool. The application layer 100 also includes a tool management module 103. The tool management module 103 can be used to maintain and update the tools and tool information, so that the question-answering system can call different tools to answer questions in different scenarios. This module is responsible for collecting and maintaining detailed information on external function improvements, including but not limited to tool descriptions, input parameters, output parameters, and calling examples.

[0097] In order to clearly illustrate the question-answering method based on tool calling in the present invention, Figure 3 And the following is to explain, Figure 3 A flowchart of a question-answering method is shown in FIG. Figure 3 As shown, the question-answering method includes the following steps:

[0098] 1. Establish a knowledge base: Specifically, the relationship between tools and vector models associated with the tools is stored in the knowledge base, and the tool information of the tools is stored at the same time.

[0099] 2.1. User questions.

[0100] 2.2. Extract question vector.

[0101] 2.3. Tool retrieval: Retrieve candidate tools from the knowledge base based on the question vector to obtain a list of candidate tools, and then assemble prompt words based on the tool information corresponding to the candidate list and the user's question.

[0102] 2.4. Select tool: The large language model determines the target tool from the candidate tools based on the prompt word.

[0103] 2.5. (Generate) call instructions: The large language model generates (tool) call instructions based on the target tool and sends them to the tool call module.

[0104] 2.6 Obtain tool response content: The tool calling module calls the tool and obtains the tool response content (output parameters), and then calls the large language model based on the tool response content.

[0105] 2.7. Answer generation: The large language model generates answers (reply content).

[0106] 2.8. Feedback: The large language model feeds back the answer to the user.

[0107] Optionally, the tool information mainly includes tool name, tool description, input parameter type, instruction template, output parameter type, etc. Storing the tool information in the knowledge base means building an association relationship between the tool information and the tool. Figure 4 It is a schematic diagram of the participation of the content in a tool information at different stages of the system, such as Figure 4 As shown,

[0108] In the tool retrieval stage, the vectors of tool description and input parameter name in the tool information are used to perform similarity matching retrieval with the vector of user questions, and a list of candidate tools (corresponding tool IDs) is output;

[0109] In the tool selection stage, the tool description, input parameter name, and instruction template in the tool information are used to construct prompt words, allowing the large language model to perform semantic reasoning to select the target tool and generate tool call instructions;

[0110] In the content extraction stage, the corresponding output parameter values ​​in the tool response content are extracted according to the tool output parameter names, assisting the large language model to summarize and generate answers.

[0111] Corresponding to the question-answering method based on tool calling of the present invention, in one embodiment of the present invention, a question-answering system is also provided. Figure 1 FIG. 1 is a schematic diagram of a question-answering system provided by an embodiment of the present invention. Figure 1 As shown, the question-answering system includes: an application layer, a service layer and a large language model, the application layer includes multiple tools and a user question-answering interface, the tool includes a question-answering model, and the service layer includes a tool retrieval module and a tool calling module;

[0112] The user question and answer interface is used to obtain the user's question information and send it to the service layer;

[0113] The tool search module is used to filter out the IDs of candidate tools from the existing tools according to the question information to obtain a list of candidate tools;

[0114] The tool calling module is used to call the large language model based on the candidate tool list and the question information;

[0115] The large language model is used to filter out the ID of the target tool associated with the question information from the candidate tool list, and generate a tool calling instruction based on the ID of the target tool;

[0116] The tool calling module is used to call the target tool according to the tool calling instruction and obtain the output parameters of the target tool;

[0117] The tool calling module is used to call the large language model based on the output parameter and the question information;

[0118] The large language model is used to generate reply content and feed it back to the user question and answer interface.

[0119] Optionally, the service layer includes a knowledge base, the knowledge base stores tool information of the tool, the tool information includes a tool description and an input parameter name,

[0120] The tool retrieval module is also used to extract a feature vector from the question information; extract the tool description and input parameter name in the tool information as a standard vector; calculate the similarity between the feature vector and the standard vector; and use the ID of the tool associated with the standard vector whose similarity is greater than a preset similarity threshold as the ID of the candidate tool.

[0121] Optionally, the service layer includes a knowledge base, the knowledge base stores tool information of the tool, the tool information includes tool description, input parameter name and instruction template;

[0122] The tool calling module is also used to obtain tool information corresponding to the tool ID in the candidate tool list from the knowledge base to obtain candidate tool information, wherein the candidate tool information includes a tool description, input parameter name and instruction template of the candidate tool; generate a first prompt word according to a preset first prompt word template, the candidate tool list, the candidate tool information and the question information; and call the large language model based on the first prompt word.

[0123] Optionally, the large language model is also used to determine the applicable scenarios of the candidate tool based on the tool description in the candidate tool information; if the number of types of the applicable scenarios is 1, the ID of a target tool associated with the question information is filtered out from the candidate tool list; if the number of types of the applicable scenarios is N, for each applicable scenario, the ID of a target tool associated with the question information is filtered out from the candidate tool list, where N is a positive integer greater than 1; if the number of types of the applicable scenarios is 0, the ID of the target tool is determined to be a null value.

[0124] Optionally, the large language model is also used to obtain the input parameter type and instruction template corresponding to the ID of the target tool based on the candidate tool information; extract the parameter value corresponding to the input parameter type from the question information; pair the input parameter type and the parameter value corresponding to the input parameter type and fill them into the instruction template to obtain a tool calling instruction.

[0125] Optionally, after extracting the parameter value corresponding to the input parameter type from the question information, the large language model is also used to determine whether the parameter value corresponding to the input parameter type is missing for each input parameter type; if so, send follow-up question information to the user question and answer interface according to the input parameter type, and receive supplementary information of the input parameter type from the user question and answer interface; add the supplementary information to the existing question information to obtain updated question information, and return to execute the step of extracting the parameter value corresponding to the input parameter type from the question information.

[0126] Optionally, the tool calling module is also used to obtain the output parameter type of the target tool; extract the target parameter from the output parameter according to the output parameter type of the target tool; generate a second prompt word according to a preset second prompt word template, the question information and the target parameter; and call the large language model based on the second prompt word.

[0127] The question-and-answer system provided in the embodiment of the present invention can execute the question-and-answer method based on tool calling provided in any embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method.

[0128] Figure 5A schematic diagram of the structure of an electronic device 40 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0129] like Figure 5 As shown, the electronic device 40 includes at least one processor 41, and a memory connected to the at least one processor 41, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 to the random access memory (RAM) 43. In the RAM 43, various programs and data required for the operation of the electronic device 40 can also be stored. The processor 41, the ROM 42, and the RAM 43 are connected to each other through a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0130] A number of components in the electronic device 40 are connected to the I / O interface 45, including: an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0131] The processor 41 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 41 executes the various methods and processes described above, such as the question-answering method based on tool calls.

[0132] In some embodiments, the tool-invoked question-answering method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 40 via the ROM 42 and / or the communication unit 49. When the computer program is loaded into the RAM 43 and executed by the processor 41, one or more steps of the tool-invoked question-answering method described above may be performed. Alternatively, in other embodiments, the processor 41 may be configured to execute the tool-invoked question-answering method in any other appropriate manner (e.g., by means of firmware).

[0133] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0135] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, system, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection according to one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0136] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0137] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0138] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0139] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0140] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A question-answering method based on tool calling, characterized in that: Applied to a question-answering system, the question-answering system includes an application layer, a service layer and a large language model, the application layer includes multiple tools and a user question-answering interface, the tool includes a question-answering model, and the service layer includes a tool retrieval module and a tool calling module; the question-answering method based on tool calling includes: The user question and answer interface obtains the user's question information and sends it to the service layer; The tool search module selects the IDs of candidate tools from the existing tools according to the question information to obtain a candidate tool list; The tool calling module calls the large language model based on the candidate tool list and the question information; The large language model selects the ID of the target tool associated with the question information from the candidate tool list, and generates a tool calling instruction based on the ID of the target tool; The tool calling module calls the target tool according to the tool calling instruction and obtains the output parameters of the target tool; The tool calling module calls the large language model based on the output parameter and the question information; The large language model generates reply content and feeds it back to the user question and answer interface.

2. The method according to claim 1, characterized in that The service layer includes a knowledge base, the knowledge base stores tool information of tools, the tool information includes tool description and input parameter name, and the tool search module selects candidate tool IDs from existing tools according to the question information, including: The tool retrieval module extracts a feature vector from the question information; Extract tool description and input parameter names from tool information as standard vectors; Calculating the similarity between the feature vector and the standard vector; The ID of the tool associated with the standard vector having a similarity greater than a preset similarity threshold is used as the ID of the candidate tool.

3. The method according to claim 1, characterized in that The service layer includes a knowledge base, the knowledge base stores tool information of tools, the tool information includes tool description, input parameter name and instruction template, and the tool calling module calls the large language model based on the candidate tool list and the question information, including: The tool calling module obtains the tool information corresponding to the tool ID in the candidate tool list from the knowledge base to obtain the candidate tool information, wherein the candidate tool information includes a tool description, an input parameter name and an instruction template of the candidate tool; A first prompt word is generated according to a preset first prompt word template, the candidate tool list, the candidate tool information and the question information; and the large language model is called based on the first prompt word.

4. The method according to claim 3, characterized in that The large language model selects the ID of the target tool associated with the question information from the candidate tool list, including: The large language model determines the applicable scenario of the candidate tool according to the tool description in the candidate tool information; If the number of types of the applicable scenarios is 1, select an ID of a target tool associated with the question information from the candidate tool list; If the number of types of the applicable scenarios is N, for each of the applicable scenarios, an ID of a target tool associated with the question information is screened out from the candidate tool list, where N is a positive integer greater than 1; If the number of types of the applicable scenarios is 0, it is determined that the ID of the target tool is a null value.

5. The method according to claim 3, characterized in that The large language model generates a tool calling instruction based on the ID of the target tool, including: The large language model acquires the input parameter type and instruction template corresponding to the ID of the target tool based on the candidate tool information; Extracting a parameter value corresponding to the input parameter type from the question information; The input parameter type and the parameter value corresponding to the input parameter type are paired and filled into the instruction template to obtain a tool calling instruction.

6. The method according to claim 5, characterized in that After extracting the parameter value corresponding to the input parameter type from the question information, the method further includes: The large language model determines, for each of the input parameter types, whether a parameter value corresponding to the input parameter type is missing; If so, sending follow-up information to the user question-and-answer interface according to the input parameter type, and receiving supplementary information of the input parameter type from the user question-and-answer interface; The supplementary information is added to the existing question information to obtain updated question information, and the step of extracting a parameter value corresponding to the input parameter type from the question information is returned to be executed.

7. The method according to claim 1, characterized in that The service layer includes a knowledge base, the knowledge base stores tool information of the tool, the tool information includes an output parameter type, and the tool calling module calls the large language model based on the output parameter and the question information, including: The tool calling module obtains the output parameter type of the target tool; extracting target parameters from the output parameters according to the output parameter type of the target tool; Generate a second prompt word according to a preset second prompt word template, the question information and the target parameter; The large language model is called based on the second prompt word.

8. A question-answering system, characterized in that: It includes an application layer, a service layer and a large language model, wherein the application layer includes multiple tools and a user question-answering interface, the tool includes a question-answering model, and the service layer includes a tool retrieval module and a tool calling module; The user question and answer interface is used to obtain the user's question information and send it to the service layer; The tool search module is used to filter out the IDs of candidate tools from the existing tools according to the question information to obtain a list of candidate tools; The tool calling module is used to call the large language model based on the candidate tool list and the question information; The large language model is used to filter out the ID of the target tool associated with the question information from the candidate tool list, and generate a tool calling instruction based on the ID of the target tool; The tool calling module is used to call the target tool according to the tool calling instruction and obtain the output parameters of the target tool; The tool calling module is used to call the large language model based on the output parameter and the question information; The large language model is used to generate reply content and feed it back to the user question and answer interface.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the question-answering method based on tool invocation according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the question-answering method based on tool calling according to any one of claims 1 to 7 when executed.

Images