Tool augmentation based large language model question and answer method and device

Abstract

This invention provides a method and apparatus for question answering using a large language model based on tool enhancement. The method includes: performing intent recognition on a user-inputted original question to obtain a dataset corresponding to the original question; determining a toolset corresponding to the original question based on the data type of the dataset; dividing the original question into multiple subtasks and determining the execution tool for each subtask; determining the parameter generation structure of the execution tool based on its tool type; determining the calling parameters of the execution tool from the dataset based on the parameter generation structure for each subtask; calling the execution tool to execute the calling parameters and obtaining the subtask execution result; and determining the output result of the original question based on the subtask execution result of each subtask. This invention can improve the accuracy of large language models in multi-dataset question answering tasks.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a tool-enhanced large language model question answering method and apparatus. Background Technology

[0002] Natural Language Processing (NLP) aims to enable computers to understand, interpret, and generate human language. It is an important method of human-computer interaction and a crucial part of artificial intelligence research. In recent years, the emergence and development of Large Language Models (LLMs) have provided effective methods for solving NLP tasks and have demonstrated superior performance in numerous NLP applications.

[0003] However, for multi-dataset question answering tasks, existing solutions lack a process for recognizing user intent, which leads to the large language model selecting a dataset that does not correspond to the question when answering, thus obtaining incorrect answers.

[0004] This shows that the large language model question answering method in related technologies has the technical problem of low question answering accuracy. Summary of the Invention

[0005] This invention provides a tool-enhanced question-answering method and apparatus for large language models, which addresses the shortcomings of low accuracy in existing large language model question-answering methods and improves the accuracy of large language models in multi-dataset question-answering tasks.

[0006] This invention provides a tool-enhanced question-answering method for a large language model, comprising the following steps: Performing intent recognition on the original question input by the user to obtain a dataset corresponding to the original question; determining a toolset corresponding to the original question based on the data type of the dataset;

[0007] The original problem is divided into multiple subtasks, and the execution tool for each subtask is determined. Based on the tool type of the execution tool, the parameter generation structure of the execution tool is determined. Based on the subtask and according to the parameter generation structure, the calling parameters of the execution tool are determined from the dataset. The execution tool is called to execute the calling parameters to obtain the subtask execution result. Based on the subtask execution result of each subtask, the output result of the original problem is determined.

[0008] According to a tool-enhanced large language model question answering method provided by the present invention, before determining the parameter generation structure of the execution tool based on the tool type of the execution tool, the method further includes: determining the tool type of the execution tool as a first type when the required calling parameters of the execution tool are less than a preset parameter threshold and the number of steps to be executed by the execution tool is less than a preset step threshold; determining the tool type of the execution tool as a second type when the required calling parameters of the execution tool are less than the preset parameter threshold and the number of steps to be executed by the execution tool is greater than the preset step threshold; and determining the tool type of the execution tool as a third type when the required calling parameters of the execution tool are greater than the preset parameter threshold and the number of steps to be executed by the execution tool is less than the preset step threshold.

[0009] According to the present invention, a large language model question answering method based on tool enhancement is provided. The step of determining the parameter generation structure of the execution tool based on the tool type includes: when the tool type of the execution tool is a first type, determining the parameter generation structure of the execution tool as a direct generation structure; when the tool type of the execution tool is a second type, determining the parameter generation structure of the execution tool as a serial generation structure; and when the tool type of the execution tool is a third type, determining the parameter generation structure of the execution tool as a parallel generation structure.

[0010] According to the present invention, a tool-enhanced large language model question answering method is provided, wherein determining the output result of the original question based on the subtask execution result of each of the plurality of subtasks includes: iteratively executing the following process until a preset task decomposer determines the output result of the original question: determining the next subtask to be executed based on the subtask execution result of the current subtask by the task decomposer; executing the subtask to be executed to obtain the subtask execution result of the subtask to be executed.

[0011] According to the present invention, a tool-enhanced large language model question answering method is provided, the method further includes: when the number of iterations exceeds a preset iteration threshold, returning a query failure result.

[0012] According to a tool-enhanced large language model question answering method provided by the present invention, the step of performing intent recognition on the original question input by the user to obtain a dataset corresponding to the original question includes: determining the keywords of the original question input by the user; and matching the dataset corresponding to the original question from a preset set of multiple datasets based on the keywords.

[0013] This invention also provides a tool-enhanced large language model question answering device, comprising the following modules: a recognition module for performing intent recognition on the original question input by the user to obtain a dataset corresponding to the original question; a filtering module for determining a toolset corresponding to the original question based on the data type of the dataset; a decomposition module for dividing the original question into multiple subtasks and determining the execution tool for each subtask; a structure module for determining the parameter generation structure of the execution tool based on the tool type of the execution tool; a parameter module for determining the calling parameters of the execution tool from the dataset based on the subtask and according to the parameter generation structure; an execution module for calling the execution tool to execute the calling parameters and obtain the subtask execution result; and an output module for determining the output result of the original question based on the subtask execution result of each subtask.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the tool-enhanced large language model question answering method as described above.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the tool-enhanced large language model question-answering method as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the tool-enhanced large language model question answering method as described above.

[0017] The tool-enhanced large language model question answering method and apparatus provided by this invention can accurately understand the user's original question through intent recognition, transform it into a computer-processable form, select question-related data from a preset dataset to provide a foundation for subsequent processing, select the most suitable toolset for processing the data based on the data type of the dataset, improve the targeting and efficiency of question processing, and avoid resource waste and erroneous results caused by using inappropriate tools, divide the original question into multiple sub-tasks, help simplify the complexity of question processing, and make each sub-task more explicit and controllable, assign specific execution tools to each sub-task, ensure that each sub-task is handled professionally, and improve the accuracy and efficiency of problem solving, determine the parameter generation structure according to the tool type of the execution tool, and ensure that the generated parameters match the requirements of the tool, call the execution tool to execute the call parameters, and obtain the sub-task execution results, and determine the output result of the original question based on the execution results of multiple sub-tasks, improve the comprehensiveness and accuracy of the final result, and thus solve the technical problem of low question answering accuracy in related large language model question answering methods. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced one by one below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the tool-enhanced question-answering method for large language models provided by this invention.

[0020] Figure 2 This is a schematic diagram of the framework of the tool-enhanced large language model question answering method provided by the present invention.

[0021] Figure 3 This is a schematic diagram of the structure of the tool-enhanced large language model question-answering device provided by the present invention.

[0022] Figure 4 This is a schematic diagram of the physical structure of the electronic device provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] Natural Language Processing (NLP) aims to enable computers to understand, interpret, and generate human language. It is an important method of human-computer interaction and a crucial part of artificial intelligence research. In recent years, the emergence and development of Large Language Models (LLMs) have provided effective methods for solving NLP tasks and have demonstrated superior performance in numerous NLP applications.

[0025] NLP tasks are diverse and broad, encompassing tasks of varying difficulty. Simply increasing the training scale of an LLM (Low-Level Model) cannot accurately solve complex tasks or acquire the latest information. Tool usage is a crucial method for improving LLM performance. By using retrieval tools, LLM can acquire more knowledge, including the latest information; by using specialized tools, its professional knowledge capabilities can be enhanced; and by using database query tools, LLM can load external databases, filter data, and ultimately obtain query results. Furthermore, tool usage can enhance the model's interpretability and robustness.

[0026] Existing solutions have made progress in tool selection and planning. Some solutions combine the reasoning capabilities of LLM with tool invocation capabilities, breaking down complex tasks into simpler subtasks and selecting appropriate tools for each subtask. This approach improves the accuracy of tool selection and tool execution order, but it has the following drawbacks.

[0027] (1) For multi-dataset question answering tasks, existing solutions lack the process of identifying user intent, which leads to the LLM selecting a dataset that does not correspond to the question when answering the question, thus resulting in incorrect answers.

[0028] (2) The parameter generation stage is the main source of errors. Due to the different difficulty of calling different tools, the existing solution using the direct generation method is prone to missing parameters, generating irrelevant parameters and incorrect parameters, thus returning incorrect execution results.

[0029] Therefore, accurately determining the dataset being queried during tool invocation, and further improving the ability of LLM tools to reduce parameter generation errors, remain important research challenges.

[0030] This invention proposes a tool-enhanced question-answering method for large language models (LLMs), addressing the challenge of multi-dataset question-answering tasks. By initially determining the user's intent, the method identifies the dataset to be queried and performs preliminary filtering of the toolset, thus narrowing the tool selection range and improving the efficiency and accuracy of LLM tool selection. Simultaneously, a "divide and conquer" parameter generation approach is employed, using multi-structure parameter generation methods for different tool invocation scenarios to enhance the accuracy of LLM-generated parameters.

[0031] This invention primarily addresses the problems of LLM (Limited Language Management) failing to correctly identify user intent and misjudging the dataset to be queried, as well as parameter errors caused by using a single parameter generation structure for tool calls of varying difficulty. By adding an intent recognition module before task decomposition and using a multi-structure parameter generation method, the intent recognition capability of LLM is effectively improved, and the accuracy of parameter generation is increased. This technology has significant objective effects, improving the accuracy of LLM in multi-dataset question answering tasks.

[0032] Optionally, the tool-enhanced large language model question answering method of this application embodiment can be executed by a server, by a terminal device, or by both a server and a terminal device. For example, the tool-enhanced large language model question answering method of this embodiment can be executed by the server where the large language model is located.

[0033] Figure 1 This is a flowchart illustrating the tool-enhanced question-answering method for large language models provided by this invention, as shown below. Figure 1 As shown, the method includes the following steps:

[0034] Step 101: Perform intent recognition on the original question input by the user to obtain the dataset corresponding to the original question.

[0035] In this embodiment of the invention, the large language model can analyze the original question input by the user and understand the user's true intention and needs. For example, the large language model finds the dataset that best matches the original question from multiple preset datasets.

[0036] Intent recognition provides a corresponding dataset for subsequent data processing steps. The large language model can select the most suitable processing tool based on the corresponding dataset, thereby optimizing the entire data processing process.

[0037] According to the present invention, a tool-enhanced large language model question answering method is provided, which performs intent recognition on the original question input by the user to obtain a dataset corresponding to the original question, including:

[0038] Determine the keywords of the original question entered by the user;

[0039] Based on keywords, the dataset corresponding to the original question is matched from multiple pre-set datasets.

[0040] In this embodiment of the invention, by identifying keywords in the original question, the large language model can more accurately locate the information points that the user wants to query or process; keyword extraction helps to reduce the interference of irrelevant information and improve the accuracy of subsequent information retrieval and processing.

[0041] By comparing keywords in user questions with tags or content in preset datasets, datasets related to the questions can be found quickly. The matching mechanism improves the efficiency of data retrieval and reduces processing time.

[0042] Step 102: Based on the data type of the dataset, determine the toolset corresponding to the original problem.

[0043] In this embodiment of the invention, after identifying the user's intent and matching the dataset to be queried, the toolset is initially screened.

[0044] For example, the data types of datasets include: numerical data, text data, image data, etc.

[0045] Numerical data typically consists of numbers and can be used for mathematical operations and statistical analysis. The corresponding toolset includes data processing tools, statistical analysis tools, and visualization tools.

[0046] Data processing tools are used for data cleaning, transformation, aggregation, and other operations; statistical analysis tools are used for descriptive statistics, inferential statistics, regression analysis, and other operations; visualization tools are used to draw charts and display data distribution and trends.

[0047] Text data consists of text and requires processing such as text parsing, word segmentation, and sentiment analysis. The corresponding toolset includes text processing tools, sentiment analysis tools, and visualization tools.

[0048] Text processing tools are used for text segmentation, stop word removal, and stemming; sentiment analysis tools are used to analyze the sentiment tendency of text; visualization tools are used to generate word clouds to show the distribution of keywords in the text.

[0049] Image data consists of pixels and requires processing such as image recognition, classification, and segmentation. The corresponding toolset includes image processing tools and image recognition tools.

[0050] Image processing tools are used for operations such as reading, cropping, and scaling images, while image recognition tools are used to train image recognition models and perform image classification and segmentation.

[0051] Through the embodiments of the present invention, user intent is identified, the dataset to be queried is identified, and the toolset is initially screened.

[0052] Step 103: Divide the original problem into multiple subtasks and determine the execution tool for each subtask.

[0053] In this embodiment of the invention, a complex problem in the form of multiple steps is decomposed into multiple simple sub-tasks, and an execution tool is selected for each sub-task, with each sub-task involving the invocation of a tool.

[0054] For example, logical analysis is performed on the original problem to identify its multiple components and the logical relationships between them; based on these logical relationships, the original problem is divided into multiple relatively independent but interconnected sub-tasks, and the inputs, outputs, and constraints of each sub-task are determined.

[0055] For each subtask, a target analysis is performed to obtain the target of each subtask. Based on the target of each subtask, the execution tools for each subtask are determined. The execution tools include: code tools, database tools, text tools, graph query tools, mathematical tools, and systems.

[0056] Step 104: Determine the parameter generation structure of the execution tool based on the tool type of the execution tool.

[0057] According to the tool-enhanced question-answering method for large language models provided by the present invention, before determining the parameter generation structure of the execution tool based on the tool type, the method further includes:

[0058] When the required calling parameters for the execution tool are less than the preset parameter threshold, and the number of steps to be executed by the execution tool is less than the preset step threshold, the tool type of the execution tool is determined to be the first type.

[0059] When the required calling parameters for the execution tool are less than the preset parameter threshold, and the number of steps to be executed by the execution tool is greater than the preset step threshold, the tool type of the execution tool will be determined as the second type.

[0060] When the required calling parameters for the execution tool are greater than the preset parameter threshold, and the number of steps to be executed by the execution tool is less than the preset step threshold, the tool type of the execution tool will be determined as the third type.

[0061] In this embodiment of the invention, when the required calling parameters of the execution tool are less than a preset parameter threshold and the number of steps to be executed by the execution tool is less than a preset step threshold, the tool type is determined to be the first type. That is, the first type of tool does not require a large number of parameter inputs, nor does it require too many steps to be executed.

[0062] When the required parameters for an execution tool are less than a preset parameter threshold, but the number of steps to be executed by the tool exceeds a preset step threshold, the tool type is determined to be Type II. That is, Type II tools require fewer parameters but need to execute a series of complex steps to complete the task.

[0063] When the required parameters for executing a tool exceed a preset parameter threshold, and the number of steps to be executed by the tool is less than a preset step threshold, the tool type is determined to be Type 3. That is, Type 3 tools require a large number of parameter inputs to precisely control their behavior, but the execution process is relatively simple with fewer steps.

[0064] According to the present invention, a large language model question answering method based on tool enhancement determines the parameter generation structure of the execution tool based on the tool type of the execution tool, including:

[0065] When the tool type of the execution tool is type 1, the parameter generation structure of the execution tool is determined to be a direct generation structure;

[0066] When the tool type of the execution tool is type 2, the parameter generation structure of the execution tool is determined to be a serial generation structure;

[0067] When the tool type of the execution tool is the third type, the parameter generation structure of the execution tool is determined to be a parallel generation structure.

[0068] When the tool type of the execution tool is Type 1, the parameter generation structure is determined to be a direct generation structure, that is, generated in a simple and direct way without complex logic or sequence. Type 1 tools require only a small number of input parameters, and there are no complex dependencies between these parameters.

[0069] When the tool type is Type II, the parameter generation structure is determined to be a serial generation structure, meaning that parameter generation involves a series of steps that need to be executed sequentially. Type II tools require more input parameters, and these parameters may have dependencies on each other; that is, the generation of one parameter may depend on the result of another parameter.

[0070] When the tool type is Type 3, the parameter generation structure is determined to be a parallel generation structure, meaning that parameter generation involves multiple steps or tasks executed in parallel. Type 3 tools require a large number of input parameters, and the generation of these parameters can potentially be performed simultaneously to improve efficiency. Furthermore, there may still be some dependencies between these parameters, but these dependencies can be optimized through parallel processing techniques.

[0071] In this embodiment of the invention, to improve the accuracy of parameter generation, different parameter generation methods are used for different tool types, and different parameter generation structures are used for different tool invocation scenarios. Three commonly used structures are described below.

[0072] Directly generated structure: For simple tools with few parameters, large language models directly generate their parameters.

[0073] Serial generation structure: For complex tools, large language models break down the parameter generation process into multiple simple steps and generate the required parameters in the last step.

[0074] Parallel generation structure: For tools with a large number of parameters, large language models are classified in a certain way (such as spatial and temporal), and the corresponding parameters are generated in parallel and finally combined.

[0075] Step 105: Based on the subtask and the structure generated according to the parameters, determine the calling parameters of the execution tool from the dataset.

[0076] In this embodiment of the invention, the required calling parameters are extracted or calculated from the dataset according to the parameter generation structure and subtasks. For direct generation structures, parameter values ​​are directly obtained from the dataset. For serial generation structures, parameters are extracted sequentially from the dataset based on the dependencies between parameters, following the serial generation order. For parallel generation structures, parameters are extracted from multiple parts of the dataset simultaneously.

[0077] Through the embodiments of the present invention, appropriate parameters can be generated for the selected tool based on the sub-task.

[0078] Step 106: Call the execution tool to execute the call parameters and obtain the subtask execution result.

[0079] Step 107: Determine the output of the original problem based on the execution results of each subtask in the multiple subtasks.

[0080] According to the tool-enhanced question-answering method for large language models provided by the present invention, the output result of the original question is determined based on the execution result of each subtask in multiple subtasks, including:

[0081] Iteratively execute the following process until the preset task decomposer determines the output of the original problem:

[0082] The task decomposer determines the next subtask to be executed based on the execution results of the current subtask.

[0083] Execute the subtask to be executed and obtain the execution result of the subtask to be executed.

[0084] In this embodiment of the invention, task decomposition and subtask processing is an iterative process. The LLM generates the next subtask based on the previous subtask and its return results. This allows for timely receipt of the results and feedback from each tool's execution, and when execution errors occur, the LLM can promptly adjust the erroneous tools and parameters.

[0085] For example, in a large language model, the task decomposer continues to generate the next subtask based on the historical subtask-result trajectory. When the task decomposer determines that the original problem has been solved, it calls the system tool Finish to output the final answer.

[0086] According to the present invention, a tool-enhanced question-answering method for large language models is provided, the method further includes:

[0087] If the number of iterations exceeds a preset iteration threshold, a query failure result will be returned.

[0088] The following describes an example of a practical application of the tool-enhanced question-answering method for large language models provided by this invention.

[0089] For complex question-answering tasks that require external tools, a multi-module collaborative tool enhancement framework for LLM is built, wherein the framework includes the following modules.

[0090] Intent recognition module: used to recognize user intent, identify the dataset to be queried, and perform preliminary screening of the toolset.

[0091] Task decomposition module: Used to break down complex problems into multiple simple subtasks in a multi-step manner, and select an appropriate tool for each subtask.

[0092] Subtask processing module: Used to generate appropriate parameters for the selected tool based on the subtask.

[0093] Tool execution module: Used to execute tools and return the results of subtasks.

[0094] To improve the accuracy of parameter generation, different parameter generation methods are used for different tool types, and different parameter generation structures are used for different tool invocation scenarios. Three commonly used structures are described below.

[0095] Directly generate structures: For simple tools with few parameters, LLM directly generates their parameters.

[0096] Serial generation structure: For complex tools, LLM breaks down the parameter generation process into multiple simple steps and generates the required parameters in the last step.

[0097] Parallel generation structure: For tools with a large number of parameters, LLM classifies them in a certain way (such as spatial and temporal), generates the corresponding parameters in parallel, and finally combines them.

[0098] The framework provided by this invention can be used to solve complex multi-dataset question-answering tasks. Users can provide the dataset they want to query and a question description. The LLM (Limited Language Management) then calls tools to answer the question and returns the query results. When the LLM receives a user's question, it performs intent recognition, matches it to the dataset the user is querying, and filters suitable query tools based on the data type of the dataset, making the tool selection range more focused and improving the efficiency of the tool selection process. By decomposing the user's complex question into more manageable subtasks, the LLM enhances its ability to solve complex tasks. Furthermore, through a multi-structure parameter generation method, more accurate parameters are generated for the selected tools, improving the accuracy of question answers.

[0099] The embodiments of the present invention have at least the following beneficial effects:

[0100] Modularity and independence: Through a modular solution process, each module can be designed with its own appropriate prompts and targeted selection of contextual examples. This provides more opportunities for contextual learning for complex and important modules, effectively improving the learning efficiency and capabilities of LLM.

[0101] Preliminary screening and solution efficiency: By identifying user intent before task decomposition, the dataset and toolset are initially screened. This avoids wasting time and resources on incorrect datasets and irrelevant tools in LLM, thus improving the solution efficiency of LLM.

[0102] Online planning: Task decomposition and subtask processing is an iterative process. LLM generates the next subtask based on the previous subtask and its return results. This allows for timely receipt of results and feedback from each tool's execution, and when execution errors occur, LLM can promptly adjust the erroneous tools and parameters.

[0103] Fine-grained parameter generation: For subtask processing modules, different parameter generation structures are designed to adapt to different tool call situations and improve parameter accuracy.

[0104] Tool preference: For tools with the same function, prompting LLM to choose the more efficient tool can effectively improve the efficiency and accuracy of LLM solutions.

[0105] For example, consider the multi-dataset question-answering task scenario based on ToolQA. This dataset is specifically designed to evaluate the ability of LLMs to perform question answering using external tools. It includes six contextual dimensions: time, space, social, scientific, mathematical, and personal, corresponding to eight datasets. It uses three data storage formats: database, graph, and text, and also includes a mathematical reasoning dataset, GSM8K. Table 1 details the relevant information for each dataset.

[0106] Refer to Table 1, which contains information on eight datasets in the ToolQA provided by this invention.

[0107] Table 1

[0108]

[0109] As shown in Table 1, ToolQA designs questions of two difficulty levels: easy and hard. Each difficulty level has a question template, with a total of 55 easy question templates and 62 hard question templates. Following these templates, there are a total of 800 easy questions and 730 complex questions. These questions can only be answered by using appropriate tools to retrieve information from the corresponding datasets. A total of 13 tools for retrieving information from the corresponding datasets are defined, as shown in Table 2.

[0110] Accuracy (Acc) is used as an evaluation metric to compare the LLM output answer with the ground truth answer and calculate the accuracy.

[0111] Table 2

[0112]

[0113] In this embodiment of the invention, a few-shot prompting method is used to design prompts for each module and tool, which include instructions, relevant information, format, historical information, examples, and the current task.

[0114] refer to Figure 2 , Figure 2 This is a schematic diagram of the framework of the tool-enhanced large language model question answering method provided by the present invention.

[0115] like Figure 2 The system mainly consists of four modules: intent recognition, task decomposition, subtask processing, and tool execution. The specific steps are as follows:

[0116] When a user inputs a question, it first passes through an intent recognition module. This module uses LLM (Limited Language Management) to extract keywords from the question and determine the dataset to be queried. Based on the data type, it provides a suggestion of candidate tools, filtering out irrelevant tools. For scenarios with a small number of tools and clear tool categories, this suggestion is manually written to ensure accuracy.

[0117] Next, we move on to the task decomposition module. This module breaks down the original problem into multiple subtasks, each involving the invocation of a tool.

[0118] The decomposed subtasks are passed to the subtask processing module, which is mainly responsible for generating the required parameters for each selected tool. It has multiple parameter generation structures, such as direct generation, parallel generation, and serial generation.

[0119] After generating the parameters, the tool execution module is entered, which calls the tool to query relevant information from the dataset and returns the results of the tool execution.

[0120] The task decomposer continues to generate the next subtask based on the historical subtask-result trajectory. When the task decomposer determines that the original problem has been solved, it calls the system tool Finish to output the final answer. Alternatively, if the predetermined number of iterations is exceeded, it returns a query failure message.

[0121] The tool-enhanced large language model question answering device provided by the present invention is described below. The tool-enhanced large language model question answering device described below can be referred to in correspondence with the tool-enhanced large language model question answering method described above.

[0122] refer to Figure 3 , Figure 3 This is a schematic diagram of the structure of the tool-enhanced large language model question-answering device provided by the present invention.

[0123] The recognition module 301 is used to perform intent recognition on the original question input by the user and obtain a dataset corresponding to the original question;

[0124] The filtering module 302 is used to determine the toolset corresponding to the original problem based on the data type of the dataset;

[0125] The decomposition module 303 is used to divide the original problem into multiple subtasks and determine the execution tool for each subtask.

[0126] Structure module 304 is used to determine the parameter generation structure of the execution tool based on the tool type of the execution tool;

[0127] Parameter module 305 is used to determine the calling parameters of the execution tool from the dataset based on the subtask and the parameter generation structure.

[0128] Execution module 306 is used to call the execution tool to execute the call parameters and obtain the execution results of the subtasks;

[0129] Output module 307 is used to determine the output result of the original problem based on the execution result of each subtask in multiple subtasks.

[0130] Specifically, the above-mentioned tool-enhanced large language model question answering device provided by the present invention can implement all the method steps implemented in the above-mentioned tool-enhanced large language model question answering method embodiment, and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0131] Figure 4 This is a schematic diagram of the physical structure of the electronic device provided by the present invention, such as... Figure 4 As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a tool-enhanced large language model question-answering method. This method includes: performing intent recognition on the user-inputted original question to obtain a dataset corresponding to the original question; determining a toolset corresponding to the original question based on the data type of the dataset; dividing the original question into multiple subtasks and determining the execution tool for each subtask; determining the parameter generation structure of the execution tool based on the tool type; determining the calling parameters of the execution tool from the dataset based on the parameter generation structure according to the subtasks; calling the execution tool to execute the calling parameters and obtaining the subtask execution result; and determining the output result of the original question based on the subtask execution results of each subtask.

[0132] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0133] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the tool-enhanced large language model question answering method provided by the above methods. The method includes: performing intent recognition on the original question input by the user to obtain a dataset corresponding to the original question; determining a toolset corresponding to the original question based on the data type of the dataset; dividing the original question into multiple sub-tasks and determining the execution tool for each sub-task; determining the parameter generation structure of the execution tool based on the tool type of the execution tool; determining the calling parameters of the execution tool from the dataset based on the sub-task according to the parameter generation structure; calling the execution tool to execute the calling parameters to obtain the sub-task execution result; and determining the output result of the original question based on the sub-task execution result of each sub-task.

[0134] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the tool-enhanced large language model question-answering method provided by the above methods. This method includes: performing intent recognition on an original question input by a user to obtain a dataset corresponding to the original question; determining a toolset corresponding to the original question based on the data type of the dataset; dividing the original question into multiple subtasks and determining the execution tool for each subtask; determining the parameter generation structure of the execution tool based on its tool type; determining the calling parameters of the execution tool from the dataset based on the parameter generation structure according to the subtasks; calling the execution tool to execute the calling parameters to obtain the subtask execution result; and determining the output result of the original question based on the subtask execution result of each subtask.

[0135] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0136] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A question-answering method for large language models based on tool enhancement, characterized in that, Applications to large language models include: The user's original question is subjected to intent recognition to obtain a dataset corresponding to the original question; Based on the data type of the dataset, determine the toolset corresponding to the original problem; The original problem is divided into multiple sub-tasks, and the execution tool for each sub-task is determined. Based on the tool type of the execution tool, determine the parameter generation structure of the execution tool; Based on the subtask and the structure generated according to the parameters, the calling parameters of the execution tool are determined from the dataset; The execution tool is invoked to execute the invocation parameters, and the subtask execution result is obtained; Based on the execution results of each of the multiple subtasks, the output result of the original problem is determined; Before determining the parameter generation structure of the execution tool based on its tool type, the method further includes: determining the tool type of the execution tool as a first type when the required calling parameters of the execution tool are less than a preset parameter threshold and the number of steps to be executed by the execution tool is less than a preset step threshold; determining the tool type of the execution tool as a second type when the required calling parameters of the execution tool are less than a preset parameter threshold and the number of steps to be executed by the execution tool is greater than a preset step threshold; and determining the tool type of the execution tool as a third type when the required calling parameters of the execution tool are greater than a preset parameter threshold and the number of steps to be executed by the execution tool is less than a preset step threshold. The step of determining the parameter generation structure of the execution tool based on the tool type includes: when the tool type of the execution tool is the first type, determining the parameter generation structure of the execution tool to be a direct generation structure; when the tool type of the execution tool is the second type, determining the parameter generation structure of the execution tool to be a serial generation structure; and when the tool type of the execution tool is the third type, determining the parameter generation structure of the execution tool to be a parallel generation structure.

2. The question-answering method based on tool enhancement for large language models according to claim 1, characterized in that, The step of determining the output result of the original problem based on the execution result of each of the multiple subtasks includes: The following process is executed iteratively until a preset task decomposer determines the output of the original problem: The task decomposer determines the next subtask to be executed based on the execution result of the current subtask. Execute the subtask to be executed to obtain the subtask execution result of the subtask to be executed.

3. The question-answering method based on tool enhancement for large language models according to claim 2, characterized in that, The method further includes: If the number of iterations exceeds a preset iteration threshold, a query failure result is returned.

4. The question-answering method based on tool enhancement for large language models according to claim 1, characterized in that, The process of performing intent recognition on the original question input by the user to obtain a dataset corresponding to the original question includes: Determine the keywords of the original question entered by the user; Based on the keywords, a dataset corresponding to the original question is matched from multiple preset datasets.

5. A tool-enhanced large language model question-answering device, characterized in that, include: The recognition module is used to identify the intent of the original question input by the user and obtain a dataset corresponding to the original question; The filtering module is used to determine the toolset corresponding to the original problem based on the data type of the dataset; The decomposition module is used to divide the original problem into multiple subtasks and determine the execution tool for each subtask. A structure module is used to determine the parameter generation structure of the execution tool based on the tool type of the execution tool; A parameter module is used to determine the calling parameters of the execution tool from the dataset based on the subtask and according to the parameter generation structure. The execution module is used to call the execution tool to execute the call parameters and obtain the subtask execution result; The output module is used to determine the output result of the original problem based on the execution result of each subtask in the plurality of subtasks; Before determining the parameter generation structure of the execution tool based on its tool type, the device is further configured to: determine the tool type of the execution tool as a first type when the required calling parameters of the execution tool are less than a preset parameter threshold and the number of steps to be executed by the execution tool is less than a preset step threshold; determine the tool type of the execution tool as a second type when the required calling parameters of the execution tool are less than a preset parameter threshold and the number of steps to be executed by the execution tool is greater than a preset step threshold; and determine the tool type of the execution tool as a third type when the required calling parameters of the execution tool are greater than a preset parameter threshold and the number of steps to be executed by the execution tool is less than a preset step threshold. The step of determining the parameter generation structure of the execution tool based on the tool type includes: when the tool type of the execution tool is the first type, determining the parameter generation structure of the execution tool to be a direct generation structure; when the tool type of the execution tool is the second type, determining the parameter generation structure of the execution tool to be a serial generation structure; and when the tool type of the execution tool is the third type, determining the parameter generation structure of the execution tool to be a parallel generation structure.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the tool-enhanced large language model question-answering method as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the tool-enhanced large language model question-answering method as described in any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the tool-enhanced large language model question-answering method as described in any one of claims 1 to 4.

Images