Large language model question answering method and device based on tool enhancement

Abstract

The invention provides a large language model question answering method and device based on tool enhancement, and the method comprises the steps: carrying out the intention recognition of an original question inputted by a user, and obtaining a data set corresponding to the original question; determining a tool set corresponding to the original problem based on the data type of the data set; dividing the original problem into a plurality of sub-tasks, and determining an execution tool of each sub-task in the plurality of sub-tasks; determining a parameter generation structure of the execution tool based on the tool type of the execution tool; determining calling parameters of the execution tool from the data set on the basis of the subtasks according to a parameter generation structure; calling an execution tool to execute the calling parameter to obtain a subtask execution result; based on the subtask execution result of each subtask in the plurality of subtasks, determining an output result of the original problem; according to the invention, the accuracy of the large language model in the multi-dataset question and answer task can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a tool-enhanced large language model question answering method and device. Background Art

[0002] Natural Language Processing (NLP) is dedicated to enabling computers to understand, interpret and generate human language. It is an important way of human-computer interaction and an important part of artificial intelligence research. In recent years, the emergence and development of large language models (LLM) have provided an effective method for solving NLP tasks and have shown excellent performance in countless NLP tasks.

[0003] However, for multi-dataset question-answering tasks, existing solutions lack the process of identifying user intent, which causes the selected dataset to not correspond to the question when the large language model answers the question, resulting in incorrect answers.

[0004] This shows that the large language model question-answering method in the related technology has the technical problem of low question-answering accuracy. Summary of the invention

[0005] The present invention provides a tool-enhanced large language model question answering method and device, which are used to solve the defect of low question answering accuracy of large language model question answering methods in the prior art, and to improve the accuracy of large language models in multi-dataset question answering tasks.

[0006] The present invention provides a tool-enhanced large language model question answering method, comprising the following steps: performing intent recognition on the original question input by the user to obtain a data set corresponding to the original question; determining a tool set corresponding to the original question based on the data type of the data set; The original problem is divided into multiple subtasks, and an execution tool of each of the multiple subtasks is determined; based on the tool type of the execution tool, a parameter generation structure of the execution tool is determined; based on the subtasks and according to the parameter generation structure, the calling parameters of the execution tool are determined from the data set; the execution tool is called to execute the calling parameters to obtain a subtask execution result; based on the subtask execution result of each subtask in the multiple subtasks, an output result of the original problem is determined.

[0007] According to a tool-enhanced large language model question-answering method provided by the present invention, before determining the parameter generation structure of the execution tool based on the tool type of the execution tool, the method also includes: when the calling parameter required by the execution tool is less than a preset parameter threshold, and the number of steps to be executed of the execution tool is less than a preset step threshold, determining the tool type of the execution tool as a first type; when the calling parameter required by the execution tool is less than the preset parameter threshold, and the number of steps to be executed of the execution tool is greater than the preset step threshold, determining the tool type of the execution tool as a second type; when the calling parameter required by the execution tool is greater than the preset parameter threshold, and the number of steps to be executed of the execution tool is less than the preset step threshold, determining the tool type of the execution tool as a third type.

[0008] According to a tool-enhanced large language model question-answering method provided by the present invention, the parameter generation structure of the execution tool is determined based on the tool type of the execution tool, including: when the tool type of the execution tool is the first type, determining the parameter generation structure of the execution tool to be a direct generation structure; when the tool type of the execution tool is the second type, determining the parameter generation structure of the execution tool to be a serial generation structure; when the tool type of the execution tool is the third type, determining the parameter generation structure of the execution tool to be a parallel generation structure.

[0009] According to a tool-enhanced large language model question and answer method provided by the present invention, the output result of the original question is determined based on the subtask execution result of each subtask in the multiple subtasks, including: iteratively executing the following process until a preset task decomposer determines the output result of the original question: determining the next subtask to be executed based on the subtask execution result of the current subtask by the task decomposer; executing the subtask to be executed to obtain the subtask execution result of the subtask to be executed.

[0010] According to a tool-enhanced large language model question answering method provided by the present invention, the method also includes: when the number of iterations executed is greater than a preset iteration threshold, returning a query failure result.

[0011] According to a tool-enhanced large language model question answering method provided by the present invention, the intent of the original question input by the user is recognized to obtain a data set corresponding to the original question, including: determining keywords of the original question input by the user; based on the keywords, matching the data set corresponding to the original question from multiple preset data sets.

[0012] The present invention also provides a large language model question-answering device based on tool enhancement, comprising the following modules: an identification module, used to identify the intent of the original question input by the user, and obtain a data set corresponding to the original question; a screening module, used to determine the tool set corresponding to the original question based on the data type of the data set; a decomposition module, used to divide the original question into multiple subtasks, and determine the execution tool of each subtask in the multiple subtasks; a structure module, used to determine the parameter generation structure of the execution tool based on the tool type of the execution tool; a parameter module, used to determine the calling parameters of the execution tool from the data set based on the subtask according to the parameter generation structure; an execution module, used to call the execution tool to execute the calling parameters and obtain the subtask execution result; an output module, used to determine the output result of the original question based on the subtask execution result of each subtask in the multiple subtasks.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the tool-enhanced large language model question-answering method as described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the tool-enhanced large language model question answering methods described above.

[0015] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the tool-enhanced large language model question answering methods described above.

[0016] The tool-enhanced large language model question-answering method and device provided by the present invention can accurately understand the user's original question through intent recognition, convert it into a computer-processable form, select data related to the question from a preset data set, and provide a basis for subsequent processing; according to the data type of the data set, select the tool set that is most suitable for processing these data, thereby improving the pertinence and efficiency of problem processing and avoiding resource waste and erroneous results caused by the use of inappropriate tools; divide the original problem into multiple subtasks, which helps to simplify the complexity of problem processing and make each subtask more clear and controllable; assign a specific execution tool to each subtask, ensuring that each subtask can be professionally processed, and improving the accuracy and efficiency of problem solving; determine the parameter generation structure according to the tool type of the execution tool, and ensure that the generated parameters match the requirements of the tool; call the execution tool to execute the call parameters to obtain the subtask execution result; based on the execution results of multiple subtasks, determine the output result of the original question, and improve the comprehensiveness and accuracy of the final result, thereby solving the technical problem of low question-answering accuracy in the large language model question-answering method in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced one by one below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flowchart of the tool-enhanced large language model question answering method provided by the present invention.

[0019] Figure 2 It is a schematic diagram of the framework of the tool-enhanced large language model question answering method provided by the present invention.

[0020] Figure 3 It is a structural diagram of the tool-enhanced large language model question-answering device provided by the present invention.

[0021] Figure 4 It is a schematic diagram of the physical structure of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Natural Language Processing (NLP) is dedicated to enabling computers to understand, interpret and generate human language. It is an important way of human-computer interaction and an important part of artificial intelligence research. In recent years, the emergence and development of large language models (LLM) have provided an effective method for solving NLP tasks and have shown excellent performance in countless NLP tasks.

[0024] NLP tasks are diverse, covering a wide range of tasks of different difficulty levels. Simply expanding the training scale of LLM cannot accurately answer complex tasks and obtain the latest information. Tool calling is an important method to improve the performance of LLM. By calling retrieval tools, LLM can obtain more knowledge, including the latest knowledge; by calling professional tools, the professional knowledge ability of LLM can be improved; by calling database query tools, LLM can load external databases, filter data, and finally obtain query results. In addition, calling tools can enhance the interpretability and robustness of the model.

[0025] Existing solutions have made progress in tool selection and tool planning. Some of them combine the reasoning ability of LLM with the tool calling ability to decompose complex tasks into simple subtasks and select appropriate tools for each subtask. This solution improves the accuracy of tool selection and the accuracy of tool execution order, but it has the following defects.

[0026] (1) For multi-dataset question answering tasks, existing solutions lack the process of identifying user intent, which causes the dataset selected by LLM when answering questions to not correspond to the question, resulting in incorrect answers.

[0027] (2) The parameter generation stage is the main source of errors. Due to the different difficulty levels of calling different tools, the existing solutions that use direct generation methods are prone to missing parameters, generating irrelevant parameters, and generating incorrect parameters, thereby returning incorrect execution results.

[0028] Therefore, in the process of tool calling, how to accurately determine the data set queried by the problem, how to further improve the ability of LLM to call tools, and reduce errors in parameter generation, remains an important research challenge.

[0029] This paper proposes a tool-enhanced large language model question-answering method to achieve a solution for multi-dataset question-answering tasks. By preliminarily judging the user's intention, identifying the data set to be queried, and preliminarily screening the tool set, the tool selection range is more concentrated, and the efficiency and accuracy of LLM selection tools are improved. At the same time, a "divide and conquer" parameter generation method is adopted, and a multi-structure parameter generation method is used for different tool call situations to improve the accuracy of LLM generated parameters.

[0030] The present invention mainly solves the problem that LLM cannot correctly identify user intent, misjudges the data set to be queried, and uses a single parameter generation structure for tool calls of different difficulty levels, resulting in parameter errors. By adding an intent recognition module before task decomposition and using a multi-structure parameter generation method, the intent recognition capability of LLM is effectively improved, and the accuracy of parameter generation is improved. This technology has significant objective effects and can improve the accuracy of LLM in multi-dataset question-answering tasks.

[0031] Optionally, the tool-enhanced large language model question-answering method of the embodiment of the present application can be executed by a server, or by a terminal device, or jointly by a server and a terminal device, taking the example of the tool-enhanced large language model question-answering method of the present embodiment being executed by the server where the large language model is located.

[0032] Figure 1 is a flow chart of the tool-enhanced large language model question answering method provided by the present invention. Figure 1 As shown, the method comprises the following steps: Step 101 , performing intent recognition on the original question input by the user to obtain a data set corresponding to the original question.

[0033] In an embodiment of the present invention, the large language model can analyze the original question input by the user and understand the user's real intention and needs. For example, the large language model finds the data set that best matches the original question from multiple preset data sets.

[0034] Intent recognition provides the corresponding data set for subsequent data processing steps. The large language model can select the most appropriate processing tool based on the corresponding data set, thereby optimizing the entire data processing process.

[0035] According to a tool-enhanced large language model question answering method provided by the present invention, the original question input by the user is subjected to intent recognition to obtain a data set corresponding to the original question, including: Determine the keywords of the original question entered by the user; Based on keywords, match the dataset corresponding to the original question from multiple preset datasets.

[0036] In an embodiment of the present invention, by identifying keywords in the original question, the large language model can more accurately locate the information points that the user wants to query or process; keyword extraction helps to reduce the interference of irrelevant information and improve the accuracy of subsequent information retrieval and processing.

[0037] By comparing the keywords in the user's question with the labels or content in the preset data set, the data set related to the question can be quickly found. The matching mechanism improves the efficiency of data retrieval and reduces processing time.

[0038] Step 102, based on the data type of the data set, determine the toolset corresponding to the original problem.

[0039] In the embodiment of the present invention, after identifying the user intention and matching the data set to be queried, the tool set is preliminarily screened.

[0040] For example, the data types of a data set include: numerical data, text data, image data, etc.

[0041] Numerical data usually consists of numbers and can be used for mathematical operations and statistical analysis. The corresponding tool sets include: data processing tools, statistical analysis tools, visualization tools, etc.

[0042] Data processing tools are used for data cleaning, transformation, aggregation and other operations; statistical analysis tools are used for descriptive statistics, inferential statistics, regression analysis, etc.; visualization tools are used to draw charts and display data distribution and trends.

[0043] Text data is composed of text and needs to be processed by text parsing, word segmentation, sentiment analysis, etc. The corresponding tool set includes: text processing tools, sentiment analysis tools, visualization tools, etc.

[0044] Text processing tools are used for text segmentation, stop word removal, stemming, etc.; sentiment analysis tools are used to analyze the sentiment tendency of the text; and visualization tools are used to generate word cloud diagrams to display the distribution of keywords in the text.

[0045] Image data is composed of pixels and requires image recognition, classification, segmentation and other processing. The corresponding tool set includes: image processing tools, image recognition tools, etc.

[0046] Image processing tools are used for operations such as reading, cropping, and scaling images, and image recognition tools are used to train image recognition models and perform image classification and segmentation.

[0047] Through the embodiment of the present invention, the user intention is identified, the data set to be queried is identified, and the tool set is preliminarily screened.

[0048] Step 103, divide the original problem into multiple subtasks, and determine the execution tool of each of the multiple subtasks.

[0049] In the embodiment of the present invention, a complex problem in the form of multiple steps is decomposed into multiple simple subtasks, and an execution tool is selected for each subtask. Each subtask involves the call of a tool.

[0050] For example, conduct a logical analysis of the original problem to determine the multiple components of the problem and the logical relationships between the multiple components; divide the original problem into multiple relatively independent but interrelated subtasks based on the logical relationships, and determine the input, output, and constraints of each subtask.

[0051] Perform target analysis on each subtask to obtain the target of each subtask, and determine the execution tool of each subtask based on the target of each subtask, wherein the execution tools include: code tools, database tools, text tools, graph query tools, mathematical tools and systems.

[0052] Step 104: Determine a parameter generation structure for the execution tool based on the tool type of the execution tool.

[0053] According to a tool-enhanced large language model question answering method provided by the present invention, before determining a parameter generation structure for executing the tool based on a tool type of the executing tool, the method further includes: When the calling parameters required for executing the tool are less than a preset parameter threshold, and the steps to be executed of the executing tool are less than a preset step threshold, determining the tool type of the executing tool as the first type; When the calling parameter required for executing the tool is less than the preset parameter threshold, and the steps to be executed of the executing tool are greater than the preset step threshold, determining the tool type of the executing tool as the second type; When the calling parameters required for executing the tool are greater than a preset parameter threshold and the number of steps to be executed of the tool is less than a preset step threshold, the tool type of the tool is determined to be the third type.

[0054] In an embodiment of the present invention, when the calling parameters required to execute the tool are less than a preset parameter threshold and the steps to be executed for executing the tool are less than a preset step threshold, the tool type is determined to be the first type, that is, a first type of tool that does not require a large amount of parameter input nor does it need to execute too many steps.

[0055] When the call parameters required to execute the tool are less than the preset parameter threshold, but the steps to be executed by the tool are greater than the preset step threshold, the tool type is determined to be the second type. That is, although the second type of tool requires fewer parameters, it needs to execute a series of complex steps to complete the task.

[0056] When the call parameters required to execute the tool are greater than the preset parameter threshold, and the steps to be executed for executing the tool are less than the preset step threshold, the tool type is determined to be type 3. That is, type 3 tools require a large number of parameter inputs to accurately control their behavior, but the execution process is relatively simple and has fewer steps.

[0057] According to a tool-enhanced large language model question answering method provided by the present invention, based on the tool type of the execution tool, a parameter generation structure of the execution tool is determined, including: When the tool type of the execution tool is the first type, determining the parameter generation structure of the execution tool to be a direct generation structure; When the tool type of the execution tool is the second type, determining the parameter generation structure of the execution tool to be a serial generation structure; When the tool type of the execution tool is the third type, the parameter generation structure of the execution tool is determined to be a parallel generation structure.

[0058] When the tool type of the execution tool is the first type, the parameter generation structure is determined to be a direct generation structure, that is, generated in a simple and direct manner without complex logic or sequence. The first type of tool only requires a small number of input parameters, and there is no complex dependency between these parameters.

[0059] When the tool type of the execution tool is the second type, the parameter generation structure is determined to be a serial generation structure, that is, the generation of parameters involves a series of steps, which need to be executed in sequence. The second type of tool requires more input parameters, and there may be dependencies between these parameters, that is, the generation of one parameter may depend on the result of another parameter.

[0060] When the tool type of the execution tool is the third type, the parameter generation structure is determined to be a parallel generation structure, that is, the parameter generation involves multiple steps or tasks executed in parallel. The third type of tool requires a large number of input parameters, and the generation of these parameters may be performed simultaneously to improve efficiency. In addition, there may still be certain dependencies between these parameters, but this dependency may be optimized through parallel processing technology.

[0061] In the embodiment of the present invention, in order to improve the accuracy of parameter generation, different parameter generation methods are used for tool types, and different parameter generation structures are used for different tool calling situations. Three commonly used structures are described below.

[0062] Directly generate structures: For simple tools with few parameters, the large language model directly generates its parameters.

[0063] Serial generation structure: For complex tools, the large language model decomposes the parameter generation process into multiple simple steps and generates the required parameters in the last step.

[0064] Parallel generation structure: For tools with a large number of parameters, large language models are classified in a certain way (such as space and time), and the corresponding parameters are generated in parallel and finally combined.

[0065] Step 105 , based on the subtask and the parameter generation structure, determining the calling parameters of the execution tool from the data set.

[0066] In the embodiment of the present invention, according to the parameter generation structure and the subtask, the required call parameters are extracted or calculated from the data set. For the direct generation structure, the parameter values ​​are directly obtained from the data set. For the serial generation structure, the parameters are sequentially extracted from the data set in the serial generation order based on the dependency relationship between the parameters. For the parallel generation structure, the parameters are extracted from multiple parts of the data set at the same time.

[0067] Through the embodiments of the present invention, suitable parameters can be generated for the selected tool according to the subtasks.

[0068] Step 106, calling the execution tool to execute the calling parameters and obtain the subtask execution result.

[0069] Step 107: Determine the output result of the original question based on the subtask execution result of each subtask in the multiple subtasks.

[0070] According to a tool-enhanced large language model question answering method provided by the present invention, based on the subtask execution result of each subtask in a plurality of subtasks, an output result of the original question is determined, including: Iterate the following process until the preset task decomposer determines the output of the original problem: Determine the next subtask to be executed based on the subtask execution result of the current subtask through the task decomposer; Execute the subtask to be executed and obtain the subtask execution result of the subtask to be executed.

[0071] In the embodiment of the present invention, task decomposition and subtask processing is an iterative process, and LLM generates the next subtask based on the previous subtask and its return result. In this way, the results and feedback of each tool execution can be obtained in time. When an execution error occurs, LLM can adjust the wrong tool and parameters in time.

[0072] For example, the task decomposer in the large language model continues to generate the next subtask based on the historical subtask-result trajectory. When the task decomposer determines that the original question has been answered, it calls the system tool Finish to output the final answer.

[0073] According to a tool-enhanced large language model question answering method provided by the present invention, the method further includes: When the number of iterations executed is greater than the preset iteration threshold, a query failure result is returned.

[0074] The following describes an example of a tool-enhanced large language model question answering method provided by the present invention in practical application.

[0075] For complex question-answering tasks that require the assistance of external tools, a multi-module collaborative tool is built to enhance the LLM framework, where the framework includes the following modules.

[0076] Intent recognition module: used to identify user intent, identify the data sets that need to be queried, and perform preliminary screening of the toolset.

[0077] Task decomposition module: used to decompose complex problems into multiple simple subtasks in a multi-step manner and select a suitable tool for each subtask.

[0078] Subtask processing module: used to generate appropriate parameters for the selected tool based on the subtask.

[0079] Tool execution module: used to execute tools and return subtask results.

[0080] In order to improve the accuracy of parameter generation, different parameter generation methods are used for different tool types, and different parameter generation structures are used for different tool calling situations. The following describes three commonly used structures.

[0081] Directly generate structure: For simple tools with few parameters, LLM directly generates their parameters.

[0082] Serial generation structure: For complex tools, LLM breaks down the parameter generation process into multiple simple steps and generates the required parameters in the last step.

[0083] Parallel generation structure: For tools with a large number of parameters, LLM classifies them in a certain way (such as space and time), generates the corresponding parameters in parallel, and finally combines them.

[0084] The above framework provided by the present invention can be used to solve complex multi-dataset question-answering tasks. The user can provide the data set to be queried and provide a problem description. LLM answers the question by calling the tool and returns the query result. When LLM receives the user's question, it identifies the intent of the question, matches it to the data set queried by the user, and filters out the appropriate query tool according to the data type of the data set, so that the tool selection range is more concentrated and the efficiency of the tool selection process is improved. LLM decomposes the user's complex problems into subtasks that are easier to handle, thereby improving the ability of LLM to solve complex tasks. In addition, through a multi-structure parameter generation method, more accurate parameters are generated for the selected tool to improve the accuracy of question answering.

[0085] The embodiments of the present invention have at least the following beneficial effects: Modularity and independence: Through the modular solution process, each module can design appropriate prompts and select context examples in a targeted manner. This provides more contextual learning opportunities for complex and important modules, effectively improving the learning efficiency and ability of LLM.

[0086] Preliminary screening and solution efficiency: By identifying the user's intention before decomposing the task, the data set and tool set are preliminarily screened. This can prevent LLM from wasting time and resources on wrong data sets and irrelevant tools, and improve LLM's solution efficiency.

[0087] Online planning: Task decomposition and subtask processing is an iterative process. LLM generates the next subtask based on the previous subtask and its return results. In this way, the results and feedback of each tool execution can be obtained in time. When an execution error occurs, LLM can adjust the wrong tool and parameters in time.

[0088] Fine-grained parameter generation: For subtask processing modules, different parameter generation structures are designed to adapt to different tool calling situations and improve parameter accuracy.

[0089] Tool preference: For tools with the same function, by prompting LLM to select a more efficient tool, adding LLM's tool preference can effectively improve LLM's solution efficiency and accuracy.

[0090] For example, the multi-dataset question answering task scenario based on ToolQA. This dataset is specifically designed to evaluate the ability of LLMs to use external tools for question answering. It contains 6 context dimensions: time, space, society, science, mathematics, and personal. It corresponds to 8 datasets, including 3 data storage formats: database, graph, and text, and a mathematical reasoning dataset GSM8K. Table 1 details the relevant information of each dataset.

[0091] Refer to Table 1, which shows the information of 8 data sets in ToolQA provided by the present invention.

[0092] Table 1

[0093] As can be seen from Table 1, ToolQA designs two types of questions, simple and difficult. Each type of question has a question template, with a total of 55 simple question templates and 62 difficult question templates. Questions are designed according to the question templates, with a total of 800 simple questions and 730 complex questions. These questions can only be answered by using appropriate tools to obtain information from the corresponding datasets. A total of 13 tools for obtaining information from the corresponding datasets are defined, as shown in Table 2.

[0094] Use accuracy (Acc) as the evaluation metric, compare the answer output by LLM with the ground truth answer, and calculate the accuracy.

[0095] Table 2

[0096] The embodiment of the present invention uses a few-shot prompting method to design prompts for each module and each tool, which include instructions, related information, format, historical information, examples and current tasks.

[0097] refer to Figure 2 , Figure 2 It is a schematic diagram of the framework of the tool-enhanced large language model question answering method provided by the present invention.

[0098] like Figure 2 As mentioned above, it mainly consists of four modules: intention recognition, task decomposition, subtask processing and tool execution. The specific steps are as follows: When a user enters an original question, it first passes through the intent recognition module, which uses LLM to extract keywords in the question and determine the data set to be queried. Based on the type of data, a prompt of candidate tools is provided to filter out irrelevant tools. For scenarios where the number of tools is small and the tool classification is clear, the prompt is manually written to ensure the accuracy of the prompt.

[0099] Then we enter the task decomposition module, which decomposes the original problem into multiple subtasks, each of which involves the call of a tool.

[0100] The decomposed subtasks are passed to the subtask processing module, which is mainly responsible for generating the required parameters for each selected tool and has multiple parameter generation structures such as direct generation, parallel generation, and serial generation.

[0101] After generating the parameters, enter the tool execution module, call the tool to query relevant information from the data set, and return the result of the tool execution.

[0102] The task decomposer continues to generate the next subtask based on the historical subtask-result track. When the task decomposer determines that the original question has been answered, it will call the system tool Finish to output the final answer. Or when the predetermined number of iterations is exceeded, the query failure information is returned.

[0103] The tool-enhanced large language model question-answering device provided by the present invention is described below. The tool-enhanced large language model question-answering device described below and the tool-enhanced large language model question-answering method described above can be referenced to each other.

[0104] refer to Figure 3 , Figure 3 It is a structural diagram of the tool-enhanced large language model question-answering device provided by the present invention.

[0105] The recognition module 301 is used to perform intent recognition on the original question input by the user to obtain a data set corresponding to the original question; A screening module 302 is used to determine a tool set corresponding to the original question based on the data type of the data set; A decomposition module 303 is used to divide the original problem into multiple subtasks and determine an execution tool for each of the multiple subtasks; A structure module 304 is used to determine a parameter generation structure of the execution tool based on the tool type of the execution tool; A parameter module 305, for determining the calling parameters of the execution tool from the data set based on the subtask according to the parameter generation structure; An execution module 306 is used to call an execution tool to execute a calling parameter and obtain a subtask execution result; The output module 307 is used to determine the output result of the original problem based on the subtask execution result of each subtask in the multiple subtasks.

[0106] Specifically, the tool-enhanced large language model question-answering device provided by the present invention can implement all the method steps implemented by the tool-enhanced large language model question-answering method embodiment, and can achieve the same technical effects. The parts and beneficial effects that are the same as those in the method embodiment will not be described in detail herein.

[0107] Figure 4 is a schematic diagram of the physical structure of the electronic device provided by the present invention, such as Figure 4As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the tool-enhanced large language model question-answering method, which includes: performing intent recognition on the original question input by the user to obtain a data set corresponding to the original question; determining a tool set corresponding to the original question based on the data type of the data set; dividing the original question into multiple subtasks, and determining an execution tool for each of the multiple subtasks; determining a parameter generation structure for the execution tool based on the tool type of the execution tool; determining a call parameter for the execution tool from the data set based on the parameter generation structure of the subtask; calling the execution tool to execute the call parameter to obtain a subtask execution result; and determining the output result of the original question based on the subtask execution result of each of the multiple subtasks.

[0108] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0109] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the tool-enhanced large language model question and answer method provided by the above methods, which includes: performing intent recognition on the original question input by the user to obtain a data set corresponding to the original question; based on the data type of the data set, determining the tool set corresponding to the original question; dividing the original question into multiple subtasks, and determining the execution tool for each of the multiple subtasks; based on the tool type of the execution tool, determining the parameter generation structure of the execution tool; based on the subtask according to the parameter generation structure, determining the calling parameters of the execution tool from the data set; calling the execution tool to execute the calling parameters to obtain the subtask execution result; based on the subtask execution result of each subtask in the multiple subtasks, determining the output result of the original question.

[0110] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the tool-enhanced large language model question-answering method provided by the above-mentioned methods, the method comprising: performing intent recognition on the original question input by the user to obtain a data set corresponding to the original question; determining a toolset corresponding to the original question based on the data type of the data set; dividing the original question into multiple subtasks, and determining an execution tool for each of the multiple subtasks; determining a parameter generation structure for the execution tool based on the tool type of the execution tool; determining a calling parameter for the execution tool from the data set based on the parameter generation structure of the subtask; calling the execution tool to execute the calling parameters to obtain a subtask execution result; and determining an output result of the original question based on the subtask execution result of each of the multiple subtasks.

[0111] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0112] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A tool-enhanced large language model question answering method, characterized in that: Applied to large language models, including: Performing intent recognition on the original question input by the user to obtain a data set corresponding to the original question; Based on the data type of the data set, determining a toolset corresponding to the original problem; Divide the original problem into multiple subtasks, and determine an execution tool for each of the multiple subtasks; Determining a parameter generation structure of the execution tool based on a tool type of the execution tool; Determining the calling parameters of the execution tool from the data set based on the subtask and generating the structure according to the parameters; Calling the execution tool to execute the calling parameters and obtaining the subtask execution result; Based on the subtask execution result of each subtask in the plurality of subtasks, an output result of the original question is determined.

2. The tool-enhanced large language model question answering method according to claim 1, characterized in that: Before determining the parameter generation structure of the execution tool based on the tool type of the execution tool, the method further includes: When the call parameter required by the execution tool is less than a preset parameter threshold, and the number of steps to be executed by the execution tool is less than a preset step threshold, determining the tool type of the execution tool as the first type; When the calling parameter required by the execution tool is less than a preset parameter threshold, and the number of steps to be executed by the execution tool is greater than a preset step threshold, determining the tool type of the execution tool to be the second type; When the calling parameters required by the execution tool are greater than a preset parameter threshold and the number of steps to be executed by the execution tool is less than a preset step threshold, the tool type of the execution tool is determined to be the third type.

3. The tool-enhanced large language model question answering method according to claim 2, characterized in that: The step of determining a parameter generation structure of the execution tool based on the tool type of the execution tool includes: When the tool type of the execution tool is the first type, determining that the parameter generation structure of the execution tool is a direct generation structure; When the tool type of the execution tool is the second type, determining that the parameter generation structure of the execution tool is a serial generation structure; When the tool type of the execution tool is the third type, the parameter generation structure of the execution tool is determined to be a parallel generation structure.

4. The tool-enhanced large language model question answering method according to claim 1, characterized in that: The step of determining the output result of the original question based on the subtask execution result of each subtask in the plurality of subtasks includes: The following process is iterated until the preset task decomposer determines the output result of the original problem: Determine the next subtask to be executed based on the subtask execution result of the current subtask by the task decomposer; Execute the subtask to be executed and obtain the subtask execution result of the subtask to be executed.

5. The tool-enhanced large language model question answering method according to claim 4, characterized in that: The method further comprises: When the number of iterations executed is greater than a preset iteration threshold, a query failure result is returned.

6. The tool-enhanced large language model question answering method according to claim 1, characterized in that: The performing of intent recognition on the original question input by the user to obtain a data set corresponding to the original question includes: Determine the keywords of the original question entered by the user; Based on the keyword, a data set corresponding to the original question is matched from a plurality of preset data sets.

7. A tool-enhanced large language model question answering device, characterized in that: Applied to large language models, including: The recognition module is used to recognize the intent of the original question input by the user and obtain a data set corresponding to the original question; A screening module, used for determining a toolset corresponding to the original problem based on the data type of the data set; A decomposition module, used to divide the original problem into multiple subtasks and determine an execution tool for each of the multiple subtasks; A structure module, used for determining a parameter generation structure of the execution tool based on a tool type of the execution tool; A parameter module, configured to determine the calling parameters of the execution tool from the data set based on the subtask and the parameter generation structure; An execution module, used for calling the execution tool to execute the calling parameters and obtain the subtask execution result; The output module is used to determine the output result of the original problem based on the subtask execution result of each subtask in the multiple subtasks.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the large language model question answering method enhanced by the tool as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the large language model question answering method enhanced by the tool as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the large language model question answering method enhanced by the tool as described in any one of claims 1 to 6 is implemented.

Images