Large language model reasoning optimization method and system based on error strategy feedback

Abstract

The invention provides a big language model reasoning optimization method and system based on error strategy feedback. The method comprises the following steps: acquiring multi-modal data of a question to be answered; according to the multi-modal data, error feedback information corresponding to the multi-modal data is obtained by using a pre-constructed error driving model, and reasoning prediction is performed on the multi-modal data according to the error feedback information to obtain reasoning prediction information; selecting and planning a tool chain corresponding to the multi-modal data in a preset tool application set by utilizing a large language model according to the multi-modal data; according to the reasoning prediction information and a tool chain corresponding to the multi-modal data, reasoning by using the large language model to obtain a reasoning answer result of the multi-modal data; by combining error feedback information and a self-adaptive selection tool chain, the capability of processing and reasoning multi-modal information of a large language model is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a large language model reasoning optimization method and system based on error strategy feedback. Background Art

[0002] Currently, reasoning is a cognitive process that uses evidence, arguments, and logical principles to draw conclusions or make evaluations. Reasoning plays a central role in many intellectual activities, such as problem solving, decision making, theorem proving, and critical thinking. Reasoning tasks can be divided into many categories, including common sense reasoning, mathematical reasoning, logical reasoning, causal reasoning, visual reasoning, audio reasoning, multimodal reasoning, agent reasoning, etc. Its main function is to help individuals connect information, evaluate associations, and draw wise conclusions or solutions.

[0003] In recent years, with the development of large language models (LLMs), there has been an increasing interest in exploring their reasoning capabilities in different tasks. Currently, there are three mainstream approaches to enhance and improve the reasoning capabilities of LLMs. The first is fully supervised fine-tuning, which adapts the parameters of a pre-trained model on a specific labeled dataset to a specific task or domain. However, this approach is only trained on a specific dataset, which may limit its generalization ability to other domains and cause the model to rely on features in the training data rather than actually reasoning. The second is prompt and context learning, including methods such as Chain-of-Thought, Reasoning Engineering, and Prompt Optimization, which guide the model to enhance its reasoning and problem-solving capabilities through structured input and optimized prompts. However, many of these methods cannot handle image information, which limits their application in text-only datasets and unimodal tasks, especially in complex mathematical operations and scientific problems that require common sense knowledge. In addition, existing prompt methods rely on manually designed templates, which are labor-intensive. While current prompt optimization methods tend to study automatic prompt optimization, focusing on the workflow of generating, scoring, and selecting prompts, most of them fail to make improvements based on error feedback, and the prompt optimization space lacks strategic planning.

[0004] In response to the shortcomings of LLMs in reasoning tasks, many studies in recent years have shown great interest in using external tools to enhance the reasoning capabilities of models. Integrated web browsers enable models to obtain and utilize the latest information from the Internet, greatly enhancing their ability to answer queries that require up-to-date knowledge. Similarly, integrated search engines enable LLMs to sift through large amounts of online data to find relevant information, thereby improving the accuracy and relevance of their responses. Calculators enhance the ability of LLMs to perform precise mathematical calculations, which is essential for tasks that require numerical reasoning. Symbolic modules and Python interpreters give models the ability to perform complex algorithms and mathematical operations, broadening their problem-solving capabilities. In addition, recent research has explored the concept of using combined tools, which represents an important innovation. This approach leverages multiple tools simultaneously to comprehensively enhance the reasoning capabilities of LLMs, allowing models to approach tasks with multifaceted strategies, combining different types of information and computational processes, and thus achieving a more comprehensive understanding and problem-solving capabilities. However, tool enhancement methods also bring challenges and limitations. These methods may introduce integration complexity and raise concerns about reliability and consistency, making large language models rigid and inflexible when calling external tools.

[0005] In summary, the existing technology is unable to generate error feedback for multimodal information during reasoning, and due to the rigidity and inflexibility of large language models when calling external tools, the accuracy of large language models is low when processing and reasoning about multimodal information. Summary of the invention

[0006] In order to solve the problem that the prior art cannot generate error feedback for multimodal information during reasoning, and that the large language model is rigid and inflexible when calling external tools, resulting in low accuracy of the large language model when processing and reasoning multimodal information, the present invention proposes a large language model reasoning optimization method based on error strategy feedback, including:

[0007] Obtain multimodal data for the questions to be answered;

[0008] According to the multimodal data, using a pre-built error-driven model, error feedback information corresponding to the multimodal data is obtained, and according to the error feedback information, inference prediction is performed on the multimodal data to obtain inference prediction information;

[0009] According to the multimodal data, using a large language model, a tool chain corresponding to the multimodal data is selected and planned in a preset tool application set;

[0010] Performing reasoning using the large language model according to the inference prediction information and a tool chain corresponding to the multimodal data to obtain an inference answer result of the multimodal data;

[0011] Wherein, the error-driven model is constructed based on the Monte Carlo tree search algorithm.

[0012] Optionally, the error-driven model includes the following construction process:

[0013] Using the sample multimodal information in the preset data set as the input data of the training data;

[0014] Using the correct answer corresponding to the sample multimodal information as output data of training data;

[0015] Based on the input data and the output data, training an initial error-driven model;

[0016] Inputting the input data into the initial error-driven model to obtain a predicted value corresponding to the input data;

[0017] Based on the predicted value and the output data, an error-driven model is constructed using a Monte Carlo tree search algorithm.

[0018] Optionally, constructing an error-driven model based on the predicted value and the output data using a Monte Carlo tree search algorithm includes:

[0019] Step S1: Initialize a decision tree, use the sample multimodal information to represent the state of each node in the decision tree, use the output data corresponding to the sample multimodal information to represent the action of the edge between each node from the current state to the next state, and use the loss function value between the output data and the predicted value to represent the feedback error of the action of each node from the current state to the next state;

[0020] Step S2: calculating the supremum confidence interval value of each node in the decision tree, and forming a node group of promising nodes with nodes having the largest supremum confidence interval value;

[0021] Step S3: Expand each node in the node group using a state-action value function to obtain a new child node corresponding to each node, and generate an answer error query based on the new child node and the corresponding feedback error;

[0022] Step S4: Based on the incorrectly answered query, error information is identified by the optimizer to obtain error feedback information of the new child node; the error feedback information is back-propagated to other nodes of the decision tree, and statistical information of each node in the decision tree is updated;

[0023] Step S5: Repeat steps S2-S4 until a predetermined stop condition is reached to obtain an error drive model.

[0024] Optionally, the calculation formula corresponding to the supremum confidence interval value is as follows:

[0025]

[0026] Among them, UCT represents the supremum confidence interval value; s t Represents the state value of the node at time t; a t represents the action value of the node at time t; C represents the adjustable parameter value; V represents the state value function value; N(s t ) indicates that the state value is s t The total number of times the node is visited; n i (s t ) indicates that the state value is s t The number of times the child node i corresponding to the node is visited.

[0027] Optionally, the expression corresponding to the state-action value function is as follows:

[0028]

[0029] Among them, V represents the state value function value output by the state-action value function; S represents the state space; A represents the action space; R represents the set of real numbers.

[0030] Optionally, the multimodal data includes: text information and additional information;

[0031] The additional information includes: image information and / or table information;

[0032] The error feedback information includes one or more of the following: visual error information, text error information, and integrated error information.

[0033] Optionally, the selecting and planning a tool chain corresponding to the multimodal data in a preset tool application set by using a large language model according to the multimodal data includes:

[0034] According to the multimodal data, obtaining a question-answering scenario corresponding to the multimodal data;

[0035] Based on the question-answering scenarios corresponding to the multimodal data, using a large language model, select and plan enhancement tools for the multimodal data in different question-answering scenarios from a preset tool application set;

[0036] The enhancement tools for the multimodal data in different question-answering scenarios are combined into a tool chain.

[0037] Optionally, the question-answering scenario includes: a multimodal scientific question-answering scenario and a table question-answering scenario;

[0038] The enhancement tools in the multimodal scientific question-answering scenario include one or more of the following: parameter knowledge acquisition tools, web page knowledge acquisition tools, and visual enhancement tools;

[0039] The enhancement tools in the table question and answer scenario include one or more of the following: parameter knowledge acquisition tool, program generator, program verifier, program executor and data filter.

[0040] Based on the same inventive concept, the present invention also provides a large language model reasoning optimization system based on error strategy feedback, comprising:

[0041] A data acquisition module, used to acquire multimodal data of questions to be answered;

[0042] An error feedback module is used to obtain error feedback information corresponding to the multimodal data based on the multimodal data using a pre-built error-driven model, and to perform inference prediction on the multimodal data based on the error feedback information to obtain inference prediction information;

[0043] A tool planning module, for selecting and planning a tool chain corresponding to the multimodal data in a preset tool application set based on the multimodal data and using a large language model;

[0044] A reasoning answer module, configured to perform reasoning using the large language model according to the reasoning prediction information and a tool chain corresponding to the multimodal data, to obtain a reasoning answer result of the multimodal data;

[0045] Wherein, the error-driven model is constructed based on the Monte Carlo tree search algorithm.

[0046] Optionally, the error-driven model in the error feedback module includes the following construction process:

[0047] Using the sample multimodal information in the preset data set as the input data of the training data;

[0048] Using the correct answer corresponding to the sample multimodal information as output data of training data;

[0049] Based on the input data and the output data, training an initial error-driven model;

[0050] Inputting the input data into the initial error-driven model to obtain a predicted value corresponding to the input data;

[0051] Based on the predicted value and the output data, an error-driven model is constructed using a Monte Carlo tree search algorithm.

[0052] Optionally, the error feedback module constructs an error-driven model based on the predicted value and the output data using a Monte Carlo tree search algorithm, including:

[0053] Step S1: Initialize a decision tree, use the sample multimodal information to represent the state of each node in the decision tree, use the output data corresponding to the sample multimodal information to represent the action of the edge between each node from the current state to the next state, and use the loss function value between the output data and the predicted value to represent the feedback error of the action of each node from the current state to the next state;

[0054] Step S2: calculating the supremum confidence interval value of each node in the decision tree, and forming a node group of promising nodes with nodes having the largest supremum confidence interval value;

[0055] Step S3: Expand each node in the node group using a state-action value function to obtain a new child node corresponding to each node, and generate an answer error query based on the new child node and the corresponding feedback error;

[0056] Step S4: Based on the incorrectly answered query, error information is identified by the optimizer to obtain error feedback information of the new child node; the error feedback information is back-propagated to other nodes of the decision tree, and statistical information of each node in the decision tree is updated;

[0057] Step S5: Repeat steps S2-S4 until a predetermined stop condition is reached to obtain an error drive model.

[0058] Optionally, the calculation formula corresponding to the supremum confidence interval value in the error feedback module is as follows:

[0059]

[0060] Among them, UCT represents the supremum confidence interval value; s t Represents the state value of the node at time t; a t represents the action value of the node at time t; C represents the adjustable parameter value; V represents the state value function value; N(s t ) indicates that the state value is s t The total number of times the node is visited; n i (s t ) indicates that the state value is s t The number of times the child node i corresponding to the node is visited.

[0061] Optionally, the expression corresponding to the state-action value function in the error feedback module is as follows:

[0062]

[0063] Among them, V represents the state value function value output by the state-action value function; S represents the state space; A represents the action space; R represents the set of real numbers.

[0064] Optionally, the multimodal data in the data acquisition module includes: text information and additional information;

[0065] The additional information includes: image information and / or table information;

[0066] The error feedback information in the error feedback module includes one or more of the following: visual error information, text error information and integrated error information.

[0067] Optionally, the tool planning module includes:

[0068] A scene recognition unit, configured to obtain a question-answering scene corresponding to the multimodal data according to the multimodal data;

[0069] A tool planning unit, configured to select and plan enhancement tools for the multimodal data in different question-answering scenarios from a preset tool application set using a large language model based on the question-answering scenarios corresponding to the multimodal data;

[0070] A tool combination unit is used to combine the enhancement tools of the multimodal data in different question-answering scenarios into a tool chain.

[0071] Optionally, the question-answering scenarios in the tool planning unit include: multimodal scientific question-answering scenarios and table question-answering scenarios;

[0072] The enhancement tools in the multimodal scientific question-answering scenario include one or more of the following: parameter knowledge acquisition tools, web page knowledge acquisition tools, and visual enhancement tools;

[0073] The enhancement tools in the table question and answer scenario include one or more of the following: parameter knowledge acquisition tool, program generator, program verifier, program executor and data filter.

[0074] In another aspect, the present invention further provides an electronic device, comprising: at least one processor and a memory; the memory and the processor are connected via a bus;

[0075] The memory is used to store one or more programs;

[0076] When the one or more programs are executed by the at least one processor, the large language model reasoning optimization method based on error strategy feedback as described above is implemented.

[0077] On the other hand, the present invention also provides a computer-readable storage medium having an execution program stored thereon, which, when executed, implements the large language model reasoning optimization method based on error strategy feedback as described above.

[0078] Compared with the prior art, the present invention has the following beneficial effects:

[0079] The present invention provides a large language model reasoning optimization method and system based on error strategy feedback, including: obtaining multimodal data of questions to be answered; according to the multimodal data, using a pre-constructed error-driven model to obtain error feedback information corresponding to the multimodal data, and according to the error feedback information, performing reasoning prediction on the multimodal data to obtain reasoning prediction information; according to the multimodal data, using a large language model, selecting and planning a tool chain corresponding to the multimodal data in a preset tool application set; according to the reasoning prediction information and the tool chain corresponding to the multimodal data, using the large language model to perform reasoning to obtain a reasoning answer result of the multimodal data; wherein the error-driven model is constructed based on a Monte Carlo tree search algorithm; the present application can generate error feedback information of multimodal data through an error-driven model constructed based on a Monte Carlo tree search algorithm; and using a large language model to adaptively plan a tool chain for multimodal data, the large language model can be enabled to adaptively use different tools; the present invention can enhance the ability of a large language model to process and reason about multimodal information by combining error feedback information and enhancement tools. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 A flowchart of a large language model reasoning optimization method based on error strategy feedback provided by the present invention;

[0081] Figure 2 A schematic diagram of a process of constructing an error-driven model using a Monte Carlo tree search algorithm in a large language model reasoning optimization method based on error strategy feedback provided by the present invention;

[0082] Figure 3 A schematic diagram for comparing traditional multimodal question answering and a large language model reasoning optimization method based on error strategy feedback provided by the present invention;

[0083] Figure 4 Schematic diagram of the overall architecture of a large language model reasoning optimization method based on error strategy feedback provided by the present invention

[0084] Figure 5 A schematic diagram of the optimization process of the error feedback model of the Monte Carlo search tree algorithm in a large language model reasoning optimization method based on error strategy feedback provided by the present invention in a table reasoning scenario;

[0085] Figure 6 An example diagram of tool selection for adaptive tool-enhanced reasoning in a table reasoning scenario in a large language model reasoning optimization method based on error strategy feedback provided by the present invention;

[0086] Figure 7 A schematic diagram of the structure of a large language model reasoning optimization system based on error strategy feedback provided by the present invention;

[0087] Figure 8 The present invention provides a schematic structural diagram of an electronic device. DETAILED DESCRIPTION

[0088] The present invention proposes a large language model reasoning optimization method and system based on error strategy feedback. The specific implementation methods of the present invention are further described in detail below in conjunction with the accompanying drawings.

[0089] Embodiment 1:

[0090] The present invention provides a large language model reasoning optimization method based on error strategy feedback, the flow chart is as follows Figure 1 As shown, including:

[0091] Step 1: Obtain multimodal data for the question to be answered;

[0092] Step 2: Based on the multimodal data, the error-driven model built in advance is used to obtain error feedback information corresponding to the multimodal data, and based on the error feedback information, the multimodal data is inferred and predicted to obtain inference prediction information;

[0093] Step 3: Based on the multimodal data, use the large language model to select and plan the tool chain corresponding to the multimodal data in the preset tool application set;

[0094] Step 4: Based on the tool chain corresponding to the inference prediction information and multimodal data, use the large language model to perform inference and obtain the inference answer result of the multimodal data;

[0095] Among them, the error-driven model is built based on the Monte Carlo tree search algorithm.

[0096] Currently, existing large language models are mainly based on unimodal data (such as plain text data) for reasoning and question answering. Large language models learn rich language patterns and semantic relationships by pre-training on large-scale text datasets. After pre-training, these models can be fine-tuned on specific tasks to perform tasks such as reasoning and question answering. Through the above training process, the large language model performs well in understanding natural language and can handle complex grammatical structures and contextual relationships. However, this unimodal large language model only relies on the text modality. When processing tasks that require the integration of visual and textual information, the model's ability to process non-text information (such as images, audio, etc.) is limited. Therefore, although the unimodal large language model has made remarkable achievements in text processing, its performance is still limited in scenarios that require the integration of multiple modal data. In addition, the existing technology fails to make full use of the training set data to effectively generate error feedback, which makes the collection efficiency of error feedback of the large language model low, resulting in a decrease in the generalization ability of the model. In response to the above problems, the present invention introduces multimodal data for reasoning question and answering, and in the process of reasoning question and answering multimodal data, selects a suitable error collection algorithm to construct an error-driven model to provide error feedback for the data in training. This not only overcomes the above limitations of the current large language model, but also effectively balances the breadth of the search space and the computational cost, greatly expands the application scope of the large language model, and improves its performance in complex tasks.

[0097] For example, the multimodal data may include: text information and additional information; the additional information may include: image information and / or table information;

[0098] In one implementation, the above error-driven model may include the following construction process:

[0099] Using the sample multimodal information in the preset data set as the input data of the training data;

[0100] The correct answer corresponding to the sample multimodal information is used as the output data of the training data;

[0101] Based on the input data and output data, an initial error-driven model is trained;

[0102] Input the input data into the initial error-driven model to obtain the predicted value corresponding to the input data;

[0103] Based on the predicted values ​​and output data, the error-driven model is constructed using the Monte Carlo tree search algorithm.

[0104] The error-driven model constructed in the above manner uses multimodal data as training data to improve the robustness of the large language model when processing different types of data. Even if the data quality of a certain modality is not high, the model can rely on data from other modalities to make accurate predictions. Secondly, the error-driven model construction process of the present invention adopts the Monte Carlo Tree Search algorithm (MCTS) to identify valuable error feedback in the training data set to find an ideal balance between the exploration and utilization of decision trees. It can help the model better explore the unknown state space during the training process, focus on navigating in a broad prompt space, and thus improve the generalization ability of the large language model on unseen data.

[0105] In this implementation, if Figure 2 As shown, the process of constructing the error-driven model based on the predicted value and the output data using the Monte Carlo tree search algorithm may specifically include:

[0106] Step S1: Initialize a decision tree, use sample multimodal information to represent the state of each node in the decision tree, use the output data corresponding to the sample multimodal information to represent the action of the edge between each node from the current state to the next state, and use the loss function value between the output data and the predicted value to represent the feedback error of the action of each node from the current state to the next state;

[0107] Step S2: Calculate the supremum confidence interval value of each node in the decision tree, and group the nodes with the largest supremum confidence interval value into a node group with promising nodes;

[0108] Step S3: Expand each node in the node group using the state-action value function to obtain a new child node corresponding to each node, and generate an answer error query based on the new child node and the corresponding feedback error;

[0109] Step S4: Based on the answer to the wrong query, the optimizer performs error information identification to obtain error feedback information of the new child node; and the error feedback information is back-propagated to other nodes of the decision tree, and the statistical information of each node in the decision tree is updated;

[0110] Step S5: Repeat steps S2-S4 until a predetermined stop condition is reached to obtain an error drive model.

[0111] For example, the above-mentioned error feedback information may include one or more of the following: visual error information, text error information and integrated error information.

[0112] The goal of this implementation method to optimize the consideration through the MCTS algorithm is to find the optimal consideration (for example, expressed by D*) to maximize the performance of the metric function (for example, expressed by R′), which is formally expressed as follows:

[0113]

[0114] Where ω represents the sample space of error feedback; D represents the error feedback information; D * represents the optimal consideration in the sample space; The input is (q i ,e i ,D)’s large language model output; where q i represents the i-th text information; e i represents the additional information corresponding to the i-th text information; R′ represents the metric function;

[0115] For example, the expression corresponding to the above state-action value function is as follows:

[0116]

[0117] Where V represents the state value function value output by the state-action value function; S represents the state space; A represents the action space; R represents the set of real numbers; In this example, the MCTS algorithm strategically expands the tree by maintaining the state-action value function, which can reduce dependence on specific training data, reduce the risk of overfitting, and show better generalization ability. MCTS implements this strategic planning by gradually building a tree structure, in which each node represents the state described by the natural language considered, and each edge represents the action of the transition state. The node class has the following definitions: access time, operation, parent node, child node list, victory, and reward. And by defining the reward as the upper confidence range applied to the nodes in the tree, and defining the win as the accuracy measure on the data set. Specifically, when building the error feedback model through the MCTS algorithm, four operations are iteratively performed: selection, expansion, simulation, and backpropagation:

[0118] Selection phase: Select the most promising nodes at each level for further expansion and exploration. In each iteration, it starts from the root node (for example, s0), traverses each tree layer, selects a child node at each layer, until it reaches the leaf node. When selecting child nodes at each layer, the present invention uses the upper confidence interval (Upper Confidence boundapplied to Trees, UCT) algorithm, which is known for its balance between selecting high-value nodes (utilization) and selecting nodes with fewer visits (exploration). For example, the upper confidence interval value obtained according to the UCT algorithm is calculated as follows:

[0119]

[0120] Among them, UCT represents the supremum confidence interval value; s t Represents the state value of the node at time t; a t represents the action value of the node at time t; C represents the adjustable parameter value; V represents the state value function value; N(s t ) indicates that the state value is s t The total number of times the node is visited; n i (s t ) indicates that the state value is s t The number of visits to the child node i corresponding to the node of . In this example, the present invention relies on the state-action value function V(s t ,α t ) to predict and estimate the current state-action pair (s t ,α t ) can select the most promising action path, thereby improving the quality of decision making and making better choices in the face of uncertainty and complexity.

[0121] Expansion phase: Promote the growth of the decision tree by adding new child nodes to the leaf nodes selected in the previous phase. The input data of the model includes q i ,e i and D, these inputs prompt the optimizer (e.g., using p B According to p B (a i ∣q i ,e i , D0) for prediction, where D0 is used as the initial consideration hint to avoid expected errors; i represents the i-th text information; e i represents the additional information corresponding to the i-th text information; D represents the error feedback information. Crucially, this stage only retains the responses that the basic LLM considers to be incorrect (i.e. Represents the output data of the model's i-th text information, a i The correct answer to the i-th text information of the model), and select a part of it to generate incorrectly answered query pairs: These pairs are input to an optimizer p B is responsible for error identification and outputs the best output of the optimizer (for example, using D i Indicates), where D i Represents the optimizer p B The best output for the i-th text information, that is, the consideration factor. It should be noted that D iIt's not the error itself, but what the model should do to avoid it.

[0122] Simulation phase: Traditional Monte Carlo Tree Search (MCTS) technology usually includes vertical simulation, just like the game of Go, the simulation will continue until the end of the game. However, given that the optimization state is different from traditional games and there is no clear end scenario, the present invention improves the traditional MCTS algorithm and enhances the traditional MCTS algorithm by adopting horizontal simulation (also called horizontal simulation strategy). This improvement enables LLM to effectively synthesize multiple errors encountered in the optimization process. This modification enables LLM to effectively synthesize multiple errors encountered in the optimization process, and can converge to the optimal solution faster, thereby improving the speed and transparency of the optimization process. Specifically, we instruct LLM to summarize these n considerations into a {S1|D1,D2,…D i …,D n In this way, we have completed the state expansion from S0 to S1, where S0 and S1 represent the state of each node in the decision tree respectively; D1 represents the consideration of the first text information; D2 represents the consideration of the second text information; D i Indicates the consideration factor of the i-th text information; D n Indicates the consideration factor of the nth text information; n represents the total number of text information;

[0123] Back propagation phase: Update the state of the current node (for example, the node state at time t is s t ) and the state-value function values ​​and visit counts of all its predecessor nodes, which is performed when the terminal state is reached, defined as completing a predetermined number of MCTS iterations. This phase is crucial in updating the values ​​and visit counts of the currently considered node and all its predecessor nodes. This update process is crucial because it reflects the knowledge and adjustments accumulated during the MCTS search process, promoting the model to learn from its interactions. More specifically, each state-action pair (s t ,α t ) will be updated. This recalibration involves summarizing the t The rewards for all possible paths issued are as follows: Among them, M represents the state s t The total number of possible future states that the node starts from. and Respectively represent from s t and α t The jth potential state and corresponding action emitted; V *represents the updated value of the state-value function; this notation highlights a method for systematically improving the model's prediction accuracy and decision-making effectiveness by obtaining feedback from a large number of simulated future scenarios.

[0124] like Figure 3 The figure shows the comparison between traditional multimodal question answering and the present invention: the left part of the figure represents traditional multimodal question answering, while the right part shows the workflow of a specific case of the present invention in multimodal scientific question answering and the corresponding generated responses. For example, suppose there is a set of labeled instances The input is a question (for example, the i-th text message question uses q i ) and the corresponding additional information (table or image, for example, the additional information corresponding to the i-th text information is represented by e i The output is the correct answer (for example, the correct answer corresponding to the i-th text information is a i ), n is the total number of data sets, Q is the text information set, E is the additional information set, and A is the correct answer set. The goal of the combined reasoning task is to train a model on the training data set or find the optimal hint of the frozen LLM. i ,e i ) Generate the answer The present invention can more accurately understand questions and related contexts by introducing additional information (such as images, tables, etc.) and a feedback mechanism for errors, thereby generating more precise answers.

[0125] Compared with previous prompt optimization methods, this invention innovatively utilizes multimodal data, including text, images, and tables, and further applies the concept of strategic planning. This method significantly improves the reasoning ability of large language models (LLMs), complements their inherent advantages, and thus enhances their ability to process and reason about multimodal information. In addition, the Monte Carlo tree search algorithm is applied to the challenge of multimodal prompt optimization, strategically navigating in the vast error-driven consideration space, while balancing exploration and exploitation in the search for high-return error feedback paths, thereby generating the most general consideration prompts (referring to the influencing factors that LLM must consider when reasoning), and then summarizing considerations from error feedback. This method effectively utilizes training set data and efficiently identifies high-return considerations.

[0126] The current research methods fail to effectively build a diverse tool set, making the use of tools quite rigid and lacking in flexibility and adaptability. Therefore, in response to the above problems, the present invention proposes a tool enhancement method, which uses large language models (LLMs) to flexibly design tool chains from a diverse tool set, so that large language models can autonomously plan different tools required for different problems, and solve different reasoning queries by adaptively selecting and planning enhanced tools and utilizing the contextual learning advantages of LLMs, so that large models can adaptively use different enhanced tools according to different data sets to solve various reasoning problems.

[0127] In one implementation, the process of selecting and planning a tool chain corresponding to the multimodal data in a preset tool application set based on the multimodal data and using a large language model in step 3 may include:

[0128] According to the multimodal data, the question-answering scenario corresponding to the multimodal data is obtained;

[0129] Based on the question-answering scenarios corresponding to the multimodal data, a large language model is used to select and plan enhancement tools for the multimodal data in different question-answering scenarios from a preset tool application set;

[0130] The enhancement tools for the multimodal data in different question-answering scenarios are combined into a tool chain.

[0131] For example, the process of selecting and planning the enhancement tools of the multimodal data in different question-answering scenarios from a preset tool application set using a large language model may include:

[0132] For the text information containing the question (for example, the i-th text information uses q i representation) and additional information (such as text information q i The corresponding additional information is e i Represents the model input, the basic large language model is based on the optimizer p B (a i |q i , O) to predict and output the inference prediction information, where O represents the enhancement tool output by the preset tool application set for the i-th text information. t When evaluating, the above inference prediction information at time t (for example, using y t The corresponding execution output formula is:

[0133] y t ←U t (x t-1 ;c t-1 )

[0134] Among them, y trepresents the inference prediction information of the large language model for multimodal data at time t; x t -1 represents the enhancement tool U at time t-1 t The input (i.e., multimodal data) is t-1 The representation encapsulates the cached information (e.g., retrieval, program, visual enhancer) accumulated from the execution at time t-1. t-1 And subsequent enhancement tools U t Cache c t -1 Update with the following calculation:

[0135] x t ←update_input(x t-1 ,y t )

[0136] c t ←update_cache(c t-1 ,y t )

[0137] Among them, update_input means modifying the input query, including the text information containing the question and additional information (such as tables, contexts, and images), and updating after the enhancement tool is executed. In contrast, update_cache means generating new data, such as information enhanced by retrieval or tables; t represents the input of the enhancement tool at time t; c t The representation encapsulates the cache information accumulated from the execution at time t. Finally, this information is combined with the optimal consideration D of the error-driven strategy. * Combined, the large language model produces the following results:

[0138] O=y T ←LLM B (x T-1 ;c T-1 ;D * )

[0139] Among them, 0 represents the enhanced tool of the preset tool application set for the output of the i-th text information; y T represents the inference prediction information of the large language model B for multimodal data at time T; x T represents the input of the enhancement tool at time T; c T Indicates that it encapsulates the cache information accumulated from the execution at time T; D *Represents the optimal consideration in the sample space; in this example, by continuously adjusting the input and cache information, the system can better cope with changes in the environment and increase its flexibility in facing different problems. This method uses the dynamic synergy between the enhancement tool and LLM to continuously improve the large language model's understanding and processing of contextual information, which not only enhances the model's ability to interpret and reason about complex data sets, but also significantly improves the model's adaptability to various tasks and challenges.

[0140] For example, the above-mentioned question-answering scenarios may include: multimodal scientific question-answering scenarios and table question-answering scenarios;

[0141] For example, the enhancement tools in the above multimodal scientific question-answering scenario include one or more of the following: parameter knowledge acquisition tools, web page knowledge acquisition tools, and visual enhancement tools;

[0142] For example, the enhancement tools in the above-mentioned table question and answer scenario include one or more of the following: parameter knowledge acquisition tool, program generator, program verifier, program executor and data filter.

[0143] In this implementation, the tool calling method used in the present invention is compared with the calling method in the prior art. The method of the present invention includes a rich tool application set, which effectively solves the flexibility and adaptability problems when calling external tools, and enables LLM to adaptively use different tools according to different data sets.

[0144] In view of the problems in the prior art that error feedback cannot be generated for multimodal information during reasoning, and that the large language model is rigid and inflexible when calling external tools, resulting in low accuracy of the large language model when processing and reasoning multimodal information, the present invention proposes a large language model reasoning optimization method based on error strategy feedback, such as Figure 4 As shown, the overall process consists of two main parts: on the left is strategic error feedback planning based on Monte Carlo Tree Search (MCTS), and on the right is adaptive tool-enhanced reasoning. The main functions are as follows: First, error-driven consideration optimization through MCTS aims to strategically navigate the huge error feedback space and balance the exploration and utilization of high-return error feedback discovered from the training data set through MCTS; then, by building an adaptive self-planning tool chain, it aims to utilize the contextual learning ability of LLM to adaptively plan from a rich tool set to form different tool chains for various reasoning problems. In the present invention, according to the data set requirements corresponding to the multimodal data of the question to be answered, the appropriate enhancement tools are adaptively planned and integrated into the reasoning process, thereby enhancing the reasoning ability of the large language model; and error feedback is integrated with external tools to overcome a series of defects in reasoning.

[0145] Embodiment 2:

[0146] A specific embodiment is used to illustrate the process of reasoning and answering questions based on a large language model reasoning optimization method based on error strategy feedback proposed by the present invention in a multimodal scientific question-answering scenario and a table question-answering scenario. Specifically, taking the existing large language model gpt-3.5-turbo as an example, answer generation and knowledge acquisition are performed through the large language model, and the temperature is set to 0.0℃ to ensure the certainty and stability of the model output; in addition, Qwen-VL-Chat is selected as a visual enhancement tool, and for the error-driven strategy consideration module, llava2-7b is selected as the basic large language model to be optimized, and GPT-4 is selected as the optimizer for Monte Carlo tree search and consideration optimization; for the simulation of MCTS, in the actual experiment, the formula {S1|D1,D2,…,D n The total number of factors considered in} is set to 5, and the accuracy is used as the evaluation index of the experimental results in the test phase to represent the final accuracy of the reasoning question answering. The following describes the process of the present invention for two problem scenarios:

[0147] 1. Multimodal scientific question answering

[0148] Take the multimodal dataset ScienceQA as an example. ScienceQA is a dataset designed for multimodal questions in the field of science. It is the first multimodal science question-answering dataset, which contains 21,208 questions and 10,332 pictures. Each picture contains an average of 12.1 questions, and each picture contains a maximum of 141 questions. The dataset covers grades 1 to 12. Compared with previous datasets, ScienceQA covers a variety of topics such as natural sciences, social sciences, and language sciences. In addition, most questions are accompanied by relevant lectures and detailed explanations to provide a more comprehensive learning experience.

[0149] In the adaptive toolchain construction reasoning part, the large language model autonomously selects submodules and toolchains suitable for the ScienceQA dataset. These toolchains include: retrieval enhancement submodule (parameter knowledge acquisition and network knowledge acquisition), and visual enhancement submodule (equipped with visual enhancement tools), as shown in Table 1.

[0150] Table 1 Adaptive selection tools

[0151]

[0152] According to the content in Table 1, the large language model contains a retrieval enhancement submodule, a visual enhancement submodule, a program enhancement submodule, and a table enhancement submodule. Among them, the retrieval enhancement submodule obtains knowledge related to the problem from the knowledge base to enhance the background knowledge of the problem. Specifically, it can include two tools: parameterized knowledge acquisition and network knowledge acquisition; the visual enhancement submodule is used to refine the important information in the image into a concise description; the program enhancement submodule is mainly composed of three tools: program generator, program verifier, and program executor; among them, the program generator is responsible for interpreting the problem description and writing a Python program that proposes a potential solution. After the generation stage, the program verifier intervenes to ensure the correctness and reliability of the generated code. Finally, the program executor is responsible for running the verified Python program; the table enhancement submodule is used to extract and present the most relevant information from the table.

[0153] In order to more directly demonstrate how the present invention gradually utilizes error descriptions (or operations) to enhance the prompts (or states) of error feedback, the present invention examines the optimization trajectory of multimodal data in the reasoning question-answering process through qualitative analysis. First, in the formula {S1|D1,D2,…,D n}, the number of factors to be considered n is set to 1, that is, each error feedback description is directly used as a status prompt. For example, Figure 5 As shown in the figure, the figure shows the three intermediate states and corresponding transitions under the optimal reward path (reward values ​​are 1.034, 1.302 and 1.467 respectively). Specifically, first, a specific description of the error of the problem is given. From the figure, it can be observed that the error description of the problem is specific, while the error feedback description abstracts the error into a guiding instruction, indicating what the model should do, which is not related to the specific problem, but is related to the data set. For example, if the error description is "misunderstanding of the stem-and-leaf plot", the corresponding error feedback will be "accurately use the provided data..."

[0154] (II) Questions and Answers on the Form

[0155] TabMWP contains 38,431 questions and pictures, with one picture for each question, showing a one-to-one correspondence. The dataset contains an average of 22.1 questions per picture, and a single picture can contain up to 92 questions. TabMWP focuses on the field of mathematics, and its content is presented through images and text in a tabular form. TabMWP's dataset responses include multiple-choice options as well as numerical answers, and provide explanations for each question.

[0156] In the adaptive toolchain construction reasoning part, unlike the multimodal science question answering (ScienceQA dataset), TabMWP provides a corresponding natural language description for each question image, so the model does not need a visual enhancement submodule. However, due to the complex calculations involving tables, the model requires a program to eliminate possible computational errors. Therefore, the toolchain that TabMWP can choose includes: retrieval enhancement submodule (parameter knowledge acquisition); program enhancement submodule (program generator; program verifier; program executor); and table enhancement submodule (data filter).

[0157] Further, we will study the effectiveness of reasoning for building an adaptive self-planning toolchain. Figure 6 An example from a table question answering scenario is shown, which explores the model's understanding of stem-and-leaf plots and tables. Based on the adaptive toolchain building module, the model selects different tools based on different question types, and each tool generates its corresponding response, such as Figure 6 shown.

[0158] The device embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the modules displayed as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of the present invention. Those of ordinary skill in the art may understand and implement them without creative effort.

[0159] The present invention proposes a method and device for enhancing LLM with adaptive tools based on strategic error feedback, which integrates error feedback with external tools, aiming to make up for the shortcomings of LLM reasoning and enhance the reasoning and problem-solving capabilities of LLM. The present invention enables large models to adaptively plan different tools for different problems, a process called adaptive self-planning. In addition, the Monte Carlo tree search algorithm is applied to the challenge of multimodal prompt optimization, strategically navigating a vast space driven by error considerations and the balance between exploration and development, effectively utilizing data from the training set, and effectively identifying considerations with high potential returns. At the same time, it covers two scenarios: multimodal scientific question answering and table question answering, which can quickly match various simple or complex problems in practical applications.

[0160] To address the rigidity, inflexibility, and lack of domain specificity in current tool invocation methods, answer enables large models to adaptively plan different tools for different problems, a process known as adaptive self-planning. In addition, experimental results show that the invention outperforms competitive benchmarks and state-of-the-art techniques, and can also improve the reasoning ability of LLMs to solve complex mathematical and scientific problems.

[0161] Embodiment 3:

[0162] The present invention based on the same inventive concept also provides a large language model reasoning optimization system based on error strategy feedback, the structural composition diagram is as follows Figure 7 As shown, including:

[0163] A data acquisition module, used to acquire multimodal data of questions to be answered;

[0164] An error feedback module is used to obtain error feedback information corresponding to the multimodal data based on the multimodal data using a pre-built error-driven model, and to perform inference prediction on the multimodal data based on the error feedback information to obtain inference prediction information;

[0165] The tool planning module is used to select and plan the tool chain corresponding to the multimodal data in the preset tool application set based on the multimodal data and using the large language model;

[0166] The reasoning answer module is used to perform reasoning based on the tool chain corresponding to the reasoning prediction information and the multimodal data using the large language model to obtain the reasoning answer result of the multimodal data;

[0167] Among them, the error-driven model is built based on the Monte Carlo tree search algorithm.

[0168] In a possible implementation, the error-driven model in the above-mentioned error feedback module may include the following construction process:

[0169] Using the sample multimodal information in the preset data set as the input data of the training data;

[0170] The correct answer corresponding to the sample multimodal information is used as the output data of the training data;

[0171] Based on the input data and output data, an initial error-driven model is trained;

[0172] Input the input data into the initial error-driven model to obtain the predicted value corresponding to the input data;

[0173] Based on the predicted values ​​and output data, the error-driven model is constructed using the Monte Carlo tree search algorithm.

[0174] In a possible implementation, the process of constructing an error-driven model in the error feedback module based on the predicted value and the output data using the Monte Carlo tree search algorithm may include:

[0175] Step S1: Initialize a decision tree, use sample multimodal information to represent the state of each node in the decision tree, use the output data corresponding to the sample multimodal information to represent the action of the edge between each node from the current state to the next state, and use the loss function value between the output data and the predicted value to represent the feedback error of the action of each node from the current state to the next state;

[0176] Step S2: Calculate the supremum confidence interval value of each node in the decision tree, and group the nodes with the largest supremum confidence interval value into a node group with promising nodes;

[0177] Step S3: Expand each node in the node group using the state-action value function to obtain a new child node corresponding to each node, and generate an answer error query based on the new child node and the corresponding feedback error;

[0178] Step S4: Based on the answer to the wrong query, the optimizer performs error information identification to obtain error feedback information of the new child node; and the error feedback information is back-propagated to other nodes of the decision tree, and the statistical information of each node in the decision tree is updated;

[0179] Step S5: Repeat steps S2-S4 until a predetermined stop condition is reached to obtain an error drive model.

[0180] For example, the calculation formula corresponding to the supremum confidence interval value in the above error feedback module is as follows:

[0181]

[0182] Among them, UCT represents the supremum confidence interval value; s t Represents the state value of the node at time t; a t represents the action value of the node at time t; C represents the adjustable parameter value; V represents the state value function value; N(s t ) indicates that the state value is s t The total number of times the node is visited; n i (s t ) indicates that the state value is s t The number of times the child node i corresponding to the node is visited.

[0183] For example, the expression corresponding to the state-action value function in the above error feedback module is as follows:

[0184]

[0185] Among them, V represents the state value function value output by the state-action value function; S represents the state space; A represents the action space; R represents the set of real numbers.

[0186] For example, the multimodal data in the above-mentioned data acquisition module includes: text information and additional information;

[0187] By way of example, the above-mentioned additional information includes: image information and / or table information;

[0188] For example, the error feedback information in the above-mentioned error feedback module includes one or more of the following: visual error information, text error information and integrated error information.

[0189] In a possible implementation, the tool planning module may include:

[0190] A scene recognition unit, used to obtain a question-answering scene corresponding to the multimodal data based on the multimodal data;

[0191] A tool planning unit is used to select and plan enhanced tools for multimodal data in different question-answering scenarios from a preset tool application set using a large language model based on the question-answering scenarios corresponding to the multimodal data;

[0192] The tool combination unit is used to combine the enhancement tools of multimodal data in different question-answering scenarios into a tool chain.

[0193] Optionally, the question-answering scenarios in the tool planning unit include: multimodal scientific question-answering scenarios and table question-answering scenarios;

[0194] For example, the enhancement tools in the above multimodal scientific question-answering scenario include one or more of the following: parameter knowledge acquisition tools, web page knowledge acquisition tools, and visual enhancement tools;

[0195] For example, the enhancement tools in the above-mentioned table question and answer scenario include one or more of the following: parameter knowledge acquisition tool, program generator, program verifier, program executor and data filter.

[0196] Embodiment 4:

[0197] like Figure 8 As shown, the present invention also provides an electronic device, which may be a computer device, a single-chip device, an intelligent mobile device, etc. The electronic device in this embodiment may include a processor, a memory, a transceiver component, etc. The memory, the processor, and the transceiver component are connected via a bus; the memory may be used to store an execution program, and an exemplary execution program may include instructions; the processor is used to execute the instructions stored in the memory. The memory may also be used to store data, which may be called and / or modified when the instructions are executed.

[0198] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in a storage medium to implement the corresponding method flow or corresponding functions, so as to implement the steps of a large language model reasoning optimization method based on error strategy feedback in the above embodiment.

[0199] Embodiment 5:

[0200] Based on the same inventive concept, the present invention also provides a readable storage medium, specifically an electronic device readable storage medium (Memory), which is a memory device in an electronic device for storing programs and data. It can be understood that the storage medium here can include both built-in storage media in electronic devices and, of course, extended storage media supported by electronic devices. The storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more execution programs (including program codes). It should be noted that the storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage. The processor loads and executes one or more instructions stored in the storage medium, which can implement the steps of a large language model reasoning optimization method based on error strategy feedback in the above embodiment.

[0201] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0202] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0203] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0204] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0205] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit its protection scope. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that after reading the present invention, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the application, but these changes, modifications or equivalent substitutions are all within the protection scope of the claims to be approved.

Claims

1. A large language model reasoning optimization method based on error strategy feedback, characterized in that: include: Obtain multimodal data for the questions to be answered; According to the multimodal data, using a pre-built error-driven model, error feedback information corresponding to the multimodal data is obtained, and according to the error feedback information, inference prediction is performed on the multimodal data to obtain inference prediction information; According to the multimodal data, using a large language model, selecting and planning a tool chain corresponding to the multimodal data in a preset tool application set; Performing reasoning using the large language model according to the inference prediction information and a tool chain corresponding to the multimodal data to obtain an inference answer result of the multimodal data; Wherein, the error-driven model is constructed based on the Monte Carlo tree search algorithm.

2. The method according to claim 1, characterized in that The error-driven model includes the following construction process: Using the sample multimodal information in the preset data set as the input data of the training data; Using the correct answer corresponding to the sample multimodal information as output data of training data; Based on the input data and the output data, training an initial error-driven model; Inputting the input data into the initial error-driven model to obtain a predicted value corresponding to the input data; Based on the predicted value and the output data, an error-driven model is constructed using a Monte Carlo tree search algorithm.

3. The method according to claim 2, characterized in that The method of constructing an error-driven model based on the predicted value and the output data using a Monte Carlo tree search algorithm includes: Step S1: Initialize a decision tree, use the sample multimodal information to represent the state of each node in the decision tree, use the output data corresponding to the sample multimodal information to represent the action of the edge between each node from the current state to the next state, and use the loss function value between the output data and the predicted value to represent the feedback error of the action of each node from the current state to the next state; Step S2: calculating the supremum confidence interval value of each node in the decision tree, and forming a node group of promising nodes with nodes having the largest supremum confidence interval value; Step S3: Expand each node in the node group using a state-action value function to obtain a new child node corresponding to each node, and generate an answer error query based on the new child node and the corresponding feedback error; Step S4: Based on the incorrectly answered query, error information is identified by the optimizer to obtain error feedback information of the new child node; the error feedback information is back-propagated to other nodes of the decision tree, and statistical information of each node in the decision tree is updated; Step S5: Repeat steps S2-S4 until a predetermined stop condition is reached to obtain an error drive model.

4. The method according to claim 3, characterized in that The calculation formula corresponding to the supremum confidence interval value is as follows: Among them, UCT represents the supremum confidence interval value; s t Represents the state value of the node at time t; a t represents the action value of the node at time t; C represents the adjustable parameter value; V represents the state value function value; N(s t ) indicates that the state value is s t The total number of times the node is visited; n i (s t ) indicates that the state value is s t The number of times the child node i corresponding to the node is visited.

5. The method according to claim 3, characterized in that The expression corresponding to the state-action value function is as follows: Among them, V represents the state value function value output by the state-action value function; S represents the state space; A represents the action space; R represents the set of real numbers.

6. The method according to any one of claims 1 to 5, characterized in that: The multimodal data includes: text information and additional information; The additional information includes: image information and / or table information; The error feedback information includes one or more of the following: visual error information, text error information, and integrated error information.

7. The method according to claim 1, characterized in that The method of selecting and planning a tool chain corresponding to the multimodal data in a preset tool application set based on the multimodal data and using a large language model includes: According to the multimodal data, obtaining a question-answering scenario corresponding to the multimodal data; Based on the question-answering scenarios corresponding to the multimodal data, using a large language model, select and plan enhancement tools for the multimodal data in different question-answering scenarios from a preset tool application set; The enhancement tools for the multimodal data in different question-answering scenarios are combined into a tool chain.

8. The method according to claim 7, characterized in that The question-and-answer scenarios include: multimodal scientific question-and-answer scenarios and table question-and-answer scenarios; The enhancement tools in the multimodal scientific question-answering scenario include one or more of the following: parameter knowledge acquisition tools, web page knowledge acquisition tools, and visual enhancement tools; The enhancement tools in the table question and answer scenario include one or more of the following: parameter knowledge acquisition tool, program generator, program verifier, program executor and data filter.

9. A large language model reasoning optimization system based on error strategy feedback, characterized in that: include: A data acquisition module, used to acquire multimodal data of questions to be answered; An error feedback module is used to obtain error feedback information corresponding to the multimodal data based on the multimodal data using a pre-built error-driven model, and to perform inference prediction on the multimodal data based on the error feedback information to obtain inference prediction information; A tool planning module, for selecting and planning a tool chain corresponding to the multimodal data in a preset tool application set based on the multimodal data and using a large language model; A reasoning answer module, configured to perform reasoning using the large language model according to the reasoning prediction information and a tool chain corresponding to the multimodal data, to obtain a reasoning answer result of the multimodal data; The error-driven model in the error feedback module is constructed based on the Monte Carlo tree search algorithm.

10. The system according to claim 9, characterized in that The error-driven model in the error feedback module includes the following construction process: Using the sample multimodal information in the preset data set as the input data of the training data; Using the correct answer corresponding to the sample multimodal information as output data of training data; Based on the input data and the output data, training an initial error-driven model; Inputting the input data into the initial error-driven model to obtain a predicted value corresponding to the input data; Based on the predicted value and the output data, an error-driven model is constructed using a Monte Carlo tree search algorithm.

11. The system according to claim 9, characterized in that The tool planning module includes: A scene recognition unit, configured to obtain a question-answering scene corresponding to the multimodal data according to the multimodal data; A tool planning unit, configured to select and plan enhancement tools for the multimodal data in different question-answering scenarios from a preset tool application set using a large language model based on the question-answering scenarios corresponding to the multimodal data; A tool combination unit is used to combine the enhancement tools of the multimodal data in different question-answering scenarios into a tool chain.

12. An electronic device, characterized in that: include: at least one processor and memory; The memory and the processor are connected via a bus; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, a large language model reasoning optimization method based on error strategy feedback as described in any one of claims 1 to 8 is implemented.

13. A computing device readable storage medium, characterized in that: An execution program is stored thereon, and when the execution program is executed, a large language model reasoning optimization method based on error strategy feedback as described in any one of claims 1 to 8 is implemented.

Images