Evolved graph reasoning table question answering method based on large language model

Abstract

The invention discloses an evolved graph reasoning table question answering method based on a large language model, which is characterized in that a table is converted into a graph G = {N, E, V} aiming at a given complex table T and a user question Q, N represents a node set, E represents an edge set, and V represents a corresponding cell value set. Step-by-step reasoning is carried out through graph representation to determine a correct answer A, in the process, a reasoning track Tr composed of visited nodes is dynamically maintained and is represented as a sub-graph, the process enables a reasoning path to be effectively guided to the answer, and irrelevant information is prevented from being interfered. According to the method, explicit reasoning is achieved through a series of intermediate reasoning steps on a graph, header cells do not need to be additionally marked, and therefore a clear reasoning path is constructed, and answers to questions are efficiently found.

Description

Technical Field

[0001] The present invention relates to the field of large language models and table question answering, and in particular to an evolutionary graph reasoning table question answering method based on a large language model. Background Art

[0002] Complex tables in documents often adopt advanced layouts such as merged cells and flexible positions of header cells. This complexity helps to present detailed information in a structured and flexible manner, making it a common feature in financial reports and other professional documents. In this context, complex table question answering (QA) has recently become a vital task to extract valuable information from complex tables. However, the merged cells and flexible layouts contained in complex tables introduce implicit semantic relationships between entities within cells, which brings great challenges to table structure understanding.

[0003] To address these challenges, current approaches focus on prompting large language models (LLMs) to perform implicit reasoning and derive answers based on carefully designed tabular representations. Existing methods propose converting tables into more accessible formats such as tuples, Markdown, and HTML. Some of these methods also involve adding annotated table headers to the converted representations to further improve the model's understanding of complex hierarchical structures. Subsequently, these techniques integrate the converted tabular representations into carefully designed prompts to unlock the reasoning potential of the LLM, enabling it to pinpoint accurate answers in complex tables.

[0004] However, current approaches suffer from two significant limitations. First, they rely heavily on pre-annotated table header cells, assuming that their positions and orientations are known in advance. This assumption does not hold in practice: given a complex table in a document, we cannot predetermine the position of the header cells (the flexibility of complex tables allows header cells to appear in different positions, not just at the top or left edge). This requires alternatives to manually annotating table headers, which is not a practical approach. Second, their reasoning process is usually implicitly embodied in table embeddings or free-form text, making it difficult to provide a structured way to represent the complex relationships between data in different (merged) columns and rows. Therefore, these two limitations greatly reduce the practicality and effectiveness of complex table question answering. Summary of the invention

[0005] In order to overcome the shortcomings and deficiencies of the prior art, the present invention provides an evolutionary graph reasoning table question answering method based on a large language model.

[0006] The technical solution adopted by the present invention is that, given a complex table T and a user question Q, the method converts the table into a graph G={N, E, V}, where N, E and V represent the node set, edge set and corresponding cell value set respectively. Using the graph representation, step-by-step reasoning is performed to determine the correct answer A. In the process, the method dynamically maintains a reasoning trajectory Tr, which is represented as a subgraph composed of visited nodes.

[0007] The model aims to establish a step-by-step reasoning process for complex tabular question answering. At each reasoning step, the model prompts the LLM to select a reasoning action from a set of predefined intermediate reasoning actions. To guide this process, the model introduces a reasoning trajectory Tr to track the nodes visited during the reasoning process, which further helps to select the next intermediate action. Finally, the process terminates when the model selects the "AnswerQuestion" action and produces the final answer.

[0008] An evolutionary graph reasoning table question answering method based on a large language model, the method comprising: Step S1: Initialization of table representation. This method converts the input table T into an undirected graph G = {N, E, V}, where N, E and V represent the node set, edge set and corresponding cell value set, respectively. The edge connects cells in the same row or column. Each node is represented as a triple (R id , C id , V i ), where R id and C id are the row and column indices in the table; each edge is represented as a triple (N i , N j , M ij ), where M ij Indicates whether the two connected nodes are in the same row or column of the table; Step S2: Graph reasoning is used for complex table question answering. This method performs the thinking, action, and update process to perform graph reasoning step by step. In order to guide the reasoning process, the reasoning trajectory Tr is introduced to track the nodes visited during the reasoning process. Step S2.1: Thinking, the method collects scattered information to establish the current reasoning state, considering not only the global information provided by the entire graph, but also local information specific to the input problem. These local information include: the output of previous "thinking" and "acting" steps; historical reasoning trajectories, key information extracted from past reasoning steps (represented as a subgraph), enabling the method to focus on details; and using the semantic understanding ability of LLM to identify header cells and infer the semantic relationship between connected nodes; Step S2.2: Action, which guides the method to select subsequent intermediate actions from the action set R according to the reasoning state.

[0009] Step S2.3: Update. After the method executes the “VisitNode” action, the reasoning trajectory Tr is updated by appending the newly visited nodes. If the newly added nodes share common neighbors with the previously visited nodes or are directly connected in the original graph, an edge is established between them. This process is used to update the local information of the subsequent “Thinking” step. In this way, the method performs iterative reasoning on the graph to find the correct answer. Step S3: Answer Generation,When the “AnswerQuestion” action is selected, the LLM is instructed,through the thought chaining technique, to generate the final answer to the user’s question,based on the question and the current local reasoning state from the,“Thinking” step, instead of using the entire graph.

[0010] Furthermore, in step S1, for the merged cells, the Rid and / or Cid values ​​are represented as a set containing the corresponding split cell indexes; the header cells and the data cells are both regarded as nodes.

[0011] Further, in step S2, the method prompts LLM to think about the current reasoning state according to the current reasoning trajectory and other relevant parameters, and the method selects an action from the reasoning action set R, and updates the trajectory after the method executes the selected action.

[0012] Beneficial effects: The present invention proposes an evolutionary graph reasoning table question answering method based on a large language model, which is named this method, and it uses graph reasoning to answer complex table questions. The method converts a complex table into an undirected graph, in which both header cells and data cells are regarded as nodes without distinction. By analyzing the semantic relationship between connected nodes through LLM, potential header cells can be inferred, which gets rid of the dependence on pre-labeled header cells and enhances the adaptability of the method to different table structures. The method guides LLM to perform step-by-step reasoning on the graph. It dynamically selects actions from a set of predefined intermediate reasoning actions to facilitate the explicit reasoning process. This process allows the method to effectively guide the reasoning path towards the answer and avoid information that is irrelevant to the given question. The method performs explicit reasoning through a series of intermediate steps on the graph without the need for additional labeling of header cells. The present method constructs a clear reasoning path and effectively identifies the answer to a given question. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a flow chart of the method of the present invention; Figure 2 It is a flow chart of the thinking, action and updating process of the present invention. DETAILED DESCRIPTION

[0014] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application may be combined with each other. The present application is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0015] like Figure 1 As shown, an evolutionary graph reasoning table question answering method based on a large language model, the method comprising: Step S1: Table representation initialization. This method converts the input table T into an undirected graph G, where edges connect cells in the same row or column. Specifically, each node is represented as a triple (R id , C id , V i ), where R id and C id are the row and column indices in the table. It is worth noting that for merged cells, their Rid and / or Cid values ​​are represented as a set containing the corresponding split cell indices.

[0016] In addition, each edge is represented as a triple (N i , N j , M ij ), where M ij Indicates whether the two connected nodes are in the same row or column of the table.

[0017] It is worth noting that both header cells and data cells are treated as nodes without distinction. This approach eliminates the reliance on pre-labeled header cells, unlike existing methods that require pre-labeled header cells or assume that header cells are only located at the top or left side of the table. This enhances the adaptability of the method to different table structures.

[0018] In order to start the reasoning process, the method LLM selects a small group of nodes that are most helpful in answering the user's question Q, and uses the retriever to select a small group of nodes that are most semantically relevant to the user's question Q, and takes the union of the two as the initial set of initializing the reasoning trajectory Tr. In the implementation, the application initially selects up to eight most relevant cells. The process can be expressed as:

[0019]

[0020]

[0021] in, is the initial node set, the node set selected by LLM and Retriever according to table T and question Q, expressed as the union of the node sets selected by the two. Represents the edge set between the initial nodes, each edge Representation Node and Whether it is in the same row or column in the table. is the initial set of the inference trajectory initialized by this method, which consists of the initial node set and the corresponding edge set composition.

[0022] Step S2: Graph Reasoning for Complex Tabular Question Answering. The method implements the Think, Act, Update process for step-by-step graph reasoning. To guide the reasoning process, it further introduces a reasoning trajectory Tr to track the nodes visited during the reasoning process. Specifically, in each iteration, the method first prompts the LLM to think about the current reasoning state based on the current reasoning trajectory and other relevant parameters. Based on this thinking process, the method selects an action from the reasoning action set R. Finally, the trajectory is updated after the method executes the selected action, which in turn affects the subsequent reasoning state.

[0023] like Figure 2 As shown, the thinking, action, and update process includes: Step S2.1: Think. The method collects scattered information to build its current reasoning state, considering not only the global information provided by the entire graph, but also local information specific to the input problem. This local information includes: the output of previous "think" and "act" steps; historical reasoning trajectories, key information extracted from past reasoning steps (represented as a subgraph), allowing the method to focus on important details; and identifying header cells using the semantic understanding ability of LLM to infer the semantic relationships between connected nodes.

[0024] Step S2.2: Action. The present application guides the method to select subsequent intermediate actions from the action set R according to the reasoning state. The present application designs a set of simple and flexible reasoning actions, covering the basic operations required for graph reasoning, as shown in Table 1. These actions act as tools to enhance the reasoning performance of the method. The "VisitNode" action helps to select and traverse nodes in the graph. In the case where multiple nodes share the same value, the method combines these nodes and the structural information (C id and R id ) are presented together for the LLM to choose from so that they can make an informed decision.

[0025] Table 1. Reasoning actions on the graph

[0026] In addition, the "GetAllNeighbours" and "GetSharedNeighbours" actions enable the method to focus on local information and analyze the neighbors around the selected node. The application also incorporates the action of answering questions into this set. This is consistent with the concept of "learning to postpone", which prompts the method to automatically evaluate whether more information is needed before answering.

[0027] It is worth noting that, unlike implicit reasoning that embeds the entire table into the prompt, the method of the present application explicitly establishes a reasoning path by performing these actions, which can effectively filter out irrelevant information and guide the reasoning to a more accurate answer.

[0028] Step S2.3: Update. After the method executes the "VisitNode" action, the reasoning trajectory Tr is updated by appending the newly visited nodes. If the newly added nodes share common neighbors with previously visited nodes or are directly connected in the original graph, an edge is established between them. This process effectively updates the local information for the subsequent "Think" steps. In this way, the method forms an iterative reasoning on the graph to find the correct answer.

[0029] Step S3: Answer generation. When the “AnswerQuestion” action is selected, the present application instructs LLM to generate the final answer to the user question based on the question and the current local reasoning state from the “Thinking” step through the thought chaining technique, instead of using the entire graph.

[0030] The goal of the present invention is to establish a clear reasoning process for complex table question and answer without relying on pre-labeled header cells. The method proposed in this application uses graph reasoning to perform complex table question and answer. The method first converts the complex table into an undirected graph, in which both header cells and data cells are regarded as nodes without distinction. By analyzing the semantic relationship between connected nodes through LLM, potential header cells can be inferred, which gets rid of the dependence on pre-labeled header cells and enhances the adaptability of the method to different table structures. Then, the method guides LLM to perform step-by-step reasoning on the graph. It dynamically selects actions from a set of predefined intermediate reasoning actions (such as GetSharedNeighbours and AnswerQuestion) to facilitate the explicit reasoning process. This process allows the method to effectively guide the reasoning path towards the answer and avoid information that is irrelevant to the given question.

[0031] It is worth noting that this method is different from existing graph-based complex table question answering methods, which usually rely on pre-labeled table header cells and GNN-style models to encode the table structure. In contrast, this method explicitly reasoned through a series of intermediate steps on the graph without the need for additional labeled table header cells.

[0032] Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various equivalent changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An evolutionary graph reasoning table question answering method based on a large language model, characterized by , Step S1: Initialize the table representation. Convert the input table T into an undirected graph G = {N, E, V}; where N, E and V represent the node set, edge set and corresponding cell value set, respectively, where the edge connects cells in the same row or column; each node is represented as a triple (R id , C id , V i ), where R id and C id are the row and column indices in the table; each edge is represented as a triple (N i , N j , M ij ), where M ij Indicates whether the two connected nodes are in the same row or column of the table; Step S2: Graph reasoning is used for complex table question answering. This method performs the thinking, action, and update process to gradually perform graph reasoning. The reasoning process is guided by introducing the reasoning trajectory Tr to track the nodes visited during the reasoning process. Step S2.1: Thinking, collecting scattered information to establish the current reasoning state, through the global information provided by the entire graph, and local information specific to the input problem, which includes: the output of previous "thinking" and "action" steps; historical reasoning trajectories, information extracted from past reasoning steps, able to focus on details; and using the semantic understanding ability of LLM to identify header cells and infer the semantic relationship between connected nodes; Step S2.2: Action, guiding the method to select subsequent intermediate actions from the action set R according to the reasoning state; Step S2.3: Update. After the method executes the "VisitNode" action, the reasoning trajectory Tr is updated by appending the newly visited nodes. If the newly added nodes share common neighbors with the previously visited nodes or are directly connected in the original graph, an edge is established between them. This process is used to update the local information of the subsequent "Thinking" step and perform iterative reasoning on the graph to find the correct answer. Step S3: Answer Generation,When the “AnswerQuestion” action is selected, the LLM is instructed,through the thinking chain technique, to generate the final answer to the user’s question,based on the question and the local reasoning state of the current,“thinking” step, instead of using the entire graph.

2. The method for answering questions using an evolutionary graph reasoning table based on a large language model as claimed in claim 1, characterized in that: In the step S1, for the merged cells, the Rid and / or Cid values ​​are represented as a set containing the corresponding split cell indexes; the header cells and the data cells are both regarded as nodes.

3. The method for answering questions using an evolutionary graph reasoning table based on a large language model as claimed in claim 1, characterized in that: In the initialization of the table representation, in order to start the reasoning process, the method instructs the LLM to select a small group of nodes that are most helpful in answering the user's question Q, and uses the retriever to select a small group of nodes that are most semantically relevant to the user's question Q. The union of the two is taken as the initial set of the initial reasoning trajectory Tr. The process expression is: in, is the initial node set, the node set selected by LLM and Retriever according to table T and question Q, expressed as the union of the node sets selected by the two. Represents the edge set between the initial nodes, each edge Representation Node and Whether they are in the same row or column in the table, is the initial set of initial reasoning trajectories, which consists of the initial node set and the corresponding edge set composition.

4. The method for answering questions using an evolutionary graph reasoning table based on a large language model as claimed in claim 1, characterized in that: In step S2, the method uses the thought chain technology to prompt LLM to generate the current reasoning state according to the current reasoning trajectory and other relevant parameters, and then selects an action from the reasoning action set R, and updates the trajectory after executing the selected action.

Images