A complex text-table question answering method based on graph structure

Abstract

This invention relates to the field of intelligent document question answering and knowledge enhancement technology, specifically to a complex text-table question answering method based on graph structure, comprising: S1: obtaining user questions and original tables and their related descriptive text; S2: generating table physical layout mapping results; S3: constructing a semantic topology subgraph of the table; S4: constructing an unstructured text subgraph; S5: constructing a question-driven association evidence graph; S6: constructing a complete global knowledge graph; S7: generating vector indices for each cell node and paragraph node; S8: converting the user question into a query vector and matching it with the vector indices of each cell node and paragraph node to obtain a ranked evidence set; S9: constructing a prompt template input and performing forward inference in a large language model fine-tuned by instructions to output the question answering result. This invention improves the evidence localization ability, answer accuracy, and system interpretability in complex text-table question answering tasks.

Description

Technical Field

[0001] This invention relates to the field of intelligent document question answering and knowledge enhancement technology, specifically to a complex text-table question answering method based on graph structure, belonging to the cross-integration field of natural language processing technology, knowledge graph technology, information retrieval technology and artificial intelligence generation technology. Background Technology

[0002] In practical applications such as financial reports, medical documents, government statistical bulletins, and audit materials, key information is often not only contained in continuous text or structured tables, but is expressed in a mixed form of "text + tables". Especially in complex documents, the meaning of numerical values ​​in tables usually depends on multiple layers of table headers, row and column headings, unit descriptions, and contextual descriptions to be accurately understood, while text paragraphs often supplement and explain the table data, provide conditional constraints, or summarize the results.

[0003] Therefore, when conducting research on intelligent question answering tasks for this type of document, large language models not only need to understand the natural language question itself, but also need to understand the text content, table structure, and the semantic relationships between the two in order to accurately locate the evidence supporting the answer and generate the correct answer. However, with the continuous growth of large-scale document data and the increasing demand for intelligent question answering, how to achieve a unified understanding and efficient utilization of complex text-table mixed information has become an important research direction in the field of intelligent question answering for complex documents.

[0004] To address the aforementioned issues, existing technologies typically employ the following approaches: One approach involves extracting the text portion of a document separately and using traditional information retrieval methods or semantic retrieval methods based on pre-trained language models to perform question-and-answer within the text. Another approach involves expanding tables into linear text sequences or converting them into simple row and column descriptions before inputting them into pre-trained models or large language models for processing. Additionally, some researchers directly input the entire document along with the table content into a large language model, allowing the model to generate answers directly based on contextual understanding. While some improved approaches introduce mechanisms such as vector retrieval and retrieval-enhanced generation, in practice, text and tables are often treated as two separate information sources, lacking a unified, structured, and interconnected representation.

[0005] However, in practical applications, existing technologies still struggle to effectively address the core challenges in complex text-table question-answering scenarios. The primary reason lies in the significant differences in information organization between text and tables. Text represents continuous, unstructured semantic expression, while tables convey meaning through rows, columns, headers, cell positions, and hierarchical relationships. Simply linearizing tables or treating them as ordinary text can easily lead to the loss of structural dependencies between cells and their rows and columns, resulting in incomplete numerical meaning. Furthermore, independently searching text and tables makes it difficult to establish a unified evidentiary link between questions, text paragraphs, and table cells, easily leading to fragmented search results, inaccurate evidence location, and insufficient information integration capabilities across text and tables. In complex question-answering tasks, answers often require a comprehensive judgment based on both textual descriptions and tabular data. Existing methods, lacking a unified semantic organization and evidence connection mechanism, struggle to collaboratively retrieve and utilize heterogeneous information, thus affecting the accuracy and interpretability of the final question-answering results.

[0006] In summary, current technologies have not yet established a unified structured representation of text-table information in complex documents, and a technical method for achieving accurate evidence location in a question-driven context. Therefore, how to unify the semantics of table structure, the semantics of text content, and the relationship between questions and evidence in complex text-table question-answering scenarios, thereby achieving joint modeling and accurate retrieval of heterogeneous information, is a core problem that current technologies urgently need to solve. Summary of the Invention

[0007] To address the shortcomings of the existing technologies, the technical problem to be solved by this invention is: how to provide a complex text-table question-answering method based on graph structure, which improves the evidence location capability, answer accuracy and system interpretability in complex text-table question-answering tasks by performing unified graph modeling of the relationship between text, tables and question evidence, and combining a hybrid retrieval and controllable answer generation mechanism under graph constraints.

[0008] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0009] A complex text-table question answering method based on graph structure includes:

[0010] S1: Obtain the user issues to be processed, as well as the original table and its associated descriptive text;

[0011] S2: Map the original table to a two-dimensional grid space to generate the table's physical layout mapping result;

[0012] S3: Construct a semantic topology subgraph of the table based on the table's physical layout mapping results;

[0013] S4: Construct an unstructured text subgraph based on the relevant descriptive text of the original table;

[0014] S5: Based on user questions, construct a question-driven association evidence graph;

[0015] S6: Construct a complete global knowledge graph based on table semantic topology subgraphs, unstructured text subgraphs, and question-driven association evidence graphs;

[0016] S7: Read the attributes of each cell node and paragraph node in the complete global knowledge graph, and generate vector indices for each cell node and paragraph node;

[0017] S8: Transform the user's question into a query vector and match it with the vector indices of each cell node and paragraph node to obtain a set of sorting evidence;

[0018] S9: Construct a prompt template based on the user's question and the ranked evidence set, and input it into the large language model that has been fine-tuned by instructions to perform forward inference, and output the question and answer results of the user's question.

[0019] Preferably, in step S2, the processing steps for generating the table physical layout mapping result include:

[0020] S201: Through function For the original table Perform structural analysis to generate a regularized mesh. ;

[0021] S202: Through the mapping matrix Record normalized grid The correspondence between each grid position and the original cell;

[0022] S202: Through function For normalized grids and mapping matrices The analysis generates a set of column semantics. Line semantic set and cell entity collection The result of the physical layout mapping of the table.

[0023] Preferably, in step S3, the processing steps for constructing the table semantic topology subgraph include:

[0024] S301: Through function For the original table Generate the root node of the table and add to the global vertex set. middle;

[0025] S302: Through function Traversing column semantic set Create a corresponding set of list header nodes for each column header. and add to the global vertex set. middle;

[0026] S303: via function At the root node of the table With list header node set Create an edge of type [type] between them and add it to the global edge collection. middle;

[0027] S304: via function Traversing line semantic set Create a corresponding set of line title nodes for each line title. and add to the global vertex set. middle;

[0028] S305: Through function At the root node of the table With the set of line header nodes Create an edge of type [type] between them and add it to the global edge collection. middle;

[0029] S306: Traverse the collection of cell entities Each cell in Through function Create a cell node for the cell Get the set of cell nodes and add to the global vertex set. In the middle; create the root node of the table. With cell nodes The cells between contain borders and add to the global edge set. middle;

[0030] S307: Through function Parse the current cell node The column semantic header node Create the current cell node. semantic header node of the column column semantic definition edge and add to the global edge set. middle;

[0031] S308: via function Parse the current cell node Corresponding line semantic title node Create the current cell node. With the corresponding line semantic title node The semantic definition of the edge of the row and add to the global edge set. middle;

[0032] S309: Based on the table root node List header node set Line title node set A collection of cell entities The generated collection of cell nodes As well as edges containing types, cell-containing edges, column semantic definition edges, and row semantic definition edges, construct a table semantic topology subgraph.

[0033] Preferably, in step S4, the processing steps for constructing the unstructured text subgraph include:

[0034] S401: For relevant descriptive text Create the text root node and add to the global vertex set. middle;

[0035] S402: Through function For relevant descriptive text Divide into segments to obtain a set of segments. ;

[0036] S403: Traversing the collection of paragraphs Each paragraph Through function Instantiate the corresponding paragraph node for each paragraph This yields the set of paragraph nodes. And add the paragraph node set to the global vertex set. middle;

[0037] S404: Create the text root node To paragraph node The containment relationship between edges and add to the global edge set. middle;

[0038] S405: Based on text root node Paragraph node set and the text root node Paragraph nodes Construct an unstructured text subgraph based on the containment relationships between edges.

[0039] Preferably, in step S5, the processing steps for constructing the problem-driven association evidence graph include:

[0040] S501: Through function For user issues Create a question and answer instance node and add to the global vertex set. ;

[0041] S502: Through function User issues Decompose the problem to obtain the problem nodes. Answer nodes Paragraph semantic nodes and table semantic nodes and add to the global vertex set. ;

[0042] S503: Through function Create question-and-answer instance nodes With the problem node Answer nodes Paragraph semantic nodes and table semantic nodes The edges that establish the relationships between them are added to the global edge set. middle;

[0043] S504: Traverse each candidate node in the candidate node set, using the function... Positioning and User Issues If the relevant cell node is successfully located, a question-and-answer instance node is created. Add the tabular evidence association edge TAB_EVI between the cell node and the node to the global edge set. middle;

[0044] S505: Traverse each candidate node in the candidate node set, using the function... Positioning and User Issues If the relevant paragraph node is successfully located, a question-and-answer instance node will be created. Add the text evidence associated edge TEX_EVI between the paragraph node and the corresponding edge to the global edge set. middle;

[0045] S506: Based on question-answering instance nodes Problem nodes Answer nodes Paragraph semantic nodes Table semantic nodes The system locates successfully placed cell and paragraph nodes, as well as associated edges, table evidence associated edges, and text evidence associated edges, and constructs a problem-driven associated evidence graph.

[0046] Preferably, in step S6, the processing steps for constructing a complete text-table knowledge graph include:

[0047] S601: Through function Create top-level document node and add to the global vertex set. ;

[0048] S602: Create top-level document node With the root node of the table Relationship between edges and top-level document nodes With question and answer instance nodes Relationship between edges and add to the global edge set. middle;

[0049] S603: Create top-level document nodes With the root node of the text The relationship edges between the nodes and other auxiliary semantic nodes are added to the global edge set. middle;

[0050] S604: Nodes and edges in a table-based semantic topological subgraph, an unstructured text subgraph, and a question-driven association evidence graph, as well as top-level document nodes. Construct a complete global knowledge graph by combining the graph with its corresponding relation edges. .

[0051] Preferably, in step S7, the processing steps for constructing a vector indexing mechanism for graph nodes include:

[0052] S701: Calculate the size of cell nodes and paragraph nodes in the complete global knowledge graph, and use a pagination batch processing mechanism to read the corresponding node attributes in batches to obtain the attributes of each cell node and paragraph node.

[0053] S702: Construct index text from the attributes of each cell node and paragraph node; for paragraph nodes, use the original text content of the corresponding paragraph as index text; for cell nodes, concatenate their table identifier, row path, column path, value and unit into structured natural language description text as index text.

[0054] Relevant descriptive text Top-level document node It contains two types of searchable nodes, namely, the set of paragraph nodes. With cell node collection Candidate node set ;

[0055] For the candidate node set Any candidate node in Its index text Represented as:

[0056] ;

[0057] In the formula: The original text content representing the paragraph node; This indicates that the cell node contains natural language descriptive text that integrates table identifiers, row paths, column paths, values, and units;

[0058] S703: Through a pre-trained sentence vector encoder Batch semantic encoding is performed on the index text of each cell node and paragraph node to obtain each candidate node. node vectors and for node vectors conduct Normalization yields the normalized node vectors for each cell node and paragraph node. ;

[0059] Pre-trained sentence vector encoder The calculation formula is expressed as:

[0060] ;

[0061] In the formula: Indicates candidate nodes Node vectors;

[0062] The normalization calculation formula is expressed as:

[0063] ;

[0064] In the formula: Indicates candidate nodes The normalized node vector;

[0065] S704: Based on the normalized node vectors of each cell node and paragraph node, construct an independent FAISS vector index for each cell node and paragraph node;

[0066] The formula is expressed as:

[0067] ;

[0068] ;

[0069] In the formula: Represents cell node FAISS vector index; Represents paragraph nodes FAISS vector index.

[0070] Preferably, in step S8, the processing steps for constructing a document-level constrained graph retrieval tool include:

[0071] S801: Through a pre-trained sentence vector encoder User issues Encode and then proceed Normalization yields user questions Normalized query vector ;

[0072] S802: Based on Normalized Query Vector Calculate user problems With top-level document nodes candidate node set Each candidate node semantic similarity score ;

[0073] The formula is expressed as:

[0074] ;

[0075] In the formula: Indicates user issues With candidate nodes Semantic similarity score; Indicates candidate nodes The normalized node vector;

[0076] S803: Calculate the candidate node set Each candidate node terms Inverse document frequency, and based on each term Inverse document frequency calculation user problem With each candidate node BM25 sparse score;

[0077] The formula for calculating inverse document frequency is expressed as:

[0078] ;

[0079] In the formula: Indicates terms Inverse document frequency; Represents the set of candidate nodes The total number of candidate nodes; Represents the set of candidate nodes Contains terms The number of candidate nodes;

[0080] The formula for calculating the BM25 sparsity score is as follows:

[0081] ;

[0082] In the formula: Indicates user issues With candidate nodes BM25 sparse score; Indicates terms In candidate nodes Corresponding word frequencies in the text; Represents the set of candidate nodes Average text length; Indicates user issues The set of word segments; , Indicates preset parameters;

[0083] S804: Regarding user issues With each candidate node semantic similarity score And BM25 sparse score After score normalization, the two normalized scores are linearly fused to obtain the candidate nodes. Mixed retrieval scores;

[0084] The formula for calculating the mixed retrieval score is expressed as follows:

[0085] ;

[0086] In the formula: Indicates user issues With candidate nodes Mixed retrieval scores; Indicates the dense search weight, taking =0.7; , represents the normalized semantic similarity score and the BM25 sparsity score, respectively;

[0087] S805: Based on user questions With each candidate node The mixed retrieval score from the candidate node set Select the evidence node to obtain the sorted evidence set.

[0088] Preferably, step S805, the processing steps for obtaining the ordered evidence set, include:

[0089] S8051: For the candidate node set The collection of cell nodes Take the top results with the highest combined search score. Each candidate node is used as an evidence node to obtain a table cell evidence set;

[0090] The formula is expressed as:

[0091] ;

[0092] In the formula: This represents a set of evidence for a table cell;

[0093] S8052: For the candidate node set The collection of paragraph nodes in Take the top results with the highest combined search score. Each candidate node is used as an evidence node to obtain a set of text paragraph evidence.

[0094] The formula is expressed as:

[0095] ;

[0096] In the formula: This represents a set of evidence from text paragraphs;

[0097] S8053: Evidence Set Based on Tabular Units and text paragraph evidence set This yields a sorted set of evidence;

[0098] The formula is expressed as:

[0099] ;

[0100] In the formula: This represents the arrangement of a set of evidence.

[0101] Preferably, in step S9, the processing steps for generating accurate end-to-end question-and-answer results include:

[0102] S901: Format each evidence node in the ordered evidence set to obtain formatted evidence, and construct a supporting context set;

[0103] The formula is expressed as:

[0104] ;

[0105] In the formula: Represents a set of supporting contexts; Indicates the first Formatted evidence;

[0106] S902: By using a predefined Prompt template, the user question, the formatted evidence in the supporting context set, and the generation constraints are organized into a model input sequence to generate a prompt template;

[0107] S903: Perform forward inference within the large language model that is fine-tuned by prompting input instructions, and output the question-and-answer results for the user's question.

[0108] Compared with existing technologies, the complex text-table question-answering method based on graph structure in this invention has the following advantages:

[0109] (1) Achieving explicit modeling of complex table structures and improving the semantic expression capability of tables: This invention first uses a table physical layout mapping model to normalize the complex structures such as spanning rows, spanning columns, merging cells, and multi-level headers in the original table into a unified two-dimensional grid, and further constructs a semantic topology subgraph of the table. Compared with the existing technology that simply linearizes the table or treats it as pure text, this invention can explicitly preserve the structural dependencies between data units in the table and row titles and column headers, so that the semantic position, numerical meaning, and context of the cells can be accurately expressed, thereby significantly improving the information representation accuracy in complex table scenarios.

[0110] (2) Achieving a unified representation of textual and tabular semantics, and improving the ability to fuse heterogeneous information: This invention not only performs graph modeling on the table structure, but also constructs unstructured text related to the table as a text subgraph, and organizes the table subgraph, text subgraph, and question-driven evidence subgraph in the same knowledge graph space through top-level document nodes. Compared with the existing technology of separating table and text modeling and independent retrieval, this invention can achieve logical alignment and joint modeling of textual and tabular semantics, thereby improving the ability to integrate cross-modal and cross-structural evidence, and is particularly suitable for complex question-answering tasks that require simultaneous reliance on table values ​​and text descriptions for answers.

[0111] (3) Constructing explicit "question-evidence" association links to improve the accuracy and interpretability of evidence location: This invention introduces question nodes and evidence association edges such as TAB_EVI and TEX_EVI to pre-construct explicit connection relationships between questions and table units and text paragraphs in the knowledge graph. Compared with traditional methods that rely solely on vector similarity or implicit attention mechanisms for evidence matching, this invention can retain a traceable path from the question to the evidence node in the graph structure, making the evidence retrieval process more structurally constrained and interpretable. Therefore, in complex question-and-answer scenarios, it can more accurately locate key evidence supporting the answer, while providing a structured basis for result tracing and process auditing.

[0112] (4) Improving the efficiency of large-scale semantic retrieval by combining graph structure constraints and vector indexing mechanism: Based on the persistent storage of knowledge graph, this invention further constructs vector indexes for cell nodes and paragraph nodes respectively. Through the encoding and normalization of the natural language description text of the nodes, the mapping of graph nodes to dense semantic space is realized. Compared with schemes that rely solely on graph traversal or solely on global vector retrieval, this invention introduces efficient approximate semantic retrieval capabilities while retaining the graph structure organization capability and node interpretability. It can quickly complete the recall of candidate evidence and reduce the noise interference caused by global retrieval.

[0113] (5) Improve the relevance and coverage of evidence recall through a hybrid retrieval mechanism with document-level constraints: This invention proposes a document-level constrained graph retrieval device that limits the retrieval scope to the document subgraph associated with the current question-and-answer instance, and integrates dense vector scoring and BM25 sparse scoring, performing linear fusion within a unified normalized space. Simultaneously, a categorized quota selection strategy is adopted for paragraph nodes and cell nodes. Compared to existing single semantic retrieval or single keyword retrieval methods, this invention can both utilize semantic similarity to discover implicitly relevant evidence and accurately locate key expressions through keyword statistical matching, thereby effectively improving the accuracy, robustness, and heterogeneous evidence coverage of evidence recall.

[0114] (6) Improving the accuracy, consistency, and controllability of answer generation: After completing evidence retrieval, this invention uniformly formats the sorted table unit evidence and text paragraph evidence into a structured supporting context and inputs it into a large language model based on instruction fine-tuning for answer generation. By strictly constraining the prompt template, evidence numbering method, and output format, the model is limited to generating concise answers based solely on the retrieved evidence. Compared to schemes that directly rely on open generation from a large language model, this invention can effectively reduce model illusion, reduce unfounded expansions and redundant reasoning outputs, thereby significantly improving the factual consistency, result stability, and output controllability of answer generation.

[0115] (7) Achieving a closed-loop processing from structured graph construction and semantic retrieval to answer generation, improving the overall practicality of the system: This invention forms a complete closed-loop processing flow from table layout parsing, semantic subgraph construction, text subgraph modeling, question-driven evidence association, global knowledge graph storage, vector index establishment, document-level hybrid retrieval, to the final answer generation based on instruction fine-tuning model. Compared with the existing technologies with fragmented modules and opaque evidence flow, the steps of this invention are closely connected, and the data representation is unified. It can support both offline knowledge construction and online question response, thus possessing strong engineering implementation capabilities and industrial application value.

[0116] (8) Enhanced adaptability and generalization in complex text-table question-answering scenarios. Because this invention uses a graph structure to uniformly express the relationships between tables, text, questions, and evidence, and enhances semantic matching capabilities through vector indexing and hybrid retrieval strategies, it is not only suitable for conventional single-table question-answering tasks, but also for scenarios involving complex table headers, multi-layered semantic paths, long text descriptions, and heterogeneous evidence joint reasoning. Compared to traditional methods designed for single data formats, this invention has better task adaptability and scenario generalization capabilities, and can be widely applied to various application scenarios that require joint understanding of text and tables, such as financial report analysis, medical document question-answering, government statistical interpretation, and enterprise knowledge management. Attached Figure Description

[0117] To make the objectives, technical solutions, and advantages of the invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0118] Figure 1 This is a logic block diagram and architecture diagram for a complex text-table question-answering method based on a graph structure.

[0119] Figure 2 This is a schematic diagram of a prompt template. Detailed Implementation

[0120] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but only to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0121] The following detailed explanation illustrates the specific implementation methods:

[0122] Example:

[0123] This embodiment discloses a complex text-table question-answering method based on graph structure.

[0124] like Figure 1 As shown, a complex text-table question-answering method based on graph structure includes:

[0125] S1: Obtain the user issues to be processed, as well as the original table and its associated descriptive text;

[0126] S2: Map the original table to a two-dimensional grid space to generate the table's physical layout mapping result;

[0127] S3: Construct a semantic topology subgraph of the table based on the table's physical layout mapping results;

[0128] S4: Construct an unstructured text subgraph based on the relevant descriptive text of the original table;

[0129] S5: Based on user questions, construct a question-driven association evidence graph;

[0130] S6: Construct a complete global knowledge graph based on table semantic topology subgraphs, unstructured text subgraphs, and question-driven association evidence graphs;

[0131] S7: Read the attributes of each cell node and paragraph node in the complete global knowledge graph, and generate vector indices for each cell node and paragraph node;

[0132] S8: Transform the user question into a (normalized) query vector and match it with the vector indices of each cell node and paragraph node to obtain a set of sorted evidence;

[0133] S9: Construct a prompt template based on the user's question and the set of ranked evidence, and input it into the large language model that has been fine-tuned by instructions to perform forward inference and output the question-and-answer results for the user's question.

[0134] This invention proposes a graph-based method for complex text-table question answering. It constructs a question answering system with a Knowledge Graph (KG) as the core data layer and Graph Retrieval Augmented Generation (GRAG) as the core reasoning mechanism. This system addresses multi-source heterogeneous information in complex text-table scenarios, such as structured tables with nested hierarchies, unstructured contexts, and logically related query intents. It achieves end-to-end question answering from the user's natural language query, through table topology modeling, text-table association evidence graph construction, document-level subgraph precise retrieval, heterogeneous evidence semantic fusion, and end-to-end answer generation based on instruction-based fine-tuning of Large Language Models (LLM). This invention solves the problems of semantic loss, structural fragmentation, and weak cross-modal text-table associations in traditional methods when handling complex nested tables by unifying the physical layout and semantic logic of the table into a graph topology.

[0135] The graph-based text-table question answering method proposed in this invention mainly comprises two stages: text-table knowledge graph construction and knowledge graph retrieval enhancement generation. The first stage unifies the modeling of complex tables, contextual text, and question-evidence relationships into a structured graph, while the second stage completes node-level semantic retrieval, evidence fusion, and answer generation based on the constructed graph.

[0136] The standardized representation of complex tables is the foundation for building a high-quality knowledge graph (KG) and a prerequisite for realizing complex text-table question answering. Since original tables typically have features such as spanning rows and columns, nested headers, multi-level headings, and contextual semantic dependencies, directly expressing them using linear text format easily leads to the loss of the original table's topological structure and semantic constraints. Therefore, this invention first employs a text-table knowledge graph construction method, which mainly consists of five sub-modules: constructing a table physical layout mapping model, constructing a table semantic topology subgraph, constructing an unstructured text subgraph, constructing a question-driven association evidence graph, and constructing a global KG with persistent storage.

[0137] The purpose of using KG (Knowledge-Based Graph) as the core data foundation for the downstream question-answering system, rather than serialized text, in this invention is that complex tables contain numerous nested levels and row-column relationships. Serialization processes, such as converting to Markdown or JSON format, often destroy the physical spatial relationships and logical hierarchy between cells. KG, through explicit modeling of nodes and relationships, can achieve a "lossless transformation" of the table's topology, ensuring accurate location of specific rows and columns in the semantic space during queries. Simultaneously, serialized text struggles to establish strong logical mappings between text paragraphs and specific table cells. This invention establishes a direct link between query intent and heterogeneous evidence within the graph space by constructing TAB_EVI and TEX_EVI relationship edges, significantly improving the semantic positioning accuracy during the retrieval stage. Furthermore, the downstream question-answering module of this invention employs a GRAG mechanism, which inherently requires the system not only to retrieve semantically relevant content but also to narrow down the candidate range using structural constraints during the retrieval process and preserve evidence logic during generation. KG possesses node-level indexing, relationship-level constraints, and subgraph-level organization capabilities, enabling good coupling with vector retrieval, subgraph filtering, and LLM generation. Therefore, compared to linearly serialized text, KG is more suitable as the core data foundation for the GRAG mechanism.

[0138] After constructing the text-table knowledge graph and achieving persistent storage in the graph database, this invention further employs a Knowledge Graph Retrieval Augmented Generation (KGRAG) method to achieve high-precision evidence localization and answer generation for complex text-table question-answering tasks. This method mainly consists of three sub-stages: KG vector indexing, KG retrieval, and LLM generation. Its purpose is to combine symbolic graph representation with vectorized semantic retrieval while preserving graph structure constraints, and to leverage LLM for fusion reasoning and natural language generation of heterogeneous evidence, thereby forming a complete KGRAG generation process.

[0139] The purpose of this invention, which employs the KGRAG mechanism instead of traditional linear retrieval augmented generation or pure LLM direct generation, is that traditional retrieval augmented generation (RAG) typically uses linear text blocks as the basic retrieval unit, relying mainly on semantic similarity in vector space for recall. This makes it difficult to fully utilize the inherent row-column relationships, hierarchical relationships, and cell affiliation relationships within complex tables. KGRAG, while retaining the advantages of vector retrieval, introduces node relationships, document subgraph boundaries, and node type constraints from a graph structure. This allows the system to not only find semantically relevant content during the retrieval phase but also filter out structurally sound evidence nodes, thereby improving the accuracy of retrieval results. Furthermore, in complex table-based question answering, answers often depend on a specific cell, a row-column intersection, or the combined support of a paragraph and a cell. KGRAG, by conducting retrieval at the node level, can directly return cell and paragraph nodes, rather than coarse-grained text fragments. This is more conducive to accurate evidence location, and the system can further trace the relationship paths between nodes, making the final answer more traceable and interpretable. Furthermore, pure LLM models are prone to illusions when faced with complex tables and long text inputs due to contextual redundancy, ambiguous structural information, or unclear evidence boundaries. KGRAG, through a "retrieve first, generate later" mechanism, uses highly relevant and structurally constrained evidence nodes as model input, significantly reducing interference from irrelevant information. It also strengthens the model's fact-based generation capabilities through explicit evidence organization, thereby reducing the risk of unfounded completions or erroneous inferences during the generation stage.

[0140] To better illustrate the technical solution of the present invention, this embodiment will be described in more detail through the following parts.

[0141] I. Construction of Table Physical Layout Mapping Model

[0142] This invention performs parsing and normalization on the original table T, aiming to uniformly map complex tables containing elements such as spanning rows and columns and merging cells into a two-dimensional grid space G, and construct a mapping matrix M, thereby providing an accurate spatial positioning foundation for the subsequent construction of the table's semantic topological subgraph. The algorithm flow is shown in Table 1.

[0143] Table 1. Algorithm Flow for Constructing the Physical Layout Mapping Model

[0144]

[0145] in, Indicates a question, Represents tables, This represents the relevant descriptive text, where V represents the set of vertices, E represents the set of edges, and so on. In a knowledge graph, H represents the column semantic set, and R represents the row semantic set. Represents a collection of cell entities. Representing a table Total number of rows.

[0146] Specifically, the processing steps for generating the physical layout mapping results of the table are as follows:

[0147] S201: Through function For the original table Perform structural analysis to generate a regularized mesh. ;

[0148] S202: Through the mapping matrix Record normalized grid The correspondence between each grid position and the original cell;

[0149] S202: Through function For normalized grids and mapping matrices The analysis generates a set of column semantics. Line semantic set and cell entity collection The result of the physical layout mapping of the table.

[0150] The purpose of this invention in parsing and normalizing table structures is to transform the structural relationships in complex tables that rely on visual layout into regular coordinate relationships, ensuring that each grid cell has a unique physical location identifier in the graph structure, thereby providing accurate geometric constraints and structural foundations for subsequent topology construction.

[0151] II. Construction of Semantic Topological Subgraphs of Tables

[0152] Based on the obtained physical layout mapping results of the table, this invention constructs a semantic topology subgraph of the table by instantiating nodes and relational edges, which explicitly represents the semantic relationships between the row and column structure, header level, and data units in the table. The algorithm flow for constructing the table semantic topology subgraph is shown in Table 2.

[0153] Table 2 shows the algorithm flow for constructing semantic topological subgraphs.

[0154]

[0155] in, Represents the set of list header nodes, Represents the set of line header nodes. This represents the column semantic header node corresponding to the current cell. This represents the row semantic header node corresponding to the current cell, where c represents the cell node.

[0156] Specifically, the processing steps for constructing a table semantic topology subgraph include:

[0157] S301: Through function For the original table Generate the root node of the table and add to the global vertex set. middle;

[0158] S302: Through function Traversing column semantic set Create a corresponding set of list header nodes for each column header. and add to the global vertex set. middle;

[0159] S303: via function At the root node of the table With list header node set Create edges containing the type of column headers to indicate that these column headers belong to the table, and add them to the global edge collection. middle;

[0160] S304: via function Traversing line semantic set Create a corresponding set of line title nodes for each line title. and add to the global vertex set. middle;

[0161] S305: Through function At the root node of the table With the set of line header nodes Create edges containing the type of these row headers to indicate that they belong to the table, and add them to the global edge set. middle;

[0162] S306: Traverse the collection of cell entities Each cell in Through function Create a cell node for the cell Get the set of cell nodes and add to the global vertex set. In the middle; create the root node of the table. With cell nodes The cells between contain borders and add to the global edge set. middle;

[0163] S307: Through function Parse the current cell node based on its position. The column semantic header node Create the current cell node. semantic header node of the column column semantic definition edge This indicates that the meaning of the value in this cell is defined by the column header and is added to the global edge collection. middle;

[0164] S308: via function Parse the current cell node Corresponding line semantic title node Create the current cell node. With the corresponding line semantic title node The semantic definition of the edge of the row This indicates that the meaning of the value in this cell is defined by the row header and is added to the global edge collection. middle;

[0165] S309: Based on the table root node List header node set Line title node set A collection of cell entities The generated collection of cell nodes As well as edges containing types, cell-containing edges, column semantic definition edges, and row semantic definition edges, construct a table semantic topology subgraph.

[0166] The purpose of this invention incorporating knowledge graphs, rather than solely relying on the parametric knowledge of large language models, is that complex table-based question answering often requires precise location of the row and column semantic space where data values ​​reside. However, the parametric knowledge within large language models struggles to reliably preserve the fine-grained structural relationships within specific table instances. This invention creates a global root node and corresponding entity nodes for table headers, row headings, and data cells. By utilizing semantic relationships to reproduce the logical connections between data values ​​and their row and column coordinates, it transforms two-dimensional table information into a searchable, reasonable, and traceable semantic network, providing a structural foundation for subsequent evidence retrieval and answer generation.

[0167] III. Construction of Unstructured Text Subgraphs

[0168] This invention processes text information related to tables. By semantically segmenting descriptive text, instantiating text blocks into paragraph nodes, and constructing unstructured text subgraphs, a structured organization of text semantic information is achieved. The algorithm flow is shown in Table 3.

[0169] Table 3. Algorithm Flow for Unstructured Text Subgraph Construction

[0170]

[0171] in, Indicates relevant descriptive text, Represents a collection of text paragraphs, Indicates paragraph nodes, Represents the root node of the text. This indicates the total number of text paragraphs.

[0172] Specifically, the steps for constructing unstructured text subgraphs include:

[0173] S401: For relevant descriptive text Create the text root node and add to the global vertex set. middle;

[0174] S402: Through function For relevant descriptive text Divide into segments to obtain a set of segments. ;

[0175] S403: Traversing the collection of paragraphs Each paragraph Through function Instantiate the corresponding paragraph node for each paragraph This yields the set of paragraph nodes. And add the paragraph node set to the global vertex set. middle;

[0176] S404: Create the text root node To paragraph node The containment relationship between edges and add to the global edge set. middle;

[0177] S405: Based on text root node Paragraph node set and the text root node Paragraph nodes Construct an unstructured text subgraph based on the containment relationships between edges.

[0178] The purpose of constructing unstructured text subgraphs in this invention is to transform linearly organized text content into explicit nodes in a graph structure, enabling descriptive text to be integrated into a unified knowledge graph representation space along with table structures. This approach achieves logical alignment of text semantics and table semantics within the same graph space, providing a foundation for subsequent joint retrieval, evidence fusion, and answer generation across table cells and text paragraphs.

[0179] IV. Problem-Driven Construction of Association Evidence Graphs

[0180] This invention models user queries and their sub-components as question nodes. By defining evidence association edges TAB_EVI and TEX_EVI, question nodes are directly linked to corresponding cell nodes or paragraph nodes, thereby pre-constructing an explicit "query-evidence" association link in the knowledge graph. The algorithm flow for question-driven association evidence graph construction is shown in Table 4.

[0181] Table 4. Problem-Driven Algorithm Flow for Constructing Association Evidence Graphs

[0182]

[0183] in, Represents question and answer instance nodes, This represents question nodes, answer nodes, paragraph semantic nodes, and table semantic nodes.

[0184] Specifically, the steps for constructing a problem-driven association evidence graph include:

[0185] S501: Through function For user issues Create a question and answer instance node and add to the global vertex set. ;

[0186] S502: Through function User issues Decompose the problem to obtain the problem nodes. Answer nodes Paragraph semantic nodes and table semantic nodes and add to the global vertex set. ;

[0187] S503: Through function Create question-and-answer instance nodes With the problem node Answer nodes Paragraph semantic nodes and table semantic nodes The edges that establish the relationships between them are added to the global edge set. middle;

[0188] S504: Traverse each candidate node in the candidate node set, using the function... Positioning and User Issues If the relevant cell node is successfully located, a question-and-answer instance node is created. Add the tabular evidence association edge TAB_EVI between the cell node and the node to the global edge set. middle;

[0189] S505: Traverse each candidate node in the candidate node set, using the function... Positioning and User Issues If the relevant paragraph node is successfully located, a question-and-answer instance node will be created. Add the text evidence associated edge TEX_EVI between the paragraph node and the corresponding edge to the global edge set. middle;

[0190] S506: Based on question-answering instance nodes Problem nodes Answer nodes Paragraph semantic nodes Table semantic nodes The system locates successfully placed cell and paragraph nodes, as well as associated edges, table evidence associated edges, and text evidence associated edges, and constructs a problem-driven associated evidence graph.

[0191] The purpose of constructing a question-driven association evidence graph in this invention is to explicitly encode the connections between questions and evidence implicit in training samples or question-answering contexts into computable relationships in a knowledge graph, thereby achieving direct alignment between query intent and heterogeneous evidence nodes. This approach not only enhances the ability to locate key evidence in subsequent retrieval stages but also provides a traceable and interpretable evidence chain for the answer generation process, enabling the question-answering system to achieve higher semantic matching accuracy and reasoning reliability in complex text-table scenarios.

[0192] V. Global KG Construction and Persistent Storage Implementation

[0193] This invention further sets up top-level document nodes to uniformly collect and manage table subgraphs, unstructured text subgraphs, and question-driven association evidence subgraphs under the same document instance, ultimately forming a complete text-table knowledge graph. This knowledge graph is persistently stored in the Neo4j graph database. The global KG construction and persistent storage algorithm flow is shown in Table 5.

[0194] Table 5. Global KG Construction and Persistent Storage Algorithm Flow

[0195]

[0196] Specifically, the processing steps for constructing a complete text-table knowledge graph include:

[0197] S601: Through function Create top-level document node and add to the global vertex set. ; through top-level document nodes Includes all subgraphs.

[0198] S602: Create top-level document node With the root node of the table Relationship between edges and top-level document nodes With question and answer instance nodes Relationship between edges and add to the global edge set. middle;

[0199] S603: Create top-level document nodes With the root node of the text The relationship edges between the nodes and other auxiliary semantic nodes are added to the global edge set. middle;

[0200] S604: Nodes and edges in a table-based semantic topological subgraph, an unstructured text subgraph, and a question-driven association evidence graph, as well as top-level document nodes. Construct a complete global knowledge graph by combining the graph with its corresponding relation edges. .

[0201] In this embodiment, the complete global knowledge graph is stored in the Neo4j graph database.

[0202] The purpose of this invention in constructing a global knowledge graph and achieving persistent storage is to provide a unified structured data foundation for the entire text-table question answering task, enabling table structure, text semantics, and question evidence relationships to be organized, accessed, and reasoned within the same graph space. Furthermore, Neo4j possesses inherent graph structure storage capabilities, flexible node-edge patterns, and an efficient Cypher graph query mechanism, providing robust underlying support for subsequent graph traversal, relationship filtering, node retrieval, and evidence path extraction. This, in turn, provides a directly accessible, locally constrained, and path-traceable semantic data foundation for downstream graph-based retrieval enhancement generation processes.

[0203] VI. Constructing a KG Vector Indexing Mechanism

[0204] After completing the persistent storage of the knowledge graph, this invention further constructs a vector indexing mechanism oriented towards graph nodes to support efficient semantic retrieval of cell and paragraph nodes. Its core idea is to transform discrete nodes in the knowledge graph into continuous semantic vectors and establish a backtrackable mapping relationship between the vector space and graph nodes, thereby achieving a unification between symbolic graph structure and dense semantic retrieval.

[0205] Specifically, the processing steps for constructing a vector indexing mechanism for graph nodes include:

[0206] S701: In the complete global knowledge graph (Neo4j graph database), the scale of cell nodes and paragraph nodes are counted separately, and the corresponding node attributes are read in batches using a pagination batch processing mechanism to ensure the scalability and stability of the processing in large-scale graph data scenarios, and to obtain the attributes of each cell node and paragraph node.

[0207] S702: Construct index text from the attributes of each cell node and paragraph node; for paragraph nodes, use the original text content of the corresponding paragraph as index text; for cell nodes, concatenate their table identifier, row path, column path, value and unit into structured natural language description text as index text.

[0208] In this embodiment, index text is constructed for different types of nodes. For cell nodes, the structured information of the table is concatenated into a natural language description, explicitly including key semantic elements such as table ID, row path, column path, and numerical value / unit, to fully express the semantic position and numerical meaning of the cell in the table. Its construction form can be represented as: Table{}, Row: {}, Col: {}, Value: {} {unit}, or as Table {} {RowHeader}of{Header cell}is {Value}. For paragraph nodes, the original text content is directly used as vectorized input, and its form can be represented as content:{} to maintain the semantic integrity of the paragraph. For example, for the table context "Table 2shows Management and financial advice fees Revenues of Years Ended December31, 2009 (in millions, except percentages) is $1,234", the semantic description can be constructed as "content:{}".

[0209] Relevant descriptive text Top-level document node It contains two types of searchable nodes, namely, the set of paragraph nodes. With cell node collection Candidate node set ;

[0210] For the candidate node set Any candidate node in Its index text Represented as:

[0211] ;

[0212] In the formula: The original text content representing the paragraph node; This indicates that the cell node contains natural language descriptive text that integrates table identifiers, row paths, column paths, values, and units;

[0213] S703: Through a pre-trained sentence vector encoder Batch semantic encoding is performed on the index text of each cell node and paragraph node to obtain each candidate node. node vectors and for node vectors conduct Normalization yields the normalized node vectors for each cell node and paragraph node. ;

[0214] Pre-trained sentence vector encoder The calculation formula is expressed as:

[0215] ;

[0216] In the formula: Indicates candidate nodes Node vectors;

[0217] The normalization calculation formula is expressed as:

[0218] ;

[0219] In the formula: Indicates candidate nodes The normalized node vector;

[0220] S704: Based on the normalized node vectors of each cell node and paragraph node, construct an independent FAISS vector index for each cell node and paragraph node;

[0221] The formula is expressed as:

[0222] ;

[0223] ;

[0224] In the formula: Represents cell node FAISS vector index; Represents paragraph nodes The FAISS vector index. IndexFlatIP is preferred as the vector index structure.

[0225] S705: Define the node retrieval scoring function:

[0226] ;

[0227] In the formula: Indicates user issues With candidate nodes Semantic similarity score; Indicates user issues Normalized query vector; Indicates candidate nodes The normalized node vector;

[0228] S706: Constructing a mapping function between index positions and graph node identifiers :

[0229] ;

[0230] In the formula: Indicates the position number in the FAISS vector index; Represents the node identifier in the complete global knowledge graph (Neo4j graph database); Indicates the document identifier; Indicates the node type label.

[0231] The purpose of this invention in constructing a knowledge graph vector indexing mechanism is as follows: While relying solely on symbolic relation queries in graph databases can accurately express the structural connections between nodes, it is difficult to directly support semantic similarity matching for natural language problems; and relying solely on dense vector retrieval makes it difficult to preserve node types, document affiliation, and structural constraints in the graph. This invention, by establishing vector indexes for knowledge graph nodes, introduces semantic retrieval capabilities while preserving the interpretability of the graph structure. This enables the system to simultaneously utilize node content semantics and graph organization information within a unified framework, thereby providing an efficient, accurate, and traceable evidence access mechanism for subsequent graph constraint-based retrieval enhancement generation.

[0232] VII. Constructing a Graph Retrieval Tool with Document-Level Constraints

[0233] This invention constructs a document-level constrained graph retrieval tool for a given user query q, enabling joint retrieval of cell nodes and paragraph nodes within the document subgraph corresponding to the current question-answering instance. Its core idea is to combine dense semantic retrieval with sparse keyword retrieval under document-range constraints, and to uniformly sort and quota-based select heterogeneous nodes, thereby providing highly relevant, high-coverage, and traceable evidential context for downstream large language models.

[0234] Specifically, the processing steps for building a document-level constrained graph retrieval engine include:

[0235] S801: Through a pre-trained sentence vector encoder User issues Encode and then proceed Normalization yields user questions Normalized query vector ;

[0236] To ensure that search results remain consistent with the context of the current question, the search scope is limited to target documents associated with the current question-and-answer instance. Specifically, based on the pre-built index mapping, the set of paragraph nodes belonging to the document is selected from the document subgraph. With cell node collection And further obtain the complete set of candidate nodes: .

[0237] Given problem With text-table context The goal is to select candidates. Return the evidence set.

[0238] S802: Based on Normalized Query Vector Calculate user problems With top-level document nodes candidate node set Each candidate node semantic similarity score ;

[0239] The formula is expressed as:

[0240] ;

[0241] In the formula: Indicates user issues With candidate nodes Semantic similarity score; Indicates candidate nodes The normalized node vector;

[0242] S803: In the sparse retrieval phase, BM25 scoring is performed on candidate nodes based on node text statistics. The candidate node set is then calculated. Each candidate node terms Inverse document frequency, and based on each term Inverse document frequency calculation user problem With each candidate node BM25 sparse score;

[0243] The formula for calculating inverse document frequency is expressed as:

[0244] ;

[0245] In the formula: Indicates terms Inverse document frequency; Represents the set of candidate nodes The total number of candidate nodes; Represents the set of candidate nodes Contains terms The number of candidate nodes;

[0246] The formula for calculating the BM25 sparsity score is as follows:

[0247] ;

[0248] In the formula: Indicates user issues With candidate nodes BM25 sparse score; Indicates terms In candidate nodes Corresponding word frequencies in the text; Represents the set of candidate nodes Average text length; Indicates user issues The set of word segments; , Indicates the preset parameters, take , ;

[0249] S804: Regarding user issues With each candidate node semantic similarity score And BM25 sparse score After score normalization, the two normalized scores are linearly fused to obtain the candidate nodes. Mixed retrieval scores;

[0250] To eliminate the scaling differences between the two types of scoring functions in the numerical space, this step further performs min-max normalization on the dense and sparse scores in the candidate set, respectively. For any retrieval method... Its normalized score is defined as:

[0251] ;

[0252] In the formula: This represents the normalized score after normalization. This represents the score before normalization. This indicates a smoothing term to prevent the denominator from being zero; , These represent the lowest and highest scores among all candidate nodes, respectively;

[0253] The formula for calculating the mixed retrieval score is expressed as follows:

[0254] ;

[0255] In the formula: Indicates user issues With candidate nodes Mixed retrieval scores; Indicates the dense search weight, taking =0.7; , represents the normalized semantic similarity score and the BM25 sparsity score, respectively;

[0256] S805: Based on user questions With each candidate node The mixed retrieval score from the candidate node set Select the evidence node to obtain the sorted evidence set.

[0257] After obtaining the fusion score, to ensure coverage of both textual and tabular evidence, this invention sorts and truncates the two types of nodes separately. Specifically, the processing steps for obtaining the sorted evidence set include:

[0258] S8051: For the candidate node set The collection of cell nodes Take the top results with the highest combined search score. Each candidate node is used as an evidence node to obtain a table cell evidence set;

[0259] The formula is expressed as:

[0260] ;

[0261] In the formula: This represents a set of evidence for a table cell;

[0262] S8052: For the candidate node set The collection of paragraph nodes in Take the top results with the highest combined search score. Each candidate node is used as an evidence node to obtain a set of text paragraph evidence.

[0263] The formula is expressed as:

[0264] ;

[0265] In the formula: This represents a set of evidence from text paragraphs;

[0266] S8053: Evidence Set Based on Tabular Units and text paragraph evidence set This yields a sorted set of evidence;

[0267] The formula is expressed as:

[0268] ;

[0269] In the formula: This represents the arrangement of a set of evidence.

[0270] In this embodiment, the default is... =20, =40.

[0271] For the retrieved evidence nodes, if the node type is a paragraph, its corresponding original text content is directly output; if the node type is a cell, its structured natural language description text is output, including table identifiers, row paths, column paths, and information such as numerical values ​​and units. This forms a heterogeneous context fragment that integrates text paragraph evidence and table cell evidence, which is then used as input to the downstream large language model for answer reasoning and generation.

[0272] The purpose of this invention, which employs a document-level constraint-based graph retrieval mechanism, is that evidence in complex text-table question-answering tasks is often locally clustered within a specific document context. Directly searching the entire graph can easily introduce cross-document noise and weaken the relevance of the evidence. By introducing document-level constraints, the retrieval scope can be controlled within the local subgraph corresponding to the current question. By combining dense recall using normalized semantic vectors with BM25 sparse scoring based on node text statistics, and performing linear fusion in the normalized space, followed by quota-based selection by paragraph and cell type, this approach simultaneously considers semantic matching ability, precise keyword matching ability, and heterogeneous evidence coverage, thereby achieving accurate and rapid evidence localization for complex text-table mixed information.

[0273] 8. Perform semantic generation based on instruction-based fine-tuning LLM

[0274] After obtaining the set of ranked evidence, this invention employs a large language model based on instruction fine-tuning as a generator to perform end-to-end answer generation. Its core objective is to unify the retrieved table unit evidence and text paragraph evidence into a structured supporting context, and generate the final answer under strict prompt constraints.

[0275] Specifically, the processing steps for generating accurate end-to-end question-and-answer results include:

[0276] S901: Format each evidence node in the ordered evidence set to obtain formatted evidence, and construct a supporting context set;

[0277] The formula is expressed as:

[0278] ;

[0279] In the formula: Represents a set of supporting contexts; Indicates the first Formatted evidence;

[0280] S902: Using a predefined Prompt template, the user question, formatted evidence from the supporting context set, and generative constraints are organized into a sequence of model inputs, generating a sequence such as... Figure 2The prompt template shown;

[0281] In this embodiment, a predefined Prompt template is used to organize the user question, formatted evidence, and generation constraints into a model input sequence. The prompt uses a dialogic structure, where system instructions explicitly require the model to act as a fact-finding responder, generating answers solely based on the provided evidence, without introducing external knowledge, outputting intermediate reasoning processes, or discussing whether the question is answerable. The user input section provides the original question q and the evidence set. For tabular evidence, the Prompt specifically stipulates that it should be treated as an indivisible semantic unit of fact. For example, "Evidence 1: Table A, Row: 2025Revenue, Col: Company X, Value: 1500 Million USD" should be understood as a complete fact, rather than being broken down into several isolated fields. The final output format is limited to concise answers, such as numerical values, entity names, or phrases, to ensure the purity and consistency of the generated results.

[0282] S903: Perform forward inference within the large language model that is fine-tuned by prompting input instructions, and output the question-and-answer results for the user's question.

[0283] The purpose of this invention, which employs instruction-based LLM for semantic generation, is that complex text-table question-answering tasks not only require the system to retrieve relevant evidence but also to achieve a unified understanding, aggregation, and expression of heterogeneous evidence. Relying solely on retrieval or rule-based methods is insufficient for semantic integration across text paragraphs and table units; however, the instruction-based large language model possesses stronger instruction-following capabilities, fact extraction capabilities, and short answer generation capabilities. By inputting highly relevant evidence into the model in a structured manner and imposing strict generation constraints, the risk of illusion can be effectively reduced, and the accuracy, consistency, and interpretability of answer generation can be improved, thereby completing a closed loop from knowledge graph evidence retrieval to final answer output.

[0284] The key technology of this invention lies in constructing an integrated method for graph structure modeling, heterogeneous evidence joint retrieval, and controllable answer generation for complex text-table question-answering scenarios. Its core lies in organizing table structure information, unstructured text information, and question-driven evidence relationships into the same semantic space through a unified knowledge graph representation. This is combined with a graph-constrained hybrid retrieval mechanism and a large language model generation mechanism based on instruction fine-tuning to achieve accurate question answering and interpretable reasoning in complex text-table hybrid scenarios. First, by performing physical layout mapping and semantic topology modeling on the original table, the relationships between multi-level headers, row titles, merged cells, and data units, originally implicit in the two-dimensional layout, are explicitly transformed into a table semantic subgraph composed of nodes and edges. This allows each data unit to establish a stable and traceable structural association with its table, corresponding row semantics, and column semantics, thereby achieving structured expression and fine-grained semantic localization of complex table content. Secondly, by semantically segmenting the descriptive text related to the table and instantiating each text segment as a paragraph node, an unstructured text subgraph is constructed. This is then combined with top-level document nodes to uniformly aggregate the table subgraph, text subgraph, and question instance subgraph, forming a complete document-level text-table knowledge graph. This achieves logical alignment and unified representation of text semantics and table semantics within the same graph space. Furthermore, by modeling user questions as question nodes and establishing evidence association edges from question nodes to table cell nodes and text paragraph nodes, explicit, computable, and traceable connections are formed between questions and candidate evidence, providing a graph structure foundation for subsequent evidence location, path tracing, and answer interpretation. Further, by constructing semantic vector indexes for cell nodes and paragraph nodes in the knowledge graph and introducing document-level constraints during retrieval, dense vector semantic recall and BM25 sparse retrieval based on node text statistics are normalized and fused. Simultaneously, categorized quotas are selected based on paragraph evidence and table evidence, ensuring retrieval relevance while also considering keyword matching ability, heterogeneous evidence coverage, and local contextual constraint capabilities. Finally, the table unit evidence and text paragraph evidence obtained from the mixed retrieval are uniformly formatted into numbered supporting contexts and input into a large language model based on instruction fine-tuning. Under strict prompt constraints, controllable answer generation is performed, so that the model outputs concise answers based only on the retrieved evidence. This effectively reduces the illusion problem caused by open generation and forms a complete closed-loop question-answering process from graph modeling, evidence retrieval to answer generation.

[0285] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit the technical solutions. Those skilled in the art should understand that any modifications or equivalent substitutions to the technical solutions of the present invention without departing from the spirit and scope of the present invention should be covered within the scope of the claims of the present invention.

Claims

1. A complex text-table question-answering method based on graph structure, characterized in that, include: S1: Obtain the user issues to be processed, as well as the original table and its associated descriptive text; S2: Map the original table to a two-dimensional grid space to generate the table's physical layout mapping result; S3: Construct a semantic topology subgraph of the table based on the table's physical layout mapping results; S4: Construct an unstructured text subgraph based on the relevant descriptive text of the original table; S5: Based on user questions, construct a question-driven association evidence graph; S6: Construct a complete global knowledge graph based on table semantic topology subgraphs, unstructured text subgraphs, and question-driven association evidence graphs; S7: Read the attributes of each cell node and paragraph node in the complete global knowledge graph, and generate vector indices for each cell node and paragraph node; S8: Transform the user's question into a query vector, match it with the vector indices of each cell node and paragraph node to obtain a set of sorting evidence; S9: Construct a prompt template based on the user's question and the ranked evidence set, and input it into the large language model that has been fine-tuned by instructions to perform forward inference, and output the question and answer results of the user's question.

2. The complex text-table question-answering method based on graph structure as described in claim 1, characterized in that: Step S2 includes the processing steps for generating the table physical layout mapping results: S201: Through function For the original table Perform structural analysis to generate a regularized mesh. ; S202: Through the mapping matrix Record normalized grid The correspondence between each grid position and the original cell; S202: Through function For normalized grids and mapping matrices The analysis generates a semantic set containing columns. Line semantic set and cell entity collection The result of the physical layout mapping of the table.

3. The complex text-table question-answering method based on graph structure as described in claim 2, characterized in that: In step S3, the processing steps for constructing the table semantic topology subgraph include: S301: Through function For the original table Generate the root node of the table and add to the global vertex set. middle; S302: Through function Traversing column semantic set Create a corresponding set of list header nodes for each column header. and add to the global vertex set. middle; S303: via function At the root node of the table With list header node set Create an edge of type [type] between them and add it to the global edge collection. middle; S304: via function Traversing the semantic set of lines Create a corresponding set of line title nodes for each line title. and add to the global vertex set. middle; S305: Through function At the root node of the table With the set of line header nodes Create an edge of type [type] between them and add it to the global edge collection. middle; S306: Traverse the collection of cell entities Each cell in Through function Create a cell node for the cell Get the set of cell nodes and add to the global vertex set. In the middle; create the root node of the table. With cell nodes The cells between contain borders and add to the global edge set. middle; S307: Through function Parse the current cell node The semantic header node of the column to which it belongs Create the current cell node. semantic header node of the column column semantic definition edge and add to the global edge set. middle; S308: via function Parse the current cell node Corresponding line semantic title node Create the current cell node. With the corresponding line semantic title node The semantic definition of the edge of the row and add to the global edge set. middle; S309: Based on the table root node List header node set Line title node set A collection of cell entities The generated collection of cell nodes As well as edges containing types, cell-containing edges, column semantic definition edges, and row semantic definition edges, construct a table semantic topology subgraph.

4. The complex text-table question-answering method based on graph structure as described in claim 1, characterized in that: In step S4, the processing steps for constructing the unstructured text subgraph include: S401: For relevant descriptive text Create the text root node and add to the global vertex set. middle; S402: Through function For relevant descriptive text Divide into segments to obtain a set of segments. ; S403: Traverse the collection of paragraphs Each paragraph Through function Instantiate the corresponding paragraph node for each paragraph This yields the set of paragraph nodes. And add the paragraph node set to the global vertex set. middle; S404: Create the text root node To paragraph node The containment relationship between edges and add to the global edge set. middle; S405: Based on text root node Paragraph node set and the text root node Paragraph nodes Construct an unstructured text subgraph based on the containment relationships between edges.

5. The complex text-table question-answering method based on graph structure as described in claim 1, characterized in that: Step S5, the processing steps for constructing the problem-driven association evidence graph include: S501: Through function For user issues Create a question and answer instance node and add to the global vertex set. ; S502: Through function User issues Decompose the problem to obtain the problem nodes. Answer nodes Paragraph semantic nodes and table semantic nodes and add to the global vertex set. ; S503: Through function Create question-and-answer instance nodes With the problem node Answer nodes Paragraph semantic nodes and table semantic nodes The edges that establish the relationships between them are added to the global edge set. middle; S504: Traverse each candidate node in the candidate node set, using the function... Positioning and User Issues If the relevant cell node is successfully located, a question-and-answer instance node is created. Add the tabular evidence association edge TAB_EVI between the cell node and the node to the global edge set. middle; S505: Traverse each candidate node in the candidate node set, using the function... Positioning and User Issues If the relevant paragraph node is successfully located, a question-and-answer instance node will be created. Add the text evidence association edge TEX_EVI between the paragraph node and the corresponding edge to the global edge set. middle; S506: Based on question-answering instance nodes Problem nodes Answer nodes Paragraph semantic nodes Table semantic nodes The system locates successfully placed cell and paragraph nodes, as well as associated edges, table evidence associated edges, and text evidence associated edges, and constructs a problem-driven associated evidence graph.

6. The complex text-table question-answering method based on graph structure as described in claim 1, characterized in that: In step S6, the processing steps for constructing a complete text-table knowledge graph include: S601: Through function Create top-level document node and add to the global vertex set. ; S602: Create top-level document node With the root node of the table Relationship between edges and top-level document nodes With question and answer instance nodes Relationship between edges and add to the global edge set. middle; S603: Create top-level document node With the root node of the text The relationship edges between the nodes and other auxiliary semantic nodes are added to the global edge set. middle; S604: Nodes and edges in a table-based semantic topological subgraph, an unstructured text subgraph, and a question-driven association evidence graph, as well as top-level document nodes. Construct a complete global knowledge graph by combining the graph with its corresponding relation edges. .

7. The complex text-table question-answering method based on graph structure as described in claim 1, characterized in that: In step S7, the processing steps for constructing a vector indexing mechanism for graph nodes include: S701: Calculate the size of cell nodes and paragraph nodes in the complete global knowledge graph, and use a pagination batch processing mechanism to read the corresponding node attributes in batches to obtain the attributes of each cell node and paragraph node. S702: Construct index text from the attributes of each cell node and paragraph node; for paragraph nodes, use the original text content of the corresponding paragraph as index text; for cell nodes, concatenate their table identifier, row path, column path, value and unit into structured natural language description text as index text. Relevant descriptive text Top-level document node It contains two types of searchable nodes, namely, the set of paragraph nodes. With cell node collection Candidate node set ; For the candidate node set Any candidate node in Its index text Represented as: ； In the formula: The original text content representing the paragraph node; This indicates that the cell node contains natural language descriptive text that integrates table identifiers, row paths, column paths, values, and units; S703: Through a pre-trained sentence vector encoder Batch semantic encoding is performed on the index text of each cell node and paragraph node to obtain each candidate node. node vectors and for node vectors conduct Normalization yields the normalized node vectors for each cell node and paragraph node. ; Pre-trained sentence vector encoder The calculation formula is expressed as: ； In the formula: Indicates candidate nodes Node vectors; The normalization calculation formula is expressed as: ； In the formula: Indicates candidate nodes The normalized node vector; S704: Based on the normalized node vectors of each cell node and paragraph node, construct an independent FAISS vector index for each cell node and paragraph node; The formula is expressed as: ； ； In the formula: Represents cell node FAISS vector index; Represents paragraph nodes FAISS vector index.

8. The complex text-table question-answering method based on graph structure as described in claim 1, characterized in that: In step S8, the processing steps for constructing a document-level constrained graph retrieval engine include: S801: Through a pre-trained sentence vector encoder User issues Encode and then proceed Normalization yields user questions Normalized query vector ; S802: Based on normalized query vectors Calculate user problems With top-level document nodes candidate node set Each candidate node semantic similarity score ; The formula is expressed as: ； In the formula: Indicates user issues With candidate nodes Semantic similarity score; Indicates candidate nodes The normalized node vector; S803: Calculate the candidate node set Each candidate node terms Inverse document frequency, and based on each term Inverse document frequency calculation user problem With each candidate node BM25 sparse score; The formula for calculating inverse document frequency is expressed as: ； In the formula: Indicates terms Inverse document frequency; Represents the set of candidate nodes The total number of candidate nodes; Represents the set of candidate nodes Contains terms The number of candidate nodes; The formula for calculating the BM25 sparsity score is as follows: ； In the formula: Indicates user issues With candidate nodes BM25 sparse score; Indicates terms In candidate nodes Corresponding word frequencies in the text; Represents the set of candidate nodes Average text length; Indicates user issues The set of word segments; , Indicates preset parameters; S804: Regarding user issues With each candidate node semantic similarity score And BM25 sparse score After score normalization, the two normalized scores are linearly fused to obtain the candidate nodes. Mixed retrieval scores; The formula for calculating the mixed retrieval score is expressed as follows: ； In the formula: Indicates user issues With candidate nodes Mixed retrieval scores; Indicates the dense search weight, take =0.7; , represents the normalized semantic similarity score and the BM25 sparsity score, respectively; S805: Based on user questions With each candidate node The mixed retrieval score from the candidate node set Select the evidence node to obtain the sorted evidence set.

9. The complex text-table question-answering method based on graph structure as described in claim 8, characterized in that: In step S805, the processing steps for obtaining the ordered evidence set include: S8051: For the candidate node set The collection of cell nodes in Take the top results with the highest combined search score. Each candidate node is used as an evidence node to obtain a table cell evidence set; The formula is expressed as: ； In the formula: This represents a set of evidence for a table cell; S8052: For the candidate node set The collection of paragraph nodes in Take the top results with the highest combined search score. Each candidate node is used as an evidence node to obtain a set of text paragraph evidence. The formula is expressed as: ； In the formula: This represents a set of evidence from text paragraphs; S8053: Evidence Set Based on Tabular Units and text paragraph evidence set This yields the ordered set of evidence; The formula is expressed as: ； In the formula: This represents the arrangement of a set of evidence.

10. The complex text-table question-answering method based on graph structure as described in claim 8, characterized in that: In step S9, the processing steps for generating accurate end-to-end question-and-answer results include: S901: Format each evidence node in the ordered evidence set to obtain formatted evidence, and construct a supporting context set; The formula is expressed as: ； In the formula: Represents a set of supporting contexts; Indicates the first Formatted evidence; S902: By using a predefined Prompt template, the user question, the formatted evidence in the supporting context set, and the generation constraints are organized into a model input sequence to generate a prompt template; S903: Perform forward inference within the large language model that is fine-tuned by prompting input instructions, and output the question-and-answer results for the user's question.

Images