Large model optimization method based on semi-structured table data questions and answers

Abstract

A large model optimization method based on semi-structured table data questions and answers comprises the following steps: aiming at an input semi-structured table, constructing an HO-Tree structure to accurately represent a multi-level association relationship between a header and content, and serializing and storing the table structure; the method comprises the following steps: splitting a complex problem proposed by a user into a plurality of sub-problems, generating a corresponding operation pipeline and a retrieval strategy, and executing the operation pipeline on an HO-Tree structure according to the retrieval strategy; after the execution correctness of each step is checked through forward verification, the overall reliability is evaluated through backward verification in an answer push-back mode, and the accuracy and consistency of final answers are ensured. According to the method, for different question types, the HO-Tree is constructed to capture the hierarchical structure and the orthogonal relation in the table, and the accuracy and efficiency of question answering are greatly improved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of large language models, and particularly relates to a large model optimization method based on semi-structured table data question answering. BACKGROUND

[0002] With the wide application of large language models (LLM) in the field of natural language processing, the question answering ability of the large language models on structured data has made remarkable progress. However, semi-structured tables are widely used in financial statements, medical records and transaction orders and the like, and the data structure of the semi-structured tables often contains complex hierarchical table headers, multi-value cells and orthogonal sub-tables, and these characteristics make the traditional structured query method (such as SQL) face the problems of information loss and semantic mismatch when processing semi-structured data. For example, NL2SQL needs to convert the semi-structured table into a completely structured format, and context information is often lost; NL2Code can perform programming operations on part of the table data, but the accuracy is low when facing complex hierarchical structures. SUMMARY

[0003] The application proposes a large model optimization method based on semi-structured table data question answering, which can effectively deal with diversified layouts, cannot perform fine-grained retrieval, understanding, question answering on complex semi-structured table content, cannot verify the reliability and accuracy of the obtained answers and cannot dynamically adapt to all user question answering, and the like, and the method can greatly improve the accuracy and efficiency of question answering by constructing an HO-Tree to capture the hierarchical structure and orthogonal relationship in the table for different problem types.

[0004] The application is implemented by the following technical scheme:

[0005] The application relates to a large model optimization method based on semi-structured table data question answering, which constructs an HO-Tree structure to accurately represent the multi-level association relationship between table headers and contents for the input semi-structured table, and serializes and stores the table structure; splits a complex question proposed by a user into multiple sub-questions, generates corresponding operation pipelines and retrieval strategies, and executes the operation pipelines on the HO-Tree structure according to the retrieval strategies; after checking the execution correctness of each step through forward verification, evaluates the overall reliability in the form of answer back-propagation through backward verification, and ensures the accuracy and consistency of the final answer.

[0006] Technical effects

[0007] The present application captures the hierarchy in complex tables by constructing HO-Tree, and dynamically generates a pipelined question analysis strategy according to the problem semantics. In addition, the two-stage verification mechanism of the present application effectively reduces the error propagation. Compared with the prior art, the present application effectively avoids the information loss problem in the traditional method, and significantly improves the efficiency of complex table question and answer. Further improve the overall question and answer accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0008] Figure 1 is a structural schematic diagram of the present application;

[0009] Figure 2 is a flowchart of the embodiment;

[0010] Figure 3 is a HO-Tree structure diagram;

[0011] Figure 4 is a flowchart of converting a table into a HO tree;

[0012] Figure 5 is a question decomposition flowchart;

[0013] Figure 6 is a comparison chart of the embodiment effect experiment data. DETAILED DESCRIPTION

[0014] As shown in Figure 1 , the present embodiment relates to a large model optimization system based on semi-structured table data question and answer, which includes a table processing unit, a question processing unit, an answer generation unit and an answer verification unit, wherein: the table processing unit performs table layout matching and splitting processing according to the header information and hierarchical nested structure information of the original table, obtains the nested sub-table information structure and constructs it into a HO-Tree structure result, the question processing unit performs question step decomposition and sub-question operation generation processing according to the HO-Tree obtained in the previous step, obtains the operation pipeline result and HO-Tree retrieval strategy result of each sub-question after decomposition, the answer generation unit performs processing of executing the operation pipeline on the HO-Tree according to the sub-question operation pipeline and retrieval strategy information obtained in the previous step, obtains the result of retrieving the relevant table content according to the sub-question semantics, and the answer verification unit performs correctness checking of each operation execution and final answer backtracking processing according to the result of executing the operation pipeline and the information of the finally generated answer, obtains the operation step correctness verification and final answer overall reliability evaluation result.

[0015] As shown in Figure 2 , the present embodiment is a large model optimization method based on semi-structured table data question and answer based on the above system, which includes:

[0016] Step 1: Process the table and generate the HO-Tree structure, specifically including:

[0017] 1.1) Meta-information extraction: After generating candidate meta-information cells based on the preprocessed semi-structured table using the Visual Language Model (VLM), the similarity between the candidate meta-information cells and all cells in the table is calculated. Cells with similarity exceeding a preset threshold are identified as valid meta-information, and their position information is used for subsequent table segmentation.

[0018] The preprocessing refers to converting the input semi-structured tables of different forms into HTML format, rendering them using a headless browser, and saving them as image format for use as input to the Visual Language Model (VLM).

[0019] The similarity is measured using n-grams or embeddings.

[0020] The aforementioned meta-Info includes header, label, and cell elements.

[0021] 1.2) Table structure analysis, specifically including: for merged cells that span an entire row or column, treat them as table headers and treat the adjacent cells below or to the right as sub-tables.

[0022] In cases where both top and left headers exist, the portion containing more cells is selected to build the Meta Tree (MT), and the remaining portion is integrated into the Body Tree (BT).

[0023] When an orthogonal table (such as an L.3 layout) is detected, the table is split by row or column and sub-tables are built for each.

[0024] 1.3) Generate as follows Figure 3 The HO-Tree shown is constructed by building a Meta-Information Tree (MT) and a Data Tree (BT) separately, and then combining the MT and BT to build a complete hierarchical orthogonal tree (HO-Tree).

[0025] The aforementioned metadata tree is used to represent the header structure of a table, including first-level headers, second-level headers, and multi-level headers, reflecting the overall layout of the table. For vertically structured tables, each path from the root to a leaf node represents a column, reflecting the vertical hierarchical relationship.

[0026] The data tree described above is used to represent the specific data content of a table, mainly including the data values ​​of each cell and their corresponding attributes. For a horizontally structured table, each leaf node represents a row, reflecting the horizontal attribute relationships.

[0027] The hierarchical orthogonal tree is used to accurately describe the table header, data and hierarchical relationship in the table. The HO-Tree can contain a nested structure, that is, some nodes in the BT can recursively contain another HO-Tree.

[0028] The step 1.3 specifically comprises:

[0029] i) According to the meta information extracted by the VLM, the input table is sub-table split to obtain a sub-table list T_list.

[0030] ii) The table processing function divides the table into different logical blocks according to the meta information, and these blocks can contain single-layer tables (L1, L2, L4) or orthogonal sub-tables (L3).

[0031] iii) An empty HO-Tree list HOTree_list is initialized to store the HO-Tree constructed subsequently.

[0032] iv) For each sub-table T_sub, the corresponding HO-Tree is constructed.

[0033] v) Determine the type of the sub-table, if the type of the sub-table is L1 (table header-single value data), L2 (table header-multiple value data) or L4 (table header-orthogonal table), directly call the tree construction function to construct the HO-Tree, and add the result to HOTree_list. Construct(T_sub) is a simple HO-Tree construction function suitable for relatively simple tables. For more complex L.3 (orthogonal sub-table) layout, a recursive depth-first search (DFS) algorithm is required to construct the HO-Tree layer by layer, ensuring that the hierarchical relationship of each sub-table is completely expressed, which specifically comprises:

[0034] ① Construct Meta Tree (MT): Establish the table header hierarchy, and each root-to-leaf node path represents a column.

[0035] ② Construct Body Tree (BT): Construct the data content tree, and each root-to-leaf node path represents a row.

[0036] ③ Mapping between trees: While constructing the MT and BT, establish the node mapping between them to ensure the association of meta data and data content.

[0037] ④DFS Backtracking: In the DFS Backtracking phase, the processing node's hierarchical containment and orthogonal relationships are handled.

[0038] ⑤After construction, the generated HO-Tree is added to the HOTree_list.

[0039] vi)After all sub-table processing is complete, the Construct function is called to integrate the HO-Tree list into a complete HO-Tree, which is returned as the final output.

[0040] Step 2, problem decomposition and operation pipeline generation, as shown in Figure 5 , the HO-Tree obtained in step 1 is used to decompose the user input original question and generate an operation pipeline for retrieving data on the HO-Tree, which specifically includes:

[0041] 2.1) Problem decomposition: decompose the user input complex natural language question containing complex multi-hop into multiple simple single-step sub-questions.

[0042] The decomposition specifically includes: outputting the HO-Tree obtained in step 1 according to the nested structure in text form, and appending the user input complex natural language question, inputting the large language model (LLM), and generating multiple decomposed questions.

[0043] For example, "What is the total salary for 2021 and 2022?" can be decomposed into "Retrieve the 2021 salary", "Retrieve the 2022 salary", and "Compute the sum of the two".

[0044] 2.2) Relevant data retrieval: each single-step sub-question is processed independently, using top-down and bottom-up retrieval strategies, and when retrieval enters the finest granularity, the cosine similarity between problem entities and BT node values is calculated to filter out the most relevant nodes through embedding-based alignment methods.

[0045] The retrieval strategy includes: node retrieval (CHL), parent node retrieval (FAT), and value retrieval (EXT).

[0046] The cosine similarity of the problem entity and the BT node value refers to: using the Embedding embedding model, the problem entity text and the text of the BT node value are respectively converted into vector representation, and the distance between the two vectors is calculated using the cosine distance, and the closer the distance, the higher the similarity.

[0047] 2.3) Data operation: filtering (Cond), calculation (Math) and comparison (Cmp) processing of the retrieved most relevant nodes to obtain results consistent with the data required by the sub-problems, ensuring that the results are consistent with the context.

[0048] The filtering refers to eliminating irrelevant data in the sub-problem description from the most relevant nodes.

[0049] The calculation refers to mathematical operation based on the value of the node to obtain new results.

[0050] The comparison refers to comparing the values in the nodes with the information in the problem content to obtain the conclusions after comparison.

[0051] 2.4) Semantic reasoning: calling a large model for context reasoning (Reason) to ensure the accuracy of the answer.

[0052] Step 3, answer generation: executing the operation pipeline generated in step 2 on the HO-Tree and obtaining the execution results of each sub-problem, inputting each sub-problem and the operation pipeline execution result of the sub-problem into a large language model (LLM) to obtain the answer of each sub-problem, and finally inputting all sub-problems and sub-problem answers into the large language model (LLM) again to generate the final question answer and output to the user, including direct results and possible context information.

[0053] For complex multi-level table structures, structured answers containing hierarchical relationships can be output to ensure the integrity and interpretability of the information.

[0054] Step 4, two-stage verification mechanism, based on the operation pipeline generated in step 2 and the question answer obtained in step 3, verifying the correctness of the operation pipeline and the reliability of the question answer, specifically including:

[0055] 4.1) Forward verification: after each operation is executed, check whether the generated intermediate result is consistent with the context of the original question to ensure the effectiveness of the operation, specifically: after each operation is executed, evaluate whether the generated data can fully answer the corresponding sub-problem through LLM.

[0056] The evaluation refers to: by matching the parameters in each operation in the operation pipeline generated in step 2 with the table cell content in the HO-Tree, obtaining the vector embedding of the texts of the two using the Embedding embedding model, calculating the cosine distance between the vectors, and taking the content in the table that is most similar to the operation parameters as the modification of the operation parameters, ensuring consistency in semantics and syntax; after executing the operation, the completeness of the operation result is verified using a large language model (LLM), and if it is found that the operation execution result is insufficient to answer the question, a new operation statement will be generated to replace the current step and be re-executed until the iteration number limit (set to 3 times) is reached.

[0057] 4.2) Backward verification: after obtaining the final answer through step 3, verify the rationality of the operation pipeline, specifically: based on a key observation: multiple different queries of the same semi-structured table may produce the same answer, but their reasoning paths (Pipeline) may be different When there is inconsistency, steps 2-3 are re-executed to ensure the reliability of the answer.

[0058] Through actual application scene experiments, in the environment of Python version 3.10, single RTX 4090 GPU (software and hardware), the large model optimization method based on semi-structured table data question and answer of the application is run in the way of calling Deepseek V3 large language model (LLM) interface and InternVL 2.5 visual language model (VLM) interface, and the experimental data obtained are as shown in Figure 6 The effect on simple tables is comparable to that of GPT-4o and Deepseek V3, the accuracy on medium tables is about 13% higher than that of the second-ranked GPT-4o model, and the accuracy on difficult tables is about 20% higher than that of the second-ranked Deepseek V3 model.

[0059] Compared with the prior art, the application represents a complex semi-structured table as a data format of HO-Tree through a table processing unit, and designs data retrieval processing operations on the HO-Tree, so that the structure understanding of the large language model for the table and the content retrieval are decoupled, the difficulty of the large language model for the complex table understanding is alleviated, and the performance index is improved. The problem decomposition and module decompose a problem that requires multiple reasoning steps to obtain an answer into multiple simpler problems and obtain answers, and a certain performance improvement is obtained. In addition, the two-stage verification strategy uses a combination of forward verification and backward verification to ensure the correctness of the operation pipeline and the reliability of the final answer generation, further improving the overall system performance.

[0060] The above specific embodiments can be partially adjusted in different ways by those skilled in the art without departing from the principles and purposes of the present application, the protection scope of the present application is subject to the claims and is not limited by the above specific embodiments, each implementation within the scope is subject to the present application.

Claims

1. A large-scale model optimization method based on semi-structured tabular data question answering, characterized in that, For the input semi-structured table, an HO-Tree structure is constructed to accurately represent the multi-level relationships between the table header and content, and the table structure is serialized and stored. The complex questions raised by the user are broken down into multiple sub-questions, and corresponding operation pipelines and retrieval strategies are generated. The operation pipelines are executed on the HO-Tree structure according to the retrieval strategies. After checking the correctness of each step through forward validation, the overall reliability is evaluated through backward validation by answer backtracking to ensure the accuracy and consistency of the final answer.

2. The large model optimization method based on semi-structured tabular data question answering as described in claim 1, characterized in that, specifically... include: Step 1: Process the table and generate the HO-Tree structure, specifically including: 1.1) Meta-information extraction: After generating candidate meta-information cells based on the preprocessed semi-structured table using the Visual Language Model (VLM), the similarity between the candidate meta-information cells and all cells in the table is calculated. Cells with similarity exceeding a preset threshold are identified as valid meta-information, and their position information is used for subsequent table segmentation. 1.2) Table structure analysis, specifically including: for merged cells that span an entire row or column, treat them as table headers and treat the adjacent cells below or to the right as sub-tables; 1.3) Generate HO-Tree: Construct the Meta-information Tree (MT) and Data Tree (BT) separately, and then combine the MT and BT to construct a complete hierarchical orthogonal tree (HO-Tree); Step 2: Problem decomposition and operation pipeline generation. Using the HO-Tree obtained in Step 1, the original problem input by the user is decomposed, and an operation pipeline for retrieving data on the HO-Tree is generated. Step 3, Answer Generation: Execute the operation pipeline generated in Step 2 on the HO-Tree and obtain the execution results of each sub-problem. Input each sub-problem and the execution results of the sub-problem operation pipeline into the Large Language Model (LLM) to obtain the answer to each sub-problem. Finally, input all the sub-problems and their answers back into the Large Language Model (LLM) to generate the final answer to the question and output it to the user, including the direct result and possible contextual information. Step 4: A two-stage verification mechanism is used to verify the correctness of the operation pipeline and the reliability of the question answers based on the operation pipeline generated in Step 2 and the question answers obtained in Step 3. The preprocessing refers to converting the input semi-structured tables of different forms into HTML format, rendering them using a headless browser, and saving them as image format for use as input to the Visual Language Model (VLM). The aforementioned similarity is measured using n-grams or embeddings. The aforementioned meta-Info includes header, label, and cell elements; The aforementioned metadata tree is used to represent the header structure of a table, including a first-level header, a second-level header, and multi-level headers, reflecting the overall layout of the table. For a vertically structured table, each path from the root to a leaf node represents a column, reflecting the vertical hierarchical relationship. The data tree is used to represent the specific data content of the table, mainly including the data value of each cell and its corresponding attributes. For a horizontally structured table, each leaf node represents a row, reflecting the horizontal attribute relationships. The hierarchical orthogonal tree is used to accurately describe the header, data and hierarchical relationships in the table. The HO-Tree contains a nested structure, that is, some nodes in the BT recursively contain another HO-Tree.

3. The large model optimization method based on semi-structured tabular data question answering as described in claim 2, characterized in that, Step 1.3 specifically includes: i) The input table is split into sub-tables based on the metadata extracted by VLM to obtain a sub-table list T_list; ii) The table processing function divides the table into different logical blocks based on the metadata. These blocks may contain single-level tables (L1, L2, L4) or orthogonal sub-tables (L3). iii) Initialize an empty HO-Tree list HOTree_list to store the HO-Trees that will be built later; iv) For each sub-table T_sub, begin constructing the corresponding HO-Tree; v) Determine the type of the sub-table. If the type of the sub-table is L1 (header - single-value data), L2 (header - multi-value data), or L4 (header - orthogonal table), directly call the tree construction function to construct the HO-Tree and add the result to HOTree_list. Construct(T_sub) is a simple HO-Tree construction function, which is suitable for tables with relatively simple structures. For more complex L.3 (orthogonal sub-table) layouts, a recursive depth-first search (DFS) algorithm needs to be called to construct the HO-Tree layer by layer to ensure that the hierarchical relationship of each sub-table is fully expressed.

4. The large model optimization method based on semi-structured tabular data question answering as described in claim 3, is characterized in that, Step v) specifically includes: ① Construct MetaTree (MT): Establish a header hierarchy, where each path from the root to a leaf node represents a column; ② Construct BodyTree (BT): Construct a data content tree, where each path from the root to a leaf node represents a row; ③ Inter-tree mapping: While constructing MT and BT, establish node mapping between them to ensure the correlation between metadata and data content; ④DFS backtracking: In the DFS backtracking phase, the hierarchical containment and orthogonal relationships of nodes are processed; ⑤ After the construction is complete, add the generated HO-Tree to HOTree_list; vi) After all sub-tables have been processed, call the Construct function to integrate the HO-Tree list into a complete HO-Tree and return it as the final output.

5. The large model optimization method based on semi-structured tabular data question answering as described in claim 2, characterized in that, Step 2 specifically includes: 2.1) Problem decomposition: The complex natural language problem with multiple hops input by the user is decomposed into multiple simple single-step sub-problems. Specifically, the HO-Tree obtained in step 1 is output as text according to the nested structure, and the complex natural language problem input by the user is appended. The input is then fed into the Large Language Model (LLM) to generate multiple decomposed problems. 2.2) Relevant data retrieval: Each sub-problem is processed independently, using top-down and bottom-up retrieval strategies. When the retrieval reaches the finest granularity, the most relevant nodes are selected by calculating the cosine similarity between the problem entity and the BT node value through an embedding-based alignment method. 2.3) Data manipulation: Filter (Cond), calculate (Math), and compare (Cmp) the most relevant nodes retrieved to obtain results that match the data required by the sub-problem, ensuring that the results conform to the context; 2.4) Semantic reasoning: Call the large model to perform contextual reasoning to ensure the accuracy of the answer.

6. The large model optimization method based on semi-structured tabular data question answering as described in claim 5, characterized in that, The retrieval strategies include: node retrieval (CHL), parent node retrieval (FAT), and value retrieval (EXT); The cosine similarity between the question entity and the BT node value refers to: using the Embedding model, converting the text of the question entity and the text of the BT node value into vector representations respectively, and using cosine distance to calculate the distance between the two vectors; the closer the distance, the higher the similarity. The filtering mentioned refers to removing irrelevant data from the most relevant nodes in the sub-problem description; The calculation refers to performing mathematical operations based on the node values ​​to obtain new results; The comparison refers to comparing the value in the node with the information in the question content to obtain a conclusion.

7. The large model optimization method based on semi-structured tabular data question answering as described in claim 2, characterized in that, Step 4 specifically includes: 4.1) Forward validation: After each operation is executed, check whether the generated intermediate results are consistent with the context of the original question to ensure the validity of the operation. Specifically, after each operation is executed, evaluate whether the generated data can fully answer the corresponding sub-question through LLM. 4.2) Backward verification: After obtaining the final answer through step 3, verify the rationality of the operation pipeline. Specifically, based on a key observation: multiple different queries on the same semi-structured table may produce the same answer, but their reasoning paths (Pipeline) may be different. When inconsistencies occur, re-execute steps 2-3 to ensure the reliability of the answer.

8. The large model optimization method based on semi-structured tabular data question answering as described in claim 7, is characterized in that, The evaluation refers to: matching the parameters in each operation of the operation pipeline generated in step 2 with the content of the table cells in the HO-Tree, using the Embedding model to obtain the vector embeddings of the two texts, calculating the cosine distance between the vectors, and using the content in the table that is most similar to the operation parameters as the correction of the operation parameters to ensure semantic and syntactic consistency. After an operation is performed, the completeness of the operation result is verified using a Large Language Model (LLM). If the operation result is found to be insufficient to answer the question, a new operation statement is generated to replace the current step and the operation is re-executed until the maximum number of iterations is reached.

9. A large model optimization system for implementing the method of any one of claims 1-8, characterized in that, include: The system comprises a table processing unit, a question processing unit, an answer generation unit, and an answer verification unit. Specifically: The table processing unit performs table layout matching and splitting based on the header and hierarchical nesting structure information of the original table, obtaining a nested sub-table information structure and constructing it into an HO-Tree structure. The question processing unit, based on the semantic information of the question, combines the HO-Tree obtained in previous steps to decompose the question steps and generate sub-question operations, obtaining the operation pipeline results for each sub-question and the HO-Tree retrieval strategy results. The answer generation unit, based on the sub-question operation pipeline and retrieval strategy information obtained in previous steps, executes the operation pipeline on the HO-Tree, obtaining the results of retrieving relevant table content based on the semantics of the sub-questions. The answer verification unit, based on the results of the executed operation pipeline and the information of the final generated answer, checks the correctness of each operation and performs a final answer backtracking process, obtaining the operation step correctness verification results and the overall reliability assessment results of the final answer.

Images