Table content structured processing method and device, medium and product

Abstract

The invention provides a table content structured processing method and device, a medium and a product, and the method comprises five steps: correcting the condition of table title recognition errors in layout analysis; a standardized storage data structure of table content in the PDF document is constructed; sub-expression expressions fitting table contents and table meanings; a comprehensive structured storage mode is standardized; and the engineering adaptation and generalization capability is good.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and engineering data processing technology, in particular to a table content structuring processing method and device, medium and product. BACKGROUND

[0002] In the field of engineering design and construction, technical specifications serve as an important technical basis for projects, which plays a crucial role in engineering quality and safety. However, due to the very detailed provisions and the close logical relationship between them, it takes a lot of time for design and construction personnel to find the corresponding provisions from the numerous technical specifications. With the development of large language models, intelligent question and answer systems based on Retrieval-Augmented Generation (RAG) technology provide a solution to the above-mentioned problems. Chinese patent CN118332072B proposes an intelligent document retrieval and generation method and system based on RAG technology, which realizes natural language query for text documents. However, if the large number of tables in the specifications are converted into text for RAG operations such as sharding and embedding, the logical structure of the tables will be lost, and the meaning of the table content cannot be understood due to the loss of table context.

[0003] To address the difficulty of directly processing tables using text-based RAG technology, there are currently several solutions: one is to use a large amount of labeled data to train an end-to-end table recognition and splitting model, which directly obtains a basic model that can convert table content into text, such as Chinese patent CN120148056A; two is to define artificial rules for the structure analysis of specific industry documents, and then generate text representations of tables according to the rules, such as Chinese patent CN118966159A; three is to use a layout recognition model and a large language model for collaborative processing, accurately identify the position and content of the table, and then use keyword matching and large language model understanding to provide the table content with complete context and reasonable text representation, such as Chinese patent CN119917654A. The first method requires a large amount of high-precision data labeling, which greatly limits the feasibility of the method. The second method has some application prospects for specific industry-specific tables, but the definition of rules needs to be changed with input, and it cannot provide the context of the table. The third method provides a relatively accurate context for the table content, but lacks the ability to handle complex problems such as multiple sub-tables, cross-page tables, and long continuous tables, and the generated text representation lacks specific data of the table due to the use of summary representation. SUMMARY

[0004] The present application aims to overcome the shortcomings of the prior art and provide a table content structuring processing method and device, medium and product, which can solve the problem of the prior art in handling multiple sub-tables, cross-page tables, and long continuous tables.

[0005] To this end, the present application adopts the following technical solutions: The table content structuring processing method and device, medium and product thereof, comprise the following steps: Step one, adopt a document layout analysis model to identify the table main body area and its related text area in the document, and output the position information and text content of each area; Step two, based on the spatial position relationship, filter out the text related to the table main body from the text content as context information, and at the same time adopt a table parsing model to parse the table main body into HTML structured data; Step three, through a large language model, the semantic discrimination of the context information is carried out, and the table title and table note are identified, so as to obtain the standard data structure of the table title, table main body and table note; Step four, the standard data structure is transmitted into the large language model to generate the text sub-expression of each row data of the table main body; Step five, the text sub-expression is vectorized by using an embedding model, and the vectorization result is matched and stored in structure with the original image of the table main body area and the HTML structured data.

[0006] On the basis of adopting the above technical solutions, the present application can also adopt the following further technical solutions, or use these further technical solutions in combination: The document layout analysis model in step one can be MinerU, LayoutLM or PP-Structure.

[0007] The table parsing model in step two can be ppstructure_table, wired_table_rec_v2 or RapidTable.

[0008] Step three comprises: before semantic discrimination, first, all areas are sorted according to the page number, and within the same page number, the areas are sorted from top to bottom according to the boundary box vertical coordinate value; the prompt words of the large language model are defined, so that the large language model can discriminate the text and assign the corresponding text to the corresponding field of the standard data structure according to the discrimination result; the table main body area is taken as the center, the text is traversed in reverse order, the traversed text is input into the large language model for discrimination, when the discrimination result is the table title, the text is assigned to the table title field of the standard data structure; after positioning the table title, it is judged whether there is other text between the table main body and the table title, if there is, the part of the text is combined and assigned to the table sub-title field of the standard data structure; the table main body area is taken as the center, a predetermined number of texts are traversed in sequence, the traversed texts are input into the large language model for discrimination, when the discrimination result is the table note, the text is assigned to the table note field of the standard data structure.

[0009] Step four includes: designing prompt words for text sub-representations to guide the large language model to output factual text sub-representations for each line of data.

[0010] The embedding model in step five can be an embedding model, including bge-large-zh-v1.5, Qwen3-Embedding-0.6B, or bge-m3.

[0011] The method is applicable to processing multi-subtables, multi-page tables, and long continuation tables, and accurately matches the table body with the corresponding table title and table notes. A computer device includes a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the method of claim 1.

[0012] A computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 1.

[0013] A computer program product, comprising a computer program / instructions, When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 1.

[0014] Compared with the prior art, the present invention has the following advantages and beneficial effects: Correcting the error in table title recognition in layout analysis: The present invention uses a large language model to perform semantic discrimination on text that is spatially related to the main content of the table, thus solving the problem of missed and misidentified recognition of table titles and table notes; A standardized data structure for storing table content in PDF documents was constructed: by constructing a multi-level standardized storage structure for table content, including main table title, subtitle, footnotes, body, and sub-descriptions, the content of multi-sub-tables, cross-page tables, and long continuation tables can be expressed in a complete and consistent manner, thereby improving the efficiency of subsequent use of table content. Sub-representations that fit the content and meaning of the table: Sub-representations of the table based on the table topic ensure that the table information does not deviate from the meaning and application scope of the table in the document when it is converted into text, reducing the complex steps of needing to connect with the context and then break it down. A standardized and comprehensive structured storage method: By combining the table body image, table HTML, and sub-expressions into a vectorized form for structured storage, it is convenient to retrieve the entire table content through sub-expression vectorization and to display the original text of subsequent reference content. Excellent engineering adaptability and generalization ability: This method is applicable to scanned and text-based PDF documents, and can be used for documents of different purposes and styles, demonstrating good versatility and potential for engineering promotion. Attached Figure Description

[0015] Figure 1 This is the overall technical roadmap of the present invention.

[0016] Figure 2 This is an example diagram showing a document containing tables spanning multiple pages and multiple sub-tables.

[0017] Figure 3 This is the intended representation of the structured data in the table. Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solutions of the present invention, preferred embodiments of the present invention are described below in conjunction with specific examples. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote elements with the same or similar functions throughout. However, it should be understood that the drawings are for illustrative purposes only and should not be construed as limiting the present invention. To better illustrate this embodiment, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product size. It is understandable for those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings. The positional relationships described in the drawings are for illustrative purposes only and should not be construed as limiting the present invention.

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments, but this should not be construed as limiting the present invention.

[0020] The present invention provides a method, equipment, medium, and product for structuring table content, comprising the following steps: Step 1: Use the document layout analysis model to identify the layout and obtain information about the table body and other text in the document.

[0021] The document layout analysis model can be a PP-Structure model based on image object detection or a LayoutLLM model based on text and image multimodal fusion. The goal is to identify text in tables and other non-table areas within the document, obtaining the page position information of the table content and other text. Figure 2 For example, this document contains tables spanning multiple pages and multiple sub-tables. The identified data results are as follows: [{ "type": "text", "bbox": [65,123,148,136], "page": 76, "content": "6.2 Stone Excavation Engineering" }, { "type": "text", "bbox": [65,151,616,166], "page": 76, "content": "6.2.1 The quota includes general rock excavation (Table 56), trench rock excavation (Table 57), pit rock excavation (Table 58), and rock transportation." }, … { "type": "text", "bbox": [519,400,604,412], "page": 76, "content": "The unit of measurement is \mathrm{100m^{3}}" }, { "type": "table_body", "bbox": [64,413,617,910], "page": 76, "image_path":"d4dfab6ffe1be4279903b31052158b881781c89fe41db2abfba7377b7457067c.jpg" }, { "type": "title", "bbox": [311,114,422,126], "page": 77, "content": Hydraulic breaker for rock excavation }, … ] In this data structure, "type" represents the type of content layout identified. Since different layout analysis models are used, the type results may vary and accuracy cannot be guaranteed. However, all models can achieve 100% accuracy in recognizing table bodies. "bbox" represents the position of the identified content within the current page, measured in pixels after the page is converted to an image, with the origin typically at the top left corner. "page" represents the page number of the document containing the identified content. "content" is the identified content; if the identified type is a table body (table_body), it is replaced by the "image_path" variable, representing a screenshot of the table body. Other content types are the text obtained from OCR.

[0022] Step 2: Extract the main information of the tables from the layout analysis results and combine it with the text information related to the spatial location to form structured data.

[0023] Extracting the table body information refers to inputting the obtained body screenshot into a table parsing model for image-to-HTML content parsing. The table parsing model can use open-source models such as ppstructure_table, wired_table_rec_v2, or RapidTable. The data structure related to the table body is redefined, adding fields such as "html", "table_title", "table_subtitles", and "table_footnotes". These correspond to the HTML after image parsing, the main table title, the subtitles, and the footnotes, respectively. For example... Figure 2 The data structure related to the main body of the first table is as follows: { "type": "table_body", "bbox": [64,413,617,910], "page":76, "html":" Item … ", "image_path": "d4dfab6ffe1be4279903b31052158b881781c89fe41db2abfba7377b7457067c.jpg", "table_title": "", "table_subtitles": " ", "table_footnotes": " " }, Step 3: Traverse the text related to the table body according to the spatial location relationship, and use the large language model to perform semantic judgment to determine whether it is a table title or a table footer, thereby obtaining the data values ​​of table title, table body and table footer.

[0024] Specifically, the content mentioned is as follows: ① Based on the data generated in the first step, sort the elements by page number and spatial position according to the second coordinate value of "page" and "bbox". Ensure that the elements are first sorted by page number from smallest to largest, and then sorted by their y-coordinate from top to bottom. Then, iterate through the elements with "type" = "table_body" for organizing the table information.

[0025] ② Define the large language model prompt for determining whether the input is a table title: "Your task is to determine only whether the input is a table name, figure name, or table footnote. If it is a title and not a continuation table or figure title, reply with 1; if it is a continuation table or figure title, reply with 2; if it is a table footnote, reply with 3; otherwise, reply with 0. No additional explanations are allowed; only numbers should be output." ③ Based on the "table_body" element located in ①, traverse the elements in reverse order. If "type"="table_body" is encountered directly on the first traversal, the "table_title" value of the encountered table body element is used as the table title; otherwise, the "content" value of the encountered "type"="text" element is passed to the large language model, and the large language model is used to determine whether the text content is a table title. If the large language model determines that it is a table title, then the "content" text value is saved as the table title, and the number of traversals J is recorded (used to determine the subtitle content).

[0026] ④ After finding the table title, collect the table subtitles. Determine if there is any other text between the table body and the table title. If so (J>1), iterate through these texts and combine them as the value of "table_subtitles" for storage.

[0027] ⑤ Locate table footnotes. This invention limits footnotes to the 6 elements following the table body, so it iterates through the 6 elements following the table body and performs a large-scale model determination to determine if they are table footnotes. If the response is 3, indicating a table footnote, then this text is stored as the value of "table_footnotes".

[0028] by Figure 2 For example, after this step, the following structured data table can be obtained: [ { "type": "table_body", "bbox": [64,413,617,910], "page":76, "html":" Item … ", "image_path": "d4dfab6ffe1be4279903b31052158b881781c89fe41db2abfba7377b7457067c.jpg", "table_title": "Table 56 General Rock Excavation", "table_subtitles": "Scope of application for blasting excavation: general rock excavation. Work content: drilling, blasting, cutting, surface clearing, and section trimming, etc. Unit of measurement is \\mathrm{100m^{3}}", "table_footnotes": " " }, { "type": "table_body", "bbox": [90,180,645,329], "page":77, "html":" Item … ", "image_path": "902b30b06eb6b39cedd1592a448d1ce23489245edb1e12c3a0451ea8268cb854.jpg", "table_title": "Table 56 General Rock Excavation", "table_subtitles": "Hydraulic breakers are suitable for rock excavation. General rock excavation. Work includes: machine positioning, rock impact, and machine transfer. The unit of measurement is 100m³". "table_footnotes": " " }, { "type": "table_body", "bbox": [90,389,644,574], "page":77, "html":" Item … ", "image_path": "902b30b06eb6b39cedd1592a448d1ce23489245edb1e12c3a0451ea8268cb854.jpg", "table_title": "Table 56 General Rock Excavation", "table_subtitles": "Excavator (hydraulic backhoe) for dredging rock and gravel: Open-pit. Work content: Excavation, stacking. 100\\mathrm{m}^{3}", "table_footnotes": " " }, { "type": "table_body", "bbox": [88,652,644,913], "page": 77, "html": " Item … ", "image_path": "78960e79e44610ff6bd118ad9980e8a4b1885615b2ffda700966b8732edbf86d.jpg", "table_title": "Table 57 Trench Rock Excavation", "table_subtitles": "Work content: drilling, blasting, cutting, slag removal, surface cleaning, section repair, etc. 100\\mathrm{m}^{3}", "table_footnotes": "" }] Step 4: Input the table title, the parsed HTML of the table body, and the table footers into the large language model to summarize the table theme and design prompt words to obtain the text description of each row of data based on the table theme.

[0029] The table title, the parsed HTML of the table body, and the table footnote data refer to the data in the table-related dictionaries obtained in step three. Each table body has one dictionary, which contains the table title `table_title="table_title" + "table_subtitles"`, the parsed HTML content `table_body="html"`, and the table footnotes `table_footnote="table_footnotes"`. The prompts for designing table sub-statements are as follows: “system”: “Role: You are a professional data analyst. Analyze the HTML table and table title I provide, and then generate a series of sub-questions based on this. The goal is to transform the information in the table into a textual description.” "user":f""" Task: Please carefully read the table content, table title, and table notes I provided to identify the table's theme. Then, generate a sub-question for each row in the table.

[0030] Require: Each sub-question must be strictly related to the topic of the table, must contain the topic keywords, and must retain as much information as possible from the table. The content cannot be summarized, and no information not found in the table should be added. Ensure that the questions are accurate factual questions based on an understanding of the table's topic, and that the answers can be obtained directly by looking at the table's content.

[0031] Table title: {table_title} Table content: {table_body} Table Notes: {table_footnote} Output format: Output your answers line by line, one question per line. Do not use item markers or numbers, and do not add serial numbers or other symbols.

[0032] """ For example: Figure 2 The sub-statements in the second table are as follows: The unit for senior technicians in the project is man-days, with 0.02 for turquoise, 0.02 for medium-hard stone, and 0.03 for hard stone.

[0033] The unit of measurement for project technicians is man-days: 0.13 for turquoise, 0.16 for medium-hard stone, and 0.20 for ordinary hard stone. The unit for general laborers on the project is man-days: 0.20 for turquoise, 0.26 for medium-hard stone, and 0.32 for ordinary hard stone.

[0034] The total unit for the project is man-days: 0.35 for turquoise, 0.44 for medium-hard stone, and 0.55 for sound peak sacred stone.

[0035] Other material costs are in yuan: 10 for turquoise, 13 for medium hard stone, and 16 for ordinary hard stone.

[0036] The unit for 100mm of a hydraulic breaker is a shift. The unit for turquoise is 3.96, for medium hard rock it is 5.04, and for ordinary hard rock it is 6.39.

[0037] The unit for a 140mm hydraulic breaker is a shift. The unit for turquoise is 3.55, for medium hard rock it is 4.51, and for ordinary hard rock it is 5.71.

[0038] The unit for transporting stone chips is m3; loose stone is 101, medium-hard stone is 101, and ordinary hard stone is 101. The compiling units did not provide specific information; turquoise corresponds to 2004, medium hard stone to 2005, and ordinary hard stone to 2006.

[0039] Step 5: Match the main image of the table, the parsed HTML of the table, and the table sub-representation vectors obtained using the embedding model, and store the three in a structured manner. The embedding model can be any open-source embedding model, such as bge-large-zh-v1.5, Qwen3-Embedding-0.6B, or bge-m3. The embedding model only vectorizes the sub-representations of the table, mapping them to the main table image and the parsed HTML data using the same ID. These arrays are stored in an Elasticsearch database for easy RAG retrieval later. The storage structure is as follows... Figure 3 As shown.

[0040] Based on the description and accompanying drawings of this invention, those skilled in the art can readily manufacture or use the table content structuring method, apparatus, medium, and products of this invention, and can achieve the positive effects described in this invention.

[0041] It should be noted that the terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this invention are intended to cover non-exclusive inclusion. Those skilled in the art can understand the specific meaning of these terms in this invention according to the specific circumstances.

[0042] Furthermore, in practicing the claims of this invention, those skilled in the art can understand and influence variations to the disclosed embodiments through a study of the drawings, the disclosure, and the appended claims. Additionally, in the claims and description, words such as "comprising" and "containing" do not exclude other elements or steps, and non-plural nouns do not exclude their plural forms.

[0043] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. All equivalent changes and modifications made in accordance with the present invention are covered by the scope of the claims of the present invention, and will not be listed here.

Claims

1. A method for structuring table content, characterized in that, Includes the following steps: Step 1: Use a document layout analysis model to identify the main table area and its related text areas in the document, and output the location information and text content of each area; Step 2: Based on spatial location relationships, select text related to the table body from the text content as context information, and at the same time use a table parsing model to parse the table body into HTML structured data; Step 3: Use a large language model to perform semantic discrimination on the context information to identify the table title and table notes, thereby obtaining the standard data structure of table title, table body and table notes; Step 4: Input the standard data structure into the large language model to generate text sub-representations of each row of data in the main body of the table; Step 5: Vectorize the text sub-representations using an embedding model, and match and store the vectorized results with the original image and HTML structured data of the main table area in a structured manner.

2. The table content structuring method as described in claim 1, characterized in that, The document layout analysis model in step one can be MinerU, LayoutLM, or PP-Structure.

3. The method for structuring table content as described in claim 1, characterized in that, The table parsing model in step two can be ppstructure_table, wired_table_rec_v2, or RapidTable.

4. The method for structuring table content as described in claim 1, characterized in that, Step three includes: Before semantic discrimination, all regions are first sorted according to page number, and within the same page number, they are sorted from top to bottom according to the y-coordinate value of the region's bounding box; defining large language model prompt words, enabling the large language model to discriminate text and assign the corresponding text to the corresponding field of the standard data structure based on the discrimination result; traversing the text backwards from the table body area as the center, inputting the traversed text into the large language model for discrimination, and when the discrimination result is a table title, assigning the text to the table title field of the standard data structure; after locating the table title, determining whether there is other text between the table body and the table title, if so, combining this part of the text and assigning it to the table subtitle field of the standard data structure; traversing a predetermined number of texts sequentially backwards from the table body area as the center, inputting the traversed text into the large language model for discrimination, and when the discrimination result is a table footer, assigning the text to the table footer field of the standard data structure.

5. The method for structuring table content as described in claim 1, characterized in that, Step four includes: designing prompts for text sub-representations to guide the large language model to understand the standard data structure, and then generating factual text sub-representations for each row of data in the table body of the standard data structure, retaining all the original data of that row without making general summaries.

6. The method for structuring table content as described in claim 1, characterized in that, The embedding model in step five can be an embedding model, including bge-large-zh-v1.5, Qwen3-Embedding-0.6B, or bge-m3.

7. The method for structuring table content as described in claim 1, characterized in that, The method is applicable to processing tables with multiple sub-tables, tables spanning multiple pages, and long continuation tables, and accurately matches the table body with the corresponding table title and table notes.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method of claim 1.

9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 1.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 1.

Images