A method and device for converting irregular table structure into JSON file

Abstract

The present invention provides a method for converting an irregular table structure into a JSON file, comprising obtaining an irregularly divided original table file, extracting each sub-cell in the original table file as an independent node to form a node set; based on the spatial position information of the node, traversing the neighbor nodes that satisfy a predetermined adjacency relationship of each node to construct an adjacency graph structure; calculating the text cosine similarity, text type matching score and context semantic score between two adjacent nodes according to the text content and corresponding type of the node, and deleting the connection edges without semantic association in the adjacency graph structure; deleting and updating the direction with the smallest number of connection edges in the adjacency graph structure; parsing and organizing the table semantic structure according to the connection relationship retained between nodes in the updated adjacency graph structure to generate a nested JSON file. Implementation of the present invention can efficiently process complex table structures, improve the accuracy of structured extraction and the semantic consistency of JSON expression.

Description

Technical Field

[0001] The present invention relates to the technical field of computer data processing, and in particular to a method and device for converting an irregular table structure into a JSON file. Background Art

[0002] With the increasing prevalence of information management and data intelligence systems, the acquisition of structured data has become a critical prerequisite for data processing. In particular, data files stored in irregularly divided tables are common in real-world scenarios such as medical documents, legal reports, financial vouchers, and government documents. These tables often originate from scanned images, complex spreadsheets, or nested output formats. Rows and columns are irregularly divided, and a single row may be split into multiple cells of varying sizes, lacking standardized structural boundary descriptions.

[0003] Currently, traditional table recognition and conversion methods typically rely on regular templates or structure inference based on fixed grids. However, these methods are significantly less adaptable to irregular tables with flexible formats and complex structures, often leading to errors in field associations, data loss, or misclassification, significantly reducing the accuracy of subsequent data mining and analysis.

[0004] Therefore, to address these issues, research has attempted to introduce visual recognition algorithms or language model-based structural recognition methods, and to use deep learning models to perform end-to-end parsing of tables. While some progress has been made in standardizing tables, there is still a lack of stable, universal solutions for irregular tables with loose structures, significant differences in cell sizes, and loose field associations. In particular, in applications that require structural restoration, semantic cleansing, and JSON-formatted data output of tables, accurately identifying the relationships between fields and constructing hierarchical data structures without template constraints has become a major bottleneck restricting the automated extraction and understanding of information.

[0005] However, existing methods often rely on manually set rules or training samples, lack transferability, and find it difficult to maintain consistent recognition accuracy in multi-type table environments.

[0006] Therefore, there is an urgent need for a table parsing method with structural adaptability, semantic understanding ability and strong versatility, which can perform node modeling, connection relationship identification, and semantic redundancy trimming for irregularly divided table files, and finally output semantically clear JSON format structured data to improve data utilization efficiency and system automation capabilities. Summary of the Invention

[0007] The technical problem to be solved by the embodiments of the present invention is to provide a method and device for converting an irregular table structure into a JSON file, which can realize structural modeling, semantic discrimination and field organization of cells in the table without relying on templates, and output JSON objects with accurate semantics and clear structure, thereby improving the accuracy of information extraction and the level of processing automation in complex table scenarios.

[0008] In order to solve the above technical problems, an embodiment of the present invention provides a method for converting an irregular table structure into a JSON file, the method comprising the following steps:

[0009] Obtaining an irregularly divided original table file, identifying each sub-cell and related information in the original table file, treating each sub-cell as an independent node, and further forming a node set carrying all sub-cell related information; wherein each node carries corresponding sub-cell related information, including spatial position information of the four corners of the sub-cell rectangular border, and text content contained within the sub-cell rectangular border and its corresponding type;

[0010] Based on the spatial position information of the four corners of the rectangular border of the sub-cell corresponding to each node, each node is traversed to find neighboring nodes that satisfy a predetermined adjacency relationship to construct an adjacency graph structure; wherein the predetermined adjacency relationship is that the two nodes should simultaneously satisfy a direction validity condition, an overlap rate validity condition, and a distance validity condition; the direction validity condition is that the vertical center position interval between the two nodes is less than a first predetermined interval value, and the horizontal center position interval between the two nodes is less than a second predetermined interval value; the overlap rate validity condition is that the overlap rate values ​​between the two nodes in the horizontal direction and the vertical direction are both greater than a preset overlap rate threshold; the distance validity condition is that the Euclidean distance between the two nodes is within a predetermined distance range;

[0011] Based on the text content carried by each node and its corresponding type, the text similarity score, text type matching score and context semantic score between each node and its neighboring nodes are calculated to further obtain the comprehensive score of the text relationship between each node and its neighboring nodes, and based on the comprehensive score of the text relationship between each node and its neighboring nodes, the edges in the adjacency graph structure that meet the predetermined semantic invalid association condition are deleted; wherein the predetermined semantic invalid association condition is that the comprehensive score of the text relationship between any node and its neighboring nodes is less than a preset score threshold;

[0012] In the adjacency graph structure, determining the number of horizontal connecting edges and vertical connecting edges corresponding to each node, and updating the adjacency graph structure after deleting the smallest number of horizontal connecting edges or vertical connecting edges in each node;

[0013] According to the connection relationship retained between nodes in the updated adjacency graph structure, the table semantic structure is parsed and organized to generate a nested JSON file that conforms to the actual semantics.

[0014] The node set is ;in,

[0015] To carry the i nodes with relevant information about child cells, and ;

[0016] For the i The text content of the node, that is, i The characters or strings contained within the rectangular border of each child cell represent the semantic information of the node;

[0017] For the i The spatial location information of nodes; No. i The coordinates of the upper left corner of the rectangular border of the sub-cell, The coordinates of the lower right corner;

[0018] For the i The type of text content of each node, including text, number, time, and mixed.

[0019] The direction validity conditions are expressed by formulas (1) and (2);

[0020] (1)

[0021] (2)

[0022] in, and Respectively i nodes and j The vertical center of the nodes, and ; and Respectively i nodes and j The horizontal center of the nodes, and ; For the i nodes and j The horizontal center position interval between nodes; No. i nodes and j The vertical center position interval between nodes; is a first predetermined interval value; is the second predetermined interval value.

[0023] The overlap rate validity conditions are expressed by formulas (3) to (5);

[0024] (3)

[0025] (4)

[0026] ; (5)

[0027] in, For the i nodes and j The overlap ratio of nodes in the horizontal direction; No. i nodes and j The overlap ratio of nodes in the vertical direction; is the preset overlap rate threshold; For the i The cell height of nodes, and ; For the i The cell width of nodes, and ; For the j The cell height of nodes, and ; For the j The cell width of nodes, and .

[0028] Among them, the distance validity condition is Within 1.5 of the average width and average height; No. i nodes and j The Euclidean distance between nodes is calculated by formula (6);

[0029] (6).

[0030] Among them, the text similarity score between each node and its neighbor nodes is calculated by formula (7);

[0031] (7)

[0032] in, For the i nodes andj The text similarity score between nodes; and Respectively i nodes and j The text content of each node; is the norm calculation;

[0033] The text type matching score between each node and its neighbor nodes is calculated using formula (8);

[0034] (8)

[0035] in, For the i nodes and j Text type matching score between nodes; and Respectively i nodes and j The type of text content in each node; It is a preset matching scoring matrix used to record matching scores;

[0036] The contextual semantic score between each node and its neighboring nodes is calculated using formula (9);

[0037] (9)

[0038] in, For the i nodes and j Contextual semantic score between nodes; A linear transformation matrix; is the weight vector in the preset attention mechanism; LeakyReLU is the activation function; softmax is used to normalize the final result.

[0039] Among them, the comprehensive score of the textual relationship between each node and its neighboring nodes is calculated by formula (10);

[0040] (10)

[0041] in, For the i nodes and j Comprehensive score of textual relations between nodes; , and are three preset coefficients, and .

[0042] An embodiment of the present invention also provides a device for converting an irregular table structure into a JSON file, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the aforementioned method for converting an irregular table structure into a JSON file are implemented.

[0043] The implementation of the embodiments of the present invention has the following beneficial effects:

[0044] In view of the characteristics of irregularly divided tables, the present invention designs a method for converting irregular table structures into JSON files. By modeling the table as a graph structure and integrating spatial adjacency rules with a multi-index connection scoring mechanism, the semantic associations and structural hierarchies between fields in the table are effectively identified, and a JSON object with clear structure and accurate semantics is automatically generated. In this way, structural modeling, semantic discrimination and field organization of cells in the table can be achieved without relying on templates, and JSON objects with accurate semantics and clear structure can be output, thereby improving the accuracy of information extraction and the level of processing automation in complex table scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, without paying any creative work, other drawings obtained based on these drawings still fall within the scope of the present invention.

[0046] Figure 1 A flowchart of a method for converting an irregular table structure into a JSON file is provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention will be described in further detail below with reference to the accompanying drawings.

[0048] like Figure 1 As shown in the figure, a method for converting an irregular table structure into a JSON file is provided in an embodiment of the present invention, and the method includes the following steps:

[0049] Step S1: Obtain an irregularly divided original table file, identify each sub-cell and related information in the original table file, and treat each sub-cell as an independent node, further forming a node set carrying all sub-cell related information; wherein each node carries corresponding sub-cell related information, including spatial position information of the four corners of the sub-cell rectangular border, and text content contained in the sub-cell rectangular border and its corresponding type;

[0050] The specific process is as follows: first, obtain the irregularly divided original table file, such as an image or spreadsheet; second, identify each sub-cell in the original table file and read the relevant information of each sub-cell, including the text content and type of text content contained in the rectangular border of the sub-cell, as well as the boundary coordinates of the four corners of the rectangular border of the sub-cell; finally, treat each sub-cell as an independent node, and further form a node set carrying the relevant information of all sub-cells as

[0051] At this time, each node As a structure, Carrying the i nodes with relevant information about child cells, and ;

[0052] in, For the i The text content of the node, that is, i The characters or strings contained within the rectangular border of each child cell represent the semantic information of the node; For the i The spatial location information of nodes; No. i The coordinates of the upper left corner of the rectangular border of the sub-cell, The coordinates of the lower right corner; For the i The type of text content of each node, including text, number, time, and mixed.

[0053] Step S2: based on the spatial position information of the four corners of the rectangular border of the sub-cell corresponding to each node, traverse the neighboring nodes of each node that satisfy a predetermined adjacency relationship to construct an adjacency graph structure; wherein the predetermined adjacency relationship is that the two nodes should simultaneously satisfy a direction validity condition, an overlap rate validity condition, and a distance validity condition; the direction validity condition is that the vertical center position interval between the two nodes is less than a first predetermined interval value, and the horizontal center position interval between the two nodes is less than a second predetermined interval value; the overlap rate validity condition is that the overlap rate values ​​between the two nodes in the horizontal direction and the vertical direction are both greater than a preset overlap rate threshold; the distance validity condition is that the Euclidean distance between the two nodes is within a predetermined distance range;

[0054] The specific process is as follows: considering that the adjacency graph structure is a set of adjacent edges constructed based on the spatial relationship between each node, when traversing each node and finding a neighboring node that meets the predetermined adjacency relationship, if the node pair simultaneously meets the direction validity condition, overlap validity condition, and distance validity condition, a connecting edge is established between the two nodes, thus obtaining the adjacency graph structure.

[0055] At this point, the process of establishing a connection edge between any two node pairs is as follows:

[0056] First, for any node, only the horizontal and vertical adjacency connections are considered. Therefore, for any two nodes and , when it satisfies formula (1) and formula (2), it is determined that there is a horizontal direction relationship and a vertical direction relationship, that is, it meets the direction validity condition.

[0057] (1)

[0058] (2)

[0059] in, and Respectively i nodes and j The vertical center of the nodes, and ; and Respectively i nodes and j The horizontal center of the nodes, and ; For the i nodes and j The horizontal center position interval between nodes; No. i nodes and j The vertical center position interval between nodes; is a first predetermined interval value; is the second predetermined interval value.

[0060] Next, for the node pairs that meet the direction validity condition, the boundary overlap rate in the vertical or horizontal direction is calculated using formulas (3) to (5) to determine whether the two are aligned, that is, whether the overlap rate validity condition is met.

[0061] (3)

[0062] (4)

[0063] ; (5)

[0064] in, For the i nodes and j The overlap ratio of nodes in the horizontal direction; No. i nodes and jThe overlap ratio of nodes in the vertical direction; is the preset overlap rate threshold; For the i The cell height of nodes, and ; For the i The cell width of nodes, and ; For the j The cell height of nodes, and ; For the j The cell width of nodes, and .

[0065] Then, the Euclidean distance between node pairs is calculated by formula (6), and The distance validity condition is met when it is within 1.5 of the average width and average height;

[0066] (6).

[0067] Finally, in i nodes and j A connecting edge is established between the nodes.

[0068] Step S3: Calculate the text similarity score, text type matching score, and context semantic score between each node and its neighboring nodes based on the text content carried by each node and its corresponding type, so as to further obtain a comprehensive score of the text relationship between each node and its neighboring nodes, and delete the edges in the adjacency graph structure that meet a predetermined semantic invalid association condition based on the comprehensive score of the text relationship between each node and its neighboring nodes; wherein the predetermined semantic invalid association condition is that the comprehensive score of the text relationship between any node and its neighboring nodes is less than a preset score threshold;

[0069] The specific process is as follows: first, the text similarity score between each node and its neighbor nodes is calculated using formula (7);

[0070] (7)

[0071] in, For the i nodes and j The text similarity score between nodes; and Respectively i nodes and jThe text content of each node; Calculate the norm.

[0072] Secondly, the text type matching score between each node and its neighbor nodes is calculated using formula (8);

[0073] (8)

[0074] in, For the i nodes and j Text type matching score between nodes; and Respectively i nodes and j The type of text content in each node; It is a preset matching scoring matrix used to record matching scores.

[0075] Then, the contextual semantic score between each node and its neighbor nodes is calculated using formula (9);

[0076] (9)

[0077] in, For the i nodes and j Contextual semantic score between nodes; A linear transformation matrix; is the weight vector in the preset attention mechanism; LeakyReLU is the activation function; softmax is used to normalize the final result.

[0078] Then, the comprehensive score of the textual relationship between each node and its neighboring nodes is calculated using formula (10);

[0079] (10)

[0080] in, For the i nodes and j Comprehensive score of textual relations between nodes; , and are three preset coefficients, and .

[0081] Finally, if you judge >Preset scoring threshold , then retain the i nodes and j The connection edge between the nodes; on the contrary, if the judgment <Preset scoring threshold , then delete the i nodes and j The connecting edges between nodes are updated.

[0082] Step S4: determining the number of horizontal connecting edges and vertical connecting edges corresponding to each node in the adjacency graph structure, and updating the adjacency graph structure after deleting the smallest number of horizontal connecting edges or vertical connecting edges in each node;

[0083] The specific process is to clear the redundant connection edges in the adjacency graph structure. , retain its connection relationship in the most important direction of connection direction and delete the connection edges in other directions.

[0084] First, statistics and nodes All connected edges, including horizontal and vertical connecting edges;

[0085] Secondly, set and Node The number of horizontal connecting edges and the number of vertical connecting edges, select the direction with the largest number as the node The main connection direction of , delete all edges that do not belong to the main connection direction, that is, delete the node The smallest number of connected edges in the direction.

[0086] Step S5: Parse and organize the table semantic structure according to the connection relationship retained between nodes in the updated adjacency graph structure to generate a nested JSON file that conforms to the actual semantics.

[0087] The specific process is to identify the label node and its corresponding value node from the updated adjacency graph structure, nest the fields according to the spatial position and semantic relationship, generate a structured and semantically cleaned JSON object, and output the final JSON file. The specific process is implemented using conventional technical means in this field and will not be described in detail here.

[0088] Corresponding to the method for converting an irregular table structure into a JSON file provided in an embodiment of the present invention, an embodiment of the present invention also provides a device for converting an irregular table structure into a JSON file, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the method for converting an irregular table structure into a JSON file provided in an embodiment of the present invention. For details, please refer to the previous related content and will not be repeated here.

[0089] The implementation of the embodiments of the present invention has the following beneficial effects:

[0090] In view of the characteristics of irregularly divided tables, the present invention designs a method for converting irregular table structures into JSON files. By modeling the table as a graph structure and integrating spatial adjacency rules with a multi-index connection scoring mechanism, the semantic associations and structural hierarchies between fields in the table are effectively identified, and a JSON object with clear structure and accurate semantics is automatically generated. In this way, structural modeling, semantic discrimination and field organization of cells in the table can be achieved without relying on templates, and JSON objects with accurate semantics and clear structure can be output, thereby improving the accuracy of information extraction and the level of processing automation in complex table scenarios.

[0091] Those skilled in the art will understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc.

[0092] The above disclosure is merely a preferred embodiment of the present invention and certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A method for converting an irregular table structure into a JSON file, characterized in that: The method comprises the following steps: Obtaining an irregularly divided original table file, identifying each sub-cell and related information in the original table file, treating each sub-cell as an independent node, and further forming a node set carrying all sub-cell related information; wherein each node carries corresponding sub-cell related information, including spatial position information of the four corners of the sub-cell rectangular border, and text content contained within the sub-cell rectangular border and its corresponding type; Based on the spatial position information of the four corners of the rectangular border of the sub-cell corresponding to each node, each node is traversed to find neighboring nodes that satisfy a predetermined adjacency relationship to construct an adjacency graph structure; wherein the predetermined adjacency relationship is that the two nodes should simultaneously satisfy a direction validity condition, an overlap rate validity condition, and a distance validity condition; the direction validity condition is that the vertical center position interval between the two nodes is less than a first predetermined interval value, and the horizontal center position interval between the two nodes is less than a second predetermined interval value; the overlap rate validity condition is that the overlap rate values ​​between the two nodes in the horizontal direction and the vertical direction are both greater than a preset overlap rate threshold; the distance validity condition is that the Euclidean distance between the two nodes is within a predetermined distance range; Based on the text content carried by each node and its corresponding type, the text similarity score, text type matching score and context semantic score between each node and its neighboring nodes are calculated to further obtain the comprehensive score of the text relationship between each node and its neighboring nodes, and based on the comprehensive score of the text relationship between each node and its neighboring nodes, the edges in the adjacency graph structure that meet the predetermined semantic invalid association condition are deleted; wherein the predetermined semantic invalid association condition is that the comprehensive score of the text relationship between any node and its neighboring nodes is less than a preset score threshold; In the adjacency graph structure, determining the number of horizontal connecting edges and vertical connecting edges corresponding to each node, and updating the adjacency graph structure after deleting the smallest number of horizontal connecting edges or vertical connecting edges in each node; Based on the preserved connection relationships between nodes in the updated adjacency graph structure, the table semantic structure is parsed and organized to generate a nested JSON file that conforms to the actual semantics. The contextual semantic score between each node and its neighboring nodes is calculated using formula (9); （9） in, For the i nodes and j Contextual semantic score between nodes; and Respectively i nodes and j The text content of each node; is a preset linear transformation matrix; is the weight vector in the preset attention mechanism; LeakyReLU is the activation function; softmax is used to normalize the final result.

2. The method for converting an irregular table structure into a JSON file according to claim 1, wherein: The node set is ;in, To carry the i nodes with relevant information about child cells, and ; For the i The text content of the node, that is, i The characters or strings contained within the rectangular border of each child cell represent the semantic information of the node; For the i The spatial location information of nodes; For the i The coordinates of the upper left corner of the rectangular border of the sub-cell, The coordinates of the lower right corner; For the i The type of text content of each node, including text, number, time, and mixed.

3. The method for converting an irregular table structure into a JSON file according to claim 2, wherein: The direction validity conditions are expressed by formulas (1) and (2); （1） （2） in, and Respectively i nodes and j The vertical center of the nodes, and ; and Respectively i nodes and j The horizontal center of the nodes, and ; For the i nodes and j The horizontal center position interval between nodes; No. i nodes and j The vertical center position interval between nodes; is a first predetermined interval value; is the second predetermined interval value.

4. The method for converting an irregular table structure into a JSON file according to claim 3, wherein: The overlap ratio validity conditions are expressed by formulas (3) to (5); （3） （4） ； （5） in, For the i nodes and j The overlap ratio of nodes in the horizontal direction; No. i nodes and j The overlap ratio of nodes in the vertical direction; is the preset overlap rate threshold; For the i The cell height of nodes, and ; For the i The cell width of nodes, and ; For the j The cell height of nodes, and ; For the j The cell width of nodes, and .

5. The method for converting an irregular table structure into a JSON file according to claim 4, wherein: The distance validity condition is Within 1.5 of the average width and average height; No. i nodes and j The Euclidean distance between nodes is calculated by formula (6); （6）。 6. The method for converting an irregular table structure into a JSON file according to claim 5, wherein: The text similarity score between each node and its neighbor nodes is calculated using formula (7); （7） in, For the i nodes and j The text similarity score between nodes; and Respectively i nodes and j The text content of each node; is the norm calculation; The text type matching score between each node and its neighbor nodes is calculated using formula (8); （8） in, For the i nodes and j Text type matching score between nodes; and Respectively i nodes and j The type of text content in each node; It is a preset matching scoring matrix used to record matching scores.

7. The method for converting an irregular table structure into a JSON file according to claim 6, wherein: The comprehensive score of the textual relationship between each node and its neighboring nodes is calculated using formula (10); （10） in, For the i nodes and j Comprehensive score of textual relations between nodes; , and are three preset coefficients, and .

8. A device for converting an irregular table structure into a JSON file, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method for converting an irregular table structure into a JSON file according to any one of claims 1 to 7 are implemented.

Images