Table Data to Machine Readable Text via Graph Knowledge Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing document question-answering (QA) systems face difficulties in efficiently extracting information from tables due to the need for correlation with multiple parts of information, which traditional methods, including deep learning, struggle to handle effectively, especially in scenarios with varied table layouts and structures.
Innovation Solution
A method that converts table information into a graph structure using joint representations of bounding box position, semantic, and image information, allowing for layout association analysis and reorganization, and utilizes a neural network to determine relative relationships between nodes, ultimately generating continuous natural language for accurate QA.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional deep learning methods are used to extract information from tables, then the system can handle some table extraction tasks, but it fails to effectively correlate multiple parts of information and handle varied table layouts and structures
Solution Approach 1:
The patent segments the table extraction process into distinct components: converting tabular data to images, detecting text areas and their positions, extracting semantic information, and reconstructing the knowledge graph. This segmentation allows each component to be optimized independently, improving both adaptability to different layouts and reliability of information extraction.
Solution Approach 2:
The patent transforms tabular data into a different dimensional representation by converting it to images with bounding boxes, then to graph structures with nodes and edges. This dimensional transformation enables the system to capture spatial relationships and semantic connections that traditional methods miss, thereby handling varied layouts more effectively while maintaining extraction accuracy.
2Measurement precision
If tabular data is converted to images with bounding boxes and reconstructed into graph form, then the system can accurately represent spatial and semantic relationships, but the processing complexity and computational requirements increase
Solution Approach 1:
The patent introduces images with bounding boxes as an intermediary representation between the original tabular data and the final graph structure. This intermediary captures spatial relationships effectively while providing a standardized format that simplifies subsequent processing steps, reducing overall system complexity despite the additional transformation step.
Solution Approach 2:
The patent creates a copy of the tabular data in image format with annotated bounding boxes, preserving the original data while enabling spatial relationship detection. This copying approach allows the system to analyze spatial relationships without modifying the original data structure, simplifying the processing pipeline.
3Reliability
If a neural network is used to determine relative relationships between nodes in the graph, then the accuracy of relationship detection improves, but the computational time and resources increase
Solution Approach 1:
The patent performs preliminary processing by converting tabular data to images, detecting text areas, and extracting semantic information before feeding data to the neural network. This preliminary action prepares the data in an optimized format, reducing the computational burden on the neural network and decreasing overall processing time while maintaining high relationship detection accuracy.
Solution Approach 2:
The patent extracts only the essential features (text area positions, bounding box coordinates, semantic information) from the tabular data before inputting them to the neural network. This extraction of relevant features reduces the dimensionality and complexity of the input data, allowing the neural network to operate more efficiently with reduced computational time.
Data Source
AI summary
A system and method for table conversion including converting a table containing text in tabular form to an image, labeling each text area of the image with a bounding box, determining for each bounding box, a position information, a semantic information, and an image information, reconstructing the image into a graph form having a plurality of nodes, wherein each node represents the bounding box of the text areas of the image, inputting at least two nodes into a trained neural network to determine a relative relationship between the at least two nodes, building a knowledge graph using the relative relationship of the at least two nodes, and translating the knowledge graph into machine readable natural language.