Neural Network Token Relationship Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in reliably and efficiently extracting structured content from documents with varying formats, requiring significant manual effort or customized automation due to limitations in recognizing complex document structures and relationships.
Innovation Solution
The use of neural networks trained on text documents to evaluate pairwise relationships between tokens, allowing for the extraction of structured content by mapping tokens to database columns and transforming unstructured content into a structured representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional technologies are used to parse document content, then simple tasks such as parsing grammar of a single sentence or recognizing fields of a single fixed document format can be addressed, but documents of a single class can vary considerably in format and require significant manual effort or customized automation
Solution Approach 1:
The patent applies a universal neural network model that can process multiple document types and formats through a single unified architecture. The model learns general patterns from training data and automatically adapts to handle diverse document structures without requiring separate customized automation systems for each document type, thereby achieving high adaptability while maintaining manageable system complexity.
Solution Approach 2:
The system dynamically adjusts its processing parameters and relationship evaluation thresholds based on the input document characteristics. The neural network modifies its internal parameters during inference to adapt to different document formats, allowing the same system to effectively handle varied document structures without manual reconfiguration or extensive customization.
2Reliability
If conventional technologies are used to extract structured content, then simple document structures can be processed, but complex document structures with multiple levels and varied formats cannot be reliably extracted
Solution Approach 1:
The patent segments the document structure extraction into multiple hierarchical levels, where the neural network evaluates relationships at different depths and granularities. This segmentation allows the system to handle complex multi-level document structures by processing them in manageable layers, improving reliability for complex documents while maintaining systematic approachability.
Solution Approach 2:
The system transitions from traditional two-dimensional document parsing to a multi-dimensional relationship evaluation framework. By adding the dimension of hierarchical levels and using neural networks to evaluate pairwise relationships across all tokens, the system can reliably extract structured content from complex multi-level documents that would be intractable for conventional two-dimensional parsing methods.
3Measurement precision
If customized automation is created for each different document style, then specific document formats can be processed accurately, but the manual effort and system complexity increase significantly
Solution Approach 1:
The system performs preliminary training on a corpus of documents to learn general structural patterns and relationships before processing new documents. This preliminary action enables the neural network to achieve high extraction precision across different document styles without requiring time-consuming manual setup or customization for each specific document type, as the model is pre-configured through training data.
Data Source
AI summary
Methods and apparatus are disclosed for extracting structured content, as graphs, from text documents. Graph vertices and edges correspond to document tokens and pairwise relationships between tokens. Undirected peer relationships and directed relationships (e.g. key-value or composition) are supported. Vertices can be identified with predefined fields, and thence mapped to database columns for automated storage of document content in a database. A trained neural network classifier determines relationship classifications for all pairwise combinations of input tokens. The relationship classification can differentiate multiple relationship types. A multi-level classifier extracts multi-level graph structure from a document. Disclosed embodiments support arbitrary graph structures with hierarchical and planar relationships. Relationships are not restricted by spatial proximity or document layout. Composite tokens can be identified interspersed with other content. A single token can belong to multiple higher level structures according to its various relationships. Examples and variations are disclosed.