Graph Neural Network Text Extraction with Limited Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current information extraction methods are limited by their reliance on complex hand-crafted rules or transformer-based neural networks, which require large amounts of training data and computational resources, and are not customizable by users.
Innovation Solution
A machine learning text extraction method using graph neural networks that allows users to define sets of keys of interest, enabling the model to learn key-value associations from limited training data and adapt to each user's specific requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If transformer-based neural networks are used for information extraction, then extraction accuracy can be improved, but the amount of training data and computational resources required increases significantly
Solution Approach 1:
The patent segments the information extraction task into two distinct stages: (1) a lightweight rule-based component that performs initial key-value extraction with basic pattern matching, and (2) a transformer-based neural network component that refines the extraction for complex cases. This segmentation allows the system to achieve high accuracy only when needed, rather than applying the resource-intensive transformer model to all documents, thus reducing overall training data requirements while maintaining extraction precision.
Solution Approach 2:
The patent applies partial action by using the transformer-based neural network selectively rather than universally. The system first attempts extraction using simpler, less resource-intensive methods, and only invokes the transformer model when the initial extraction is insufficient or when dealing with particularly complex document structures. This partial application of the powerful but data-hungry model achieves high accuracy for difficult cases without requiring the model to be trained on vast amounts of data for all possible scenarios.
2Measurement precision
If transformer-based neural networks are used for information extraction, then extraction accuracy can be improved, but computational resources and processing time increase
Solution Approach 1:
The patent divides the processing workload into two segments: a lightweight rule-based extraction layer that handles straightforward cases with minimal computational resources, and a transformer-based neural network layer that processes only the subset of documents requiring more sophisticated analysis. This segmentation ensures that high computational power is consumed only when necessary, reducing overall resource consumption while maintaining high accuracy for complex extractions.
Solution Approach 2:
The system applies the computationally intensive transformer model partially, only when the simpler extraction methods fail to achieve sufficient accuracy or when document complexity thresholds are exceeded. This selective application of excessive computational power achieves high extraction accuracy for difficult cases without incurring the full computational cost of applying the transformer model universally to all documents.
3Power
If hand-crafted rules are used for information extraction, then computational resources are reduced, but the system cannot adapt to user-specific requirements
Solution Approach 1:
The patent implements a dynamic, multi-layered extraction system where the rule-based component provides a static, resource-efficient foundation, while the transformer-based neural network component introduces adaptability through learned patterns. The system dynamically selects which component to apply based on document characteristics and extraction requirements, enabling user-specific customization without permanently increasing computational overhead for all operations. Users can configure extraction parameters and the system adapts its resource usage accordingly.
Solution Approach 2:
The patent creates a universal extraction framework that combines multiple approaches (rule-based and neural network-based) into a single system capable of handling diverse extraction scenarios. The rule-based component handles common, straightforward cases efficiently, while the transformer-based component handles complex, user-specific cases. This multi-functional design allows the system to adapt to different user requirements and document types without requiring separate systems for each scenario, achieving versatility without proportional increases in computational resource requirements.
Data Source
AI summary
Training and using a machine learning model for data extraction is provided. The method comprises receiving keys of interest received from a user through an interface and receiving a batch of documents containing unstructured text. Unstructured text of a first document is processed to extract structured text. The model predicts text classifications of the structured text according to the keys of interest. The predicted text classifications are output to the user through the interface. Annotations to correct any incorrect predictions are received from the user, and the model is retrained according to the annotations. The above steps are repeated for less than ten additional documents from the batch until the model has been trained to predict text classifications with a specified level of accuracy. The trained model then classifies extracted structured text in the remaining documents in the batch.