Frequent Logic Rule Sets for ML Document Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current information extraction methods from unstructured documents face challenges due to layout variability, content variability, and incompleteness, leading to imperfect accuracy and the need for human intervention, particularly in identifying cell-items from tables with varying formats and terminologies.
Innovation Solution
The integration of logic rules with machine learning models for frequent itemset mining to determine frequent rule sets over word-items, allowing for semi-automated labeling and identifying areas requiring human revision by assessing which rules hold for cell-item predictions in new documents.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If machine learning models are used for cell-item extraction, then extraction speed and automation are improved, but accuracy and reliability deteriorate due to content variability and layout differences
Solution Approach 1:
The patent introduces logic rules as an intermediary layer between the machine learning model and the final extraction results. The ML model generates initial predictions, which are then refined by applying logic rules that encode domain knowledge about document structures and relationships. This intermediary step corrects errors and improves reliability without sacrificing the automated speed of the ML approach.
Solution Approach 2:
The patent merges two different approaches—machine learning and logic rule-based methods—into a hybrid system. The ML component handles speed and initial pattern recognition, while the logic rule component handles accuracy and constraint satisfaction. By combining these complementary approaches, the system achieves both high productivity and high reliability.
2Reliability
If rule-based approaches are used for document extraction, then accuracy for specific document types is improved, but adaptability to different layouts and organizations deteriorates
Solution Approach 1:
The patent makes the rule system dynamic by allowing logic rules to be generated and adapted based on the specific document being processed. Rather than using a fixed set of rules for all documents, the system can generate relevant logic rules on-the-fly based on the document's structure, organization, and content characteristics. This dynamic adaptation maintains high accuracy while achieving versatility across different document types.
Solution Approach 2:
The patent creates a universal framework that can handle multiple document types and layouts through a common logic rule mechanism. The same underlying system—combining ML predictions with logic rule refinement—works across different organizations, document formats, and table structures. The logic rules themselves are designed to be generalizable patterns that can adapt to various specific cases.
3Reliability
If more comprehensive rules are applied to cover all document variations, then extraction accuracy is improved, but system complexity and computational cost increase
Solution Approach 1:
The patent applies partial action by selectively generating and applying only the logic rules that are relevant to the current document and ML predictions, rather than exhaustively applying all possible rules. The system identifies which rules are necessary to correct the specific errors present in the ML output and applies only those, reducing computational overhead while maintaining accuracy.
Solution Approach 2:
The patent performs preliminary action by pre-defining a library of common logic rules that capture frequent document patterns and relationships. These pre-established rules can be quickly applied without requiring complex real-time reasoning, reducing system complexity while maintaining high accuracy for common document types.
Data Source
AI summary
One example method includes receiving a rule-set, including a combination of rules, that was determined to occur in a set of ground truth documents, applying the rule-set to a new document that was not included in the set of ground truth documents, determining whether or not a rule in the rule-set succeeded or failed when applied to a word in the new document, and when the rule is determined to have failed, identifying the failed rule, identifying a confidence level in the determination that the rule failed, and when the confidence level is below a threshold confidence level, identifying the word, to which the failed rule was applied, as a candidate for verification by a human.