Out-of-Distribution Element Detection in Information Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models for information extraction from unstructured documents are not accurate in distinguishing between in-distribution and out-of-distribution elements, leading to high false-positive rates, especially when dealing with noise content and elements with similar properties to in-distribution elements.
Innovation Solution
A novel training technique that combines a regular training dataset with an augmented training dataset to train machine learning models to classify elements as either in-distribution or out-of-distribution, using visual cues and attributes such as location, color, font, and context, to improve classification accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing ML models are trained to classify elements from unstructured documents, then information extraction can be performed, but the models misclassify out-of-distribution elements as in-distribution elements leading to high false-positive rates
Solution Approach 1:
The patent segments the classification task into two distinct parts: (1) determining whether an element is in-distribution or out-of-distribution, and (2) classifying in-distribution elements into specific categories. This is achieved by training separate ML models - one for distribution detection and another for category classification - thereby preventing out-of-distribution elements from being misclassified and reducing false positives
Solution Approach 2:
The patent introduces an intermediary classification step that acts as a gatekeeper between element extraction and final categorization. The first ML model serves as an intermediary that filters elements by determining their distribution status before they reach the second classification model, thus preventing out-of-distribution elements from contaminating the final classification results
2Device complexity
If ML models classify all extracted elements as in-distribution elements, then processing is simplified, but noise content and out-of-distribution elements are misclassified leading to degraded user experience
Solution Approach 1:
The patent divides the extraction system into two specialized models: a distribution detection model that identifies whether elements belong to the training distribution, and a category classification model that categorizes only in-distribution elements. This segmentation maintains reasonable complexity while dramatically improving reliability by preventing misclassification of out-of-distribution and noise elements
3Ease of manufacture
If traditional OCR techniques are used for information extraction, then the process is straightforward, but accuracy is insufficient when dealing with varied unstructured document layouts
Solution Approach 1:
The patent replaces traditional mechanical OCR techniques with machine learning-based classification systems. Instead of relying on fixed pattern matching and rule-based extraction, the system uses trained ML models that can adapt to varied document layouts, fonts, and formats, thereby maintaining ease of implementation while dramatically improving extraction accuracy
Data Source
AI summary
Techniques for extracting information from unstructured documents that enable an ML model to be trained such that the model can accurately distinguish in-distribution (“in-D”) elements and out-of-distribution (“OO-D”) elements within an unstructured document. Novel training techniques are used that train an ML model using a combination of a regular training dataset and an enhanced augmented training dataset. The regular training dataset is used to train an ML model to identify in-D elements, i.e., to classify an element extracted from a document as belonging to one of the in-D classes contained in the regular training dataset. The augmented training dataset, which is generated based upon the regular training dataset may contain one or more augmented elements which are used to train the model to identify OO-D elements, i.e., to classify an augmented element extracted from a document as belonging to an OO-D class instead of to an in-D class.