Specialized NLP Engines for Form Data Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional natural language processing (NLP) techniques struggle with accurately extracting features from medical records containing forms, often resulting in false positives due to misinterpretation of form data, especially when forms with lists of symptoms or diseases are processed alongside text data.
Innovation Solution
The method involves using different processing engines trained for specific types of data within a document, such as form data and sentence or paragraph data, to extract features more accurately, and combining these features to improve the accuracy of information extraction from healthcare documents.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional NLP techniques are used to process all document content uniformly, then the system is simple to operate, but accuracy deteriorates due to false positives from form data misinterpretation
Solution Approach 1:
The patent divides the document into different types (form documents vs. non-form documents) and processes each type with a specialized NLP model. Form documents are processed by a first NLP model trained to ignore form-specific patterns, while non-form documents are processed by a second NLP model trained for standard text extraction, thereby improving accuracy without requiring a single overly complex system
Solution Approach 2:
The patent applies different processing qualities to different parts of the input based on document type. The first NLP model is specifically trained with form data to recognize and exclude form patterns locally, while the second model maintains standard extraction capabilities for regular text, allowing each model to have optimized local expertise for its specific document type
2Reliability
If a single NLP model processes all document types, then the system has low device complexity, but reliability deteriorates due to inconsistent form data interpretation
Solution Approach 1:
The system segments the processing task by creating two distinct NLP models: a first model trained on form data to reliably identify and exclude form patterns, and a second model trained on standard text for reliable feature extraction from non-form documents. This segmentation ensures each model is reliable for its specific domain
Solution Approach 2:
The patent introduces an intermediary classification step that determines whether a document is a form document or non-form document before routing to the appropriate NLP model. This intermediary classification mechanism ensures that the correct specialized model is used, maintaining high reliability across different document types
3Measurement precision
If conventional NLP processes form lists as valid findings, then processing is straightforward, but measurement precision worsens due to false positives from symptom and disease lists
Solution Approach 1:
The first NLP model performs preliminary action by being pre-trained on form data to recognize form patterns before actual feature extraction occurs. This preliminary training enables the model to identify and exclude form-specific patterns (such as symptom lists and disease lists) before they can be misinterpreted as valid findings, thereby reducing false positives
Data Source
AI summary
Aspects include receiving a document and classifying at least a subset of the document as having a first type of data. Features are extracted from the document. The extracting includes initiating processing of the at least a subset of the document by a first processing engine that was previously trained to extract features from the first type of data. The extracting also includes initiating processing of a remaining portion of the document not included in the at least a subset of the document by a second processing engine that was previously trained to extract features from a second type of data. The first type of data is different than the second type of data. Features are received from one or both of the first processing engine and the second processing engine. The received features are stored as features of the document.