ML Model Training via Source-Specific Clustering for Entity Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI/ML systems face challenges in accurately extracting named entities from diverse documents, such as invoices, due to variations in format and structure, leading to errors and inefficiencies in information processing.
Innovation Solution
An iterative, lightly supervised training process is employed to train AI/ML systems, where a small manually labeled dataset is used to generate vendor-specific or universal ML models for named entity recognition, leveraging clustering techniques and confidence scoring to improve prediction accuracy over iterations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional AI/ML systems are used to extract named entities from diverse documents, then the system can process documents automatically, but the extraction accuracy deteriorates due to format and structure variations across different document sources
Solution Approach 1:
The patent segments the document corpus by source (e.g., vendor-specific clustering) and trains separate ML models for each segment. This allows each model to specialize in the particular format and structure characteristics of its source, thereby maintaining high extraction accuracy across diverse document types while preserving automatic processing capability
Solution Approach 2:
The patent changes the parameter of model specialization by training vendor-specific or source-specific ML models with parameters optimized for each document source's unique characteristics. This enables the system to adapt to format variations without sacrificing automation, resolving the contradiction between productivity and precision
2Measurement precision
If a large manually labeled dataset is used to train AI/ML systems for named entity recognition, then extraction accuracy improves, but the training time and computational resources increase significantly
Solution Approach 1:
The patent performs preliminary clustering of documents by source before training, organizing the data structure in advance. This preliminary action enables efficient subsequent training by grouping similar documents together, reducing the overall training time required to achieve high accuracy compared to training on the entire diverse corpus simultaneously
Solution Approach 2:
By segmenting the training process into source-specific models trained on smaller, homogeneous subsets of data, the patent reduces the computational burden and training time for each individual model while collectively achieving high accuracy across all document sources
3Measurement precision
If vendor-specific ML models are trained for each document source, then named entity extraction accuracy for that source improves, but the system complexity increases due to multiple models to maintain
Solution Approach 1:
The patent implements a universal framework that can handle multiple document sources through a standardized interface and process. While multiple source-specific models are trained, they are managed within a single multi-functional system that provides consistent extraction capabilities across all vendors, reducing the operational complexity despite the number of specialized models
4Reliability
If confidence scoring is used to filter machine-labeled entities, then extraction reliability improves, but the processing time increases due to iterative validation
Solution Approach 1:
The patent implements a self-service mechanism where the ML model automatically generates confidence scores for its own predictions and uses these scores to filter results. This self-validation process improves reliability by enabling the system to identify and flag low-confidence extractions without requiring external verification, while the automated nature minimizes additional processing time
Data Source
AI summary
A method and system are provided for training a machine-learning (ML) system/module and to provide an ML model. In one embodiment, a method includes using a labeled entities set to train a machine learning (ML) system, to obtain an ML model, and using the trained ML model to predict labels for entities in an unlabeled entities set, yielding a machine-labeled entities set. One or more individual ML models may be trained and used in this way, where each individual ML model corresponds to a respective document source. The document sources can be identified via classification of a corpus of documents. The prediction of labels provides a respective confidence score for each machine-labeled entity. The method also includes selecting from the machine-labeled entities set, a subset of machine-labeled entities having a respective confidence score at least equal to a threshold confidence score; and updating the labeled entities set by adding thereto the selected subset of machine-labeled entities. The method further includes removing from the machine-labeled entities set the selected subset of machine-labeled entities and deleting labels assigned to the entities in the updated machine-labeled entities set to provide the unlabeled entities set for a next iteration. The method also includes, if a termination condition is not reached, repeating the steps above and, otherwise, storing the ML model.