Pseudo-Labeling for Key-Value Extraction from Visually Rich Documents
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training machine learning models for key-value extraction from visually rich documents is hindered by the need for large amounts of labeled data, which is time-consuming and costly to produce, especially due to the diversity of document formats.
Innovation Solution
A pseudo-labeling approach that generates model pseudo-labels and graph pseudo-labels using a pretrained model and graph matching, allowing for the creation of updated graphs that can be used to train a machine learning model for key-value extraction without extensive human annotation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If human annotation is used to create labeled documents, then the quality and accuracy of training data is improved, but the time consumption and cost increase significantly
Solution Approach 1:
The system performs preliminary actions by using a pre-trained model to generate pseudo-labels for training data before actual model training begins. This preliminary labeling step creates an initial set of labeled documents that can be used to train the model without requiring extensive human annotation, thereby reducing time consumption while maintaining acceptable labeling quality
Solution Approach 2:
The system implements self-service by enabling the model to generate its own training data through pseudo-labeling. The pre-trained model labels unlabeled documents, creating training examples that are then used to fine-tune and improve the model itself, forming a self-reinforcing cycle that reduces dependency on human annotators
2Adaptability or versatility
If more labeled data is collected to improve model generalization, then the model performance on diverse documents is improved, but the complexity and cost of data preparation increases
Solution Approach 1:
The system uses copying by generating synthetic training examples through pseudo-labeling. Instead of manually creating diverse labeled documents from scratch, the system copies and adapts patterns from the pre-trained model to generate varied training data that maintains structural consistency while covering different document types and formats
Solution Approach 2:
The system implements universality by creating a multi-functional pipeline that handles multiple document types (invoices, receipts, forms, etc.) through a single pseudo-labeling framework. The graph-based representation and matching algorithms work universally across different document formats, eliminating the need for separate data preparation processes for each document type
3Manufacturing precision
If graph matching is used to generate pseudo-labels, then the structural information is better preserved, but the computational complexity increases
Solution Approach 1:
The system applies segmentation by dividing the graph matching process into distinct components: node feature extraction, edge relationship identification, and label assignment. This segmentation allows each component to be optimized independently, reducing overall computational complexity while maintaining structural accuracy in the generated pseudo-labels
Data Source
AI summary
A computing device may access visually rich documents comprising an image and metadata. A graph, based on the image or metadata, can be generated for a visually rich document. The graph's nodes can correspond to words from the visually rich document. Features for nodes can be determined by the device. The device may generate model labeled graphs by assigning a pseudo-label to nodes using a pretrained model. The device may generate a plurality of graph labeled graphs by assigning a pseudo-label to nodes by matching a first node from a first graph to at least a second node from a second graph. The device may generate a plurality of updated graphs by cross referencing labels from the model labeled graphs and the graph labeled graphs. Until a change in labels is below a threshold, a model can be trained to perform key-value extraction using the updated graphs.