Document Parsing With VLLM Auto-Labeling and eForm Structure
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The process of creating and updating document parsing models in machine learning is neither quick nor easy, requiring significant human intervention and coding expertise.
Innovation Solution
The use of Visual Large Language models and eForms to generate structural representations of documents, auto-label documents, and train AI models, with the option for human review and correction, simplifying the labeling process and enabling the use of multi-modal transformers for improved performance and fidelity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual labeling of documents is performed to train machine learning models, then the accuracy and fidelity of parsing results can be achieved, but the process requires significant human intervention and time
Solution Approach 1:
The system performs preliminary actions by using VLLM to automatically generate structural representations, field labels, and values from documents before the actual model training process. This preliminary automated labeling reduces the manual intervention needed later while maintaining high parsing accuracy through subsequent human review and correction of the generated labels.
2Measurement precision
If manual labeling and model updating processes are performed to achieve desired fidelity, then parsing accuracy is improved, but the process requires coding expertise and is not easy to perform
Solution Approach 1:
The system enables self-service by allowing business users to perform model creation and updating without requiring coding expertise. The VLLM automatically generates structural representations and labels, and the platform provides intuitive interfaces for reviewing and correcting these generated labels, enabling non-technical users to maintain high parsing accuracy through simple drag-and-drop or click-based operations.
3Productivity
If Visual Large Language models are used to generate structural representations and auto-label documents, then the speed and ease of model creation is improved, but the process requires new approaches different from traditional methods
Solution Approach 1:
The system introduces an intermediary layer between the VLLM and the final parsed output. The VLLM generates structural representations and field labels, which then pass through an intermediary review and correction stage where users can adjust the labels, before being used to train the final parsing model. This intermediary layer manages the complexity by providing a controlled interface between automated generation and final output.
4Reliability
If traditional manual labeling methods are used, then the process is well-understood and reliable, but it is neither quick nor easy to perform
Solution Approach 1:
The system performs preliminary automated labeling using VLLM to generate structural representations, field labels, and values before the traditional model training process. This preliminary action maintains reliability by allowing human review and correction of the generated labels, while simultaneously improving productivity by eliminating the need for entirely manual labeling from scratch.
Data Source
AI summary
Document parsers, document parsing methods, and products are provided that use Visual Large Language models and/or eForms to generate structural representations to train artificial intelligence used in intelligent document processing. These structural representations are enhanced with the Visual Large Language models with geometry data from the documents and the results are correlated with a training sample. The data set is then curated for errors and omissions and reintegrated into the initial structure of the form. Auto-generated synthetic documents can be used in certain embodiments. Standardized outputs such as eForms and from an Electronic Document Interchange can be used in certain embodiments to enhance efficiency and synchronization of intelligent document processing. A multi-modal transformer-based machine learning model is built that can then be used to create an output in intelligent document processing.