Cross-Modal Document Reading Model Training with OCR Layout Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing natural language processing (NLP) technologies for document reading comprehension require separate information extraction solutions for documents from different sources, which is time-consuming and labor-intensive.
Innovation Solution
A method for training a document reading comprehension model using cross-modal information, including acquiring question and document samples, performing optical character recognition (OCR) to extract text and layout information, and training the model based on predicted and real answers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If separate information extraction solutions are designed for documents from different sources, then extraction accuracy is improved, but development time and labor costs increase
Solution Approach 1:
The patent applies universality by designing a unified information extraction framework that can process multiple document types (PDF, Word, Excel, HTML, Markdown) through a single interface. The system automatically detects document formats and routes them to appropriate processing pipelines, eliminating the need to develop separate extraction solutions for each document source while maintaining high extraction accuracy through format-specific processing modules within the unified framework.
Solution Approach 2:
The patent introduces an intermediary component - a document format detection and routing module - that sits between the input interface and processing pipelines. This intermediary automatically identifies the document format and directs it to the appropriate extraction module, enabling a single entry point to handle multiple document types without requiring separate user-facing solutions for each format.
2Measurement precision
If separate information extraction solutions are designed for documents from different sources, then extraction accuracy is improved, but system complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the information extraction system into independent, modular processing pipelines for different document formats (PDF processor, Word processor, Excel processor, etc.). Each module handles a specific format independently, and the results are aggregated by a unified post-processing module. This segmentation allows the system to maintain high accuracy for each format while managing complexity through modularity and clear separation of concerns.
3Adaptability or versatility
If multiple extraction solutions are maintained for different document sources, then comprehensive coverage is improved, but maintenance effort increases
Solution Approach 1:
The patent implements universality through a standardized interface layer that all document processing modules must adhere to. This unified interface defines consistent methods for document loading, extraction, and result formatting. When a new document format needs to be supported, developers only need to implement the format-specific processing logic while automatically conforming to the standard interface, reducing maintenance effort and enabling easier addition of new formats.
Data Source
AI summary
A method for training a document reading comprehension model includes: acquiring a question sample and a rich-text document sample, in which the rich-text document sample includes a real answer of the question sample; acquiring text information and layout information of the rich-text document sample by performing OCR processing on image information of the rich-text document sample; acquiring a predicted answer of the question sample by inputting the text information, the layout information and the image information of the rich-text document sample into a preset reading comprehension model; and training the reading comprehension model based on the real answer and the predicted answer. The method may enhance comprehension ability of the reading comprehension model to the long rich-text document, and save labor cost.