Chatbot Document Information Extraction via Segmented ML Modules
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional chatbot systems face difficulties in accurately extracting key information and tables from documents, especially in distinguishing important information from non-important and noise data.
Innovation Solution
A chatbot system equipped with a set of trained machine-learning models, including a key information extraction module, a table detection module, and a table extraction module, which are trained separately using different processes to extract key information and tables from documents.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional chatbot systems are used to extract information from documents, then the system structure remains simple, but the extraction accuracy and ability to distinguish important information from noise deteriorates
Solution Approach 1:
The system divides the information extraction task into multiple specialized modules: a key information extraction module for extracting important text, a table detection module for identifying tables, and a table extraction module for extracting table data. Each module is trained separately on specific tasks, allowing high precision for each function while managing overall system complexity through modular architecture.
Solution Approach 2:
The system employs multiple machine learning models with different parameters and architectures optimized for specific extraction tasks. By changing model parameters and selecting different model types for different extraction objectives, the system achieves high extraction accuracy for various information types without requiring a single overly complex model.
2Measurement precision
If multiple trained machine-learning models are used to extract key information and tables, then the extraction precision improves, but the model training and system implementation complexity increases
Solution Approach 1:
The training process is segmented into separate tasks for each module. The key information extraction module is trained independently on key field extraction, the table detection module is trained separately on table identification, and the table extraction module is trained independently on table data extraction. This segmentation allows each model to be optimized for its specific task without requiring complex joint training.
Solution Approach 2:
The system performs preliminary actions by training each machine learning model separately on its specific task before deployment. This preliminary training of individual modules simplifies the overall implementation process compared to training a single comprehensive model, as each module can be developed, tested, and optimized independently.
3Reliability
If separate training processes are used for different extraction modules, then the extraction reliability improves, but the training time and processing duration increases
Solution Approach 1:
The training process is segmented into parallel independent tasks for each extraction module. This allows different modules to be trained simultaneously on different hardware resources, reducing the total training duration while maintaining the reliability benefits of specialized training for each module.
Data Source
AI summary
Techniques for extracting key information from a document using machine-learning models in a chatbot system is disclosed herein. In one particular aspect, a method is provided that includes receiving a set of data, which includes key fields, within a document at a data processing system that includes a table detection module, a key information extraction module, and a table extraction module. Text information and corresponding location data are extracted via optical character recognition. The table detection module detects whether one or more tables are present in the document and, if applicable, a location of each of the tables. The key information extraction module extracts text from the key fields. The table extraction module extracts each of the tables based on input from the optical character recognition and the table detection module. Extraction results include the text from the key fields and each of the tables can be output.