Textual and Numerical Document Retrieval with Segmented Transformer Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models are inaccurate in processing documents with high numerical content, unstructured formats, and consume excessive computational resources, leading to inefficiencies in predicting the meaning of numbers and words.
Innovation Solution
Employing modified transformer models, such as BERT, to process documents in parallel, perform object detection, and use knowledge distillation to reduce model size, thereby improving accuracy and reducing resource consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing machine learning models are used to process documents with high numerical content, then the models can handle the processing task, but the accuracy in determining meaning is poor
Solution Approach 1:
The patent segments the document processing task into two distinct phases: first, an object detection model identifies and classifies numerical elements and their meanings (e.g., invoice number, total amount, date); second, an NLP model processes the textual content. This segmentation allows each model to specialize in its strengths, improving overall accuracy for numerical documents
Solution Approach 2:
The patent introduces an intermediary object detection model that acts as a bridge between raw document input and NLP processing. This intermediary model extracts and structures numerical information before it reaches the NLP model, enabling more accurate meaning determination by providing pre-processed, context-aware numerical data
2Productivity
If existing machine learning models process documents, then processing can be performed, but computational resources are excessively consumed
Solution Approach 1:
The patent divides the document processing workload into two separate models: an object detection model for numerical elements and an NLP model for text. This segmentation allows parallel processing and prevents the computationally intensive NLP model from processing every numerical element, thereby reducing overall computational resource consumption while maintaining productivity
Solution Approach 2:
The patent applies partial action by using the object detection model to identify and extract only the relevant numerical elements that require further NLP processing. Not all numerical content undergoes full NLP analysis, which reduces computational overhead while maintaining processing efficiency for the most important elements
3Speed
If existing machine learning models are used, then processing can occur, but latency is high
Solution Approach 1:
The patent segments processing into two parallel tracks: object detection for numerical elements and NLP for text. The object detection model quickly identifies and classifies numerical elements with low latency, while NLP processes text independently. This parallel segmentation reduces overall processing time and latency compared to sequential processing
4Measurement precision
If existing machine learning models process unstructured documents, then processing can be performed, but accuracy is poor
Solution Approach 1:
The patent introduces an object detection model as an intermediary that structures unstructured numerical content into organized data elements with identified meanings. This intermediary layer transforms unstructured numerical text into structured data that NLP models can process accurately, thereby improving accuracy while maintaining adaptability to unstructured formats
Data Source
AI summary
The accuracy of existing machine learning models, software technologies, and computers are improved by using one or more machine learning models to predict a type of data that one or more numerical characters and/or one or more natural language word characters of a document correspond to. For instance, a Question Answering systems can be used to predict that a particular number value corresponds to a date, a billing amount, a page number, or the like.