Page Relevance Scoring for Tabular Content Identification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in accurately identifying and extracting domain-based tabular content from documents using machine learning and deep learning techniques.
Innovation Solution
A methodology utilizing a combination of term, embeddings, summary, and neighborhood models to generate a combined score for page relevance, leveraging frequency, positioning, text, image, and layout features to identify tabular content within documents.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple models (term, embeddings, summary, neighborhood) are combined to improve identification accuracy, then the accuracy of identifying tabular content is improved, but the device complexity increases
Solution Approach 1:
The identification system is divided into four independent models (term model, embeddings model, summary model, neighborhood model), each responsible for a specific aspect of tabular content identification. This segmentation allows each model to be optimized independently while contributing to the overall accuracy through their combined scores.
Solution Approach 2:
The patent combines the outputs of four separate models into a unified identification result by integrating their scores. The term model score, embeddings model score, summary model score, and neighborhood model score are merged to produce the final identification accuracy, leveraging the strengths of each individual model.
2Measurement precision
If multiple features (frequency, positioning, text, image, layout) are extracted and processed, then the measurement precision is improved, but the loss of time increases
Solution Approach 1:
Frequency features, positioning features, text embeddings, image embeddings, and layout embeddings are extracted and processed in advance before the final relevance scoring. This preliminary extraction allows the models to work with pre-processed data, reducing the time required during the actual identification process.
Solution Approach 2:
The feature extraction process operates continuously across all pages and models, with each model receiving and processing the extracted features without interruption. This continuous processing ensures that all features are utilized efficiently to maintain high measurement precision while minimizing delays.
Data Source
AI summary
A method computes page relevance for tabular contents from a document. The method includes receiving a table type and executing a term model to generate a term score of the page. The method further includes executing an embeddings model to generate an embeddings score of the page. The method further includes executing a summary model using a page summary generated with a language model to generate a summary score of the page. The method further includes executing a neighborhood model using a set of pages from the document to generate a neighborhood score of the page. The method further includes executing a combination model using the term score, the embeddings score, the summary score, and the neighborhood score to generate a combined score of the page. The method further includes presenting the combined score to indicate the page includes the table of the table type.