S-BERT Document Segmentation for Efficient Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing document classification systems using Bidirectional Encoder Representational Transformers (BERT) require significant computational resources and time for training, and they do not efficiently process and categorize non-training documents into original configurations.
Innovation Solution
A method involving a pretrained Sentence Bidirectional Encoder Representational Transformers (S-BERT) model that scans and character recognizes training documents, processes them to generate a trained S-BERT model capable of automatically categorizing and assembling non-training documents into original configurations, using a system with multiple scanners and a template OCR engine for content segmentation and encoding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a pretrained S-BERT model is used to process training documents, then the computational burden is reduced and training efficiency is improved, but the model requires segmentation of page stream documents to function effectively
Solution Approach 1:
The patent applies segmentation by dividing page stream documents into individual page images, then further segmenting each page into content regions (text, images, tables) and eventually into discrete text lines or blocks. This hierarchical segmentation enables the S-BERT model to process documents in manageable units, resolving the contradiction between improved training efficiency and the complexity of document segmentation by making the segmentation process structured and systematic.
Solution Approach 2:
The patent implements preliminary action through preprocessing steps performed before S-BERT processing: scanning documents to create digital images, applying OCR to extract text, segmenting pages into content regions, and organizing text into lines. These preliminary actions prepare the document data in advance, allowing the S-BERT model to focus solely on classification without dealing with raw document structure, thus improving efficiency while managing segmentation complexity.
2Measurement precision
If a BERT model is used for document classification, then accurate categorization is achieved, but significant computational resources and training time are required
Solution Approach 1:
The patent applies preliminary action by performing document segmentation, OCR text extraction, and text line organization before feeding data to the S-BERT model. This preprocessing prepares the input data in advance, allowing the model to focus computational resources on classification rather than also processing raw document structure, thereby reducing training time while maintaining accuracy.
Solution Approach 2:
The patent extracts only the necessary text content from documents using OCR, separating it from the visual document layout. This extraction allows the S-BERT model to work with clean text sequences rather than entire document images, significantly reducing computational requirements and training time while preserving classification accuracy through focused text analysis.
3Adaptability or versatility
If traditional document classification systems are used, then comprehensive processing is achieved, but they do not efficiently assemble non-training documents into original configurations
Solution Approach 1:
The patent implements self-service by training the S-BERT model to automatically learn document structure patterns and assembly rules from training documents. The model autonomously determines how to categorize and reassemble non-training documents into original configurations without requiring explicit programming of assembly rules, thereby improving both adaptability and productivity simultaneously.
Solution Approach 2:
The patent applies parameter changes by transforming documents from their original multi-dimensional structure (pages, regions, lines, text) into flattened text sequences for S-BERT processing, then using the model's output to reconstruct documents in target configurations. This parameter transformation enables efficient processing while maintaining the ability to reconstruct complex document structures through learned patterns.
Data Source
AI summary
Systems and methods relate generally to performing a machine learning task on training documents to generate an output. In an example method, a pretrained Sentence Bidirectional Encoder Representational Transformers (“S-BERT”) model is obtained. The training documents are scanned by a plurality of scanners. Content of the training documents is recognized with character recognition. The content is templated responsive to the character recognition. The content is processed with the pretrained S-BERT model for training thereof. A trained S-BERT model is generated from the processing of the content as the output. The trained S-BERT model is configured to automatically categorize and assemble non-training documents into original configurations thereof.