Composite Scanned Document Segmentation via Vector Similarity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The practice of scanning and merging component documents into single digital documents can strain memory and CPU/GPU usage, increase process times, and complicate automated contract analysis systems, especially when dealing with multiple languages or document-specific analyses.
Innovation Solution
A method, system, and computer program product that detects a target composite scanned document, extracts document features from sequential pages, iteratively generates sub-documents, generates vector representations based on these features, compares them to a knowledgebase of document vectors to calculate similarity scores, and clusters pages based on these scores to decompose the document into separate files.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If component documents are scanned and merged into single digital documents for storage and distribution, then storage efficiency and ease of distribution are improved, but memory usage, CPU/GPU usage, and process times increase
Solution Approach 1:
The patent segments the merged composite document into individual component documents by detecting boundaries between them. The system processes the composite document in smaller manageable units rather than treating it as one large document, reducing memory and computational resource requirements while maintaining storage efficiency benefits.
2Ease of operation
If component documents are scanned and merged into single digital documents, then ease of distribution is improved, but process times for analysis increase
Solution Approach 1:
The system divides the composite document into segmented components based on detected boundaries, allowing parallel processing of multiple smaller documents rather than sequential processing of one large document, thereby reducing overall analysis time while preserving distribution便利性.
Solution Approach 2:
The patent performs preliminary detection of document boundaries and segmentation before the actual analysis process. By pre-identifying where component documents begin and end within the composite document, the system prepares the data structure for efficient subsequent processing, reducing the time required during the main analysis phase.
3Productivity
If automated analysis is performed on merged composite documents, then storage efficiency is improved, but analysis accuracy and system performance deteriorate
Solution Approach 1:
The patent segments composite documents into their constituent component documents by detecting boundaries between them. This segmentation allows each component to be analyzed in its original context and format, preserving analysis accuracy while maintaining the storage efficiency benefits of consolidated document management.
Solution Approach 2:
The system applies different analysis approaches to different segments of the composite document based on their specific characteristics. By treating each component document with appropriate analysis methods suited to its content and format, the system maintains high analysis accuracy across diverse document types while benefiting from consolidated storage.
Data Source
AI summary
An embodiment for decomposing composite scanned documents. The embodiment may detect a target composite scanned document. The embodiment may extract, for sequential pages of the target composite scanned document, a series of document features. The embodiment may iteratively generate a series of sub-documents by iteratively adding a next page from the target composite scanned document to a series of one or more pages preceding the added next page. The embodiment may generate vector representations for each of the iteratively generated series of sub-documents, where each of the generated vector representations is based on the extracted series of document features. The embodiment may calculate similarity scores by comparing the generated vector representations with a knowledgebase of document vectors. The embodiment may cluster the sequential pages of the target composite scanned document based on the calculated similarity scores. The embodiment may output separate files including the clustered sequential pages.