Document Disaggregation Using Content Vectors and Page Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems are inefficient and require subject matter experts to identify document boundaries in digitized documents, making the process time-consuming and resource-intensive.
Innovation Solution
A method and system utilizing a machine-learning model to generate content vectors, determine nearest labeled pages based on vector distances, and apply dynamic programming to automatically segment documents into their original groups.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional systems are used to identify document boundaries, then subject matter experts can manually determine document boundaries, but the process becomes time-consuming and resource-intensive
Solution Approach 1:
The patent replaces manual mechanical processes (subject matter experts manually reviewing documents) with an automated machine learning-based system. The system extracts content items from documents, generates content vectors using a machine learning model, and automatically determines document boundaries based on vector distance thresholds, eliminating the need for manual expert intervention while maintaining segmentation accuracy
Solution Approach 2:
The system performs self-service by automatically processing document segmentation without requiring external human expertise. The machine learning model autonomously extracts content, generates vectors, and determines boundaries based on predefined criteria, allowing the system to service itself rather than requiring subject matter experts to manually perform the segmentation task
2Reliability
If manual review by subject matter experts is performed, then document boundaries can be identified with expertise, but computational resources and time are consumed
Solution Approach 1:
The patent substitutes manual expert review with an automated computational system that uses machine learning models to extract content vectors and determine document boundaries. This replacement reduces the need for human computational effort while maintaining reliable segmentation through algorithmic processing based on content similarity and vector distances
3Ease of operation
If all scanned pages from multiple documents are aggregated into a single PDF, then document processing is simplified, but identifying original document boundaries becomes challenging
Solution Approach 1:
The patent replaces manual boundary detection with an automated machine learning system that processes aggregated documents. The system extracts content items, generates content vectors, and automatically identifies document boundaries by analyzing vector distances between pages, making the detection process objective and scalable without requiring manual intervention
Solution Approach 2:
The patent introduces content vectors as an intermediary representation between the aggregated document pages and the boundary detection process. By transforming page content into vector space, the system creates a measurable metric (vector distance) that facilitates automatic boundary detection, bridging the gap between the aggregated document structure and the need for precise boundary identification
Data Source
Figure 1
Figure 2
Figure 3
AI summary
In some examples, systems and methods for disaggregating a set of documents are provided. An example method includes receiving the set of documents. In some examples, the set of documents include a plurality of pages. In some examples, the method further includes extracting a plurality of content items from the plurality of pages and providing the plurality of extracted content items to a machine-learning model. In some examples, the machine-learning the model is trained to generate content vectors. In some examples, the method further includes receiving, from the machine learning model, a plurality of content vectors corresponding to the plurality of extracted content items, determining, for each page of the plurality of pages, a plurality of potential nearest labelled pages in one or more labelled documents and a plurality of vector distances from the plurality of potential nearest labelled pages, based on the plurality of content vectors, and determining a segmentation option based at least in part on the plurality of vector distances. In some examples, the segmentation option indicates that a group of pages in the plurality of pages belong to a specific document.