Document Processing Memory Optimization via Page Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Providing efficient and scalable document processing services, particularly in cloud environments, is challenging due to the significant memory requirements and high Input/Output (I/O) loads associated with processing large documents.
Innovation Solution
The method involves converting documents into a first format, splitting them into individual pages, parsing and processing each page to identify a predefined set of items, and storing only the essential information in a second format, which reduces memory footprint and I/O load, allowing for efficient document processing and storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If documents are processed in their original format without splitting, then processing completeness is maintained, but memory footprint and I/O load increase significantly
Solution Approach 1:
The patent applies segmentation by dividing documents into individual pages and further into page objects (text, images, tables, etc.). This allows the system to process and store only the essential information from each page rather than the entire document, significantly reducing memory footprint while maintaining processing completeness through systematic handling of all page elements.
Solution Approach 2:
The patent extracts only the necessary information from each page (predefined set of items such as text content, metadata, and essential structural elements) while discarding redundant data. This extraction process reduces the quantity of data stored and processed, directly addressing the memory footprint issue while preserving processing completeness for the extracted essential elements.
2Loss of information
If full documents are transferred for processing, then data completeness is ensured, but I/O load increases
Solution Approach 1:
The system extracts only the predefined set of essential items from each page before transfer and processing. This extraction ensures that data completeness is maintained for all necessary information while minimizing I/O load by eliminating redundant data transfer.
Solution Approach 2:
By segmenting documents into pages and then into individual page objects, the system enables selective transfer of only the essential extracted information. This segmentation approach maintains data completeness for processing while significantly reducing the overall I/O load compared to transferring full documents.
3Productivity
If large memory resources are reserved for processing, then processing capacity is sufficient, but resource efficiency decreases
Solution Approach 1:
The patent extracts only the essential information needed for processing from each page, which reduces the memory resources required to hold and process document data. This extraction maintains sufficient processing capacity for the essential elements while improving resource efficiency by eliminating the need to allocate memory for redundant information.
Solution Approach 2:
Segmentation of documents into pages and page objects allows the system to process smaller, manageable units in memory. This approach maintains processing capacity by enabling systematic handling of all essential elements while improving resource efficiency through reduced memory footprint compared to loading entire documents.
Data Source
AI summary
The invention is notably directed to a computer-implemented method for processing a plurality of documents. The method comprises providing the plurality of documents in a first format and splitting each of the plurality of documents of the first format into one or more individual pages. The method further comprises individually parsing the one or more individual pages of the plurality of documents. The parsing comprises identifying a predefined set of items of the one or more individual pages. Further processing comprises gathering the predefined set of items of each of the one or more individual pages of the plurality of documents into individual page files of a second format and performing the document processing service with the individual page files of the second format. The invention further concerns a corresponding computing system and a related computer program product.