Document Flagging via Secondary Table Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing document flagging technologies face challenges in efficiently updating large numbers of documents with correct flags in near-real-time, especially in collaborative environments with massive document corpora, leading to scalability issues and high computational costs.
Innovation Solution
The method involves creating a secondary flag table for flagged documents when the number exceeds a threshold, with a complementary secondary flag table storing inverse flag data, which is then merged into the primary flag table, reducing computational costs and improving memory efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If flag data is stored directly in the primary flag table for all documents, then flagging information is easily accessible, but memory usage and computational costs increase significantly for large document sets
Solution Approach 1:
The patent divides the flag data storage into multiple generations (primary flag table and secondary flag tables). Each generation stores flag data for a specific time period or batch of documents, allowing the system to manage large volumes of flag data without loading everything into memory simultaneously. This segmentation enables efficient memory usage while maintaining fast access to current flagging information.
2Reliability
If all flag data is updated in the primary flag table in real-time, then collaborative flagging is reflected immediately, but system performance degrades with massive document corpora
Solution Approach 1:
The patent implements periodic merging of secondary flag tables into the primary flag table. Instead of continuously updating the primary flag table with every flagging action, the system accumulates flag data in secondary tables and periodically merges them. This periodic action maintains real-time collaboration accuracy while preventing system performance degradation that would result from continuous updates to the primary table.
Solution Approach 2:
The secondary flag tables act as intermediary structures between user flagging actions and the primary flag table. When users flag documents, the changes are first recorded in secondary tables rather than directly updating the primary table. This intermediary layer buffers the update operations, allowing real-time collaboration to function correctly while protecting the primary table from excessive write operations that would harm system performance.
3Loss of information
If the system processes and updates flag data for all documents simultaneously, then complete flagging information is maintained, but processing time increases for large document sets
Solution Approach 1:
The patent segments flag data processing into multiple generations stored in separate tables. Each secondary flag table handles a specific batch or time period of flagging operations, allowing the system to process and store flag data incrementally rather than all at once. This ensures complete flagging information is maintained across all generations while significantly reducing the processing time required for any single operation.
Solution Approach 2:
The system performs preliminary storage of flag data in secondary tables before merging into the primary table. This preliminary action allows flagging information to be captured and preserved immediately without the overhead of simultaneous processing and merging operations. The actual merging into the primary table occurs later when resources are available, reducing processing time while maintaining information completeness.
Data Source
AI summary
A first user request which specifies a target document set wherein a first subset of the documents is flagged by a user. A primary flag table is created for the target document set. A first document subset is created matching the first user request. It is determined whether a number of flagged documents exceeds a first threshold. If so, a secondary flag table is created for the first document subset and flag data corresponding to the first document subset is stored in the secondary flag table. The flag data in the secondary flag table is merged into the primary flag table.