Document Segmentation for Full Text Index Search Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Full text searching in large databases is time-consuming, leading to undesired latency in document retrieval, as existing methods search entire databases without segmenting, resulting in inefficient use of resources and prolonged wait times for users.
Innovation Solution
The method involves partitioning a document collection into segments based on document attributes, allowing only relevant segments to be searched based on user-defined criteria, such as date ranges, thereby reducing the search scope and improving efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full text searching is performed on entire databases without segmenting, then comprehensive search coverage is achieved, but search time and latency increase significantly
Solution Approach 1:
The patent divides the document collection into multiple segments based on document attributes (e.g., date ranges, categories). Each segment is stored separately and can be searched independently. When a search request arrives, the system identifies which segments are relevant based on the search criteria and only searches those specific segments rather than the entire database, thereby reducing search time while maintaining completeness.
2Loss of time
If document collection is partitioned into multiple segments, then search time is reduced by limiting search scope, but system complexity increases due to segment management
Solution Approach 1:
The patent introduces a segment metadata structure that acts as an intermediary between the search system and the segmented document collection. This metadata structure stores information about each segment (such as attribute ranges, segment identifiers), allowing the system to quickly determine which segments to search without complex real-time analysis. The metadata layer simplifies segment management by providing a standardized interface for segment identification and selection.
3Productivity
If segments are determined based on document attributes, then search efficiency improves by filtering irrelevant segments, but additional processing is required to match search criteria with attribute ranges
Solution Approach 1:
The patent pre-processes document collections by organizing documents into segments based on their attributes before searching occurs. During this preliminary organization phase, the system creates a metadata structure that records the attribute ranges and characteristics of each segment. When a search request arrives, the system can quickly match the search criteria against this pre-established metadata to identify relevant segments, avoiding the need for complex real-time attribute analysis during the search operation.
Data Source
AI summary
A system, method and computer program product for searching documents within a collection includes partitioning the collection into a plurality of document segments based on a range of values for a document attribute. Each document segment is associated with a different value range for the document attribute and includes documents each with a value for the document attribute within the associated value range. The system, method and computer program product also determines one or more document segments to search based on one or more search criteria within a search request pertaining to the document attribute. The system, method and computer program product further searches the determined one or more document segments for information satisfying the search request.