Parallel Metadata Indexing for Rapid Re-indexing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing search engine systems face challenges in efficiently re-indexing metadata without requiring a full re-indexing of all documents, which is time-consuming and costly due to the difficulty in changing indexing settings.
Innovation Solution
A computer-implemented method and system that dynamically re-indexes metadata by maintaining separate indexes for content and metadata, allowing for quick re-indexing of metadata without re-processing the entire document set by storing metadata in native form and generating parallel indexes, enabling seamless replacement of indexes without interrupting search functionality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the entire body of documents is re-indexed to change indexing definitions, then the indexing settings can be updated, but the process becomes very expensive and time consuming
Solution Approach 1:
The patent divides the indexing system into separate segments: a first index for content data and a second index for metadata. This segmentation allows the metadata index to be re-indexed independently without re-indexing the entire document collection, thus reducing re-indexing time while maintaining flexibility in indexing settings.
Solution Approach 2:
The patent extracts metadata from the full document index and creates a separate metadata index. This extracted metadata index can be maintained and re-indexed independently, allowing changes to metadata indexing definitions without requiring re-indexing of all documents, thereby reducing time loss while maintaining adaptability.
2Adaptability or versatility
If the entire body of documents is re-indexed to change indexing definitions, then the indexing settings can be updated, but the cost increases significantly
Solution Approach 1:
The patent segments the indexing system into separate content and metadata indexes. This segmentation reduces re-indexing costs by allowing only the metadata index to be re-indexed when metadata indexing definitions change, rather than re-indexing the entire document collection, thus maintaining flexibility while reducing costs.
Solution Approach 2:
The patent extracts and separates metadata indexing from the main document index. This extraction allows independent maintenance and re-indexing of metadata, reducing the computational resources and costs associated with re-indexing while preserving the ability to update indexing settings.
3Device complexity
If a single index is used for both content and metadata, then the system is simpler, but re-indexing requires processing all documents
Solution Approach 1:
The patent implements segmentation by creating separate indexes for content and metadata. While this increases system complexity slightly, it dramatically improves re-indexing efficiency by allowing selective re-indexing of only the metadata portion when needed, thus resolving the contradiction between simplicity and productivity.
Solution Approach 2:
The patent creates a dynamic indexing system where the metadata index can be independently updated and maintained separate from the content index. This dynamic structure allows the system to adapt re-indexing operations based on specific needs, improving productivity by avoiding unnecessary re-indexing of entire document collections.
Data Source
AI summary
A method and indexing system indexes the content of a body of documents into a content index, and the metadata of the documents into a metadata index which is a parallel index to the content index. The metadata is copied into a data store that is easily accessible by the indexing system and is stored in native form. The indexing system can dynamically re-index the metadata from the native metadata in the data store to produce a new metadata index which is used to replace the original metadata index. Search queries received by a search engine associated with the indexing system are applied to both the content and metadata index and the results are merged for return.