One-Pass Inverse Indexing for Database Text Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Text search in large databases is often slow and resource-intensive due to the need for multi-pass indexing, which involves scanning the index table multiple times, impacting efficiency and performance.
Innovation Solution
Implementing a one-pass indexing scheme that scans the inverse index table only once, using a match-count based approach to identify documents matching search tokens, thereby avoiding multi-pass indexing overhead and enhancing text search performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multi-pass indexing is used to ensure accurate text search results, then search reliability is improved, but search speed and resource efficiency deteriorate
Solution Approach 1:
The patent pre-computes and stores document frequency information and term statistics in the inverse index structure during database construction. This preliminary action eliminates the need for multiple passes over the index during search operations, as all necessary statistical information is readily available in a single pass, thereby maintaining accuracy while improving speed.
2Reliability
If multi-pass indexing is performed to ensure complete document matching, then search reliability is improved, but resource consumption increases
Solution Approach 1:
The system pre-calculates and stores document frequencies, term lengths, and other statistical metrics in the inverse index during the indexing phase. This preliminary computation ensures that search operations can rely on pre-computed data without requiring additional resource-intensive passes, thus maintaining search completeness while reducing runtime resource consumption.
3Measurement precision
If traditional multi-pass indexing is used, then accurate text search is achieved, but execution time increases
Solution Approach 1:
The inverse index structure pre-stores document frequency counts, term occurrence statistics, and positional information during database construction. This preliminary action enables the search algorithm to retrieve accurate results in a single pass by leveraging pre-computed statistics, thereby maintaining measurement precision while significantly reducing execution time compared to multi-pass approaches.
Data Source
AI summary
A system and method for a text search of a database, including converting a text search expression to a query plan and implementing the text search as the query plan on the database. The implementing of the text search includes a one-pass indexing as a single scan of an inverse index table associated with the database.