One-Pass Indexing for Text Search Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Text search in large databases is slow and resource-intensive due to the inefficiencies in multi-pass indexing and separate handling of filter conditions, leading to high overhead and reduced performance.
Innovation Solution
Implementing a one-pass indexing scheme that filters an inverted word index with a single scan and embeds application-specific SQL conditions into the index search, allowing for interactive filtering between the base table and index table, thereby reducing the number of index scans and enhancing indexing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If multi-pass indexing is used to search the database, then the search can be performed with separate filter conditions, but the search speed decreases and resource consumption increases
Solution Approach 1:
The patent combines the indexing operation and filter condition application into a single integrated process. The inverted index is built while simultaneously applying filter conditions, eliminating the need for separate multi-pass operations. This merging of operations reduces the number of database scans from multiple passes to a single pass, directly improving search speed while maintaining full search capability.
2Ease of operation
If multi-pass indexing is used to search the database, then the search can be performed with separate filter conditions, but resource consumption increases
Solution Approach 1:
The patent merges the indexing process with filter condition application into a single operational pass. By building the inverted index and applying filters simultaneously, the system avoids the repeated resource consumption of multiple database scans. This single-pass approach significantly reduces CPU, memory, and I/O resource consumption while preserving complete search functionality.
3Ease of manufacture
If separate handling of filter conditions is used, then the indexing process is simple, but the overhead increases and performance decreases
Solution Approach 1:
The patent integrates filter condition handling directly into the indexing process itself. Rather than treating filtering as a separate post-indexing step, the system applies filters during index construction. This integration maintains the logical simplicity of the indexing process while dramatically improving query execution time by eliminating redundant operations.
4Reliability
If multiple index scans are performed, then the search can be thorough, but the search becomes slow
Solution Approach 1:
The patent applies filter conditions preliminarily during the indexing phase itself, before the actual search query is executed. By pre-filtering the data during index construction, the system ensures that only relevant documents are indexed. This preliminary action maintains search thoroughness for the remaining documents while dramatically reducing the search space, thereby improving search speed without sacrificing completeness.
Data Source
AI summary
A system and method for a text search of a database. A text search expression is converted to a query plan having multiple search tokens. A one-pass indexing of an invested word index filters the inverted word index based on a search condition and identifies the applicable documents having the multiple search tokens.