Pre-compiling Invariant Full Text Query Parts for Search Server
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Relational database management systems face inefficiencies in processing full text searches due to the need for repeated computation and execution of invariant queries, leading to slower result set materialization and increased computational costs.
Innovation Solution
Pre-compiling and reusing the invariant part of full text queries in a text indexing and search server, maintaining an active connection with the RDBMS for query processing, and optimizing query expansion for expensive queries like wildcard and fuzzy searches, ensuring the posting list remains open during query evaluation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full text queries are re-executed for each row in scalar search mode, then query accuracy is maintained, but query processing time increases significantly
Solution Approach 1:
The patent pre-compiles the invariant portion of the full text query before scalar search execution. This preliminary compilation creates a cached representation that can be reused across multiple row evaluations, eliminating the need to re-compile the same query structure repeatedly while maintaining accurate search results.
Solution Approach 2:
The patent creates a compiled copy of the invariant query portion and stores it in cache. This compiled copy is then referenced and reused during scalar search operations instead of re-compiling the original query each time, significantly reducing processing time while preserving query accuracy through the use of the pre-compiled search structure.
2Measurement precision
If the text indexing and search server maintains detailed query compilation state, then query processing accuracy improves, but system memory consumption increases
Solution Approach 1:
The patent extracts only the invariant portion of the full text query for compilation and caching, separating it from the variable portion that changes with each row evaluation. This extraction approach allows the system to cache only the necessary compiled state without storing redundant information, reducing memory consumption while maintaining processing accuracy.
Solution Approach 2:
The patent applies different treatment to different parts of the query: the invariant portion is compiled and cached with high detail for accuracy, while the variable portion (such as row-specific identifiers) is handled dynamically without compilation. This local quality approach optimizes memory usage by applying compilation only where it provides value.
3Manufacturing precision
If scalar search mode is used for selective queries, then query selectivity improves, but result set materialization speed decreases
Solution Approach 1:
The patent performs preliminary compilation of the invariant query portion before entering scalar search mode. This pre-compilation prepares the search structure in advance, allowing the scalar search to focus only on evaluating individual rows against the pre-compiled criteria, thereby maintaining high selectivity while improving materialization speed.
Data Source
AI summary
Method, system, and computer program product for query processing are provided. In one implementation, the method comprises receiving a query that includes a full text portion. An invariant part of the full text portion is then compiled and stored. Additional queries received are executed by reusing the compiled invariant part.