XQuery Full-Text Search Evaluation Using Iterator-Based FTSelection Trees
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
XQuery 1.0 lacks an efficient evaluation strategy for full-text search queries, leading to performance issues when executing XQuery FT code that defines the semantics of the full-text extension.
Innovation Solution
Implementing an iterator evaluation model for FTSelection trees, allocating memory for execution states at compile time, and integrating word and phrase search with inverted index lists to avoid materializing all Match instances and optimize memory and CPU usage, while ensuring at most a single scan of documents and sharing FTSelection trees among concurrent processing threads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If XQuery FT code is executed using standard XQuery evaluation, then full-text search functionality is achieved, but memory and CPU requirements become excessively high
Solution Approach 1:
The patent segments the evaluation process by separating the compilation phase (which creates the FTSelection tree) from the execution phase (which evaluates the tree using optimized iterators). This allows the expensive full-text search operations to be performed once during compilation and cached, rather than being re-executed for each query, thereby reducing memory and CPU requirements during actual query processing while maintaining full-text search functionality.
Solution Approach 2:
The patent performs preliminary actions by pre-compiling the XQuery FT code into an FTSelection tree structure during a compilation phase before actual query execution. This pre-processing creates an optimized representation that can be efficiently evaluated multiple times without re-parsing or re-compiling, significantly reducing the computational resources required during runtime while preserving the full-text search capability.
2Loss of information
If all Match instances are materialized during FTSelection tree evaluation, then complete search results are obtained, but memory consumption increases significantly
Solution Approach 1:
The patent extracts only the essential information needed for full-text search evaluation from the Match instances, rather than materializing complete result sets. The FTSelection tree evaluation uses iterators that process Match instances on-demand, extracting only the necessary matching criteria information while discarding redundant data, thereby maintaining search result completeness while minimizing memory consumption.
Solution Approach 2:
The patent employs temporary, disposable data structures during the evaluation process that are created and discarded as needed. The iterator-based evaluation uses lightweight temporary objects to hold intermediate matching results during traversal, rather than maintaining persistent storage for all Match instances, allowing complete search results to be obtained through sequential processing while keeping memory consumption low.
3Reliability
If multiple scans of documents are performed during full-text search, then thorough search coverage is achieved, but query execution time increases
Solution Approach 1:
The patent merges multiple document scan operations into a single unified traversal of the FTSelection tree. By combining the evaluation of multiple full-text search conditions into one pass through the document collection using shared iterators, the system achieves thorough search coverage equivalent to multiple separate scans while reducing query execution time by eliminating redundant document access and processing overhead.
4Ease of manufacture
If FTSelection trees are not optimized for concurrent access, then implementation simplicity is maintained, but performance degrades under concurrent processing
Solution Approach 1:
The patent designs the FTSelection tree structure to serve multiple concurrent query processing threads simultaneously. The tree is compiled once and can be evaluated by multiple iterators in parallel, making it a universal structure that supports both sequential and concurrent execution modes. This multi-functional design maintains implementation simplicity while enabling high concurrent processing performance through shared read-only tree structures and thread-safe iterator evaluation.
Data Source
AI summary
Techniques are provided for efficiently evaluating XML queries that conform to an extension of an XML language (e.g., XQuery or XPath). The extension allows XML queries to have full-text search capabilities. Such an XML query is compiled to generate a tree of nodes that correspond to one or more conditions in the full-text portion of the query. In one technique, the amount of memory for the execution state of the tree is determined at compile time and allocated only once throughout execution of the query. In another technique, to ensure at most a single scan of a document, all the words or phrases in the full-text portion of an XML query are located before any of the other conditions in the full-text portion are evaluated. In another technique, the elements of the full-text portion of an XML query are analyzed to determine, based at least in part on cost, which evaluation strategy, of a plurality of evaluation strategies, should be employed.