Wildcard Search Indexing via Reverse and N-gram Structures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud computing database systems face inefficiencies in searching data with wildcard operations, as existing data structures perform differently based on the positioning of wildcard operators, necessitating improved search techniques that function consistently regardless of operator placement.
Innovation Solution
The implementation of specialized indexes such as reverse, n-gram, and permutation indexes, which are created and maintained asynchronously to enhance the efficiency of leading wildcard queries, allowing for synchronous user functionality while minimizing resource usage. These indexes are dynamically generated based on workload types and pre-processing operations are offloaded from the database to application servers to reduce resource intensity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional search data structures are used for wildcard queries, then the system can handle basic search operations, but the search performance deteriorates when wildcard operators are positioned at the beginning of search terms
Solution Approach 1:
The patent segments the search index into multiple specialized indexes based on wildcard position: standard indexes for trailing wildcards, reverse indexes for leading wildcards, and n-gram indexes for intermediate wildcards. Each index type is optimized for specific wildcard patterns, allowing the system to maintain high search performance regardless of wildcard positioning.
Solution Approach 2:
The patent applies inversion by creating reverse indexes that store search terms in reverse order. This allows leading wildcard queries to be transformed into equivalent trailing wildcard queries on the reversed data, enabling efficient search performance for leading wildcards using the same optimization techniques as trailing wildcards.
2Productivity
If specialized indexes are created to improve leading wildcard search efficiency, then search performance for leading wildcards is improved, but resource usage increases due to maintaining multiple index types
Solution Approach 1:
The patent implements dynamic index selection where the system automatically determines which specialized index to use based on the wildcard position in the query. The query optimizer analyzes the search pattern and selects the most appropriate index type (standard, reverse, or n-gram), ensuring efficient resource utilization by only accessing necessary indexes rather than maintaining all indexes active for all query types.
Solution Approach 2:
The patent changes the parameter of index structure based on the query characteristics. Different index structures (reverse, n-gram, permutation) are created with specific parameters optimized for different wildcard positions. The system selects and applies the appropriate index structure parameter set based on the detected wildcard pattern, optimizing both performance and resource usage.
3Productivity
If pre-processing operations are performed on the database to create specialized indexes, then search efficiency is improved, but the complexity of the database system increases
Solution Approach 1:
The patent performs preliminary actions by pre-processing search terms during data insertion and update operations. Specialized indexes (reverse, n-gram, permutation) are built and maintained in advance, so that when queries are executed, the pre-computed index structures are already available for immediate efficient searching, eliminating the need for complex real-time processing during query execution.
Solution Approach 2:
The patent introduces intermediary components including a query optimizer that analyzes search patterns and selects appropriate indexes, and an index management system that coordinates the creation and maintenance of multiple specialized indexes. These intermediaries abstract the complexity from users while enabling efficient multi-index operations.
Data Source
AI summary
Described are mechanisms for creating and utilizing a specialized index to improve the efficiency of various types of leading wildcard searches without having to unduly burden the system with write requests. The mechanisms may identify particular fields to be included in one or more types of specialized indexes that are accessed in response to particular types of wildcard queries including various combinations of leading, trailing, and intermediate wildcard operators. In addition, the mechanism may perform various pre-processing by an application when creating such indexes to conserve resources of an underlying database system. The mechanisms may also execute such wildcard queries in a manner that appears to be synchronous to a user while maintaining such indexes asynchronously for efficiency purposes.