Heuristic Database Querying with Dynamic Partitioning for Lower Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing query optimization techniques for cloud and SaaS application resources are inflexible and inefficient due to their lack of applicability across different database types, leading to high latencies and suboptimal performance in retrieving resource data.
Innovation Solution
Dynamic partitioning of the search space by augmenting submitted queries with designated dynamic partition keys based on heuristics, such as recency and relevance, to generate candidate query plans that are evaluated for optimal execution, combined with pagination for result presentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing query optimization techniques are used, then query execution can be performed on the database, but high latencies and suboptimal performance occur due to inflexibility and lack of applicability across different database types
Solution Approach 1:
The system dynamically partitions the search space by augmenting submitted queries with designated dynamic partition keys based on heuristics such as recency and relevance. This dynamic approach allows the query optimization strategy to adapt to different database types and query patterns, resolving the contradiction between maintaining query execution capability and reducing latency through flexible, context-aware partitioning rather than static optimization techniques
Solution Approach 2:
The system changes the parameters of query execution by introducing dynamic partition keys and evaluating multiple candidate query plans based on heuristic criteria. This parameter transformation enables the system to optimize query performance across different database types without altering the underlying database structure, thereby improving productivity while minimizing latency loss
2Productivity
If dynamic partitioning with heuristic query augmentation is implemented, then search performance is enhanced by reducing latencies, but query processing complexity increases
Solution Approach 1:
The system segments the query processing task into multiple candidate query plans, each augmented with different dynamic partition keys. By evaluating these segmented candidates based on heuristic criteria and selecting the optimal plan, the system enhances search performance through structured complexity management rather than monolithic processing
Solution Approach 2:
The system employs self-service mechanisms by using heuristic evaluation to automatically select optimal query plans without requiring external intervention or complex manual configuration. This self-service approach to query optimization improves search performance while keeping the system manageable despite increased processing complexity
3Adaptability or versatility
If dynamic partitioning is applied without altering database structure, then adaptability across various database types is improved, but the complexity of query augmentation and evaluation increases
Solution Approach 1:
The system achieves universality by designing a query augmentation mechanism that works across different database types without requiring structural modifications to the databases themselves. The dynamic partition keys and heuristic evaluation framework serve multiple database systems universally, improving adaptability while managing augmentation complexity through a unified approach
Solution Approach 2:
The system introduces an intermediary layer of query augmentation that sits between the query submission and database execution. This intermediary applies dynamic partition keys and heuristic evaluation without altering the underlying database structure, enabling cross-database adaptability while containing the complexity within the mediation layer rather than propagating it to the database level
Data Source
AI summary
Dynamic partitioning of a search space of queries is implemented for flexible, heuristic database querying. Search space partitioning refers to dividing the search space for a submitted query into smaller parts by augmenting the queries to append thereto an additional predicate comprising a dynamic partition key and a value(s) selected based on heuristics (e.g., recency and/or relevancy of the value(s)). A plurality of candidate augmentations of the query and corresponding query plans are generated and evaluated based on additional heuristics to determine which can be executed to yield the best results in terms of result quality and latency. This query plan is selected and executed for retrieval of results that satisfy the query, with pagination utilized for presentation of the results. The procedure of generating candidate query plans, selecting one of the candidates for execution, and paginating results is repeated until a search termination criterion is satisfied.