Query Scheduling in Heterogeneous Database Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database management systems (DBMS) face challenges in optimizing workload distribution due to reliance on pre-established rules and lack of transparency in heterogeneity, leading to inefficient query execution and system performance.
Innovation Solution
A computer-implemented method for scheduling query execution in a heterogeneous database system, which involves obtaining queries, generating cost analytics for query blocks, identifying similar cost analytics across queries, applying a cost model to identify query blocks for result reuse, and distributing queries based on execution statistics to optimize resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If a DBMS distributes workloads based on pre-established rules, then the system provides automated workload distribution, but the system performance is not optimized due to lack of adaptive intelligence
Solution Approach 1:
The system collects execution statistics from multiple sources including query execution history, system metrics, and performance data. This feedback loop enables the DBMS to continuously learn from actual system behavior and adapt workload distribution decisions dynamically, transforming static rule-based distribution into an adaptive intelligent system that optimizes performance while maintaining automation.
Solution Approach 2:
The system performs preliminary analysis of query patterns and execution statistics before making workload distribution decisions. By pre-processing and analyzing execution statistics to identify patterns and optimize query plans in advance, the system prepares optimized distribution strategies proactively rather than reactively, improving overall system performance while maintaining automated operation.
2Adaptability or versatility
If a DBMS collects various execution statistics to inform workload distribution, then the intelligence for optimization is improved, but the complexity of the system increases
Solution Approach 1:
The system segments the collection and processing of execution statistics by source and type, organizing them into distinct categories such as query execution statistics, system metrics, and performance data. This segmentation allows the DBMS to manage complex statistical information in a structured manner, processing different types of statistics through specialized handlers while maintaining overall system manageability and reducing operational complexity.
Solution Approach 2:
The system implements a universal statistics collection framework that gathers diverse execution statistics through a unified interface and processing pipeline. This multi-functional approach allows the same infrastructure to handle multiple types of statistical data (query execution, system metrics, performance data) simultaneously, reducing the need for separate specialized systems and thereby managing complexity while maintaining high adaptability.
3Ease of operation
If a DBMS provides transparent workload distribution in heterogeneous systems, then user experience is improved, but the difficulty of managing heterogeneity increases
Solution Approach 1:
The system introduces an intermediary layer between the user and the heterogeneous database systems that abstracts away the complexity of heterogeneity management. This intermediary translates user queries into appropriate workload distribution decisions across different database systems, handling the complexity of detecting and managing heterogeneity internally while presenting a simple, transparent interface to users, thereby improving ease of operation without requiring users to understand or manage the underlying complexity.
Data Source
AI summary
Computer implemented methods, systems, and computer program products include program code executing on a processor(s) to: obtain queries (each query including workload(s)); obtain execution statistics for the one or more workloads; normalize one or more queries. Normalizing includes generating cost analytics, identifying similar cost analytics for query blocks across the queries, and applying a cost model to identify at least one query block for result reuse across. The program code distributes the queries to system resources by assigning the at least one query block (in more than one query/workload) to a first resource to execute (as part of a first workload) against database tables and assigning the at least one query block to a second resource to execute (as part of a first workload) against a shared cache of a heterogenous database system.