Hybrid Query Engine for OLTP and OLAP Workloads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional database systems face inefficiencies in handling combined workloads of online transaction processing (OLTP) and online analytical processing (OLAP), leading to resource contention, poor throughput, and high costs due to the need for separate databases and asynchronous data updates, resulting in suboptimal performance and data staleness.
Innovation Solution
A hybrid query engine is implemented using both transactional and analytical engines, which receives database queries, performs initial processing, and assigns operations to the appropriate engines for execution, optimizing query plans to leverage the strengths of each engine, thereby improving performance and reducing the need for separate databases.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If separate databases are maintained for OLTP and OLAP workloads, then throughput for both workloads is improved, but device complexity and cost increase due to data synchronization requirements
Solution Approach 1:
The patent merges OLTP and OLAP workloads into a single database system with a unified query engine that can handle both transactional and analytical queries. This eliminates the need for separate databases and their associated synchronization mechanisms, while maintaining high throughput for both workload types through intelligent query routing and execution planning.
Solution Approach 2:
The query engine is designed with multi-functionality to handle both OLTP and OLAP queries simultaneously. It incorporates different execution strategies and resource allocation mechanisms within a single system, allowing the same database to serve both transactional processing and analytical processing requirements without requiring separate specialized systems.
2Ease of operation
If OLAP queries are executed on OLTP databases, then data access is simplified, but resource contention occurs and OLTP throughput deteriorates
Solution Approach 1:
The patent segments query execution by analyzing query characteristics and routing them to appropriate execution paths within the unified engine. Transactional queries are processed with emphasis on speed and consistency, while analytical queries are processed with emphasis on aggregation and data scanning, preventing resource contention by isolating their execution paths.
Solution Approach 2:
The query engine dynamically adjusts resource allocation and execution strategies based on the specific query type and current system state. It can shift resources between transactional and analytical processing in real-time, ensuring that OLTP throughput is maintained while still enabling comprehensive data access for OLAP workloads.
3Productivity
If separate OLTP and OLAP databases are used, then workload isolation is improved, but data freshness deteriorates due to asynchronous updates
Solution Approach 1:
By merging both workloads into a single database system with a unified storage layer, the patent eliminates the asynchronous update problem entirely. Data is accessed and updated through a single consistent source, ensuring real-time freshness for both transactional and analytical queries while maintaining workload isolation through query routing mechanisms.
Data Source
AI summary
A hybrid query engine using different query engines may be implemented to provide access to a database. A database query to a database may be received at a hybrid query engine. A plan to perform the database query may be generated that when performed by the hybrid query engine causes one query engine to perform an assigned portion of the plan by accessing data that describes the database. Another portion of the plan assigned to another query engine may be performed by the hybrid query engine to cause the other query engine to perform the other portion of the plan by accessing the same data that describes the database.