Database Query Splitting Across Heterogeneous Services
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As cloud services become increasingly complex and feature-rich, the demand for improved performance and availability of the data tier in cloud service platforms grows, particularly due to the increased complexity and dataset relationships in databases.
Innovation Solution
The implementation of a database query engine that automatically splits database queries into component queries, optimizing their execution across multiple database services, such as column-store and row-store database services, which are synchronized to maintain consistent records.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single database service is used to execute queries, then the system structure remains simple, but query performance deteriorates as dataset complexity increases
Solution Approach 1:
The patent divides the database system into multiple specialized database services (e.g., row-store database service for transactional operations, column-store database service for analytical operations). Each service is optimized for specific query types and data access patterns, allowing queries to be executed in parallel across multiple services rather than sequentially in a single service, thereby improving overall query performance without requiring complete system redesign
Solution Approach 2:
The query engine is designed with multi-functionality to automatically analyze incoming queries, determine optimal execution strategies, and route queries to appropriate database services. It can execute queries using a single database service or split and coordinate execution across multiple services based on query characteristics, making the system adaptable to different workload scenarios without manual configuration
2Productivity
If database queries are split across multiple database services, then query execution performance improves, but system complexity increases
Solution Approach 1:
The query engine acts as an intermediary between the client and multiple database services. It receives queries, analyzes their requirements, determines which database services should execute which portions, coordinates the execution across services, and aggregates results. This intermediary layer manages the complexity of multi-service coordination internally while presenting a simplified interface to clients, thereby improving performance without proportionally increasing user-facing system complexity
Solution Approach 2:
The system dynamically determines query execution strategies based on real-time query characteristics, data distribution, and service availability. The query engine can adaptively choose to execute queries using a single database service or split them across multiple services, and can adjust execution plans based on observed performance metrics, making the system flexible rather than statically complex
3Reliability
If multiple database services are implemented with synchronized records, then data consistency is maintained across services, but synchronization overhead increases
Solution Approach 1:
Different database services maintain different local qualities or specializations optimized for their specific functions (e.g., row-store for high-write operations, column-store for high-read operations). Synchronization occurs only for specific data elements or record changes that affect consistency requirements, rather than continuous full-data synchronization. This allows each service to operate independently with its own optimizations while maintaining necessary consistency through targeted synchronization events
Data Source
AI summary
A query is received. The query is executed at a database instance that supports a first database service and a second database service. Based on historical performance data, it is determined that an optimal execution path of the query includes splitting the query across the first database service and the second database service. In response to determining the optimal execution path, the query is split into a first component query in a first query language compatible with the first database service and a second component query in a second query language compatible with the second database service. The first component query is executed at the first database service. The second component query is executed at the second database service.