Distributed Query Execution Plans for Parallel Database Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face limitations in processing speed due to hardware constraints, data storage methods, and restricted co-processing options, particularly when handling large volumes of data.
Innovation Solution
A distributed query execution plan is implemented, where operators are distributed to execute at a lower level of the query plan, optimizing data partitioning, storage, and retrieval processes across multiple computing devices, enabling parallel processing and concurrent query execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is stored and processed in a centralized database system, then data consistency and simplicity of management are maintained, but processing speed and scalability are limited by hardware constraints
Solution Approach 1:
The database system is segmented into multiple computing devices, each handling portions of data storage and query processing. The query execution plan is divided into multiple operators distributed across different devices, enabling parallel processing while maintaining overall system consistency through coordinated execution.
2Productivity
If distributed parallel processing is implemented to improve processing speed, then productivity increases, but system complexity and coordination overhead increase
Solution Approach 1:
The query execution plan is pre-compiled and distributed to relevant computing devices before actual query execution. Operators are prepared in advance with their data partitions, eliminating the need for complex runtime coordination and enabling immediate parallel processing when queries arrive.
3Adaptability or versatility
If data is partitioned across multiple computing devices, then processing capacity and scalability are improved, but data retrieval complexity and communication overhead increase
Solution Approach 1:
Different computing devices are assigned different portions of the query execution plan and different data partitions based on their capabilities and data holdings. Each device processes local data independently, reducing communication overhead while maintaining overall query correctness through coordinated results aggregation.
Data Source
AI summary
A method for execution by at least one processor of a database system includes determining whether a first operator of a query execution plan is an input to a second operator of the query execution plan such that the second operator is trapped at a non-leaf level of the query execution plan. When the second operator is trapped at the non-leaf level of the query execution plan, the method further includes generating an updated query execution plan, wherein the updated query execution plan includes the first operator being distributed to produce a plurality of distributed first operators such that the second operator executes at a lower level than the non-leaf level of the query execution plan. The method further includes executing the updated query execution plan to produce a query resultant.