Disjunction Probability Query Planning for Parallel Database Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face limitations in processing speed due to hardware constraints, data storage methods, and restricted co-processing options, leading to inefficiencies in query execution.
Innovation Solution
A database system architecture that employs parallelized data input, storage, and processing subsystems, along with optimized query planning and execution, utilizing a network of computing devices and nodes to distribute and manage data efficiently, enabling lock-free and parallel execution of administrative and configuration operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If parallelized data input, storage, and processing subsystems are employed, then query execution efficiency is improved, but device complexity increases
Solution Approach 1:
The database system is divided into multiple independent subsystems including parallelized data input subsystems, storage subsystems, and processing subsystems. Each subsystem operates independently to handle specific portions of data processing tasks, enabling concurrent execution and improving overall query processing efficiency while managing complexity through modular architecture.
Solution Approach 2:
The system transitions from sequential processing to parallel processing by adding a temporal dimension to data processing operations. Multiple data processing operations are executed simultaneously across different subsystems, effectively increasing throughput and reducing query execution time through concurrent operations.
2Loss of time
If multiple nodes are used for parallel processing, then processing time is reduced, but network communication overhead increases
Solution Approach 1:
Data is segmented and distributed across multiple nodes in the parallelized processing system. Each node processes specific portions of data independently, reducing the processing time for large datasets while the distributed architecture naturally manages network communication overhead through localized processing.
Solution Approach 2:
The query planner performs preliminary actions to optimize data distribution and processing paths before execution begins. By pre-computing execution plans and optimizing data routing, the system reduces unnecessary network communication overhead during actual data processing operations.
Data Source
AI summary
A database system operates by: determining a query for execution against a dataset that indicates a filtering predicate denoting a disjunction between a first range-based predicate and a second range-based predicate; accessing distribution data for the dataset indicating a plurality of kernels for a plurality of points in a multi-dimensional space; identifying a first sub-region within the multi-dimensional space corresponding to the first range-based predicate; identifying a second sub-region within the multi-dimensional space corresponding to the second range-based predicate; computing a disjunction probability approximation value based on an average portion summation value across a plurality of portion summation values generated for the plurality of kernels; and executing the query based on the disjunction probability approximation value.