Null-Aware Parallel Join Processing for Faster Query Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face limitations in processing speed due to hardware constraints, data storage methods, and restricted co-processing options, particularly during join operations in query execution.
Innovation Solution
A parallelized database system architecture that includes a parallelized data input, storage, retrieval, and processing subsystem, along with a query and response subsystem, utilizing a query execution plan that distributes query execution across multiple nodes with redundancy encoding and shuffle networks to ensure accurate and efficient data handling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If parallelized database system architecture is used, then processing speed and throughput are improved, but system complexity increases
Solution Approach 1:
The database system is divided into multiple independent nodes that can process queries in parallel. Each node handles a portion of the data and query workload, enabling concurrent processing while maintaining individual node simplicity. The system architecture segments the monolithic database into distributed components that operate independently but coordinate through standardized interfaces.
2Loss of time
If distributed query execution is implemented, then query processing time is reduced, but coordination overhead increases
Solution Approach 1:
The system performs query planning and execution strategy determination before actual query execution. The query optimizer analyzes the query and data distribution in advance to pre-determine the execution plan, including which nodes should process which portions of the query. This preliminary action reduces coordination overhead during execution by establishing clear task assignments upfront.
3Reliability
If redundancy encoding is used for data storage, then data reliability is improved, but storage space requirements increase
Solution Approach 1:
The system employs error correction codes and redundancy encoding schemes that add controlled amounts of redundant data to ensure data reliability. By optimizing the redundancy parameter (e.g., using efficient erasure codes rather than simple replication), the system achieves high reliability with minimal storage overhead, balancing the trade-off between reliability and storage space requirements.
Data Source
AI summary
A database system is operable to determine a query that includes a join expression for execution. A set of input rows are dispersed for processing via a plurality of parallelized join processes in conjunction with executing the join expression based on identifying a first proper subset of the set of input rows based on a null-handling strategy. First rows in the first proper subset are dispersed for processing across the plurality of parallelized join processes in accordance with the null-handling strategy. second rows in a set difference between the set of input rows and the first proper subset are dispersed across the plurality of parallelized join processes in accordance with a join key-based assignment strategy. The set of input rows are processed via the plurality of parallelized join processes.