Parallel Database Query Execution via Segmented Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database systems face limitations in processing speed due to hardware constraints, data storage methods, and restricted co-processing options, which hinder efficient execution of queries in large-scale data processing networks.
Innovation Solution
A parallelized database system architecture that includes a parallelized data input sub-system, data store and process sub-system, query and response sub-system, and administrative sub-system, which divide data into segments, optimize query plans, and execute queries across multiple nodes and storage clusters to improve processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored in a traditional database system with sequential processing, then hardware resources are简单易用, but query execution speed is slow
Solution Approach 1:
The patent divides the database system into multiple storage clusters, each containing multiple nodes. Data is segmented and distributed across these clusters, allowing parallel query execution. Each cluster can independently process portions of queries, significantly improving execution speed while maintaining manageable complexity through modular architecture.
Solution Approach 2:
The patent transitions from sequential single-node processing to multi-dimensional parallel processing across multiple storage clusters and nodes. By adding spatial distribution dimensions (clusters × nodes), the system achieves exponential speedup in query execution without proportionally increasing operational complexity.
2Productivity
If multiple queries are executed sequentially on single nodes, then system architecture is simple, but processing efficiency is low
Solution Approach 1:
The database system is segmented into multiple independent storage clusters, each with multiple nodes capable of autonomous query processing. This segmentation enables concurrent execution of multiple queries across different clusters, dramatically improving productivity while keeping each individual cluster relatively simple to manage.
Solution Approach 2:
The patent combines multiple storage clusters into a unified distributed system that presents a single logical database interface to users. Behind the scenes, queries are automatically distributed and executed in parallel across clusters, merging their individual processing capabilities to achieve high productivity without exposing complexity to end users.
3Productivity
If data is concentrated on few nodes, then data access is fast for single queries, but concurrent query processing is limited
Solution Approach 1:
Data is segmented and distributed across multiple storage clusters and nodes, with each node holding portions of the overall dataset. This segmentation enables multiple queries to access different data segments simultaneously, improving concurrent processing capacity while maintaining simple data access through automatic query routing and result aggregation.
Data Source
AI summary
A node of a database system is operable to determine a plurality of query operator execution flows for execution of a plurality of queries. Execution of the plurality of queries is facilitated based on performing a plurality of sequential operator execution steps based on power. At least one of the plurality of sequential operator execution steps includes selecting one of the plurality of queries for execution at the each of the plurality of sequential operator execution steps, selecting one of the plurality of operators of the one of the plurality of query operator execution flows corresponding to the one of the plurality of queries, facilitating execution of the one of the plurality of operators on a queued set of data blocks of the one of the plurality of operators to generate at least one output data block, and advancing to a next one of the plurality of sequential operator execution steps.