Array Field Distribution for Faster Database Query Planning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face limitations in processing speed due to hardware constraints, data storage methods, and restricted co-processing options, leading to inefficiencies in query execution.
Innovation Solution
A database system architecture that employs parallelized data input, storage, retrieval, and query processing across multiple computing devices, utilizing a parallelized data input sub-system, query and response sub-system, and administrative sub-system to optimize query execution through distributed processing and efficient data storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored in traditional database systems with sequential processing, then data integrity and reliability are maintained, but processing speed and query execution time deteriorate
Solution Approach 1:
The patent divides the database system into multiple computing devices forming a cluster, where each device processes portions of queries independently. The data is partitioned across these devices, allowing parallel processing of different query components simultaneously, thereby increasing processing speed while maintaining manageable complexity through modular architecture
Solution Approach 2:
The patent transitions from single-device sequential processing to multi-device parallel processing by adding the dimension of spatial distribution across a network. Queries are distributed across multiple computing devices that process data in parallel, effectively moving from one-dimensional sequential execution to multi-dimensional concurrent execution
2Productivity
If hardware resources are increased to improve processing speed, then query execution speed improves, but system cost and complexity increase
Solution Approach 1:
Each computing device in the cluster serves multiple functions: storing data partitions, processing query operations, and participating in distributed transactions. This multi-functionality allows the system to achieve high productivity without proportionally increasing complexity, as the same hardware resources perform diverse operations
Solution Approach 2:
The distributed database system automatically manages data distribution, query routing, and transaction coordination across computing devices without requiring centralized control for every operation. Each device independently manages its local data and processing, reducing overall system complexity while maintaining high execution efficiency
3Loss of time
If data is stored using optimized storage methods, then retrieval speed improves, but storage complexity and space requirements worsen
Solution Approach 1:
Data is divided into partitions distributed across multiple computing devices, with each device storing and managing its own segment independently. This segmentation enables parallel retrieval operations where different data portions can be accessed simultaneously from different devices, reducing overall retrieval time while keeping individual storage management simple
Data Source
AI summary
A method includes generating array field distribution data for an array field of a dataset. A query expression for execution that includes a query predicate indicating the array field of the dataset is received. The array field distribution data for the array field is utilized to generate query plan data for the query expression based on the query expression including the query predicate indicating the array field. Execution of the query expression is facilitated in accordance with the query plan data.