Data Slab Partitioning for Lock-Free Parallel Database Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems are limited by hardware, data storage methods, and co-process options, leading to inefficiencies in processing speed and execution time.
Innovation Solution
A parallelized database system architecture that includes a parallelized data input, store, retrieve, and process sub-system, along with a query and response system, utilizing a network of computing devices with independent processing core resources, and a database operating system to optimize query plans and execute operations in parallel, reducing processing time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored in a traditional sequential manner in existing database systems, then data storage is simple and straightforward, but processing speed and execution time are limited and inefficient
Solution Approach 1:
The patent divides data into multiple data slabs that are distributed across multiple computing devices in a parallelized system. Each computing device independently processes its assigned data slabs, enabling concurrent data processing operations that significantly improve processing speed compared to traditional sequential processing in single-system database architectures.
2Productivity
If a parallelized database system architecture is implemented to improve processing speed, then execution efficiency is enhanced, but system complexity increases due to multiple computing devices and coordination requirements
Solution Approach 1:
The patent combines multiple computing devices into a unified parallelized database system where data slabs are distributed across the devices. The system integrates data input, storage, retrieval, and processing functions across multiple devices, enabling parallel execution of database operations while maintaining coordinated access through a shared data slab structure.
3Loss of time
If lock-free parallel execution is implemented in the parallelized system, then processing time is reduced and efficiency is improved, but ensuring data consistency and coordination becomes more difficult
Solution Approach 1:
The patent uses data slabs as an intermediary structure that enables parallelized computing devices to access and process data without requiring traditional locking mechanisms. The data slab architecture provides a coordinated access model that allows multiple devices to read and write data concurrently while maintaining data consistency, eliminating the need for locks that would otherwise be required to ensure reliability in parallel systems.
Data Source
AI summary
A data input sub-system of a parallelized database system includes lead processing core resources of a plurality of computing nodes that are operable to receive sub-segments of segments of segment groups of dataset partitions, each partition including rows of columnar data. The lead processing core resources are operable to divide the sub-segments along columnar lines to produce divisions of data slabs, each data slab corresponding to a column of data. The lead processing core resources are further operable to store first divisions of the data slabs and transmit other divisions of the data slabs to additional processing core resources of the plurality of computing nodes.