Partitioned Database Query Processing for Lock-Free Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face limitations in processing speed due to hardware constraints, data storage methods, and restricted co-processing options, leading to inefficiencies in data processing and query execution.
Innovation Solution
A database system architecture that employs parallelized data input, storage, and query processing subsystems, utilizing Single Producer Single Consumer (SPSC) buffers and partitioned memory allocation to enable lock-free and parallel execution of operations across multiple processing core resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional database systems use sequential processing and locking mechanisms, then data consistency is maintained, but processing speed and throughput are limited
Solution Approach 1:
The patent segments the database into multiple partitions that can be processed independently across different processing core resources. Each partition can be accessed by specific consumers without requiring locks, enabling parallel processing while maintaining consistency through partition-based isolation. This segmentation allows simultaneous operations on different data partitions without interference.
Solution Approach 2:
The patent introduces a mediator mechanism that coordinates access to shared resources without using traditional locking. The mediator manages producer-consumer relationships and data flow between partitions, enabling consistent data access patterns while allowing parallel execution. This intermediary layer resolves conflicts without requiring explicit locks on data structures.
2Productivity
If database systems implement parallel processing, then processing speed increases, but system complexity increases
Solution Approach 1:
The patent implements a universal data structure that serves multiple functions: it acts as both a data container and a synchronization mechanism. The same data structure is used across all processing core resources, eliminating the need for separate locking mechanisms and coordination systems. This multi-functionality reduces overall system complexity while enabling parallel processing.
Solution Approach 2:
The patent implements self-service mechanisms where processing core resources automatically manage their own data access and coordination. Each resource independently manages its partition access and consumer relationships without requiring external locking or complex coordination protocols. This self-service approach simplifies the system by eliminating centralized control mechanisms.
3Reliability
If traditional locking mechanisms are used, then data consistency is ensured, but execution time for queries increases
Solution Approach 1:
The patent divides data into segments that can be accessed independently by different consumers simultaneously. This segmentation eliminates the need for locking when multiple consumers access different segments, reducing execution time while maintaining consistency through segment isolation. Queries can execute in parallel without waiting for locks.
Solution Approach 2:
The patent performs preliminary actions by pre-partitioning data and pre-establishing consumer assignments before query execution. This preliminary organization allows queries to execute without dynamic locking or coordination during execution, significantly reducing execution time while maintaining data consistency through the pre-established partition structure.
4Speed
If hardware resources are optimized for single-threaded processing, then hardware simplicity is maintained, but processing speed is limited
Solution Approach 1:
The patent segments computational tasks across multiple processing core resources, enabling each core to process independent partitions simultaneously. This segmentation leverages parallel hardware capabilities without requiring complex hardware modifications, improving processing speed through software-based partitioning while keeping hardware architecture relatively simple.
Solution Approach 2:
The patent adds a dimensional layer of parallelism by introducing multiple processing core resources that operate simultaneously on different data partitions. This dimensional change from sequential to parallel processing exploits hardware capabilities without fundamentally altering the core processing architecture, enabling speedup while maintaining hardware simplicity.
Data Source
AI summary
A computing device is operable to receive a plurality of partition allocation requests based on execution of a corresponding plurality of query operations in conjunction with execution of a corresponding query. Processing a first partition allocation request of the plurality of partition allocation requests corresponding to execution of a first operation of the corresponding plurality of query operations is based on allocating a set of partitions of a plurality of partitions. Content is loaded into the set of partitions for access in executing the first operation. Access to the content via the set of partitions is facilitated in conjunction with execution of the first operation. The set of partitions are released based on determining the first operation has completed access of the content in conjunction with the execution of the first operation.