Parallel Database Row Durability Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face limitations in processing speed due to hardware constraints, data storage methods, and restricted co-processing options, which affect the efficiency of data processing and query execution.
Innovation Solution
The implementation of a parallelized database system architecture that includes sub-systems for data input, storage, retrieval, and query processing, utilizing multiple computing devices and processing core resources to divide and process data in parallel, thereby optimizing storage and processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional database systems are used with sequential processing, then hardware simplicity is maintained, but processing speed and efficiency are limited
Solution Approach 1:
The database system is divided into multiple processing core resources (first processing core resource, second processing core resource, etc.), each capable of independently processing different storage transactions in parallel. This segmentation enables concurrent execution of multiple operations without requiring a single complex sequential processor, thereby improving processing speed while distributing system complexity across multiple simpler units.
Solution Approach 2:
The patent transitions from sequential single-core processing to parallel multi-core processing by adding a temporal dimension to the processing architecture. Multiple storage transactions are executed simultaneously across different processing core resources rather than one after another, effectively moving from a 1D sequential model to a 2D parallel model that improves throughput and processing speed.
2Reliability
If data is stored in traditional formats without row durability tracking, then storage simplicity is maintained, but data integrity and transaction reliability are compromised
Solution Approach 1:
A row durability data structure is introduced as an intermediary component that tracks the durability state of individual rows within storage transactions. This mediator structure enables the system to maintain detailed integrity information without requiring complete reprocessing or complex verification mechanisms, thereby improving reliability while managing storage structure complexity through targeted tracking.
Solution Approach 2:
Instead of applying uniform durability tracking to entire transactions or databases, the patent implements row-level durability tracking where only specific rows that require integrity monitoring are tracked with detailed durability data. This local quality approach applies enhanced reliability measures selectively to individual rows rather than universally, improving data integrity for critical rows while avoiding unnecessary complexity across the entire storage system.
3Productivity
If parallel processing architecture is implemented, then processing efficiency is improved, but system complexity and coordination overhead increase
Solution Approach 1:
Each processing core resource is equipped with autonomous capabilities to generate and manage its own row durability data independently without requiring constant coordination or communication with other processing cores. This self-service approach allows parallel processing to proceed efficiently while minimizing the coordination overhead that would otherwise be required to manage shared durability tracking state across multiple processors.
Data Source
AI summary
A record processing and storage system is operable to generating a set of pages that includes corresponding records included in a plurality of row data via a plurality of processing resources based on each processing resource in the plurality of processing resources generating a corresponding subset of the set of pages. A single storage transaction is performed to store the set of pages based on writing the corresponding records included in the ones of the plurality of row data to a page storage system. Based on completing the single storage transaction, row durability data indicating a least favorably ordered row number of a plurality of row numbers corresponding to the plurality of row data is generated and transmitted to a computing device.