Record-Aware Storage Using Rowblocks for Faster Distributed Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed database systems face challenges in managing uneven workloads and transaction conflicts due to the lack of record-level processing and efficient data management, leading to increased computational burden and reduced performance.
Innovation Solution
A record-aware distributed storage system that utilizes rowblocks for storage, implementing copy-on-write technique, transaction conflict detection, and multiversion concurrency control (MVCC) to manage record versions and reduce computational load on database access applications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If distributed database systems use traditional storage methods without record-level processing, then device complexity is reduced, but productivity decreases due to uneven workloads and increased computational burden
Solution Approach 1:
The patent segments data storage into rowblocks (groups of records) that can be independently managed, cached, and processed. This segmentation enables record-level operations without requiring complex distributed coordination for each individual record, thus improving productivity while controlling complexity.
Solution Approach 2:
The patent introduces rowblocks as an intermediary layer between the storage system and database applications. This intermediary enables efficient caching and compression operations at the rowblock level, reducing the computational burden on database nodes while maintaining simplicity in the storage architecture.
2Productivity
If distributed database systems implement record-level processing and transaction conflict detection, then productivity improves through better transaction processing efficiency, but device complexity increases due to additional data management mechanisms
Solution Approach 1:
The patent implements preliminary action by detecting transaction conflicts at the rowblock level before actual record modifications occur. This early detection mechanism improves transaction processing efficiency by avoiding unnecessary record-level operations, while the complexity is contained within the rowblock management layer rather than propagating to each database node.
Solution Approach 2:
The patent uses copying mechanisms to maintain multiple versions of records within rowblocks, enabling efficient MVCC (Multi-Version Concurrency Control). This approach improves productivity by allowing concurrent transactions to access different versions without conflicts, while the copying complexity is managed centrally in the storage system rather than distributed across all nodes.
3Measurement precision
If distributed database systems perform full data scans for transaction conflict detection, then measurement precision of conflicts is improved, but loss of time increases due to excessive computational operations
Solution Approach 1:
The patent segments the data space into rowblocks, enabling conflict detection to operate on smaller, manageable units rather than scanning entire tables. This segmentation maintains measurement precision by ensuring all relevant records are checked within their rowblock, while significantly reducing the time required compared to full data scans across distributed nodes.
Solution Approach 2:
The patent applies local quality by performing conflict detection and data management operations locally at the rowblock level rather than globally across the entire distributed database. This local approach maintains accurate conflict detection for relevant records while minimizing the time and computational resources required, as each node only processes its local rowblocks.
Data Source
AI summary
Requests for records are performed at a record-aware distributed storage system. A request at a storage service engine for records received from a database access application has a time value identified corresponding to a state of a table and storage nodes that store the records. Requests are sent to the storage nodes to obtain the records. The storage nodes may identify rowblocks with respective record identifier ranges and time value ranges corresponding to the records and the time value for the access request. A result may be returned to the database access application based on the records received from the storage nodes.