Transactional Database Layer Adds ACID to Distributed Key/Value Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud-based NoSQL databases lack full support for ACID transactions, multiple indexes, and query isolation, limiting their functionality compared to traditional database technologies.
Innovation Solution
Implementing a local computing system with a database engine that interacts with a remote key/value store to perform ACID transactions by using transaction state stored as key/value pairs, allowing transactions to span multiple round trips and cross storage account boundaries, and supporting snapshot isolation for readers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If cloud-based NoSQL databases are used for extreme scalability, then scalability is improved, but transaction support and ACID compliance deteriorate
Solution Approach 1:
The patent segments the database system into a distributed key/value store layer (providing scalability) and a transactional database layer (providing ACID compliance). The transactional layer sits above the key/value store and manages transactions across multiple keys and accounts, while the key/value store handles scalable data storage. This segmentation allows each layer to optimize for its specific function without compromising the other.
2Productivity
If cloud-based NoSQL databases are used for extreme scalability, then scalability is improved, but query isolation and multiple indexes deteriorate
Solution Approach 1:
The patent adds a new dimension to the data model by introducing accounts as a hierarchical layer above individual keys. This allows queries to be isolated at the account level while maintaining the scalable key/value store underneath. Multiple indexes can be created on different fields within accounts without affecting the underlying storage structure, enabling versatile querying while preserving scalability.
3Reliability
If traditional database technologies are used for full-featured functionality, then transaction support and query isolation are improved, but scalability deteriorates
Solution Approach 1:
The patent introduces an intermediary transactional database layer that mediates between the application and the distributed key/value store. This intermediary layer implements ACID transaction protocols, manages multiple indexes, and provides query isolation, while the underlying key/value store maintains its scalable architecture. The intermediary translates high-level transactional operations into low-level key/value operations.
Data Source
AI summary
A storage system for transactional data storage includes a key/value store comprising a first plurality of rows and a second plurality of rows, each of the first plurality of rows including a first key associated with a respective row of a database and a transaction state associated with the respective row, and each of the second plurality of rows including a second key associated with a respective row of the database and a data version of the respective row. The storage system is configured to provide a transaction state to and receive an updated transaction state from a plurality of different systems, each comprising a database engine to perform transactional database operations on the key/value store.


