Transaction Grouping for Overwrite Merge in Key-Value Stores
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In key value stores, frequent updates to the same key result in unnecessary transactions due to conflicting key value updates, leading to inefficiencies and potential data inconsistency, especially during crashes where multiple transactions need to be rolled back to ensure data consistency.
Innovation Solution
Implement transaction grouping and inline overwrite merging, where transactions with common key value updates are grouped and assigned transaction IDs, allowing only linked transactions in the same group to be reattempted after a crash, and delaying metadata writes until all operations in a group are completed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple transactions are processed sequentially in a pending queue, then data consistency is maintained, but write efficiency deteriorates due to unnecessary repeated updates to the same key
Solution Approach 1:
The patent merges conflicting transactions that update the same key into a single transaction. When multiple transactions in the pending queue target the same key, they are combined so that only one update operation is executed, eliminating redundant writes while preserving the final consistent state of the data
2Reliability
If all transactions are processed to ensure data consistency after a crash, then reliability is improved, but loss of time increases due to rolling back and reattempting multiple transactions
Solution Approach 1:
By merging conflicting transactions into a single transaction before execution, the patent ensures that if a crash occurs, only one transaction needs to be rolled back and reattempted rather than multiple separate transactions, significantly reducing crash recovery time while maintaining data consistency
3Productivity
If transactions with common key updates are grouped together, then write efficiency is improved by eliminating redundant updates, but device complexity increases due to transaction grouping and merging logic
Solution Approach 1:
The patent segments the pending queue into groups of transactions that update the same keys. By organizing transactions into these segments and processing them as unified units, the system reduces redundant writes while the complexity is managed through structured grouping rather than uncontrolled complexity
Data Source
AI summary
Provided is a method of data storage, the method including identifying a plurality of transactions in a pending queue, the transactions having one or more key value updates respectively corresponding to a plurality of keys, identifying a commonly associated key of the plurality of keys associated with commonly associated key value updates of the key value updates belonging to different ones of the transactions, respectively assigning transaction group IDs to the transactions based on respective transaction IDs assigned to the transaction group IDs, grouping the transactions into a respective transaction group of a plurality of transaction groups based on the assigned transaction group ID, and merging conflicting data writes corresponding to the commonly associated key value updates of the commonly associated key for grouped transactions of the transactions that are in a same one of the transaction groups.