Transaction Commit Indexing for Consistent Concurrent Snapshots
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Concurrent transactions in cloud-based data management systems often result in stale data due to the difficulty in ascertaining a current data value, as transactions are manipulated across multiple devices, leading to inconsistent data snapshots.
Innovation Solution
Maintain an index value indicating the highest transaction identifier without intermediary transactions, ensuring only transactions with identifiers less than or equal to this index are considered for data reads, thereby preventing the introduction of stale data during concurrent commits.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If concurrent transactions are processed across multiple devices, then productivity is improved, but data consistency deteriorates
Solution Approach 1:
The system pre-assigns monotonically increasing transaction identifiers to transactions before they are committed. This preliminary action establishes a global ordering of transactions that can be used later to determine visibility, allowing concurrent commits without sacrificing consistency. The identifier assignment happens in advance, enabling parallel processing while maintaining a deterministic order for conflict resolution.
Solution Approach 2:
The patent introduces an intermediary mechanism (the transaction identifier and commit queue index) that mediates between concurrent transactions and data reads. This intermediary allows the system to process transactions concurrently while providing a consistent view to readers by filtering which committed transactions are visible based on the index, thus resolving the contradiction between concurrency and consistency.
2Measurement precision
If all committed transactions are made visible, then data freshness is improved, but stale data introduction increases
Solution Approach 1:
The system pre-establishes a commit queue index that tracks the highest transaction identifier that should be visible to readers. This preliminary tracking mechanism allows the system to selectively make transactions visible based on their identifiers, ensuring that only consistent snapshots are exposed while maintaining data freshness within the bounds of consistency.
Solution Approach 2:
The patent changes the parameter of transaction visibility by introducing an identifier-based filtering mechanism. Instead of making all committed transactions immediately visible, the system uses the commit queue index as a parameter to control which transactions are visible, thereby preventing stale data while maintaining freshness for valid transactions.
3Reliability
If transaction commits are serialized, then data consistency is improved, but processing speed deteriorates
Solution Approach 1:
The patent segments the commit process into independent units that can be processed in parallel. Each transaction is assigned an identifier and added to a commit queue independently, allowing multiple transactions to be committed concurrently across different devices. The segmentation enables parallel processing while the identifier-based ordering maintains consistency, resolving the contradiction between serialization and throughput.
Solution Approach 2:
The system dynamically manages the commit queue index, allowing it to advance as transactions are committed. This dynamic approach enables the system to adapt to the rate of concurrent commits while maintaining consistency, rather than using a static serialized approach. The index dynamically reflects the current state of committed transactions, enabling both concurrency and consistency.
Data Source
AI summary
The present disclosure relates to a system and techniques for preventing corruption of snapshot data by limiting the visibility of committed data. To do this, the system may maintain an index that indicates the highest transaction identifier value such that no future commits will have a transaction identifier less than or equal to the indexed transaction identifier value. In embodiments, if a read is performed, only transactions having a transaction identifier less than or equal to the index value can be read. Each time that a transaction is committed, the index value is updated to the transaction identifier for the transaction having the highest transaction identifier without any intermediary transactions.