Transaction Segmentation for Reliability-Throughput Trade-off
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional transactional data processing systems face performance issues due to rigid ACID properties, which can slow system performance, especially when dealing with short-lived data items and high throughput, and existing systems struggle to manage transactions spanning multiple reliability levels effectively.
Innovation Solution
The method involves analyzing transactions to determine the reliability levels of operations, dividing them into sub-transactions based on these levels, and dispatching each sub-transaction to a persistence manager for execution, allowing for varying degrees of persistence and flexibility in data management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If rigid ACID properties are enforced for all transactions, then data integrity is guaranteed, but system performance and throughput deteriorate
Solution Approach 1:
The patent segments transactions into sub-transactions based on reliability levels. Each sub-transaction is assigned a specific reliability level (persistent, non-persistent, or semi-persistent), allowing different durability guarantees for different parts of the same transaction. This segmentation enables the system to apply ACID properties only where necessary while improving overall throughput.
Solution Approach 2:
The patent applies different reliability qualities to different operations within a transaction based on local requirements. Each operation can be marked with its own reliability level, allowing the system to enforce strict durability only for critical operations while using more performance-friendly approaches for non-critical operations.
2Adaptability or versatility
If varying reliability levels are permitted for different operations, then system flexibility and throughput improve, but transaction management complexity increases
Solution Approach 1:
The patent introduces dynamic transaction management where the reliability level of each operation can be determined at runtime based on data item characteristics and system state. The system dynamically adjusts persistence requirements, creating sub-transactions with appropriate durability guarantees rather than using a static one-size-fits-all approach.
Solution Approach 2:
The patent changes the reliability parameter of operations based on the reliability level of the data items they operate on. By associating reliability levels with data items and propagating these levels to operations, the system automatically adjusts transaction durability parameters without requiring complex manual configuration.
3Reliability
If all operations in a transaction are hardened to persistent store, then durability is ensured, but processing speed and concurrency deteriorate
Solution Approach 1:
The patent applies partial hardening to persistent storage by only persisting operations that require durability guarantees. Non-critical operations can be processed and committed more quickly without full persistent storage hardening, achieving sufficient durability with faster processing speeds through selective persistence.
Data Source
AI summary
Method, apparatus, computer program, and computer program product for managing the durability of a plurality of transactions, each transaction including a plurality of operations, each operation for applying to a data item, each operation having a reliability level associated therewith. Each transaction is divided into one or more sub-transactions based upon reliability level of the operations. Sub-transactions are then dispatched to a persistence manager for execution against a persistent store.