Database Workload Isolation via Quota-Based Throttling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face challenges in ensuring concurrent and isolated execution of transactions in distributed environments, leading to performance degradation and unfair resource utilization among OLTP databases hosted on the same data cluster.
Innovation Solution
The implementation of a concurrency control mechanism combining multi-version concurrency control (MVCC) for read operations and locking for write operations, along with targeted isolation levels, ensures that each transaction executes against a different snapshot of the database, and write locks are held until transaction commit, while quotas are specified for transaction workloads to manage throughput and prevent resource saturation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple OLTP databases share the same data cluster to improve resource utilization, then productivity increases, but performance degradation and unfair resource utilization occur due to lack of isolation
Solution Approach 1:
The system segments the data cluster into isolated workgroups, where each workgroup functions as an independent unit with dedicated resource allocation. This segmentation prevents interference between different OLTP databases while maintaining shared infrastructure, thus improving both productivity through resource sharing and reliability through isolation.
Solution Approach 2:
The system applies different quality characteristics to different parts of the system by allowing each workgroup to have customized isolation levels and resource quotas. This enables each OLTP database to receive appropriate resource guarantees and isolation characteristics tailored to its specific performance requirements, resolving the contradiction between shared resources and performance stability.
2Ease of operation
If user code executes directly on the database system to improve ease of operation, then security risks and performance degradation occur due to lack of isolation
Solution Approach 1:
The system introduces workgroups as intermediary layers between user code and the core database system. These workgroups act as isolated execution environments that mediate between user applications and database operations, enabling easy code execution while preventing security risks through enforced isolation boundaries.
Solution Approach 2:
The system changes the isolation parameter by allowing configurable isolation levels within workgroups. This enables user code to execute with appropriate security constraints while maintaining ease of operation, as users can specify the desired level of isolation based on their security requirements and performance needs.
3Productivity
If concurrent transactions are allowed to improve productivity, then transaction conflicts and data consistency issues occur
Solution Approach 1:
The system dynamically adjusts concurrency control mechanisms based on workload characteristics and isolation level requirements. By making the concurrency control strategy adaptive rather than static, the system can maintain high transaction throughput while ensuring data consistency, as the isolation mechanisms are applied dynamically based on actual transaction needs.
Solution Approach 2:
The system changes isolation parameters to resolve conflicts between concurrent transactions. By allowing configurable isolation levels and dynamically adjusting them based on workload patterns, the system maintains data consistency while maximizing transaction throughput, thus resolving the contradiction between productivity and manufacturing precision.
4Productivity
If resource allocation is shared to improve productivity, then unfair resource utilization and performance degradation occur among tenants
Solution Approach 1:
The system segments resource allocation by creating isolated workgroups for each tenant, where resources are shared within the workgroup but isolated between workgroups. This segmentation enables fair resource utilization by ensuring that one tenant's workload cannot unfairly impact another tenant's performance, while still maintaining productivity through resource sharing within isolated boundaries.
Solution Approach 2:
The system applies different resource allocation characteristics to different workgroups, allowing each tenant to have customized resource guarantees and isolation levels. This local quality approach ensures fair resource utilization by tailoring resource allocation to each tenant's specific needs while maintaining overall system productivity through efficient resource sharing.
Data Source
AI summary
The subject technology receives, by a proxy interface, the request to determine the read version of the transaction. The subject technology determines, based on the request, that the transaction is associated with the tag included in the request. The subject technology generates a sequence number for the request. The subject technology places the request in a queue associated with the tag based on the sequence number, the queue including a set of requests to determine a particular read version of a particular transaction. The subject technology determines, using a token bucket, that the request in the queue should be throttled based on information related to a quota for the tag stored in a distributed database key store.