Local SSD Transaction Log Replicas for Low-Latency Cloud Databases
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face limitations in scalability, availability, and latency due to the coupling of compute and storage functions, particularly in cloud architectures, which affect write operations in online transactional processing (OLTP) workloads.
Innovation Solution
A network system with a compute service generating transaction logs, a log replica service replicating logs to local SSDs, and a page server service updating remote storage, utilizing a shared nothing architecture to decouple compute and storage, enabling local log replicas across multiple availability zones for reduced latency and resilience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If storage is remote from compute function, then scalability and availability are improved, but write latency increases significantly
Solution Approach 1:
The system segments the storage function into multiple independent log replica services distributed across different servers, each with local SSD storage. This allows the compute service to write to multiple remote locations simultaneously, achieving both scalability and reduced latency through parallel operations.
Solution Approach 2:
The patent introduces a new dimension by implementing log replicas across multiple availability zones and servers. Instead of a single remote storage location, the system creates a distributed architecture where data is replicated across multiple dimensions (servers, zones), enabling parallel write operations while maintaining availability.
2Speed
If a single machine is dedicated to DBMS with both compute and storage, then write latency is reduced, but database size is limited by hardware constraints
Solution Approach 1:
The system separates compute and storage functions into different services running on different servers. The compute service generates transaction logs while log replica services store them on separate servers with SSDs, enabling the database to scale beyond single-machine hardware constraints while maintaining fast write performance through distributed parallel operations.
Solution Approach 2:
The log replica service is designed as a universal component that can be deployed on multiple servers and availability zones. It handles both storage and replication functions, allowing the system to expand database size by adding more replica nodes without increasing compute complexity.
3Reliability
If local log replicas are deployed across multiple availability zones, then resilience to failures is improved, but system complexity increases
Solution Approach 1:
The system segments the log storage function across multiple independent log replica services distributed in different availability zones. Each replica service operates independently with its own SSD storage, providing resilience through geographic distribution while managing complexity through standardized service templates.
Solution Approach 2:
The patent implements copying by creating multiple identical log replica services across different availability zones. Each replica contains the same functionality and storage mechanism, providing failover capability through simple replication rather than complex active-active synchronization.
Data Source
AI summary
The present application relates to database management in a cloud environment. A compute service generates a transaction log for database actions. The compute service writes at least a first portion of the transaction log to a log replica service. The log replica service replicates at least the first portion of the transaction log to two or more local log replicas persisted on solid state drives. A page server service updates database data in a remote storage based on at least a second portion of the transaction log retrieved from at least one of the log replicas. The page server service moves at least the second portion of the transaction log to the remote storage. The log replica service truncates the local log replicas by purging the second portion.