Distributed Database Replication via Segmented Log and Page Stores
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed database systems face challenges in managing data storage efficiently, particularly in balancing data persistency and read availability with increasing costs and latency due to the need for multiple storage nodes.
Innovation Solution
Implementing different replication policies for page stores and log stores, where log records are synchronously written to log store replicas and asynchronously written to page store replicas, allowing for high read availability with a relatively small number of page store replicas.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the number of storage nodes is increased to improve data persistency and read availability, then the reliability and availability of the database system are improved, but the cost and device complexity increase
Solution Approach 1:
The patent segments the storage system into two distinct components: page store replicas dedicated to serving read requests and log store replicas dedicated to maintaining data persistency. This segmentation allows the system to achieve high reliability through multiple log store replicas while using fewer page store replicas, thereby reducing overall device complexity and cost while maintaining data persistency.
2Reliability
If the number of storage nodes is increased to improve read availability, then the availability of the database system is improved, but the cost and device complexity increase
Solution Approach 1:
The patent separates read operations from write operations by dedicating page store replicas to read requests and log store replicas to write operations. This allows the system to achieve high read availability through multiple page store replicas while using a separate set of log store replicas for persistency, optimizing the balance between availability and device complexity.
Solution Approach 2:
The log store replicas serve multiple functions: they maintain data persistency, enable recovery operations, and support read availability. This multi-functionality allows the system to achieve high read availability without proportionally increasing the number of storage nodes, as log store replicas contribute to both persistency and availability.
3Reliability
If the number of storage nodes is increased to ensure data persistency, then the reliability of the database system is improved, but the cost increases
Solution Approach 1:
The patent divides the storage system into page store replicas and log store replicas with distinct roles. Page store replicas store actual data pages for read operations, while log store replicas store log records for recovery and persistency. This segmentation ensures that storage capacity is used efficiently, with log store replicas providing persistency guarantees without duplicating full page data, thereby reducing overall storage costs.
4Reliability
If synchronous replication is used for page store replicas to ensure consistency, then the reliability is improved, but the latency and processing time increase
Solution Approach 1:
The patent segments replication operations by type: log store replicas receive synchronous replication to ensure data consistency and persistency, while page store replicas receive asynchronous replication to reduce latency. This segmentation allows the system to maintain reliability through synchronous log replication while improving response time through asynchronous page replication.
Solution Approach 2:
The patent applies synchronous replication selectively only to log store replicas where data consistency is critical, rather than applying it to all replicas. This partial application of synchronous replication maintains necessary reliability while avoiding the latency penalty across the entire system, as page store replicas use faster asynchronous replication.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method includes receiving an indication of a change to a page of a database and adding a new log record corresponding to the page to a common log comprising log records, the new log record describing the change made to the page and assigned a distinct version number. The method further includes synchronously writing the new log record to each log store replica in a set of log store replicas, and asynchronously writing the new log record to all page store replicas for the page to update the page that is stored on each of the page store replicas, where each store replica for the page serves reads for the page. In response to receiving, from a predetermined number of the page store replicas, an acknowledgement of the writing of the log record, discarding the new log record from the common log.