Read Replica Cache Invalidation for Consistent Data Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face challenges in distributing databases while maintaining ACID properties, leading to costly and complex deployments, especially in scaling out read processing, where read replicas operate with lag and incur data loss upon promotion.
Innovation Solution
Implementing a system with read replicas that receive write requests, send redo log records to a distributed storage service, and invalidate cached data, allowing read replicas to retrieve current data from storage instead of cache, enabling seamless conversion to primary nodes without data loss.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If read replicas are used to scale out read processing, then read capacity is improved, but data consistency and availability are worsened due to lag and data loss upon promotion
Solution Approach 1:
The patent segments the database system into primary nodes and read replica nodes, each with specific responsibilities. The primary node handles write operations and generates redo logs, while read replicas handle read operations and maintain cached copies of data. This segmentation allows read capacity to be scaled out without compromising the reliability of the primary node, as each segment operates independently with clearly defined roles.
Solution Approach 2:
The patent implements preliminary action by maintaining redo logs on read replicas before potential promotion to primary node. The read replicas continuously receive and store redo logs from the primary node, ensuring that if a promotion is needed, the data is already synchronized and no data loss occurs. This preliminary synchronization action eliminates the lag and data loss issues associated with traditional read replica models.
2Productivity
If traditional read replica models are used, then read scaling is improved, but network traffic and data synchronization overhead are worsened
Solution Approach 1:
The patent extracts the redo log transmission function from the traditional data synchronization mechanism. Instead of continuously synchronizing entire data sets or frequent updates, the system only transmits compact redo log records from primary nodes to read replicas. This extraction of the essential change information (redo logs) reduces network traffic significantly while maintaining read scaling capabilities.
Solution Approach 2:
The patent uses copying by creating read replicas that maintain cached copies of data and redo logs from the primary node. These copies allow read operations to be served locally without continuous network communication, reducing network traffic overhead. The copying mechanism ensures that read scaling is achieved while minimizing the energy loss associated with data synchronization.
3Device complexity
If read replicas operate with asynchronous log shipping, then system complexity is reduced, but data loss occurs upon promotion to primary node
Solution Approach 1:
The patent applies preliminary action by ensuring that read replicas continuously receive and store redo logs from the primary node before any potential failure or promotion event. This continuous preliminary synchronization ensures that when a read replica needs to be promoted to primary node, it already has all the necessary redo logs and can resume operations without data loss, while maintaining the simplicity of asynchronous operation.
Solution Approach 2:
The patent implements feedback by having read replicas continuously monitor and receive redo logs from the primary node, maintaining an up-to-date state. This feedback mechanism ensures that the read replica is always synchronized with the primary node's state, eliminating data loss upon promotion while keeping the system complexity low through the use of simple log shipping and cache invalidation protocols.
Data Source
AI summary
A database system may receive a write request that specifies a modification to be made to a particular data record stored by the database system. A log record representing the modification to be made to the particular data record may be sent to a storage service of the database system. An indication (e.g., log record or other indication) that indicates a cached version of the particular data record stored in a read replica's cache is stale may be sent to a read replica. For a subsequent read of the particular data record received by the read replica, the read replica may request the particular data record from the storage service.