Cluster Database Data Block Transfer with Delayed Log Flush
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In cluster database systems, existing methods delay data block transfers until all redo records are durably stored, which can lead to inefficiencies and potential data loss if the source instance crashes before writing its redo records, affecting transaction consistency and recovery processes.
Innovation Solution
Implementing a mechanism where data blocks can be transferred without waiting for redo records to be durably stored, using block change timestamps and dependency timestamp arrays to manage and ensure the durability of changes, allowing conditional transfers and subsequent durability checks to guarantee data integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data block transfers are delayed until all redo records are durably stored, then data integrity and transaction consistency are ensured, but transfer efficiency and system productivity deteriorate
Solution Approach 1:
The patent applies preliminary action by transferring data blocks before all redo records are durably stored. The system performs the transfer operation in advance and then completes the durability requirement later, allowing the transfer to proceed without unnecessary delay while still ensuring data integrity through subsequent durability checks and conditional transfers.
Solution Approach 2:
The patent changes the parameter of transfer timing from strict sequential (wait for durability) to conditional (transfer when ready, with durability checked later). By introducing a conditional transfer mechanism where the source instance can transfer data blocks at any time and the destination instance performs durability checks, the system optimizes transfer efficiency while maintaining reliability through parameter modification rather than rigid constraint.
2Productivity
If data blocks are transferred without waiting for redo durability, then transfer efficiency improves, but the risk of data loss increases if source instance crashes
Solution Approach 1:
The patent implements feedback by having the destination instance receive and process durability information from the sourceinstance. The source instance provides feedback about which redo records are durably stored, and the destinationinstance uses this feedback to determine whether to apply changes conditionally. This feedback mechanism allows efficient transfers while managing data loss risk through informed conditional application of changes.
Solution Approach 2:
The patent applies beforehand cushioning by preparing the destinationinstance to handle potential data loss scenarios. The destinationinstance receives durability information in advance and is configured to conditionally apply changes only when durability is confirmed or to roll back if durability fails. This preparatory cushioning protects against data loss without requiring synchronous durability completion.
3Reliability
If all redo records must be written to storage before transfer, then transaction consistency is guaranteed, but system response time and processing speed worsen
Solution Approach 1:
The patent applies preliminary action by performing data block transfers before the durability of all redo records is complete. The system initiates the transfer operation in advance and continues with other processing activities while durability is being established, thereby reducing response time and improving system performance while still maintaining transaction consistency through conditional transfer mechanisms.
Data Source
AI summary
Techniques for processing changes in a cluster database system are provided. A first instance in the cluster transfers a data block to a second instance in the cluster before a redo record that stores one or more changes that the first instance made to the data block is durably stored. The first instance also transfers, to the second instance, a block change timestamp that indicates when a redo record for the one or more changes was generated by the first instance. The first instance also separately sends, to the second instance, a last store timestamp that indicates when the last redo record that was durably stored was generated by the first instance. The block change timestamp and the last store timestamp are used by the second instance when creating redo records for changes (made by the second instance) that depend on the redo record generated by the first instance.


