Cluster Database Data Block Transfer with Delayed Log Flush

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

In cluster database systems, existing methods delay data block transfers until all redo records are durably stored, which can lead to inefficiencies and potential data loss if the source instance crashes before writing its redo records, affecting transaction consistency and recovery processes.

Innovation Solution

Implementing a mechanism where data blocks can be transferred without waiting for redo records to be durably stored, using block change timestamps and dependency timestamp arrays to manage and ensure the durability of changes, allowing conditional transfers and subsequent durability checks to guarantee data integrity.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If data block transfers are delayed until all redo records are durably stored, then data integrity and transaction consistency are ensured, but transfer efficiency and system productivity deteriorate

Engineering Contradiction:
Improvedata integrityVSAvoidtransfer efficiency
Core Design Contradiction:
ReliabilityVSProductivity

Solution Approach 1:

The patent applies preliminary action by transferring data blocks before all redo records are durably stored. The system performs the transfer operation in advance and then completes the durability requirement later, allowing the transfer to proceed without unnecessary delay while still ensuring data integrity through subsequent durability checks and conditional transfers.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent changes the parameter of transfer timing from strict sequential (wait for durability) to conditional (transfer when ready, with durability checked later). By introducing a conditional transfer mechanism where the source instance can transfer data blocks at any time and the destination instance performs durability checks, the system optimizes transfer efficiency while maintaining reliability through parameter modification rather than rigid constraint.

Inventive Principle:
Principle #35Parameter changes

2Productivity

If data blocks are transferred without waiting for redo durability, then transfer efficiency improves, but the risk of data loss increases if source instance crashes

Engineering Contradiction:
Improvetransfer efficiencyVSAvoiddata loss risk
Core Design Contradiction:
ProductivityVSReliability

Solution Approach 1:

The patent implements feedback by having the destination instance receive and process durability information from the sourceinstance. The source instance provides feedback about which redo records are durably stored, and the destinationinstance uses this feedback to determine whether to apply changes conditionally. This feedback mechanism allows efficient transfers while managing data loss risk through informed conditional application of changes.

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

The patent applies beforehand cushioning by preparing the destinationinstance to handle potential data loss scenarios. The destinationinstance receives durability information in advance and is configured to conditionally apply changes only when durability is confirmed or to roll back if durability fails. This preparatory cushioning protects against data loss without requiring synchronous durability completion.

Inventive Principle:
Principle #11Beforehand cushioning (Prior cushioning)

3Reliability

If all redo records must be written to storage before transfer, then transaction consistency is guaranteed, but system response time and processing speed worsen

Engineering Contradiction:
Improvetransaction consistencyVSAvoidresponse time
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The patent applies preliminary action by performing data block transfers before the durability of all redo records is complete. The system initiates the transfer operation in advance and continues with other processing activities while durability is being established, thereby reducing response time and improving system performance while still maintaining transaction consistency through conditional transfer mechanisms.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS9563521B2Data transfers between cluster instances with delayed log file flush
Publication Date: 2017.02.07 ORACLE INT CORP
  • US9563521B2 patent drawing
  • US9563521B2 patent drawing
  • US9563521B2 patent drawing

AI summary

Techniques for processing changes in a cluster database system are provided. A first instance in the cluster transfers a data block to a second instance in the cluster before a redo record that stores one or more changes that the first instance made to the data block is durably stored. The first instance also transfers, to the second instance, a block change timestamp that indicates when a redo record for the one or more changes was generated by the first instance. The first instance also separately sends, to the second instance, a last store timestamp that indicates when the last redo record that was durably stored was generated by the first instance. The block change timestamp and the last store timestamp are used by the second instance when creating redo records for changes (made by the second instance) that depend on the redo record generated by the first instance.