Cross-Region File Replication with B-Tree Checkpoint Recovery

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing file storage replication technologies face challenges in maintaining data consistency and efficiency during failure events, particularly in cross-region replication scenarios, due to issues like system crashes, thread failures, and data corruption, which hinder effective recovery and performance.

Innovation Solution

The solution involves dividing a B-tree into multiple key ranges with independent checkpoints and parallel processing, using generation numbers to prevent data corruption, and employing a concise manifest file format for efficient recovery, ensuring data consistency and scalability.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If parallel processing is used to improve replication performance, then productivity is improved, but data consistency and reliability deteriorate due to failure events

Engineering Contradiction:

Improvereplication performanceVSAvoiddata consistency

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The B-tree is divided into multiple key ranges, with each range processed by a separate processing thread. Independent checkpoints are created for each key range, allowing parallel processing while maintaining data consistency through isolated recovery points.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Checkpoints are created in advance for each key range before processing begins. These pre-established checkpoints serve as recovery points that allow the system to resume from a consistent state if a failure occurs during parallel processing.

Inventive Principle:

Principle #10Preliminary action

2Reliability

If checkpoints are used to maintain data consistency during failure, then reliability is improved, but loss of time increases due to recovery overhead

Engineering Contradiction:

Improvedata consistencyVSAvoidrecovery time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

Instead of creating a single checkpoint for the entire B-tree, the system creates multiple independent checkpoints for different key ranges. This segmentation allows recovery to proceed in parallel across multiple ranges simultaneously, reducing overall recovery time while maintaining data consistency.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system creates checkpoint copies of key ranges that can be independently processed and recovered. These checkpoint copies enable parallel recovery operations without interfering with each other, thereby reducing the total time required to restore data consistency after a failure.

Inventive Principle:

Principle #26Copying

3Reliability

If the entire B-tree is processed by a single processing thread to ensure data consistency, then reliability is improved, but productivity deteriorates due to sequential processing

Engineering Contradiction:

Improvedata consistencyVSAvoidprocessing speed

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The B-tree is segmented into multiple key ranges, each assigned to a separate processing thread. Independent checkpoints are established for each range, enabling parallel processing while maintaining data consistency through isolated recovery points for each segment.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system transitions from a single-threaded sequential processing model to a multi-threaded parallel processing model by adding the dimension of concurrent key range processing. Each thread operates independently on its assigned key range with its own checkpoint, achieving both parallelism and consistency.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

4Productivity

If multiple processing threads are used to improve performance, then productivity is improved, but data corruption risk increases due to failure events

Engineering Contradiction:

Improveprocessing performanceVSAvoiddata corruption risk

Core Design Contradiction:

ProductivityVSObject-affected harmful factors

Solution Approach 1:

The B-tree is divided into separate key ranges with independent checkpoints for each range. This segmentation isolates potential data corruption to specific key ranges, preventing it from affecting the entire B-tree and enabling targeted recovery without compromising overall data integrity.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system converts the potential harm of thread failures into a benefit by using independent checkpoints for each key range. When a thread fails, the isolation provided by separate checkpoints allows the failed thread to be restarted without affecting other threads, turning a potential data corruption risk into an opportunity for clean recovery.

Inventive Principle:

Principle #22Blessing in disguise (Convert harm into benefit)

Data Source

PatentUS12579109B2Techniques for maintaining data consistency during disaster recovery

Publication Date: 2026.03.17 ORACLE INT CORP

AI summary

Techniques are described for maintaining data consistency when failure events occur during file storage replications between file systems in different cloud infrastructure regions. In certain embodiments, two generation numbers (or different identifications) are assigned to two groups of processed B-tree key-value pairs, one before and one after a failure event, within a key range. In some embodiments, the two generation numbers are assigned to a group of B-tree key-value pairs processed by a failed thread and another group of B-tree key-value pairs processed by a substitute thread taking over the failed thread to avoid potential data corruption.