Coordinated Checkpoint Replication for Distributed Data Integrity

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing storage systems face challenges in efficiently managing data across distributed storage environments, particularly in ensuring data integrity and availability during failures, and in optimizing operations without redundant write processes.

Innovation Solution

Implementing checkpoint-based replication with coordinated snapshots and erasure coding across multiple storage systems, utilizing non-volatile random access memory (NVRAM) for fast data buffering and proactive data rebuilding, and employing a distributed architecture with authorities for data management and redundancy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If checkpoint-based replication is implemented across distributed storage systems, then data availability and integrity are improved during failures, but system complexity and coordination overhead increase

Engineering Contradiction:

Improvedata availabilityVSAvoidsystem complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The system divides the distributed storage environment into distinct storage systems, each independently capable of maintaining checkpoints. This segmentation allows each system to manage its own replication state without requiring complex global coordination, thereby improving reliability while controlling system complexity.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system performs preliminary actions by detecting when a first storage system has completed a write operation to a first data structure before initiating replication to the second storage system. This preliminary detection mechanism ensures data integrity is established before replication begins, improving reliability without requiring complex real-time coordination during failure events.

Inventive Principle:

Principle #10Preliminary action

2Reliability

If coordinated snapshots are implemented across multiple storage systems, then data integrity is improved, but storage operation time and synchronization overhead increase

Engineering Contradiction:

Improvedata integrityVSAvoidstorage operation time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system detects completion of write operations to the first data structure before initiating snapshot replication. This preliminary detection ensures that only consistent, completed data is replicated, maintaining data integrity while avoiding the time penalty of waiting for full synchronization confirmation.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system creates snapshots as copy operations rather than requiring complex coordinated updates. By copying data structures between storage systems, the patent maintains integrity through simple replication semantics, reducing the time overhead compared to coordinated update protocols.

Inventive Principle:

Principle #26Copying

3Reliability

If erasure coding is implemented for data redundancy, then data availability is improved, but computational overhead and processing time increase

Engineering Contradiction:

Improvedata availabilityVSAvoidprocessing speed

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent applies erasure coding by dividing data into segments and distributing encoded portions across multiple storage systems. This segmentation allows parallel processing of encoding operations and enables efficient recovery by reconstructing data from available segments, thereby maintaining high data availability while minimizing processing overhead through distributed computation.

Inventive Principle:

Principle #1Segmentation

4Productivity

If asynchronous checkpoint writing is implemented, then application performance is improved, but data consistency and integrity become more difficult to ensure

Engineering Contradiction:

Improveapplication performanceVSAvoiddata consistency

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system performs preliminary detection of write operation completion before initiating asynchronous replication. By detecting when the first storage system has completed writing to the first data structure, the patent ensures data consistency is established before the asynchronous process begins, maintaining reliability while allowing the application to continue with improved performance.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements feedback mechanisms where the completion status of write operations is detected and used to trigger replication actions. This feedback loop ensures that asynchronous checkpoint writing maintains data consistency by only replicating confirmed, completed writes, thereby preserving reliability while enabling performance optimization.

Inventive Principle:

Principle #23Feedback

Data Source

PatentEP4424000B1Coordinated checkpoints among storage systems implementing checkpoint-based replication

Publication Date: 2025.07.02 PURE STORAGE INC

Figure 1A
Figure 1B
Figure 1C

AI summary

Coordinated checkpoints among storage systems implementing checkpoint-based replication, including orchestrating one or more coordinated lightweight checkpoints for a source dataset stored across two or more source storage systems; and coordinating a replication of the one or more coordinated lightweight checkpoints from the two or more source storage systems to two or more target storage systems.