Cluster Snapshot Workflow Using SAN Barriers and PIT Markers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current storage systems face inefficiencies in creating snapshots of volumes, particularly in ensuring consistency and managing multiple snapshots across a cluster, which complicates data management and recovery processes.
Innovation Solution
A snapshot creation workflow is administered by a Storage Area Network (SAN) administration layer, utilizing barriers and point-in-time markers to delimit and record I/O requests, ensuring crash consistency and efficient creation of snapshots for individual logical units (LUNs) or groups of LUNs organized as consistency groups, through a 2-phase transaction protocol and updates to cluster database records.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a snapshot is created for a single LUN, then the snapshot creation is simple and fast, but the consistency across multiple LUNs cannot be ensured
Solution Approach 1:
The patent combines multiple LUN snapshot operations into a single consistency group snapshot workflow. The SAN administration layer coordinates barrier installation, PIT marker creation, and table updates across all LUNs in the consistency group as a unified transaction, ensuring crash consistency while managing complexity through centralized coordination.
Solution Approach 2:
The snapshot creation process is divided into distinct phases: barrier installation phase, PIT marker creation phase, and table update phase. Each phase is executed as a separate transaction that must complete successfully before proceeding to the next phase, ensuring consistency while simplifying error handling and rollback operations.
2Reliability
If barriers are installed for all LUNs in a consistency group, then crash consistency is ensured, but the snapshot creation time increases
Solution Approach 1:
The barrier installation is performed as a preliminary action before PIT marker creation. By installing barriers first and verifying their successful installation before proceeding to the next phase, the system ensures that all subsequent operations will be captured consistently. This preliminary barrier installation enables parallel processing of subsequent operations.
Solution Approach 2:
Once barriers are installed and PIT markers are created, the system continuously processes table updates and snapshot creation without interruption. The workflow maintains continuous progress through coordinated transactions, ensuring that all LUNs in the consistency group are processed together while minimizing idle waiting time.
3Reliability
If a 2-phase transaction protocol is used for snapshot creation, then data consistency is ensured, but the operational complexity increases
Solution Approach 1:
The SAN administration layer acts as an intermediary that manages the 2-phase transaction protocol. It coordinates between the persistence layer (which installs barriers and creates PIT markers) and the volume layer (which creates snapshots and updates tables). This intermediary abstraction hides the transactional complexity from individual components while ensuring consistent data management across the system.
Data Source
AI summary
A technique efficiently creates a snapshot for a logical unit (LUN) served by a storage input/output (I/O) stack executing on a node of a cluster that organizes data as extents referenced by keys. In addition, the technique efficiently creates one or more snapshots for a group of LUNs organized as a consistency group (CG) and served by storage I/O stacks executing on a plurality of nodes of the cluster. To that end, the technique involves a plurality of indivisible operations (i.e., transactions) of a snapshot creation workflow administered by a Storage Area Network (SAN) administration layer (SAL) of the storage I/O stack in response to a snapshot create request issued by a host. The SAL administers the snapshot creation workflow by initiating a set of transactions that includes, inter alia, (i) installation of barriers for LUNs (volumes) across all nodes in the cluster that participate in snapshot creation, (ii) creation of point-in-time (PIT) markers to record those I/O requests that are included in the snapshot, and (iii) updating of records (entries) in snapshot and volume tables of a cluster database (CDB).