Remote Data Copy Using Diff Bitmaps for Parallel Node Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing dual node storage appliances with an AO-ANO path configuration face limitations in efficiently performing direct data copies or transfers in a replication environment, as they lack the ability to leverage the advantages of an AO-AO path configuration for simultaneous data servicing.
Innovation Solution
The disclosed techniques enable a dual node appliance to receive and service storage IO requests over optimized paths at both nodes simultaneously, involving the generation and utilization of diff bitmaps with varying granularities to perform parallel copy operations for subsets of data slices owned by different nodes, thereby optimizing data transfers in an AO-AO path configuration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single source node performs direct data copy to a remote destination node in a dual node appliance with AO-ANO path configuration, then the replication process is simple to implement, but data transfer efficiency is limited because only one node can service storage IO requests
Solution Approach 1:
The patent segments the data copy operation by dividing the storage object into multiple slices, with each slice owned by a specific node. This allows different nodes to perform copy operations on different slices simultaneously, transforming a single-node sequential operation into a multi-node parallel operation, thereby improving data transfer efficiency without requiring complex centralized coordination
Solution Approach 2:
The patent transitions from a single-node copy operation to a multi-node parallel copy operation by utilizing the node dimension. Each node independently performs copy operations on its owned slices, adding a dimensional aspect of parallelism to the replication process, which significantly enhances throughput without proportionally increasing coordination complexity
2Productivity
If a dual node appliance uses AO-AO path configuration to receive storage IO requests at both nodes simultaneously, then data servicing capability is improved, but the ability to perform direct data copies is compromised due to ownership ambiguity
Solution Approach 1:
The patent applies local quality by assigning specific ownership of data slices to specific nodes. Each node has local authority and responsibility for the slices it owns, which resolves the ownership ambiguity issue in AO-AO configuration. This localized ownership model enables nodes to independently perform copy operations on their owned slices without requiring complex coordination, thus maintaining ease of operation while leveraging the enhanced data servicing capability of AO-AO configuration
Solution Approach 2:
Each node autonomously performs copy operations on its owned slices without requiring centralized coordination or intervention from other nodes. This self-service approach simplifies the copy operation process in AO-AO configuration, as each node independently manages its own data replication, making the system easier to operate while fully utilizing the parallel data servicing capability
3Productivity
If diff bitmaps are obtained and exchanged between nodes for parallel copy operations, then copy efficiency is improved through parallel processing, but communication overhead increases due to bitmap exchange
Solution Approach 1:
The patent extracts only the necessary change information (diff bitmaps) from the source node and transmits it to the destination node, rather than transferring all data. This selective extraction of change metadata enables parallel copy operations at the destination node while minimizing communication overhead, as only compact bitmap representations of data changes need to be exchanged, not the actual data blocks
Data Source
AI summary
Techniques for optimizing a remote data copy using multiple storage nodes. The techniques include receiving a copy command for a volume at a first node, a first subset of VOL slices being owned by the first node, and a second subset of VOL slices being owned by a second node. The techniques include obtaining, by the first node, respective diff bitmaps for the first subset of slices and the second subset of slices. The techniques include sending the diff bitmap for the second subset of slices to the second node. The techniques include performing, by the first node, a first copy operation based on the diff bitmap for the first subset of slices, and performing, by the second node, a second copy operation based on the diff bitmap for the second subset of slices, the second copy operation being performed at least partially in parallel with the first copy operation.