Metadata Packet Replication for Storage High Availability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data replication methods in storage area networks (SANs) incur substantial memory utilization and data transmission bandwidth overhead by replicating entire data streams, which can disrupt high availability during replicator failures.
Innovation Solution
Replicating metadata instead of data, allowing for efficient memory use and reduced bandwidth requirements, with replicators using queues to store data and metadata until it is persisted at a target site, enabling continuous data protection without compromising high availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If entire data streams are replicated across multiple replicators, then high availability is provided, but memory utilization and data transmission bandwidth overhead increase substantially
Solution Approach 1:
The patent extracts only the essential metadata from the data stream for replication, rather than replicating entire data streams. Metadata packets contain LUN identifiers and region information that identify which data blocks need to be replicated, allowing the system to replicate only necessary data portions while significantly reducing memory utilization overhead.
Solution Approach 2:
The patent segments the data replication process into two distinct packet types: metadata packets (containing LUN and region identifiers) and data packets (containing actual data blocks). This segmentation allows replicators to handle metadata and data separately, optimizing memory usage by storing only metadata in queues while data flows through the replication pipeline.
2Reliability
If entire data streams are replicated across multiple replicators, then high availability is provided, but data transmission bandwidth overhead increases substantially
Solution Approach 1:
The patent extracts and transmits only metadata packets (containing LUN and region identifiers) separately from data packets. This extraction reduces data transmission bandwidth overhead because metadata packets are significantly smaller than full data streams, while still providing the information needed for high availability replication.
Solution Approach 2:
By segmenting transmission into metadata packets and data packets, the system can optimize bandwidth usage. Metadata packets are transmitted to multiple replicators to establish replication paths, while actual data transmission occurs only through the primary replicator, reducing overall bandwidth consumption compared to replicating entire data streams to all replicators.
3Quantity of substance
If metadata is replicated instead of data, then memory and bandwidth requirements are reduced, but system complexity increases
Solution Approach 1:
The patent creates a universal metadata packet format that serves multiple functions: identifying data blocks, tracking replication status, enabling failover, and managing queue operations. This multi-functional metadata structure reduces overall system complexity by consolidating multiple control functions into a single standardized packet type that works across all replicators.
Solution Approach 2:
The metadata packet acts as an intermediary between the data stream and the replication management system. It carries essential information (LUN, region identifiers) that mediates the replication process, allowing replicators to make decisions about data replication without directly processing entire data streams, thus simplifying the control logic while maintaining reduced memory requirements.
4Reliability
If replicators use queues to store data and metadata, then continuous data protection is enabled, but memory utilization increases
Solution Approach 1:
The patent segments queue storage requirements by placing only metadata packets in replicator queues, while data packets flow through the replication pipeline without being queued at intermediate replicators. This segmentation dramatically reduces memory utilization in queues while maintaining continuous data protection capabilities, as the metadata queue enables failover without requiring storage of entire data streams.
Data Source
AI summary
A method for data replication may include obtaining data, generating a data packet including the data and a logical unit number (LUN) corresponding to a logical unit of a storage array, generating a metadata packet including the LUN and metadata that identifies a region of the logical unit, transmitting the data packet and the metadata packet to a first replicator, transmitting the metadata packet to a second replicator, and transmitting the data packet to a target site.