Distributed Consistency Group Splitter for VM IO Latency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The interception of Input/Outputs (IOs) by a splitter in virtual machine (VM) environments introduces latency and reduces throughput, as IOs are not written to disk until an acknowledgement is received from the Data Protection Appliance (DPA), limiting the performance of high-demand VM applications.

Innovation Solution

Implementing a distributed consistency group (CG) with multiple DPAs, where IOs are distributed by a splitter across multiple grid-CGs, allowing each grid-CG to operate independently and reducing latency by eliminating the need for a single DPA acknowledgement, thereby increasing throughput.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If a single DPA is used to intercept and process IOs, then data protection consistency is maintained, but latency increases and throughput decreases

Engineering Contradiction:

Improvedata protection consistencyVSAvoidIO latency

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent divides a single consistency group into multiple sub-consistency groups (sub-CGs), each handled by a separate DPA. This segmentation allows parallel processing of IOs across multiple DPAs, reducing the latency bottleneck of a single DPA while maintaining overall data protection consistency through coordinated bookmarking across all sub-CGs.

Inventive Principle:

Principle #1Segmentation

2Device complexity

If a single DPA processes all IOs, then system complexity is low, but throughput is limited

Engineering Contradiction:

Improvesystem complexityVSAvoidIO throughput

Core Design Contradiction:

Device complexityVSProductivity

Solution Approach 1:

The consistency group is segmented into multiple sub-CGs distributed across multiple DPAs, enabling parallel IO processing that increases throughput. The splitter distributes IOs to appropriate sub-CGs, and coordinated bookmarking maintains consistency, achieving high throughput with managed complexity.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a new dimension of distribution by organizing DPAs into a grid structure with multiple rows and columns, where each DPA handles specific sub-CGs. This dimensional organization allows scalable throughput improvement while maintaining manageable system complexity through structured distribution.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

3Reliability

If IOs are written to disk only after DPA acknowledgement, then data protection is ensured, but performance of high-demand VM applications is limited

Engineering Contradiction:

Improvedata protectionVSAvoidVM application performance

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

By segmenting the consistency group into multiple sub-CGs handled by different DPAs, the system can process and acknowledge IOs in parallel across multiple DPAs. This reduces the total time for IO completion while maintaining data protection, thereby improving VM application performance without sacrificing reliability.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11671465B2Distributed consistency group at splitter

Publication Date: 2023.06.06 EMC IP HLDG CO LLC

AI summary

One example method includes intercepting an IO that is directed to a volume, writing the IO to a grid-CG that is one of a plurality of grid-CGs that collectively form a distributed CG, adding the IO to a stream journal associated with the grid-CG, receiving a bookmark request, in response to the bookmark request, inserting a bookmark in the stream journal, incrementing a splitter counter and broadcasting a splitter counter value to the grid-CGs in the distributed CG, and inserting the bookmark in respective stream journals of the grid-CGs in the distributed CG.