Self-Descriptive Data Buffer for Wire-Efficient Replication

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data replication and migration processes in storage arrays with data efficiency features, such as deduplication and compression, face inefficiencies due to the need for multiple API calls and increased latency, especially when data is not decompressed or rehydrated locally, and the complexity of connecting metadata with data.

Innovation Solution

The implementation of self-descriptive data blocks using a prefix descriptor in a page buffer that indicates attributes like deduplication, compression, and pattern recognition, allowing for wire-efficient IO operations through a single API, reducing the need for additional metadata retrieval APIs and simplifying metadata connection.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If data efficiency features (deduplication, compression) are implemented in storage arrays, then data storage efficiency is improved, but replication and migration processes become more complex due to multiple API calls and increased latency

Engineering Contradiction:

Improvedata storage efficiencyVSAvoidreplication process complexity

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

The patent combines metadata and data into a single self-descriptive data structure. The prefix descriptor containing attributes (deduplication status, compression status, pattern information) is merged with the actual data block, eliminating the need for separate metadata retrieval operations and simplifying the replication process.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The prefix descriptor acts as an intermediary that carries all necessary information about the data block's efficiency features. This intermediary structure enables the replication system to understand and handle compressed or deduplicated data without requiring complex coordination between multiple APIs.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Loss of energy

If data is not decompressed or rehydrated locally during replication, then network bandwidth is saved, but the system cannot verify data integrity or apply further efficiency features

Engineering Contradiction:

Improvenetwork bandwidth consumptionVSAvoiddata integrity verification

Core Design Contradiction:

Loss of energyVSReliability

Solution Approach 1:

The prefix descriptor is prepared in advance with all necessary information about the data block's characteristics. This preliminary structuring allows the replication system to make informed decisions about data handling without needing to decompress or rehydrate the data first, maintaining both bandwidth efficiency and verification capability.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The prefix descriptor provides feedback information about the data block's state (compression ratio, deduplication status, pattern matches). This feedback enables the replication system to verify data integrity and determine whether further efficiency features can be applied without actually decompressing the data.

Inventive Principle:

Principle #23Feedback

3Measurement precision

If multiple API calls are used to retrieve metadata and process data efficiency features, then data processing accuracy is improved, but operation latency increases

Engineering Contradiction:

Improvedata processing accuracyVSAvoidoperation latency

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent merges metadata retrieval and data processing into a single operation by embedding the prefix descriptor directly with the data block. This eliminates the need for sequential API calls to retrieve metadata first, then process the data, thereby reducing latency while maintaining processing accuracy.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The data structure is segmented into a prefix descriptor portion and a data payload portion. This segmentation allows the system to quickly access critical metadata information in the prefix without needing to process the entire data block, reducing operation latency while maintaining accuracy.

Inventive Principle:

Principle #1Segmentation

4Measurement precision

If separate metadata structures are used to track deduplication and compression information, then data tracking accuracy is improved, but system resource consumption increases

Engineering Contradiction:

Improvedata tracking accuracyVSAvoidsystem resource consumption

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent combines separate metadata structures into a single integrated prefix descriptor that tracks all data efficiency features (deduplication, compression, patterns). This merging reduces the total amount of metadata storage required while maintaining comprehensive tracking accuracy through a compact, self-contained structure.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS11100018B2System and method for supporting wire efficient replication using self-descriptive data buffer

Publication Date: 2021.08.24 EMC IP HLDG CO LLC

AI summary

A method, computer program product, and computer system for receiving, by a computing device, an IO request. It may be determined that data of the IO request includes an indication of an attribute. A prefix descriptor may be constructed for the data in a self-descriptive page buffer based upon, at least in part, determining that the data of the IO request includes the indication of the attribute. An appropriate Application Programming Interface (API) may be called to process the IO request based upon, at least in part, the prefix descriptor.