Storage Backend Transformer for Block-File Data Transfer

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The challenge lies in efficiently transferring data between block and file storage systems, particularly when heavy IO operations impact storage performance, and existing solutions often cause negative performance issues due to the need for cloud gateways to read data from storage sides.

Innovation Solution

A method and system that utilize a storage backend transformer below the cache level to transfer data by determining extents and strides, copying them into a stride buffer, and transmitting block extent files to a remote server, thereby reducing the impact on host IO bandwidth and enabling faster data transfer.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If cloud gateways read data from storage sides to transfer data between block and file storage systems, then data transfer functionality is achieved, but storage I/O bandwidth is consumed and performance deteriorates

Engineering Contradiction:

Improvedata transfer efficiencyVSAvoidstorage I/O bandwidth consumption

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The patent extracts the data transfer functionality from the host system and implements it at the storage array level through a storage backend transformer. This separation removes the I/O burden from the host system, allowing data transfers between block and file storage systems without consuming host I/O bandwidth resources.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent introduces a storage backend transformer as an intermediary component within the storage array that handles data format conversion and transfer operations. This intermediary processes data locally at the storage level, eliminating the need for host system involvement in the actual data transfer operations.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If data transfer operations are performed through host or remote servers, then data can be moved between storage systems, but the process is slow and impacts overall system performance

Engineering Contradiction:

Improvedata transfer speedVSAvoiddata transfer time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent performs preliminary actions by pre-processing data at the storage array level before transfer, including format conversion and organization. This preliminary processing eliminates the need for time-consuming operations at the host or remote server level during actual data transfer.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent replaces the traditional mechanical approach of host-system-mediated data transfer with an automated storage-level transformation system. The storage backend transformer automatically handles data conversion and transfer without requiring host system intervention, significantly reducing transfer time.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS10701134B2Transferring data between block and file storage systems

Publication Date: 2020.06.30 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

Provided is a method, computer program product, and system for transferring data between block and file storage systems. A remote server may receive, from a host device, a request to restore data to the host device. The remote server may store the data as one or more objects, with each object corresponding to an extent of a logical volume on the host device. A set of strides on the host device that correspond to the one or more objects may be determined using metadata for the one or more objects. Each of the one or more objects may be split into a set of data pieces using the metadata. Each data piece may then be transmitted from the remote server to the host device.