Data storage array scaling method and system with minimal data movement

Abstract

A method for rearranging data blocks in a data storage system when adding new storage devices to create an expanded data storage system. A temporary configuration is selected for which the exchange of one or more data blocks between the temporary configuration and the source configuration produces the destination configuration before identifying and exchanging data blocks between the temporary configuration and the source configuration to produce the destination configuration. A single data element transfer chain embodiment provides superior performance in an embodiment that maintains (does not reduce) the source array data storage efficiency at the destination array after scaling. When adding a single new device to an existing array, the required data movement is minimized and does not exceed the capacity of the new device.

Description

BACKGROUND OF THE INVENTION[0001]This invention relates generally to scaling with minimal data movement in large data storage systems having a plurality of storage blocks organized as stripes with redundancy and, more specifically, to a method for expanding a data storage array by adding new storage blocks while conserving state during the minimal data movements required to reorganize the expanded data storage system.[0002]In a data storage subsystem storing data for a computer system, throughput and reliability are important system requirements. A Redundant Array of Inexpensive or Independent Disks (herein denominated RAID or merely “array”) system meets these requirements. Viewed from a host computer, a RAID system, having a plurality of hard disk drive devices (herein denominated HDDs or merely “disks”), operates as a single logical disk. For example, a RAID-5 system is characterized in that data and corresponding parity data are stored together to improve the reliability. That i...

AI Technical Summary

Benefits of technology

[0029]Various embodiments of the system and method of this invention solve one or more of these problems by introducing a method for rearranging data blocks in a data storage system when adding new storage devices to create an expanded data storage system. When adding a single new device to an existing array, the required data movement is minimized and does not exceed the capacity of the new device. In one embodiment, the theoretical minimum data movement is sufficient (less than the capacity of one disk per new disk added) for any number of disks in the source array. In another aspect, the use of a strong array code (sufficient Hamming distance), such as (3+3P), permits the new capacity to be made available instantly, without waiting for completion of the data movement. In yet another aspect, a single data element transfer chain embodiment provides superior performance. In another aspect, data transfer performance is sharply increased using an unexpectedly advantageous embodiment that retains (does not reduce) the source array data storage efficiency at the destination array after scaling.

Problems solved by technology

The auxiliary storage backup is usually accomplished regularly during normal operation but even if no time is required to update the backup files, the rearrangement is very time-consuming, requiring hours to complete.

Despite this feature, completing the data movement still requires a very long time (many hours) because almost every block in the array must be moved.

Such conventional methods have the following problems.

Because RAID systems are expected to be always available, the down-time required for reconfiguration and initialization of the RAID system is a serious problem.

Also, the requisite auxiliary storage of large capacity for the temporary back-up of data adds to the cost of the data storage system.

First, the distribution of data reduces the HDD accesses per HDD for a larger the number of HDDs constituting the RAID.

Secondly, the complexity of controlling a plurality of separate RAID systems lowers overall data storage system performance.

Finally, the “data storage efficiency” is decreased from 80% in FIG. 2A to 75% in FIG. 1B, which is generally undesirable in the storage arts.

Standard RAID arrays, such as RAID-5 and RAID-6 make this a tedious prospect.

During this expansion (scaling) process, the existing data may be available from an in-situ copy but the new data storage system capacity is unavailable to the host computer until the process is completed.

Images