Data movement between storage levels in a cluster storage system based on a predicted data access frequency trend pattern

Abstract

A device includes a processing device configured to generate a predicted data access frequency trend pattern for a storage object over a specified period of time, the storage object being stored in a first storage tier of a cluster storage system. The processing device is further configured to classify the storage object based on the predicted data access frequency trend pattern and to determine a given storage tier in the cluster storage system for storing the storage object during the specified period of time. The processing device is further configured to, in response to the given storage tier being different from the first storage tier, select a type of data movement based on the classification of the storage object and to move the storage object to the given storage tier using the selected type of data movement.

Description

Technical Field

[0001] This field generally relates to information processing, and more specifically, to storage in an information processing system. Background Art

[0002] Storage arrays and other types of storage systems are typically shared by multiple host devices over a network. Applications running on the host devices each include one or more processes that perform the application functionality. Such processes issue input-output (IO) operation requests to be delivered to the storage system. The storage controller of the storage system services such IO operation requests. In some information processing systems, multiple storage systems can be used to form a storage cluster. Summary of the Invention

[0003] Exemplary embodiments of the present disclosure provide techniques for moving data between storage tiers of a cluster storage system based on a predicted data access frequency trend pattern.

[0004] In one embodiment, a device includes at least one processing device, the processing device including a processor coupled to a memory. The at least one processing device is configured to perform the following steps: generate a predicted data access frequency trend pattern for a given storage object over a specified period of time, the given storage object being stored using a first type of storage resource in a given storage system of two or more storage systems in a cluster storage system, the first type of storage resource being associated with a first storage tier of two or more storage tiers in the cluster storage system. The at least one processing device is further configured to perform the following steps: classify the given storage object into a given storage object category of two or more storage object categories at least in part based on the predicted data access frequency trend pattern of the given storage object over the specified period of time; and determine a given storage tier of two or more storage tiers in the cluster storage system for storing the given storage object during the specified period of time at least in part based on the predicted data access frequency trend pattern of the given storage object over the specified period of time. The at least one processing device is further configured to perform the following steps: in response to the given storage tier being different from the first storage tier, select a type of data movement for a second type of storage resource for moving the given storage object to one or more storage systems in the cluster storage system, the second type of storage resource being associated with the given storage tier in the cluster storage system, the selected type of data movement being at least in part based on the given storage object category associated with the given storage object. The at least one processing device is further configured to perform the following steps: move the given storage object to the second type of storage resource of one or more storage systems in the cluster storage system using the selected type of data movement.

[0005] These and other illustrative embodiments include, but are not limited to, methods, devices, networks, systems, and processor-readable storage media. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 is a block diagram of an information processing system for data movement between storage tiers of a cluster storage system based on a predicted data access frequency trend pattern in an illustrative embodiment.

[0007] Figure 2 is a flowchart of an exemplary process for data movement between storage tiers of a cluster storage system based on a predicted data access frequency trend pattern in an illustrative embodiment.

[0008] Figure 3 shows a graph of a storage access data pattern trend in an illustrative embodiment.

[0009] Figure 4 A graph showing the trend of cyclic storage access data patterns in an illustrative embodiment.

[0010] Figure 5 A graph showing the trend of irregular storage access data patterns in an illustrative embodiment.

[0011] Figure 6 A graph showing the trend of reduced storage access data patterns in an illustrative embodiment.

[0012] Figure 7 A table showing the available capacities of different storage arrays on different levels of a multi-level cluster storage system.

[0013] Figure 8 A process flow showing how to optimize the performance of a storage cluster by moving data across storage devices within a storage cluster and across storage arrays in the storage cluster in an illustrative embodiment.

[0014] Figure 9A and Figure 9B A diagram showing data movement between storage arrays and storage levels in a multi-level cluster storage system in an illustrative embodiment.

[0015] Figure 10 and Figure 11 An example of a processing platform that can be utilized to implement at least a portion of an information processing system in an illustrative embodiment. Detailed Description

[0016] Exemplary information processing systems and associated computers, servers, storage devices, and other processing devices will be referred to herein to describe illustrative embodiments. However, it should be understood that the embodiments are not limited to use with the specific illustrative system and device configurations shown. Thus, the term "information processing system" as used herein is intended to be interpreted broadly so as to encompass, for example, processing systems that include cloud computing and storage systems, as well as other types of processing systems that include various combinations of physical and virtual processing resources. Thus, an information processing system can include, for example, at least one data center or other type of cloud-based system that includes one or more cloud-hosted tenants accessing cloud resources.

[0017] Figure 1An information processing system 100 is shown, which is configured to provide functionality for moving data between storage tiers of a cluster storage system based on a predicted data access frequency trend pattern according to an illustrative embodiment. The information processing system 100 includes one or more host devices 102-1, 102-2,......102-N (collectively referred to as host devices 102) that communicate with one or more storage arrays 106-1, 106-2,......106-M (collectively referred to as storage arrays 106) via a network 104. The network 104 may include a storage area network (SAN).

[0018] As Figure 1 shown, the storage array 106-1 includes a plurality of storage devices 108, and each storage device stores data utilized by one or more applications running on the host device 102. The storage devices 108 are illustratively arranged in one or more storage pools. The storage array 106-1 also includes one or more storage controllers 110 that facilitate IO processing of the storage devices 108. The storage array 106-1 and its associated storage devices 108 are examples of what is more generally referred to herein as a "storage system". This storage system in this embodiment is shared by the host device 102 and is therefore also referred to herein as a "shared storage system". In an embodiment where there is only a single host device 102, the host device 102 may be configured to exclusively use the storage system.

[0019] The host device 102 illustratively includes a corresponding computer, server, or other type of processing device capable of communicating with the storage array 106 via the network 104. For example, at least a subset of the host devices 102 may be implemented as corresponding virtual machines of a computing service platform or other type of processing platform. The host device 102 in such an arrangement illustratively provides computing services, such as executing one or more applications on behalf of each of one or more users associated with the corresponding host device among the host devices 102.

[0020] The term "user" herein is intended to be broadly interpreted so as to cover a multitude of arrangements of people, hardware, software, or firmware entities, and combinations of such entities.

[0021] Computing and / or storage services may be provided to users under a platform as a service (PaaS) model, an infrastructure as a service (IaaS) model, and / or a function as a service (FaaS) model, but it should be understood that numerous other cloud infrastructure arrangements may be used. Moreover, the illustrative embodiments may be implemented outside the context of cloud infrastructure, such as in the case of a stand-alone computing and storage system implemented within a given enterprise.

[0022] The storage device 108 of the storage array 106-1 can implement a logical unit (LUN) configured to store objects of a user associated with the host device 102. These objects can include files, blocks, or other types of objects. The host device 102 interacts with the storage array 106-1 using read and write commands and other types of commands transmitted through the network 104. In some embodiments, such commands more specifically include Small Computer System Interface (SCSI) commands, but in other embodiments, other types of commands can be used. As the term "IO operation" is widely used herein, the term illustratively includes one or more such commands. References to terms such as "input-output" and "IO" herein should be understood to refer to input and / or output. Thus, an IO operation involves at least one of input and output.

[0023] Moreover, the term "storage device" as used herein is intended to be interpreted broadly so as to cover, for example, logical storage devices such as LUNs or other logical storage volumes. A logical storage device can be defined in the storage array 106-1 as including different portions of one or more physical storage devices. Thus, the storage device 108 can be regarded as including the corresponding LUN or other logical storage volume.

[0024] The storage device 108 of the storage array 106-1 can be implemented using solid state drives (SSDs). Such SSDs are implemented using non-volatile memory (NVM) devices such as flash memory. Other types of NVM devices that can be used to implement at least a portion of the storage device 108 include non-volatile random access memory (NVRAM), phase change RAM (PC-RAM), and magnetic RAM (MRAM). These and various combinations of multiple different types of NVM devices or other storage devices can also be used. For example, hard disk drives (HDDs) can be used in combination with or in place of SSDs or other types of NVM devices. Thus, numerous other types of electronic or magnetic media can be used to implement at least a subset of the storage device 108.

[0025] In Figure 1In the information processing system 100, it is assumed that the storage array 106 is part of the storage cluster 105 (for example, where the storage array 106 can be used to implement one or more storage nodes in a cluster storage system including multiple storage nodes interconnected by one or more networks), and it is assumed that the host device 102 submits an IO operation to be processed by the storage cluster 105. It is assumed that at least one of the storage controllers in the storage array 106 (for example, the storage controller 110 of the storage array 106-1) implements functionality for intelligent data movement across the storage devices 108 of the storage array 106-1 (for example, between different storage devices in the storage device 108 or portions thereof providing different storage levels in the storage cluster 105) and between the storage array 106-1 and one or more other storage arrays among the storage arrays 106-2 to 106-M. Such intelligent data movement functionality is provided via the storage object access trend classification module 112 and the storage object movement module 114.

[0026] As described above, it is assumed that Figure 1 in the implementation of, the storage array 106 is part of the storage cluster 105. It is assumed that the storage cluster 105 provides or implements multiple different storage levels of a multi-level storage system. For example, a given multi-level storage system can include a fast level or performance level implemented using flash storage devices or other types of SSDs, and a capacity level implemented using HDDs, where one or more such levels may be server-based. As will be apparent to those skilled in the art, in other implementations, a variety of other types of storage devices and multi-level storage systems can be used. The specific storage devices used in a given storage level can vary according to the specific requirements of a given implementation, and multiple different storage device types can be used within a single storage level. As previously indicated, the term "storage device" as used herein is intended to be interpreted broadly and can thus cover, for example, SSDs, HDDs, flash drives, hybrid drives, or other types of storage products and devices or portions thereof, and illustratively includes logical storage devices such as LUNs.

[0027] It should be understood that a multi-level storage system can include more than two storage levels, such as one or more "performance" levels and one or more "capacity" levels, where the performance levels illustratively provide increased IO performance characteristics relative to the capacity levels, and the capacity levels are illustratively implemented using storage that is relatively lower in cost compared to the performance levels. There can also be multiple performance levels, each providing a different level of service or performance as needed, or there can be multiple capacity levels.

[0028] The storage object access trend classification module 112 is configured to classify storage objects (e.g., one or more files, directories, file systems, LUNs, etc.) stored on the storage devices of the storage array 106-1. In some embodiments, the storage objects are classified into one of two categories. The first category of storage objects includes storage objects having an IO temperature trend during a defined time period, and the storage objects are predicted to be above a high watermark threshold or below a low watermark threshold (e.g., it is predicted that the IO temperature of the storage objects in the first category is "hot" or "cold" during the defined time period). The second category of storage objects includes storage objects having an IO temperature trend during a defined time period, and the storage objects are predicted to be between the high watermark threshold and the low watermark threshold (e.g., it is predicted that the IO temperature of the storage objects in the second category is not "hot" or "cold" during the defined time period).

[0029] The storage object movement module 114 is configured to determine the type of data movement to be used for the storage objects at least in part based on the classification provided by the storage object access trend classification module 112. For example, the storage objects in the first category can be considered suitable for movement or migration within the storage array 106-1 and between the storage array 106-1 and one or more other storage arrays among the storage arrays 106-2 to 106-M (e.g., because they have a relatively stable "hot" or "cold" predicted access trend during the defined time period, and thus are less likely to require frequent movement during the defined time period). In other words, the storage objects in the first category can be moved via the internal data movement functionality within the storage array or using the cluster-level or inter-storage-array data movement functionality.

[0030] The storage objects in the second category can be considered unsuitable for movement or migration between the storage array 106-1 and one or more other storage arrays among the storage arrays 106-2 to 106-M (e.g., because they do not have a relatively stable "hot" or "cold" predicted access trend during the defined time period, and thus may require frequent movement during the defined time period). However, the storage objects in the second category can be considered suitable for movement or migration within the storage array 106-1 (e.g., using the internal data movement functionality within the storage array), which has a lower cost (e.g., less resource-intensive) than the cluster-level or inter-storage-array data movement functionality.

[0031] Then, when such a movement is applicable, the storage object movement module 114 will move the storage object using the data movement type associated with the category associated with it. For example, consider a given storage object currently stored on a storage device in a storage device 108 of the storage array 106-1, where the storage device provides storage for a first storage tier, but the predicted data access frequency of the given object over a defined time period corresponds to storage in a second storage tier different from the first storage tier or is reasonably stored in the second storage tier (e.g., the given storage object may currently be stored in a portion of a storage device that provides storage for a capacity tier, but its predicted data access frequency over a defined time period is high enough that it is reasonable to store it in a performance tier). In this case, it is desired to move the given storage object to storage associated with the second storage tier.

[0032] As described above, the internal data movement functionality within a storage array is generally less costly or resource-intensive compared to the cluster-level or inter-storage-array data movement functionality. Therefore, the storage object movement module 114 can determine whether there is available capacity in one or more of the storage devices 108 of the storage array 106-1 that provide storage for the second storage tier. If so, the storage object movement module 114 can use the internal data movement functionality within the storage array to move the storage object. If the storage array 106-1 does not have available capacity in one or more of the storage devices 108 that provide storage for the second tier, then the category of the given storage object is identified to determine whether the predicted data access trend of the given storage object over a defined time period justifies using the cluster-level or inter-storage-array data movement functionality. Thus, if the given storage object has a first category, it can be moved to a storage device on another storage array among the storage arrays 106-2 to 106-M that has available storage in the second storage tier. If the given storage object has a second category, the cluster-level or inter-storage-array data movement functionality will not be used to move it.

[0033] There may be multiple storage objects whose predicted data access trends during a defined time period do not match their current storage tier. In such cases, the multiple storage objects can be "ranked" based on the difference between their predicted access trends and their current storage locations. As an example, consider a multi-tier storage system with three storage tiers (capacity, performance, and highest performance). If there are two storage objects in storage array 106-1 that are currently stored in the capacity tier but have predicted data access trends that would justify storage in the highest performance tier, both of these storage objects would benefit from being moved. However, if there is not enough available capacity to move both of these storage objects to the highest performance tier, the storage object with the higher predicted data access trend can be moved to the highest performance tier, and the storage object with the lower predicted data access trend can remain in its original location (or potentially be moved to the performance tier if the performance tier has available capacity).

[0034] The ranking of storage objects described above can also take into account the classification of two storage objects. As an example, consider that a first storage object of two storage objects has a first category (and is thus suitable for intra-array internal data movement or inter-array data movement), while a second storage object of the two storage objects has a second category (and is thus suitable for intra-array internal data movement but not suitable for inter-array data movement). Assume that both the first storage object and the second storage object are currently on storage array 106-1, the predicted data access trend of the second storage object is lower than that of the first storage object, the available capacity of storage array 106-1 is not sufficient to move both the first storage object and the second storage object to be stored in the highest performance tier in storage device 108, and there is sufficient available capacity in one or more other storage arrays among storage arrays 106-2 to 106-M to store the storage objects in the highest performance tier. Given the above, it is possible to accommodate the movement of both the first storage object and the second storage object by using inter-array data movement functionality to move the first storage object with the first category to one or more other storage arrays among storage arrays 106-2 to 106-M and using intra-array data movement functionality to move the second storage object with the second category within storage array 106-1.

[0035] Although at Figure 1In an embodiment, the storage object access trend classification module 112 and the storage object movement module 114 are shown to be implemented inside the storage array 106-1 and outside the storage controller 110. However, in other embodiments, one or both of the storage object access trend classification module 112 and the storage object movement module 114 may be at least partially implemented inside the storage controller 110, or at least partially implemented outside the storage array 106-1, such as being implemented in one of the host devices 102, one or more of the other storage arrays 106-2 to 106-M, or one or more servers outside the host device 102 and the storage array 106 (e.g., including being implemented on a cloud computing platform or other types of information technology (IT) infrastructure), etc. Additionally, although Figure 1 is not shown, the other storage arrays among the storage arrays 106-2 to 106-M may implement corresponding instances of the storage object access trend classification module 112 and the storage object movement module 114.

[0036] At least some portions of the functionality of the storage object access trend classification module 112 and the storage object movement module 114 may be at least partially implemented in the form of software stored in a memory and executed by a processor.

[0037] Assume that at least one processing platform is used to implement Figure 1 the host device 102 and the storage array 106 in the embodiment, where each processing platform includes one or more processing devices, and each processing device has a processor coupled to a memory. Such processing devices may illustratively include a specific arrangement of computing, storage, and network resources. For example, in some embodiments, the processing device is at least partially implemented using virtual resources such as virtual machines (VMs) or Linux containers (LXCs), or a combination of both, as in an arrangement where Docker containers or other types of LXCs are configured to run on a VM.

[0038] Although the host device 102 and the storage array 106 may be implemented on corresponding different processing platforms, numerous other arrangements are also possible. For example, in some embodiments, one or more of the host device 102 and one or more of the storage array 106 are implemented on the same processing platform. Thus, one or more of the storage arrays 106 may be at least partially implemented within at least one processing platform that implements at least one subset of the host device 102.

[0039] Network 104 can be implemented using multiple different types of networks to interconnect storage system components. For example, network 104 can include a SAN that is part of a global computer network such as the Internet, but other types of networks can be part of a SAN, including wide area networks (WANs), local area networks (LANs), satellite networks, telephone or cable networks, cellular networks, wireless networks such as WiFi or WiMAX networks, or various portions or combinations of these and other types of networks. Thus, in some embodiments, network 104 includes a combination of multiple different types of networks, each network including processing devices configured to communicate using Internet Protocol (IP) or other related communication protocols.

[0040] As a more specific example, some embodiments can utilize one or more high-speed local area networks, where the associated processing devices communicate with each other using peripheral component interconnect high-speed (PCIe) cards of those devices and networking protocols such as InfiniBand, Gigabit Ethernet, or Fibre Channel. As will be appreciated by those skilled in the art, numerous alternative networking arrangements are possible in a given embodiment.

[0041] Although in some embodiments, certain commands used by host device 102 to communicate with storage array 106 illustratively include SCSI commands, in other embodiments, other types of commands and command formats can be used. For example, some embodiments can utilize command features and functionality associated with Non-Volatile Memory Express (NVMe) to implement I / O operations, as described in Revision 1.3 of the NVMe Specification of May 2017, which is incorporated herein by reference. Other storage protocols of this type that can be utilized in the illustrative embodiments disclosed herein include: fabric-based NVMe, also known as NVMeoF; and TCP-based NVMe, also known as NVMe / TCP.

[0042] Assume that in this embodiment, storage array 106-1 includes persistent memory implemented using flash memory or other types of non-volatile memory of storage array 106-1. More specific examples include NAND-based flash memory or other types of non-volatile memory such as resistive RAM, phase change memory, spin torque transfer magnetoresistive RAM (STT-MRAM), and Intel Optane based on 3D XPoint TM memory TMApparatus. Further assume that the persistent memory is separate from the storage devices 108 of the storage array 106, but in other embodiments, the persistent memory may be implemented as one or more designated portions of one or more of the storage devices in the storage device 108. For example, in some embodiments, the storage device 108 may include a flash-based storage device, such as in embodiments involving all-flash storage arrays, or may be implemented in whole or in part using other types of non-volatile memory.

[0043] As mentioned above, the communication between the host device 102 and the storage array 106 may utilize a PCIe connection or other types of connections implemented through one or more networks. For example, illustrative embodiments may use interfaces such as Internet Small Computer System Interface (iSCSI), Serial Attached SCSI (SAS), and Serial ATA (SATA). In other embodiments, numerous other interfaces and associated communication protocols may be used.

[0044] In some embodiments, the storage array 106 may be implemented as part of a cloud-based system.

[0045] Therefore, it should be understood that the term "storage array" as used herein is intended to be interpreted broadly and may encompass multiple different instances of commercially available storage arrays.

[0046] In illustrative embodiments, other types of storage products that may be used to implement a given storage system include software-defined storage devices, cloud storage devices, object-based storage devices, and scale-out storage devices. In illustrative embodiments, a combination of multiple storage types among these and other storage types may also be used to implement a given storage system.

[0047] In some embodiments, the storage system includes a first storage array and a second storage array arranged in an active-active configuration. For example, such an arrangement may be used to ensure that data stored in one of the storage arrays is replicated to the other storage array using a synchronous replication process. Such replication of data across multiple storage arrays may be used to facilitate fault recovery in the system 100. Thus, one of the storage arrays may operate as a production storage array relative to the other storage array, and the other storage array may operate as a backup or recovery storage array.

[0048] However, it should be understood that the embodiments disclosed herein are not limited to an active-active configuration or any other specific storage system arrangement. Therefore, the illustrative embodiments herein may be configured using a variety of other arrangements, including, for example, active-passive arrangements, active-active Asymmetric Logical Unit Access (ALUA) arrangements, and other types of ALUA arrangements.

[0049] These and other storage systems can be part of what is more generally referred to herein as a processing platform, which includes one or more processing devices, each of which includes a processor coupled to a memory. A given such processing device can correspond to one or more virtual machines or other types of virtualized infrastructure (such as Docker containers or other types of LXC). As indicated above, communication between such elements of system 100 can occur over one or more networks.

[0050] The term "processing platform" as used herein is intended to be interpreted broadly so as to encompass (by way of illustration and not limitation) multiple sets of processing devices configured to communicate over one or more networks and one or more associated storage systems. For example, a distributed implementation of host device 102 is possible, where some of the host devices in host device 102 reside in one data center at a first geographic location, while other host devices in host device 102 reside in one or more other data centers at one or more other geographic locations that may be remote from the first geographic location. Storage array 106 can be implemented at least in part at the first geographic location, the second geographic location, and one or more other geographic locations. Thus, in some embodiments of system 100, different host devices and storage arrays in host device 102 and storage array 106 can reside in different data centers.

[0051] Numerous other distributed implementations of host device 102 and storage array 106 are possible. Thus, host device 102 and storage array 106 can also be implemented in a distributed manner across multiple data centers.

[0052] The following will be described in conjunction with Figure 10 and Figure 11 Additional examples of processing platforms utilized to implement portions of system 100 in the illustrative embodiments will be described in more detail.

[0053] It should be understood that Figure 1 the specific set of elements shown for moving data between storage tiers of a cluster storage system based on a predicted data access frequency trend pattern is presented only by way of illustrative example, and in other embodiments, additional or alternative elements may be used. Thus, another embodiment can include additional or alternative systems, devices, and other network entities, as well as different arrangements of modules and other components.

[0054] It should be understood that these and other features of the illustrative embodiments are presented only by way of example and should not be construed in any way as limiting.

[0055] Reference will now be made toFigure 2 The flowchart of Figure 2 describes in more detail an exemplary process for moving data between storage levels of a cluster storage system based on a predicted data access frequency trend pattern. It should be understood that this specific process is merely an example, and in other embodiments, additional or alternative processes for moving data between storage levels of a cluster storage system based on a predicted data access frequency trend pattern may be used.

[0056] In this embodiment, the process includes steps 200 to 208. It is assumed that these steps are executed by the storage object access trend classification module 112 and the storage object movement module 114. The process begins at step 200, generating a predicted data access frequency trend pattern for a given storage object over a specified time period. The given storage object is stored in a given storage system of two or more storage systems in the cluster storage system using a first type of storage resource. The first type of storage resource is associated with a first storage level among two or more storage levels in the cluster storage system.

[0057] Step 200 may include generating a prediction function for predicting the total data access volume of a given storage object over a specified time period. The predicted data access frequency trend pattern may include one of an increasing data access trend pattern and a decreasing data access trend pattern, and a least squares algorithm may be used to generate the prediction function. The predicted data access frequency trend pattern may include a cyclic data access trend pattern, and at least one of an autocorrelation algorithm and a discrete Fourier transform algorithm may be used to generate the prediction function. The predicted data access frequency trend pattern may include an irregular data access pattern, and the average value of the historical data access of the given storage object in a previous time period may be used to generate the prediction function.

[0058] In step 202, a given storage object is classified into a given storage object category among two or more storage object categories based at least in part on a predicted data access frequency trend pattern of the given storage object over a specified time period. The two or more storage object categories may include: a first storage object category that includes storage objects having a predicted data access frequency trend pattern above a first data access frequency threshold or below a second data access frequency threshold over the specified time period; and a second storage object category that includes storage objects having a predicted data access frequency trend pattern between the first data access frequency threshold and the second data access frequency threshold over the specified time period. In some embodiments, storage objects classified into the first storage object category are permitted to utilize (i) intra-storage-system data movement within a single storage system among two or more storage systems in a cluster storage system and (ii) inter-storage-system data movement between two or more different storage systems among two or more storage systems in a cluster storage system, while storage objects classified into the second storage object category are permitted to utilize (i) intra-storage-system data movement within a single storage system among two or more storage systems in a cluster storage system but are not permitted to utilize (ii) inter-storage-system data movement between two or more different storage systems among two or more storage systems in a cluster storage system.

[0059] Figure 2 The process proceeds to step 204, where a given storage tier among two or more storage tiers in the cluster storage system is determined for storing the given storage object during the specified time period, based at least in part on the predicted data access frequency trend pattern of the given storage object over the specified time period. In response to the given storage tier being different from a first storage tier, a type of data movement of a second type of storage resource for moving the given storage object to one or more storage systems in the cluster storage system is selected in step 206. The second type of storage resource is associated with the given storage tier in the cluster storage system. The selected type of data movement is based at least in part on the given storage object category associated with the given storage object. In step 208, the given storage object is moved to the second type of storage resource of the one or more storage systems in the cluster storage system using the selected type of data movement. The one or more storage systems in the storage system may include the given storage system (e.g., moving the given storage object using intra-storage-system data movement between different storage devices providing storage for the first and the given storage tiers), or may be at least one storage system among two or more storage systems different from the given storage system (e.g., moving the given storage object using inter-storage-system data movement).

[0060] Step 208 may include: determining rankings for two or more storage systems in a cluster storage system for a given storage tier, at least partially based on the available storage capacity and load handling capabilities of each of two or more storage systems for the given storage tier, and may select the one or more storage systems in the cluster storage system at least partially based on the determined rankings of the two or more storage systems.

[0061] In some embodiments, step 208 includes determining whether a given storage system has a second type of storage resource with an available amount sufficient to store a given storage object, and in response to determining that the given storage system has a second type of storage resource with an available amount sufficient to store the given storage object, moving the given storage object to the second type of storage resource in the given storage system. Step 208 may include: determining whether a given storage system has a second type of storage resource with an available amount sufficient to store a given storage object; and in response to determining that the given storage system does not have a second type of storage resource with an available amount sufficient to store the given storage object, determining whether the given storage object category associated with the given storage object permits data movement between different storage systems among two or more storage systems in the cluster storage system. In response to determining that the given storage object category associated with the given storage object permits data movement between different storage systems among two or more storage systems in the cluster storage system, identifying one or more other storage systems among the two or more storage systems having a second type of storage resource with an available amount sufficient to store the given storage object, and moving the given storage object to the second type of storage resource in one of the identified storage systems.

[0062] In a data center or other types of IT infrastructure (including cloud computing platforms), there may be many different storage arrays, each with its own characteristics and advantages for different types of workloads. For example, a data center associated with a business or other enterprise can provide high-quality comprehensive services to customers or other users by leveraging the different characteristics of such storage arrays (including the different characteristics of the storage devices within each of the storage arrays). In some cases, a storage array is part of a cluster storage system (also referred to herein as a storage cluster, storage array cluster, or array cluster).

[0063] At the array cluster level, as business requirements change, the "hotness" of different data may change continuously. Hotness or I / O temperature can characterize data access frequency (e.g., the number of I / O requests within a specified threshold of the current time) or other types of I / O metrics. Due to such hotness changes, data currently residing on high-performance storage arrays may become cold, while data residing on low-performance storage arrays may become hot. To optimize performance, therefore, self-service data movement is needed between different arrays in the array cluster (e.g., based on data access frequency). Such self-service data movement can include breaking down the barriers between storage arrays in the array cluster, moving hot data to high-performance storage arrays, archiving cold data to lower-performance storage arrays, etc. This advantageously provides various benefits to customers or other end-users (e.g., improving performance, reducing costs, improving storage utilization efficiency, and accelerating customer business processes in the array cluster, etc.).

[0064] For some storage arrays and array clusters, data movement functionality can be used to extend storage tiering between different storage arrays or platforms and cloud storage platforms, move data between heterogeneous storage resources, and make full use of storage resources at the storage cluster level and even potentially across data centers. In some storage arrays, such data movement functionality may include FAST technology, also known as FAST Sideways or FAST.X. For example, Dell EMC VMAX storage arrays can implement FAST Cache, which provides a way for users to accelerate mission-critical processes based on business priorities and other service level objectives (SLOs). Advantageously, FAST Cache is application-aware and uses storage and performance analyzers to monitor the read and write states of different workloads to send cache to the storage array for data that may be accessed within a given time period. IT administrators can create FAST Cache profiles, which are assigned priorities and are scheduled once, continuously, or periodically (e.g., daily, weekly, monthly, etc.) and have an expected execution duration. Such cache can be provided via the analysis tab of the storage array's analysis software (e.g., the database storage analyzer interface for VMAX arrays in Unisphere).

[0065] Based on the provided hints, the performance analyzer can monitor increased workload demands before taking actions. For example, FAST can receive hints from the database storage analyzer through the hint manager application programming interface (API), and actively adjust the mix of storage devices according to a set of priorities (e.g., a mix of flash and serial-attached SCSI (SAS) storage devices). The mix of storage devices may be automatically adjusted, and it is necessary to maintain the integrity of the SLO and not overwrite the previous SLO. FAST or other data movement functionality provides the ability to deliver leading application-aware functionality to customers or end users who require the best response time for mission-critical applications during specific business periods.

[0066] In addition to being optimized through FAST hints, data services can also extend beyond the storage array itself and, through FAST.X, to the entire data center. FAST.X has advantageously evolved storage tiering and service level management and extended it to other storage platforms as well as cloud storage. FAST.X enables data movement across storage technologies provided by various block devices (e.g., including Dell EMC XtremIO, VNX, CloudArray, and other types of storage). FAST.X simplifies management and operations and integrates heterogeneous storage under its control. FAST.X also further extends SLO management to external storage arrays. Implementing SLO management across external storage arrays enables easy integration of different devices according to the demands of different workloads and requirements. FAST.X can simplify management on a large scale, thus providing workload optimization across storage arrays with the same simplicity as internal SLO settings. The advanced automation of FAST optimizes the workload of customers or other end users to automatically apply the necessary amount of resources, and FAST.X extends this functionality to external storage arrays according to the same specified performance and availability criteria. As mentioned above, FAST.X can also be integrated with cloud storage, such as to move less-active workloads to more cost-effective cloud storage.

[0067] Data movement across multiple storage arrays in an array cluster (e.g., using the above FAST / FAST.X functionality) is a resource-intensive task. In addition to computing resources on the different storage arrays where data is being moved, such data movement can also result in additional network load, thereby degrading the overall performance of the array cluster. Data movement between arrays can have a higher level of granularity (e.g., moving storage objects such as LUNs or file systems) and be of large size (e.g., one to several thousand gigabytes (GB) or larger). Thus, data movement across storage arrays is heavier or more resource-intensive than data movement within a storage array (e.g., between different storage devices of a single storage array). Accordingly, it is desirable to reduce inter-array data movement in an array cluster.

[0068] Exemplary embodiments provide techniques for analyzing data activity from a long-term perspective and evaluating the overall data activity trend over a future time period to more intelligently move data across storage arrays of an array cluster. Some embodiments utilize techniques that leverage time series data theory and, based on historical IO access frequencies, will predict the future load trend of storage arrays in an array cluster. Storage objects can be classified according to their corresponding IO trends. In some embodiments, storage objects are classified into one of two categories: (1) storage objects of a first category having a long-term IO temperature trend that is cold or hot; and (2) storage objects of a second category having a long-term IO temperature trend that is neither cold nor hot. Storage objects in the first category are suitable for migration within or between storage arrays because such storage objects do not need to be moved frequently (e.g., such objects can be selected for FAST operations at the cluster level). Storage objects in the second category are not suitable for migration between storage arrays but can be internally migrated within a storage array (e.g., internal storage array FAST). In this way, exemplary embodiments can reduce unnecessary frequent data relocation, thereby improving data movement efficiency at the cluster level.

[0069] In a storage system, most customer or end-user storage objects exhibit a data access frequency that shows a pattern with a time series. The data access frequency pattern may be specific to a particular business domain or department. Non-stationary time series data is of interest because this may exhibit valuable predictions. Non-stationary time series data may have various patterns, including various trend patterns, cyclic patterns, irregular patterns, etc. Data IO frequency information can be obtained from each storage array in an array cluster.

[0070] A trend pattern refers to an increasing or decreasing frequency of IO access. For example, various data may shut down most of its activities at its creation, and as the data ages, the activities become colder and the access frequency decreases. Other data may shut down infrequent activities at its creation, and as the data ages, the activities become hotter and the access frequency increases. Figure 3 Figure 300 is shown, and Figure 300 shows an example of a trend pattern for increasing and decreasing data access frequencies. Although Figure 300 shows a linear trend pattern, the trend pattern can also be non-linear (e.g., exponential). In some embodiments, based on a data access sample set, the least squares method is used to generate a prediction function f(t) = a0 + a1t + a2t 2 +…+ a m t m . The prediction function can be used to predict the total number of data accesses in a future period T using the following equation (1):

[0071]

[0072] A cyclic pattern refers to a data access pattern in which the data access frequency rises and falls with some regularity (e.g., hourly, daily, monthly, quarterly, annually, etc.). For example, as shown in the graph 400 of Figure 4 , some data may be accessed frequently during holiday or festival periods. To determine whether the data is cyclic, some embodiments utilize an autocorrelation function or a discrete Fourier transform method to detect periodicity based on data access sampling. Then, spectral analysis can be used to obtain the period or frequency of the cyclic or seasonal time series data to obtain a Fourier series prediction function This can be used to predict the total number of data accesses in a future period T using equation (2):

[0073]

[0074] An irregular data pattern refers to a data access pattern in which the data access frequency varies randomly, and generally refers to data that does not have a trend function or periodicity. Figure 5 Figure 500 is shown, and Figure 500 shows an irregular data access pattern. For an irregular data access pattern, the historical data access samples can be averaged Then, this can be used to predict the total number of data accesses in a future period T using equation (3):

[0075]

[0076] An implementation of an algorithm for enhancing the efficiency of data movement in an array cluster including a storage array will now be described, where the array cluster classifies storage resources as belonging to one of three storage tiers: Tier 1, also referred to as the "highest performance" tier (e.g., including SAS flash, SSD, NVMe drives); Tier 2, also referred to as the "performance" tier (e.g., including SAS drives); and Tier 3, also referred to as the "capacity" tier (e.g., including nearline SAS (NL-SAS) drives or low-cost cloud storage).

[0077] The following symbols will be used to describe the algorithm for enhancing the efficiency of inter-array data movement in an array cluster:

[0078] S ai-ssd : The available SAS flash or SSD capacity of storage array i;

[0079] The total available SAS flash or SSD capacity of the array cluster, representing the sum of the SAS flash or SSD capacities of individual storage arrays;

[0080] S ai-sas : The available SAS capacity of storage array i;

[0081] The total available SAS capacity of the array cluster, representing the sum of the SAS capacities of individual storage arrays;

[0082] S ai-nlsas : The available NL-SAS capacity of storage array i;

[0083] The total available NL-SAS capacity of the array cluster, representing the sum of the NL-SAS capacities of individual storage arrays;

[0084] The ratio of the available flash capacity of storage array i in the array cluster, where R ai-ssd The larger the value, the larger the available flash capacity of storage array i in the highest performance tier of the array cluster;

[0085] The ratio of the available SAS capacity of storage array i in the array cluster, where R ai-sas The larger the value, the larger the available SAS capacity of storage array i in the performance tier of the array cluster;

[0086] The ratio of the available NL-SAS capacity of storage array i in the array cluster, where R ai-nlsas The larger the value, the larger the available NL-SAS capacity of storage array i in the capacity tier of the array cluster;

[0087] U ai-ssd : The flash memory capacity used by the storage array i;

[0088] U ai-sas : The SAS capacity used by the storage array i;

[0089] U ai-nlsas : The NL-SAS capacity used by the storage array i;

[0090] L ai-ssd : The historical cumulative IO requests for the flash memory capacity of storage array i;

[0091] L ai-sas : The historical cumulative IO requests for the SAS capacity of storage array i;

[0092] L ai-nlsas : The historical cumulative IO requests for the NL-SAS capacity of storage array i;

[0093] The average IO requests per unit flash memory storage space of storage array i;

[0094] The average IO requests per unit SAS storage space of storage array i;

[0095] The average IO requests per unit NL-SAS storage space of storage array i;

[0096] The inverse ratio of the normalized average IO load of storage array i in the highest performance tier, where if the cumulative historical IO value of storage array i for the highest performance tier is small, it means that for the highest performance tier, the load of storage array i is small, and according to the load balancing review, storage array i should or can handle more IO for the highest performance tier in the future.

[0097] The inverse ratio of the normalized average IO load of storage array i in the performance tier, where if the cumulative historical IO value of storage array i for the performance tier is small, it means that for the performance tier, the load of storage array i is small, and according to the load balancing review, storage array i should or can handle more IO for the performance tier in the future.

[0098] The inverse of the normalized average IO load of storage array i in the capacity tier, where if the cumulative historical IO value of storage array i for the capacity tier is small, it means that for the capacity tier, storage array i has a low load, and according to the load balancing review, storage array i should or can handle more IOs for the capacity tier in the future.

[0099] C ai-ssd = ω S .R ai-ssd + ω L .F ai-ssd : For the highest performance tier, the comprehensive score of storage array i, which combines the storage and load capabilities of storage array i, where if storage array i has high available capacity and low load in the highest performance tier, then storage array i is a good candidate to handle more IOs for the highest performance tier, and the sum of the weights of size and load capabilities is 1 (e.g., ω S + ω L = 1);

[0100] C ai-sas = ω S .R ai-sas + ω L .F ai-sas : For the performance tier, the comprehensive score of storage array i, which combines the storage and load capabilities of storage array i, where if storage array i has high available capacity and low load in the performance tier, then storage array i is a good candidate to handle more IOs for the performance tier, and the sum of the weights of size and load capabilities is 1 (e.g., ω S + ω L = 1);

[0101] C ai-nlsas = ω S .R ai-nlsas + ω L .F ai-nlsas : For the performance tier, the comprehensive score of storage array i, which combines the storage and load capabilities of storage array i, where if storage array i has high available capacity and low load in the performance tier, then storage array i is a good candidate to handle more IOs for the performance tier, and the sum of the weights of size and load capabilities is 1 (e.g., ω S + ω L = 1);

[0102] T0: The time point to start predicting the data access trend; and

[0103] Predict the total number of IO requests H for data j on storage array i in the future period T using a time series data pattern model based on sample data ai，j, denoted as d ai，j .

[0104] Algorithms for enhancing the efficiency of data movement in an array cluster may include four stages or steps: (1) data collection; (2) data heat prediction and tier checking; (3) analyzing and ranking the storage arrays in the array cluster; and (4) determining a data movement solution that achieves the long-term storage efficiency of the array cluster.

[0105] In step or stage (1), necessary data is collected from the array cluster. One or more monitoring services or APIs are used to collect storage array information in the array cluster, where the storage array information may include: the type of drive or storage device in each storage array; the available capacity of the drives or devices in each tier of each storage array in the array cluster (e.g., S ai-ssd , S ai-sas , S ai-nlsas ); the used space of the storage arrays in the array cluster in each tier (e.g., U ai-ssd , U ai-sas , U ai-nlsas ); the historical IO load of each storage array in the array cluster in each tier (e.g., L ai-ssd , L ai-sas , L ai-nlsas ); sampling the user data access frequency; and so on.

[0106] In step or stage (2), the data collected in step or stage (1) is used to perform time series data pattern analysis to predict the heat of data (e.g., storage objects) in the array cluster and to check whether the data is in the appropriate tier. The time series data patterns may include the different types of data patterns mentioned above (e.g., increasing or decreasing data patterns, cyclic data patterns, irregular data patterns, etc.). The time series data patterns can be used to use Equation (4) to predict the total IO requests for data within a future period (e.g., the next day, next week, next month, next quarter, next year, etc.):

[0107]

[0108] This can use the above Equations (1)-(3) for each data pattern. Consider, for example, a decreasing data access pattern. According to the decreasing trend, curve fitting as a linear function (e.g., f(t) = a0 + a1t) can be used to calculate the total IO access count for the jth data residing on storage array i after the current time point T0 within a future period T according to the following equation:

[0109]

[0110] Figure 6 shows a graph 600 that shows a data pattern trend for reduced data access, the expected IO access frequency starting from time point T0 and lasting for a time period T using a linear curve fitting function f(t). For an array cluster, the total access frequency H can be predicted based on long-term ai，j (e.g., from largest to smallest) sort the data and add level labels to each tier in the array cluster. Continuing with the above exemplary array cluster, there will be labels for three levels (e.g., first, second, and third) mapped to three tiers (e.g., highest performance, performance, capacity).

[0111] In step or stage (3), analyze and rank the storage arrays in the array cluster. Figure 7 shows a table 700 that indicates the available capacity for each tier of each storage array in the array cluster. It should be noted that if a particular storage array does not have a drive or device to provide storage for a particular tier, then for that tier S ai = 0. For example, if storage array 1 does not have a drive or device with performance characteristics suitable for use in the highest performance tier, then S a1-ssd = 0.

[0112] For each storage array and each tier type, the available size ratio in the array cluster can be represented as a 3xN matrix:

[0113]

[0114] For each storage array and each tier type, the load handling capacity or the ratio of the load that can be handled in the cluster can also be represented as a 3xN matrix:

[0115]

[0116] In each tier, a storage array with a larger R value means there is more available capacity on these storage arrays. It should be noted that for an array cluster with a different number of storage tiers, the sizes of matrices R and F will be different (e.g., for an array cluster with four tiers, the matrix will be 4xN). More generally, matrices R and F can be MxN matrices, where M represents the number of storage tiers in the array cluster.

[0117] The values of R and F can be combined to rank the storage arrays in the array cluster according to the following equation:

[0118] C = ω S *R + ω L *F

[0119] C is an MxN matrix, where the value of M is the number of storage tiers in the array cluster. Continuing with the example of an exemplary array cluster that includes three storage tiers (e.g., a highest performance tier, a performance tier, and a capacity tier), C is a 3xN matrix similar to R and F. N is the number of storage arrays in the array cluster. In each tier m, the storage array n with a high value of C m，n is a good choice (e.g., for an increased IO load in tier m).

[0120] In step or phase (4), determine a data movement solution that meets the end-user goal (e.g., achieving long-term storage efficiency). As described above, "long-term" can be defined by the storage administrator of the array cluster or other end-users. For example, the storage administrator may seek to achieve storage efficiency within the next day, next week, next month, next quarter, next year, or some other time period. For example, consider data j located in storage array i at tier 1 (e.g., the highest performance tier), and predict the access frequency or popularity H ai，j is also at a high level corresponding to tier 1, then data j may remain in its original location.

[0121] As another example, consider data j located in storage array i at tier 2 or tier 3 (e.g., the performance tier or the capacity tier), but its predicted access frequency or popularity H ai，j is at a high level corresponding to tier 1. In this example, data j should be moved. If the in-array data movement functionality (e.g., FAST) of storage array i where data j currently resides cannot be utilized to move data j to a tier 1 type of storage in storage array i (e.g., where storage array i has no tier 1 type of storage, where storage array i has tier 1 type of storage, but the available storage of the tier 1 type of storage is insufficient, etc.), then data j should be moved to a tier 1 storage on another storage array n (e.g., the storage array n with the highest or largest C 1，n value), and then C can be recalculated. This assumes that data j is considered suitable for inter-array data movement (e.g., it is in the first category).

[0122] As a further example, consider data j located in storage array i at tier 1 (e.g., the highest performance tier), but its predicted access frequency or popularity H ai，jAt a level corresponding to level 2 or level 3. In this example, data j should be moved. If the in-array data movement functionality (e.g., FAST) of storage array i where data j currently resides cannot be utilized to move data j to level 2 or level 3 type storage of storage array i (e.g., where storage array i has no level 2 or level 3 type storage, where storage array i has level 2 or level 3 type storage but there is insufficient available storage of level 2 or level 3 type, etc.), then data j should be moved to level 2 or level 3 storage on another storage array n (e.g., storage array n with the highest or maximum C 2，n or C 3，n value), and then C can be recalculated. Furthermore, this assumes that data j is considered suitable for inter-array data movement (e.g., it is in the first category).

[0123] The above processing can be used to determine plans, guidelines, or recommendations to move highly active data to high-performance storage (e.g., on the same or different storage arrays in an array cluster), and to move low-active data to low-performance storage (e.g., on the same or different storage arrays in an array cluster). Such recommendations can be used to automatically move storage objects or data while considering long-term data trends, thereby optimizing the performance of the array cluster and better utilizing the storage media across the array cluster.

[0124] Figure 8 A process flow 800 showing an algorithm for enhancing the efficiency of data movement in an array cluster is shown. The process starts at step 801, and array information is collected and the user data pattern is sampled at step 803. At step 805, a time series data pattern is detected to generate a data access prediction function for calculating the IO frequency in a future period T and ranking the in the array cluster. At step 807, the capacity and load handling capacity C(m, i) of storage array i are calculated and updated.

[0125] At step 809, it is determined whether the data D ai，j is in the appropriate level m according to the prediction of the data. If the result determined in step 809 is no, the process flow 800 proceeds to step 811, where it is determined whether the current storage array i of data D ai，j can utilize its internal data movement functionality (e.g., FAST) to move data D ai，j to the appropriate level according to the prediction of If the result of step 811 is no, then the process flow 800 proceeds to step 813, where the data D is moved to its appropriate level. ai，j Move to the highest or largest Cm′，n The target level m' on another storage array n of the value. As described above, the inter-array data movement may depend on the data D classified as the first category. ai，j .

[0126] If the result of step 809 is yes, then there is no need to move the data D ai，j , and the processing flow 800 proceeds to step 815. If the result of step 811 is yes, the internal data movement functionality of the current storage array i is used to move the data D ai，j Prediction to move the data to its correct level m, and the process flow 800 proceeds to step 815. In step 815, which may be performed after step 813 (or after step 809 or 811 if either determination 809 or 811 is yes), it is determined whether all data j in array i have been evaluated. If the result of the determination in step 815 is no, the process flow 800 loops back to step 807. If the result of the determination in step 815 is yes, the process flow 800 ends in step 817.

[0127] Figure 9A and Figure 9B A group of storage arrays 901-1, 901-2, and 901-3 (collectively referred to as storage array 901) in an array cluster is shown, the array cluster including three storage tiers: a highest performance tier 903, a performance tier 905, and a capacity tier 907. In this example, storage array 901-1 includes storage in each of the three tiers (e.g., highest performance tier 903, performance tier 905, and capacity tier 907), storage array 901-2 includes storage in performance tier 905 and capacity tier 907, and storage array 901-3 includes storage in highest performance tier 903 and performance tier 905.

[0128] Figure 9A Shown in use Figure 8Before the data movement in process flow 800, data items (e.g., storage objects) in storage array 901 in the array cluster, where the data items are associated with one of three IO levels 909-1, 909-2, and 909-3. The highest IO level 909-1 corresponds to data items whose appropriate tier is the highest performance tier 903, the medium IO level 909-2 corresponds to data items whose appropriate tier is the performance tier 905, and the lowest IO level 909-3 corresponds to data items whose appropriate tier is the capacity tier 907. Figure 9B is shown in using Figure 8 the process flow 800 for data movement, the data items in storage array 901 in the array cluster.

[0129] For illustration, the movement of data items represented as 911-1 to 911-5 will now be described. Assume that each of the data items 911-1 to 911-5 is suitable for both intra-array data movement and inter-array data movement. Data items 911-1 and 911-2 have the lowest IO level 909-3, but as Figure 9A shown, they are currently stored in the highest performance tier 903 on storage array 901-1. Therefore, data items 911-1 and 911-2 should be moved. In this example, assume that there is not enough available storage in the capacity tier 907 of storage array 901-1 for data item 911-1, but there is enough available storage in the capacity tier 907 of storage array 901-2. Therefore, the intra-array data movement functionality of storage array 901-1 cannot be used to move data item 911-1 to the capacity tier 907, but it can be used for data item 911-2. Therefore, as Figure 9B shown, data item 911-1 is moved to the capacity tier 907 storage on storage array 901-2, while data item 911-2 is moved to the capacity tier 907 storage on storage array 901-1.

[0130] Data items 911-3 and 911-4 have the highest IO level 909-1 and the lowest IO level 909-3 respectively, but both are currently stored in the performance tier 905 on storage array 901-2, as Figure 9A shown. Since storage array 901-2 does not have the highest performance tier 903 storage, data item 911-3 must be moved to another storage array (e.g., storage array 901-1 or 901-3). As Figure 9BAs shown, data item 911-3 is illustratively moved to the highest performance tier 903 storage on storage array 901-3. Assume that storage array 901-2 has sufficient capacity tier 907 for data item 911-4, and thus data item 911-4 is moved to the capacity tier 907 storage on storage array 901-2, as Figure 9B shown.

[0131] Data item 911-5 has the lowest IO level 909-3, but is currently in the highest performance tier 903 storage on storage array 901-3, as Figure 9A shown. Since storage array 901-3 does not have capacity tier 907 storage, data item 911-5 must be moved to another storage array (e.g., storage array 901-1 or 901-2). As Figure 9B shown, data item 911-5 is illustratively moved to the capacity tier 907 storage on storage array 901-2.

[0132] Although only five specific data movement examples are given above for data items 911-1 to 911-5 in Figure 9A and Figure 9B , it should be noted that various other data items are moved between storage tiers 903, 905, and 907, and in the examples of Figure 9A and Figure 9B , they all move within storage array 901 and across storage arrays 901.

[0133] It should be understood that the specific advantages described above and elsewhere in this document are associated with specific illustrative embodiments and need not be present in other embodiments. Moreover, the specific types of information processing system features and functionality shown in the figures and described above are merely exemplary, and numerous other arrangements may be used in other embodiments.

[0134] Now, an illustrative embodiment of a processing platform will be described in more detail with reference to Figure 10 and Figure 11 , which is used to implement the functionality of moving data between storage tiers in a cluster storage system based on a predicted data access frequency trend pattern. Although described in the context of system 100, in other embodiments, these platforms can also be used to implement at least some parts of other information processing systems.

[0135] Figure 10 An exemplary processing platform including cloud infrastructure 1000 is shown. Cloud infrastructure 1000 includes a combination of physical and virtual processing resources, which can be utilized to implement Figure 1At least a portion of the information processing system 100. The cloud infrastructure 1000 includes a plurality of virtual machines (VMs) and / or sets of containers 1002-1, 1002-2, ..., 1002-L implemented using the virtualization infrastructure 1004. The virtualization infrastructure 1004 runs on the physical infrastructure 1005 and illustratively includes one or more hypervisors and / or operating system-level virtualization infrastructure. The operating system-level virtualization infrastructure illustratively includes kernel control groups of a Linux operating system or other types of operating systems.

[0136] The cloud infrastructure 1000 further includes sets of applications 1010-1, 1010-2, ..., 1010-L that run on corresponding ones of the VM / container sets 1002-1, 1002-2, ..., 1002-L under the control of the virtualization infrastructure 1004. The VM / container sets 1002 may include corresponding VMs, corresponding one or more sets of containers, or corresponding one or more sets of containers running within a VM.

[0137] In Figure 10 Some implementations of the embodiment, the VM / container sets 1002 include corresponding VMs implemented using the virtualization infrastructure 1004 that includes at least one hypervisor. A hypervisor platform can be used to implement a hypervisor within the virtualization infrastructure 1004, where the hypervisor platform has an associated virtual infrastructure management system. The underlying physical machine may include one or more distributed processing platforms, and the distributed processing platforms include one or more storage systems.

[0138] In Figure 10 Other implementations of the embodiment, the VM / container sets 1002 include corresponding containers implemented using the virtualization infrastructure 1004 that provides operating system-level virtualization functionality (such as support for Docker containers running on bare metal hosts or Docker containers running on VMs). The containers are illustratively implemented using corresponding kernel control groups of the operating system.

[0139] As is apparent from the above, one or more of the processing modules or other components of the system 100 may each run on a computer, server, storage device, or other processing platform element. A given such element can be regarded as an example of an element more generally referred to herein as a "processing device". Figure 10 The cloud infrastructure 1000 shown in Figure 11 may represent at least a portion of a processing platform. Another example of such a processing platform is

[0140] In this embodiment, the processing platform 1100 includes a part of system 100 and includes a plurality of processing devices represented as 1102-1, 1102-2, 1102-3,......, 1102-K, and the plurality of processing devices communicate with each other through network 1104.

[0141] Network 1104 can include any type of network, including, for example, the global computer network (such as the Internet), WAN, LAN, satellite network, telephone or wired network, cellular network, wireless network (such as WiFi or WiMAX network) or parts or combinations of these and other types of networks.

[0142] The processing device 1102-1 in the processing platform 1100 includes a processor 1110 coupled to a memory 1112.

[0143] Processor 1110 can include a microprocessor, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a video processing unit (VPU) or other types of processing circuits, and parts or combinations of such circuit elements.

[0144] Memory 1112 can include random access memory (RAM), read only memory (ROM), flash memory or other types of memory in any combination. The memory 1112 and other memories disclosed herein should be regarded as illustrative examples of what is more generally referred to as "processor-readable storage media" storing executable program code for one or more software programs.

[0145] An article of manufacture including such processor-readable storage media is considered an illustrative embodiment. A given such article of manufacture can include, for example, a storage array, a storage disk or an integrated circuit containing RAM, ROM, flash memory or other electronic memory, or any of a variety of other types of computer program products. As used herein, the term "article of manufacture" should be understood to exclude transient propagated signals. Numerous other types of computer program products including processor-readable storage media can be used.

[0146] The processing device 1102-1 also includes a network interface circuit 1114 for interfacing the processing device with network 1104 and other system components, and can include a conventional transceiver.

[0147] It is assumed that the other processing devices 1102 of the processing platform 1100 are configured in a manner similar to that shown for the processing device 1102-1 in the figure.

[0148] Moreover, the specific processing platform 1100 shown in the figures is presented by way of example only, and system 100 may include additional or alternative processing platforms, as well as numerous different processing platforms in any combination, where each such platform includes one or more computers, servers, storage devices, or other processing devices.

[0149] For example, other processing platforms for implementing the illustrative embodiments may include converged infrastructure.

[0150] Accordingly, it should be understood that in other embodiments, different arrangements of additional or alternative components may be used. At least a subset of these components may be implemented together on a common processing platform, or each such component may be implemented on a separate processing platform.

[0151] As previously indicated, the components of the information processing system as disclosed herein may be implemented, at least in part, in the form of one or more software programs stored in a memory and executed by a processor of a processing device. For example, at least a portion of the functionality disclosed herein for moving data between storage tiers of a cluster storage system based on a predicted data access frequency trend pattern is illustratively implemented in the form of software running on one or more processing devices.

[0152] It should be emphasized again that the above embodiments are presented for illustrative purposes only. Many variations and other alternative embodiments may be used. For example, the disclosed techniques may be applicable to many other types of information processing systems, storage systems, storage clusters, etc. Moreover, the specific configurations of the system and apparatus elements illustratively shown in the figures and the associated processing operations may vary in other embodiments. Additionally, the various assumptions made above in the course of describing the illustrative embodiments should also be considered exemplary rather than requirements or limitations of the present disclosure. Numerous other alternative embodiments within the scope of the appended claims will be apparent to those skilled in the art.

Claims

1. An apparatus, comprising: at least one processing device, including a processor coupled to a memory; the at least one processing device is configured to perform the following steps: generate a predicted data access frequency trend pattern of a given storage object within a specified time period, the given storage object being stored using a first type of storage resource in a given storage system of two or more storage systems in a cluster storage system, the first type of storage resource being associated with a first storage tier among two or more storage tiers in the cluster storage system; classify the given storage object into a given storage object category among two or more storage object categories at least partially based on the predicted data access frequency trend pattern of the given storage object within the specified time period; determine a given storage tier among the two or more storage tiers in the cluster storage system for storing the given storage object during the specified time period at least partially based on the predicted data access frequency trend pattern of the given storage object within the specified time period; in response to the given storage tier being different from the first storage tier, select a data movement type for a second type of storage resource for moving the given storage object to one or more storage systems in the storage system of the cluster storage system, the second type of storage resource being associated with the given storage tier in the cluster storage system, the selected type of data movement being at least partially based on the given storage object category associated with the given storage object; and use the selected type of data movement to move the given storage object to the second type of storage resource of the one or more storage systems in the storage system of the cluster storage system.

2. The apparatus according to claim 1, wherein one or more storage systems in the storage system include the given storage system.

3. The apparatus according to claim 1, wherein one or more storage systems in the storage system include at least one of the two or more storage systems different from the given storage system.

4. The apparatus according to claim 1, wherein the two or more storage object categories include: a first storage object category, including storage objects having a predicted data access frequency trend pattern higher than a first data access frequency threshold or lower than a second data access frequency threshold within the specified time period; and a second storage object category, including storage objects having a predicted data access frequency trend pattern between the first data access frequency threshold and the second data access frequency threshold within the specified time period.

5. The apparatus according to claim 4, wherein a storage object classified as the first storage object category is allowed to utilize (i) in-system data movement within a single storage system among the two or more storage systems in the cluster storage system, and (ii) inter-system data movement between two or more different storage systems among the two or more storage systems in the cluster storage system.

6. The apparatus according to claim 4, wherein a storage object classified as the second storage object category is allowed to utilize (i) in-system data movement within a single storage system among the two or more storage systems in the cluster storage system, but is not allowed to utilize (ii) inter-system data movement between two or more different storage systems among the two or more storage systems in the cluster storage system.

7. The apparatus according to claim 1, wherein moving the given storage object to the storage resources of the second type in the one or more storage systems in the cluster storage system by using the selected type of data movement comprises: Determine a ranking of the two or more storage systems in the cluster storage system for the given storage tier, at least in part based on the available storage capacity and load handling capacity of each of the two or more storage systems for the given storage tier.

8. The apparatus according to claim 7, wherein one or more of the storage systems in the cluster storage system are selected at least in part based on the determined ranking of the two or more storage systems.

9. The apparatus according to claim 1, wherein generating the predicted data access frequency trend pattern of the given storage object within the specified time period includes: Generate a prediction function for predicting the total data access volume of the given storage object over the specified time period.

10. The apparatus according to claim 9, wherein the predicted data access frequency trend pattern includes one of an increasing data access trend pattern and a decreasing data access trend pattern, and wherein the prediction function is generated using a least squares algorithm.

11. The apparatus according to claim 9, wherein the predicted data access frequency trend pattern includes a cyclic data access trend pattern, and wherein the prediction function is generated using at least one of an autocorrelation algorithm and a discrete Fourier transform algorithm.

12. The apparatus according to claim 9, wherein the predicted data access frequency trend pattern includes an irregular data access pattern, and wherein the prediction function is generated using an average of historical data accesses of the given storage object over a previous time period.

13. The apparatus according to claim 1, wherein the second type of storage resource for moving the given storage object to one or more of the storage systems in the cluster storage system using the selected type of data movement includes: Determine whether the given storage system has an available amount of the second type of storage resource sufficient to store the given storage object; and In response to determining that the given storage system has the available amount of the second type of storage resource sufficient to store the given storage object, move the given storage object to the second type of storage resource in the given storage system.

14. The apparatus according to claim 1, wherein moving the given storage object to the second type of storage resource in the one or more storage systems in the cluster storage system by using the selected type of data movement includes: determining whether the given storage system has an available amount of the second type of storage resource sufficient to store the given storage object; responsive to determining that the given storage system does not have an available amount of the second type of storage resource sufficient to store the given storage object, determining whether the given storage object category associated with the given storage object permits data movement between different storage systems among the two or more storage systems in the cluster storage system; responsive to determining that the given storage object category associated with the given storage object permits data movement between different storage systems among the two or more storage systems in the cluster storage system, identifying one or more other storage systems among the two or more storage systems having an available amount of the second type of storage resource sufficient to store the given storage object; and moving the given storage object to the second type of storage resource in one of the identified storage systems.

15. A computer program product comprising a non-transitory processor-readable storage medium having program code of one or more software programs stored therein, wherein the program code, when executed by at least one processing device, causes the at least one processing device to perform the following steps: generating a predicted data access frequency trend pattern of a given storage object over a specified period of time, the given storage object being stored using a first type of storage resource in a given storage system among two or more storage systems in a cluster storage system, the first type of storage resource being associated with a first storage tier among two or more storage tiers in the cluster storage system; classifying the given storage object into a given storage object category among two or more storage object categories at least in part based on the predicted data access frequency trend pattern of the given storage object over the specified period of time; determining a given storage tier among the two or more storage tiers in the cluster storage system for storing the given storage object during the specified period of time at least in part based on the predicted data access frequency trend pattern of the given storage object over the specified period of time; responsive to the given storage tier being different from the first storage tier, selecting a type of data movement for moving the given storage object to a second type of storage resource in one or more storage systems in the cluster storage system, the second type of storage resource being associated with the given storage tier in the cluster storage system, the selected type of data movement being at least in part based on the given storage object category associated with the given storage object; and Use the selected type of data movement to move the given storage object to the second type of storage resource in one or more of the storage systems in the cluster storage system.

16. The computer program product according to claim 15, wherein the two or more storage object categories include: A first storage object category that includes storage objects having a predicted data access frequency trend pattern above a first data access frequency threshold or below a second data access frequency threshold during the specified time period; And A second storage object category that includes storage objects having a predicted data access frequency trend pattern between the first data access frequency threshold and the second data access frequency threshold during the specified time period.

17. The computer program product according to claim 16, wherein: Storage objects classified as the first storage object category are allowed to use (i) in-system data movement within a single storage system among the two or more storage systems in the cluster storage system, and (ii) inter-system data movement between two or more different storage systems among the two or more storage systems in the cluster storage system; and Storage objects classified as the second storage object category are allowed to use (i) in-system data movement within a single storage system among the two or more storage systems in the cluster storage system, but are not allowed to use (ii) inter-system data movement between two or more different storage systems among the two or more storage systems in the cluster storage system.

18. A method, comprising: Generating a predicted data access frequency trend pattern of a given storage object during a specified time period, the given storage object being stored using a first type of storage resource in a given storage system among two or more storage systems in a cluster storage system, the first type of storage resource being associated with a first storage tier among two or more storage tiers in the cluster storage system; Classifying the given storage object as a given storage object category in two or more storage object categories at least in part based on the predicted data access frequency trend pattern of the given storage object during the specified time period; Determining a given storage tier among the two or more storage tiers in the cluster storage system for storing the given storage object during the specified time period at least in part based on the predicted data access frequency trend pattern of the given storage object during the specified time period; In response to the given storage tier being different from the first storage tier, selecting a type of data movement for moving the given storage object to a second type of storage resource in one or more of the storage systems in the cluster storage system, the second type of storage resource being associated with the given storage tier in the cluster storage system, the selected type of data movement being at least in part based on the given storage object category associated with the given storage object; And Use the selected type of data movement to move the given storage object to the storage resources of the second type in the one or more storage systems in the cluster storage system; Wherein the method is executed by at least one processing device, the at least one processing device including a processor coupled to a memory.

19. The method according to claim 18, wherein the two or more storage object categories include: A first storage object category, which includes storage objects whose predicted data access frequency trend pattern in the specified time period is higher than a first data access frequency threshold or lower than a second data access frequency threshold; And A second storage object category, which includes storage objects whose predicted data access frequency trend pattern in the specified time period is between the first data access frequency threshold and the second data access frequency threshold.

20. The method according to claim 19, wherein: Storage objects classified as the first storage object category are allowed to use (i) in-system data movement within a single storage system among the two or more storage systems in the cluster storage system, and (ii) inter-system data movement between two or more different storage systems among the two or more storage systems in the cluster storage system; and Storage objects classified as the second storage object category are allowed to use (i) in-system data movement within a single storage system among the two or more storage systems in the cluster storage system, but are not allowed to use (ii) inter-system data movement between two or more different storage systems among the two or more storage systems in the cluster storage system.

Images