Multi-criteria based workload balancing among storage resources

Abstract

An apparatus includes a processing device configured to determine a workload level for each storage resource in a set of two or more storage resources, the workload level based at least in part on a processor performance metric, a memory performance metric, and a load performance metric. The processing device is further configured to identify a performance imbalance rate for the set of two or more storage resources and to perform workload balancing for the set of two or more storage resources in response to: (i) the performance imbalance rate for the set of two or more storage resources exceeding a specified imbalance rate threshold; and (ii) a workload level for at least one storage resource in the set of two or more storage resources exceeding a specified threshold workload level.

Description

Technical Field

[0001] This field relates generally to information processing, and more specifically to storage in information processing systems. Background Technology

[0002] Storage arrays and other types of storage systems are typically shared by multiple host devices over a network. Applications running on these host devices each consist of one or more processes that perform application functionality. These processes issue input / output (I / O) operation requests to the storage system. The storage system's storage controller serves these I / O operation requests. In some information processing systems, multiple storage systems can be used to form a storage cluster. Summary of the Invention

[0003] The illustrative embodiments of this disclosure provide techniques for performing multi-metric-based workload balancing among storage resources.

[0004] In one embodiment, an apparatus includes at least one processing means, the processing means including a processor coupled to memory. The at least one processing means is configured to perform the step of determining a workload level for each of two or more storage resources in a set, the workload level being at least partially based on processor performance metrics, memory performance metrics, and load performance metrics. The at least one processing means is further configured to perform the step of identifying a performance imbalance rate of the set of two or more storage resources and performing workload balancing on the set of two or more storage resources in response to: (i) the performance imbalance rate of the set of two or more storage resources exceeding a specified imbalance rate threshold; and (ii) the workload level of at least one storage resource in the set of two or more storage resources exceeding a specified threshold workload level. Performing workload balancing on the set of two or more storage resources includes migrating one or more workloads from the at least one storage resource in the set of two or more storage resources whose workload level exceeds the specified threshold workload level to one or more other storage resources in the set of two or more storage resources.

[0005] These and other illustrative embodiments include, but are not limited to, methods, devices, networks, systems, and processor-readable storage media. Attached Figure Description

[0006] Figure 1 This is a block diagram of an information processing system in an illustrative embodiment, which is configured to perform multi-metric workload balancing among storage resources.

[0007] Figure 2This is a flowchart of an exemplary process for performing multi-metric workload balancing among storage resources, as illustrated in the illustrative embodiments.

[0008] Figure 3 A storage array with multiple storage processors implementing a network-attached storage server and an associated file system is shown in an illustrative embodiment.

[0009] Figure 4 A memory architecture including main components, a cache memory, and a buffer is shown in an illustrative embodiment.

[0010] Figure 5A and Figure 5B The process flow for achieving workload balancing is shown in an illustrative embodiment.

[0011] Figure 6 and Figure 7 An example of a processing platform that can be used to implement at least a portion of an information processing system is shown in an illustrative embodiment. Detailed Implementation

[0012] This document will describe exemplary embodiments with reference to exemplary information processing systems and associated computers, servers, storage devices, and other processing devices. However, it should be understood that the embodiments are not limited to use with the specific illustrative system and device configurations shown. Therefore, the term "information processing system" as used herein is intended to be interpreted broadly to encompass, for example, processing systems including cloud computing and storage systems, as well as other types of processing systems including various combinations of physical and virtual processing resources. Thus, an information processing system may include, for example, at least one data center or other type of cloud-based system that includes one or more clouds hosting tenants accessing cloud resources.

[0013] Figure 1 An information processing system 100 is illustrated, configured according to an illustrative embodiment to provide functionality for performing multi-metric workload balancing among storage resources. The information processing system 100 includes one or more host devices 102-1, 102-2, ..., 102-N (collectively referred to as host devices 102) communicating via a network 104 with one or more storage arrays 106-1, 106-2, ..., 106-M (collectively referred to as storage array 106). The network 104 may include a storage area network (SAN).

[0014] Storage array 106-1 (e.g.) Figure 1The storage array 106-1 (shown) includes a plurality of storage devices 108, each storing data utilized by one or more applications running on the host device 102. The storage devices 108 are illustratively arranged in one or more storage pools. The storage array 106-1 also includes one or more storage controllers 110 that facilitate I / O processing of the storage devices 108. Storage controllers 110 are examples of what are generally referred to herein as storage processors. The storage array 106-1 and its associated storage devices 108 are examples of what are generally referred to herein as a “storage system.” In this embodiment, this storage system is shared by the host device 102 and is therefore also referred to herein as a “shared storage system.” In embodiments with only a single host device 102, the host device 102 may be configured to exclusively use the storage system.

[0015] Host device 102 illustratively includes a corresponding computer, server, or other type of processing device capable of communicating with storage array 106 via network 104. For example, at least a subset of host device 102 may be implemented as a corresponding virtual machine of a computing service platform or other type of processing platform. In such an arrangement, host device 102 illustratively provides computing services, such as executing one or more applications on behalf of each of one or more users associated with a corresponding host device in host device 102.

[0016] The term “user” in this article is intended to be interpreted broadly to encompass numerous arrangements of human, hardware, software, or firmware entities, and combinations thereof.

[0017] Computing and / or storage services can be provided to users based on Platform as a Service (PaaS), Infrastructure as a Service (IaaS), and / or Function as a Service (FaaS) models; however, it is understood that many other cloud infrastructure deployments can be used. Furthermore, illustrative embodiments can be implemented outside the context of cloud infrastructure, such as in the case of stand-alone computing and storage systems implemented within a given enterprise.

[0018] Storage device 108 of storage array 106-1 can implement logical units (LUNs) configured to store user objects associated with host device 102. These objects may include files, blocks, or other types of objects. Host device 102 interacts with storage array 106-1 using read / write commands and other types of commands transmitted over network 104. In some embodiments, such commands more specifically include Small Computer System Interface (SCSI) commands, but other types of commands may be used in other embodiments. As used extensively herein for a given I / O operation, the term descriptively includes one or more of these commands. References to terms such as “input-output” and “IO” herein should be understood to refer to input and / or output. Thus, an I / O operation involves at least one of input and output.

[0019] Furthermore, the term "storage device" as used herein is intended to be interpreted broadly to encompass, for example, logical storage devices, such as LUNs or other logical storage volumes. A logical storage device in storage array 106-1 can be defined as a distinct portion comprising one or more physical storage devices. Therefore, storage device 108 can be considered to include a corresponding LUN or other logical storage volume.

[0020] The storage device 108 of the storage array 106-1 can be implemented using a solid-state drive (SSD). Such an SSD is implemented using a non-volatile memory (NVM) device, such as flash memory. Other types of NVM devices that can be used to implement at least a portion of the storage device 108 include non-volatile random access memory (NVRAM), phase-change RAM (PC-RAM), and magnetic RAM (MRAM). Various types of NVM devices or other storage devices, as well as various combinations thereof, can also be used. For example, a hard disk drive (HDD) can be used in conjunction with or in place of an SSD or other type of NVM device. Therefore, at least one subset of the storage device 108 can be implemented using a multitude of other types of electronic or magnetic media.

[0021] exist Figure 1In the information processing system 100, it is assumed that storage array 106 implements a network attached storage (NAS) server or other type of storage server, through which host device 102 accesses file systems. The NAS server, for example, provides a file-level storage server for hosting file systems (e.g., a NAS server is used to create file systems shared using Server Message Block (SMB) or Network File System (NFS)). The NAS server can be implemented or run on different storage processors (e.g., different storage controllers 110) of storage array 106-1. Workload balancing between such storage processors can be achieved by moving the NAS server between storage processors. This workload balancing functionality is provided via performance imbalance determination module 112 and workload balancing module 114.

[0022] The performance imbalance determination module 112 is configured to measure or otherwise determine the performance of each storage processor in storage array 106-1, wherein multiple metrics (e.g., CPU utilization, memory utilization, I / O load) may be used to characterize the performance. In some embodiments, the performance imbalance determination module 112 is also configured to measure or otherwise determine the performance of one or more storage processors in one or more other storage arrays in storage arrays 106-2 to 106-M. The performance imbalance determination module 112 is further configured to determine whether a performance imbalance rate greater than a specified threshold exists between two or more storage processors in storage array 106-1 (or between one or more storage processors in storage array 106-1 and one or more storage processors in one or more other storage arrays in storage arrays 106-2 to 106-M). When a performance imbalance rate greater than the specified threshold is determined to exist, the workload balancing module 114 initiates one or more workload balancing operations. Such workload balancing operations may include, for example, moving one or more NAS servers between different storage processors in storage array 106-1 (or between one or more storage processors in storage array 106-1 and one or more storage processors in one or more other storage arrays in storage arrays 106-2 to 106-M).

[0023] The functionality of the performance imbalance determination module 112 and the workload balancing module 114 can be implemented, at least in part, in the form of software stored in memory and executed by the processor.

[0024] Despite Figure 1The embodiments are shown to be implemented both inside storage array 106-1 and outside storage controller 110. However, it should be understood that the performance imbalance determination module 112 and the workload balancing module 114 may be implemented at least partially inside storage controller 110 or at least partially outside storage array 106-1 (such as on one or more of host devices 102 or on a server or other processing platform outside storage array 106-1, e.g., including cloud computing platforms). Furthermore, it should be understood that other storage arrays in storage arrays 106-2 through 106-M may implement instances of the performance imbalance determination module 112 and the workload balancing module 114.

[0025] Figure 1 In the embodiments, the host device 102 and storage array 106 are assumed to be implemented using at least one processing platform, wherein each processing platform includes one or more processing units, each processing unit having a processor coupled to memory. Such processing units may illustratively include specific arrangements of computing, storage, and networking resources. For example, in some embodiments, the processing units are implemented at least in part using virtual resources such as VMs or Linux containers (LXC) or a combination of both, such as in an arrangement in which Docker containers or other types of LXC are configured to run on a VM.

[0026] Although the host device 102 and the storage array 106 can be implemented on correspondingly different processing platforms, many other arrangements are possible. For example, in some embodiments, at least a portion of one or more of the host device 102 and the storage array 106 are implemented on the same processing platform. One or more of the storage arrays 106 can therefore be implemented, at least in part, within at least one processing platform that implements at least one subset of the host device 102.

[0027] Network 104 can be implemented using multiple different types of networks to interconnect storage system components. Network 104 may include a SAN as part of a global computer network such as the Internet, but other types of networks may be part of a SAN, including wide area networks (WANs), local area networks (LANs), satellite networks, telephone or wired networks, cellular networks, wireless networks (such as WiFi or WiMAX networks), or various portions or combinations of these and other types of networks. Therefore, in some embodiments, network 104 includes a combination of multiple different types of networks, each including processing means configured to communicate using Internet Protocol (IP) or other related communication protocols.

[0028] As a more specific example, some embodiments may utilize one or more high-speed local area networks, where associated processing devices communicate with each other using peripheral interconnect high-speed (PCIe) cards and networking protocols such as InfiniBand, Gigabit Ethernet, or Fibre Channel. As those skilled in the art will understand, numerous alternative networking arrangements are possible in a given embodiment.

[0029] While in some embodiments certain commands used by host device 102 to communicate with storage array 106 illustratively include SCSI commands, other types of commands and command formats may be used in other embodiments. For example, some embodiments may utilize command features and functionality associated with NVM Express (NVMe) as described in NVMe specification revision 1.3 of May 2017, which is incorporated herein by reference. Other storage protocols of this type that may be utilized in the illustrative embodiments disclosed herein include: architecture-based NVMe, also known as NVMoF; and Transmission Control Protocol (TCP)-based NVMe, also known as NVMe / TCP.

[0030] Assuming that the storage array 106-1 in this embodiment includes persistent memory implemented using flash memory or other types of non-volatile memory, more specific examples include NAND-based flash memory or other types of non-volatile memory such as resistive RAM, phase-change memory, spin torque transfer magnetoresistive RAM (STT-MRAM), and 3D XPoint-based... TM Intel Optane memory TM The apparatus further assumes that the persistent memory is separate from the storage device 108 of the storage array 106-1; however, in other embodiments, the persistent memory may be implemented as one or more designated portions of one or more of the storage devices 108. For example, in some embodiments, such as those involving an all-flash memory array, the storage device 108 may include a flash-based storage device, or may be implemented wholly or partially using other types of non-volatile memory.

[0031] As mentioned above, communication between host device 102 and storage array 106 can utilize PCIe connections or other types of connections implemented on one or more networks. For example, illustrative embodiments may use interfaces such as Internet SCSI (iSCSI), Serial Attached SCSI (SAS), and Serial ATA (SATA). In other embodiments, numerous other interfaces and associated communication protocols may be used.

[0032] In some embodiments, storage array 106 and other parts of system 100 may be implemented as part of a cloud-based system.

[0033] The storage device 108 of the storage array 106-1 can be implemented using a solid-state drive (SSD). Such an SSD is implemented using a non-volatile memory (NVM) device, such as flash memory. Other types of NVM devices that can be used to implement at least a portion of the storage device 108 include non-volatile random access memory (NVRAM), phase-change RAM (PC-RAM), and magnetic RAM (MRAM). Various types of NVM devices or other storage devices, as well as various combinations thereof, can also be used. For example, a hard disk drive (HDD) can be used in conjunction with or in place of an SSD or other type of NVM device. Therefore, at least one subset of the storage device 108 can be implemented using a multitude of other types of electronic or magnetic media.

[0034] Storage array 106 can be additionally or alternatively configured to implement multiple different storage tiers in a multi-tier storage system. For example, a given multi-tier storage system may include a speed or performance tier implemented using flash storage devices or other types of SSDs and a capacity tier implemented using HDDs, wherein one or more of such tiers may be server-based. It will be apparent to those skilled in the art that a variety of other types of storage devices and multi-tier storage systems may be used in other embodiments. The specific storage device used in a given storage tier may vary depending on the specific requirements of a given embodiment, and a variety of different storage device types may be used in a single storage tier. As previously indicated, the term “storage device” as used herein is intended to be interpreted broadly and therefore may cover, for example, SSDs, HDDs, flash drives, hybrid drives, or other types of storage products and devices or portions thereof, and illustratively includes logical storage devices such as LUNs.

[0035] As another example, storage array 106 can be used to implement one or more storage nodes in a clustered storage system that includes multiple storage nodes interconnected by one or more networks.

[0036] Therefore, it should be apparent that the term “memory array” as used herein is intended to be interpreted broadly and may cover several different instances of commercially available memory arrays.

[0037] Other types of storage products that can be used to implement a given storage system in the illustrative embodiments include software-defined storage, cloud storage, object-based storage, and scale-out storage. In the illustrative embodiments, combinations of these and several other storage types can also be used to implement a given storage system.

[0038] In some embodiments, the storage system includes a first storage array and a second storage array arranged in an active-active configuration. For example, such an arrangement can be used to ensure that data stored in one storage array is copied to another storage array using a synchronous replication process. Such data replication across multiple storage arrays can be used to facilitate fault recovery in system 100. Thus, one of the storage arrays can act as a production storage array relative to another storage array that serves as a backup or recovery storage array.

[0039] However, it should be understood that the embodiments disclosed herein are not limited to active-active configurations or any other particular storage system arrangement. Therefore, the illustrative embodiments herein can be configured using a variety of other arrangements, including, for example, active-passive arrangements, active-active asymmetric logical unit access (ALUA) arrangements, and other types of ALUA arrangements.

[0040] These and other storage systems may be part of what is more generally referred to herein as a processing platform, which includes one or more processing units, each including a processor coupled to memory. A given such processing unit may correspond to one or more virtual machines or other types of virtualization infrastructure, such as Docker containers or other types of LXC. As indicated above, communication between such elements of system 100 may take place on one or more networks.

[0041] As used herein, the term "processing platform" is intended to be interpreted broadly to include, for example, but not limited to, multiple sets of processing devices and one or more associated storage systems configured to communicate over one or more networks. For example, a distributed implementation of host device 102 is possible, wherein some host devices of host device 102 reside in a data center located in a first geographic location, while other host devices of host device 102 reside in one or more other data centers located in one or more other geographic locations that may be far from the first geographic location. Storage array 106 may be implemented at least partially in the first geographic location, the second geographic location, and one or more other geographic locations. Thus, in some implementations of system 100, different host devices of host device 102 and storage array 106 may reside in different data centers.

[0042] Numerous other distributed implementations of host device 102 and storage array 106 are possible. Therefore, host device 102 and storage array 106 can also be implemented in a distributed manner across multiple data centers.

[0043] The following will combine Figure 6 and Figure 7 Additional examples of the processing platform used to implement a portion of system 100 in the illustrative embodiments are described in more detail.

[0044] It should be understood that Figure 1 The specific set of components shown for performing multi-metric workload balancing among storage resources is presented by way of illustrative example only, and additional or alternative components may be used in other embodiments. Therefore, another embodiment may include additional or alternative systems, devices, and other network entities, as well as different arrangements of modules and other components.

[0045] It should be understood that these and other features of the illustrative embodiments are presented by way of example only and should not be construed as limiting in any way.

[0046] Now refer to Figure 2 The flowchart describes in more detail an exemplary process for performing multi-metric-based workload balancing among storage resources. It should be understood that this particular process is merely an example, and additional or alternative processes for performing multi-metric-based workload balancing among storage resources may be used in other embodiments.

[0047] In this embodiment, the process includes steps 200 to 204. These steps are assumed to be performed by the performance imbalance determination module 112 and the workload balancing module 114. The process begins at step 200: determining the workload level of each storage resource in a set of two or more storage resources, the workload level being at least partially based on processor performance metrics, memory performance metrics, and load performance metrics. The set of two or more storage resources may include two or more storage processors from a single storage system, or two or more storage processors from two or more storage systems.

[0048] Processor performance metrics may include a weighted sum of two or more memory utilization metrics associated with two or more distinct time intervals. The two or more processor utilization metrics associated with two or more distinct time intervals may include: a first processor utilization metric representing the current state of processor utilization for a given memory resource within a first time interval relative to the current time; a second processor utilization metric representing the most recent state of processor utilization for a given memory resource within a second time interval relative to the current time, where the second time interval is longer than the first time interval; and a third processor utilization metric representing the persistent state of processor utilization for a given memory resource within a third time interval relative to the current time, where the third time interval is longer than the second time interval.

[0049] Memory performance metrics may include a weighted sum of two or more memory utilization metrics associated with two or more distinct times. The two or more memory utilization metrics associated with two or more distinct times may include: a first memory utilization metric representing the percentage of memory utilization of a given storage resource at a first point in time relative to the current time; a second memory utilization metric representing the percentage of memory utilization of a given storage resource at a second point in time relative to the current time, the second point in time being later than the first point in time; and a third memory utilization metric representing the percentage of memory utilization of a given storage resource at a third point in time relative to the current time, the third point in time being later than the second point in time. Each of the two or more memory utilization metrics includes the sum of main memory utilization, cache memory utilization, and buffer memory utilization at one of the two or more distinct times.

[0050] Load performance metrics may include a weighted sum of two or more load metrics associated with two or more different times. The two or more load metrics associated with two or more different times include: a first load metric representing the percentage of load on a given storage resource at a first point in time relative to the current time; a second load metric representing the percentage of load on a given storage resource at a second point in time relative to the current time, later than the first point in time; and a third load metric representing the percentage of load on a given storage resource at a third point in time relative to the current time, later than the second point in time. Each of the two or more load metrics may include the sum of an input-output operations per second (I / O operations per second) metric and a bandwidth metric within one of the two or more different times, wherein the I / O operations per second metric may be expressed as a percentage of a specified maximum I / O operations per second, and the bandwidth metric may be expressed as a percentage of a specified maximum bandwidth.

[0051] Figure 2 The process continues to step 202: identifying the performance imbalance rate of a set of two or more storage resources. In step 204, workload balancing is performed on the set of two or more storage resources in response to: (i) the performance imbalance rate of the set of two or more storage resources exceeds a specified imbalance rate threshold; and (ii) the workload level of at least one storage resource in the set of two or more storage resources exceeds a specified threshold workload level. Performing workload balancing on the set of two or more storage resources includes migrating one or more workloads from the at least one storage resource in the set of two or more storage resources whose workload level exceeds the specified threshold workload level to one or more other storage resources in the set of two or more storage resources.

[0052] Step 204 may include migrating one or more NAS servers running on at least one storage resource in a set of two or more storage resources whose workload levels exceed a specified threshold workload level to one or more other storage resources in the set of two or more storage resources. Migrating one or more NAS servers running on at least one storage resource in a set of two or more storage resources whose workload levels exceed a specified threshold workload level to one or more other storage resources in the set of two or more storage resources may include performing one or more iterations of the following: selecting the given network-attached storage server based at least in part on the sum of the workloads of file systems created on the first storage resource running on the first storage resource whose first workload level exceeds the specified threshold workload level; pausing one or more replication sessions on the given network-attached storage server; changing the owner of the given network-attached storage server from the first storage resource to a second storage resource, the second storage resource having a second workload level lower than the first workload level; recalculating the workload levels of the first and second storage resources; and rolling back the change of owner of the given network-attached storage server in response to the second workload level of the second storage resource exceeding the recalculated first workload level of the first storage resource.

[0053] Load balancing between storage arrays or storage servers can focus on using link aggregation to balance network load. For example, NAS servers can provide load balancing functionality that focuses on using link aggregation to balance network load. However, balancing network load cannot address imbalances in the utilization of memory, CPU, I / O, or other storage processors (SPs) of different storage arrays or storage servers. Such imbalances in memory, CPU, I / O, or other components can lead to system-level problems such as SP overload, out-of-memory (OOM) conditions, deadlocks, SP emergencies, etc.

[0054] The illustrative embodiments provide techniques for using multi-metric-based workload balancing algorithms in storage systems. Multi-metric-based workload balancing algorithms consider multiple metrics, such as CPU utilization, memory (e.g., main memory, cache memory, buffers, etc.) utilization, I / O load (e.g., IOPS, bandwidth), etc. Upon detecting a specified imbalance, the multi-metric-based workload balancing algorithm can trigger a workload balancing action. Workload balancing actions can be initiated between storage services (SPs) (e.g., within the same storage array or storage system, or between two or more storage arrays or storage systems). For example, this could include moving NAS servers between SPs to achieve better file performance and overall system reliability. Advantageously, such methods provide improved balancing compared to methods that only consider link aggregation and utilize network interface card (NIC) grouping or bundling to achieve network-level balancing.

[0055] In certain use case scenarios, users (e.g., storage administrators, storage customers, or other end users) may create multiple NAS servers and file systems. When configuring a storage array to achieve load balancing among the Service Providers (SPs) running NAS servers on it, users need to estimate the workload of these NAS servers and file systems. However, generating accurate estimates becomes a challenging task, especially as the number of NAS servers and file systems increases. Furthermore, such estimates are static. Loads can change dynamically over time, so if static workload estimates are used to configure NAS servers and file systems, workload imbalance is possible or unavoidable. If the workload imbalance between SPs becomes too large, it can lead to various performance issues, including degraded I / O Quality of Service (QoS). For SP overload, workload imbalance can also cause various system-level problems such as Out of Memory (OOM), deadlocks, SP emergencies, etc.

[0056] Figure 3 An example of a storage array 300 comprising multiple SPs 305-1, 305-2, ..., 305-P (collectively referred to as SP 305) is shown, each SP running one or more NAS servers 310-1, 310-2, ..., 310-P (collectively referred to as NAS server 310) with an associated set of file systems 315-1, 315-2, ..., 315-P (collectively referred to as file system 315). Although Figure 3An example set of implementations or running NAS servers 310 in each SP 305 is shown, but this is not required. For example, a storage array may include multiple SPs, where only a subset of SPs may implement or run NAS servers. Furthermore, each SP 305 may implement or run a different number of NAS servers 310 and associated file systems 315. For example, SP 305-1 may run two NAS servers 310, while SP 305-2 may run a single NAS server 310 (or more than two NAS servers 310). Additionally, each of the NAS servers 310 may be associated with any desired number of file systems 315. In some cases, there is a one-to-one relationship between NAS servers 310 and file systems 315 (e.g., each NAS server 310 is associated with one of the file systems 315). In other cases, one or more of the NAS servers 310 may be associated with more than one file system of the file systems 315. Furthermore, the NAS servers and file systems are not limited to being deployed or running on a single storage array. Users can configure multiple storage arrays, each including one or more SPs running or implementing one or more NAS servers and associated file systems. In some embodiments, the NAS server can be implemented outside the storage array (e.g., on a dedicated server, using a cloud computing platform, etc.).

[0057] In some embodiments, a dynamic multi-metric workload balancing algorithm is used for NAS or other types of storage servers or more generally storage systems. This algorithm considers multiple metrics (including CPU utilization, memory utilization (e.g., main memory utilization, cache utilization, buffer utilization), and I / O load (e.g., IOPS, bandwidth, etc.)) to trigger workload balancing operations between SPs. Such workload balancing operations may include moving NAS servers between SPs when the workload metric between a first SP and a second SP exceeds a specified workload imbalance threshold. A minimum load threshold may also be defined, where balancing is not triggered if the SP's load is below the minimum load threshold (e.g., assuming the SP can handle its own load below the minimum load threshold). In some cases, different minimum load thresholds may be defined for each SP, each storage array, etc. The dynamic multi-metric workload balancing algorithm can leverage SP resources to achieve better file performance and system reliability, thereby avoiding various adverse conditions (e.g., OOM problems).

[0058] Various metrics can be used to measure the performance of a storage array or system, including CPU utilization, memory utilization, and I / O load. These metrics can be defined in more detail below. If any one or a combination of these metrics is too high (e.g., as defined using appropriate thresholds), it may cause performance problems and potentially errors in the associated storage array or system. To evaluate the performance or total workload of a storage array (SP), a weighted metric approach is used based on CPU utilization, memory utilization, and I / O load:

[0059] C=ω CPU ·P CPU +ω Mem ·P Mem +ω IO ·P IO

[0060] Where P CPU P Mem and P IO These represent the percentage of CPU utilization, the percentage of memory utilization, and the percentage of I / O load, respectively, where ω CPU ω Mem and ω IO These represent the weights of CPU utilization, memory utilization, and I / O load, respectively, where ω CPU +ω Mem +ω IO =1. A better overall balance can be achieved by adjusting the weights. Specific values ​​for the weights can be chosen based on the use case. In some embodiments, ω... CPU ω Mem and ω IO The default assigned values ​​are 0.34, 0.33, and 0.33, respectively. In some use cases, default weight values ​​can be used initially, where the weight values ​​are adjusted over time based on real-world usage data to achieve a better overall balance. Such adjustment processes can be used for some or all of the weight sets described in this paper. The higher the total workload assessment value C, the worse the performance of SP.

[0061] For CPU performance evaluation, some implementations consider average CPU utilization, which represents the average system load over a period of time. In Unix- and Linux-based systems, average CPU utilization may be presented as three numbers representing the system load during the last minute, the last five minutes, and the last fifteen minutes (e.g., denoted as PCPU respectively). ,1分钟 PCPU ,5分钟 and P CPU,15分钟 Typically, P CPU,1分钟 P reflects the current state of the system. CPU,5分钟Reflects the latest state of the system, and P CPU,15分钟 Reflects the persistent state of the system. If P CPU,15分钟 P CPU,5分钟 and P CPU,1分钟 If an increasing or decreasing trend is shown, it means that CPU performance is getting worse or better over time. If P CPU,1分钟 ≈P CPU,5分钟 ≈P CPU,15分钟 This means that CPU performance remains constant over time (e.g., whether busy or idle).

[0062] In different time series of CPU utilization (e.g., random, periodic, etc.), CPU 15分钟 CPU 5分钟 and CPU 1分钟 The parameter may have different meanings for overall or total CPU performance. In some embodiments, a weighted average CPU utilization percentage of the last minute, last five minutes, and last fifteen minutes is used:

[0063] P CPU平均 =λ1·P CPU,1分钟 +λ2·P CPU,5分钟 +λ3·P CPU,15分钟

[0064] Here, λ1, λ2, and λ3 represent the weights of CPU utilization percentages during the last minute, last five minutes, and last fifteen minutes, respectively, and λ1 + λ2 + λ3 = 1. By adjusting the weights λ1, λ2, and λ3, a better overall average CPU utilization performance metric can be obtained. λ1, λ2, and λ3 are assigned default values ​​of 0.44, 0.33, and 0.23, respectively. It should be noted that although the above description assumes the use of CPU utilization percentage metrics during the last minute, last five minutes, and last fifteen minutes, this is not mandatory. Other embodiments may utilize only subsets of these time periods, or one or more other time periods may be used to supplement or replace one or more of these time periods. Advantageously, CPU performance metrics can be obtained using the system application programming interface (API).

[0065] Memory performance evaluation considers several factors, including main memory utilization, cache utilization, and buffer utilization. The CPU typically has direct access to main memory, often simply referred to as memory. The CPU can sequentially read instructions stored in main memory and execute them as needed. Data for any valid operation is also stored in main memory in a uniform manner. Cache memory is used to reduce the average time to access data from main memory and can be used to improve the performance of transferring repeatedly transferred data. Buffers are used to compensate for speed differences between processes or devices exchanging data. Both cache memory and buffers can have a significant impact on system performance. Figure 4 An exemplary structure of CPU 400, main memory 405, buffer 410 and cache memory 415 is shown.

[0066] Similar to CPU usage performance evaluation, different memory utilization states may have different implications for overall or systemic memory performance. Considering that memory utilization and release can take some time to materialize, a weighted average percentage of memory utilization can be considered at various points in time (e.g., now, 15 minutes ago, and 30 minutes ago).

[0067] P Mem平均 =μ1·P Mem,现在 +μ2·P Mem,15分钟 +μ3·P Mem,30分钟

[0068] and

[0069] P Mem,时间i =P mainMem,时间i +P cacheMem,时间i +P buffer,时间i

[0070] Where P Mem,现在 P Mem,15分钟 and P Mem,30分钟 These represent the percentage of memory usage now (e.g., currently), 15 minutes ago, and 30 minutes ago, respectively, where μ1, μ2, and μ3 represent the weights of memory usage now, 15 minutes ago, and 30 minutes ago, respectively, and μ1 + μ2 + μ3 = 1. By adjusting these weights, a better overall average memory usage performance metric can be obtained. Users can select specific values ​​for these weights according to the needs of specific use case scenarios. μ1, μ2, and μ3 are assigned default values ​​of 0.44, 0.33, and 0.23, respectively. It should be noted that although the above description assumes the use of current or current, 15 minutes ago, and 30 minutes ago memory usage percentage metrics, this is not necessary. Other embodiments may utilize only subsets of these times, or one or more other times may be used to supplement or replace one or more of these times.

[0071] I / O load performance can be assessed by considering several factors, including IOPS and bandwidth. IOPS is a metric used to characterize the I / O performance of a storage device, while bandwidth is the amount of data that can be transferred within a fixed time period (e.g., it can be expressed as bits per second (bps), bytes per second, etc.). Similar to CPU and memory performance evaluations, I / O load performance can be considered at multiple points in time (e.g., now, 5 minutes ago, and 15 minutes ago).

[0072] P IO平均 =ξ1·P IO,现在 +ξ2·P IO,5分钟 +ξ3·P IO,15分钟

[0073] and

[0074] P IO,时间i =P IOPS,时间i +P 带宽,时间期限i

[0075] Where P IO,现在 P IO,5分钟 and P IO,15分钟 These represent the current (e.g., present), 5 minutes ago, and 15 minutes ago percentages of I / O load, respectively, where ξ1, ξ2, and ξ3 represent the weights of the current, 5 minutes ago, and 15 minutes ago I / O load, and where ξ1 + ξ2 + ξ3 = 1. Adjusting these weights can achieve better overall average I / O load performance. Users can select specific values ​​for these weights based on the needs of their specific use case scenarios. ξ1, ξ2, and ξ3 are assigned default values ​​of 0.44, 0.33, and 0.23, respectively. It should be noted that although the above description assumes the use of current or present, 5 minutes ago, and 15 minutes ago percentages of memory usage, this is not necessary. Other embodiments may utilize only subsets of these times, or one or more other times may be used to supplement or replace one or more of these times.

[0076] To represent IOPS and bandwidth metrics in a percentage format similar to CPU and memory metrics, IOPS and bandwidth metrics can be enhanced or adjusted as follows:

[0077]

[0078]

[0079] IOPS 最大 This represents the maximum theoretical IOPS value, and the bandwidth within it... 最大 This represents the maximum theoretical bandwidth value. The I / O load of the NAS server can be determined by summing the loads of each file system associated with the NAS server.

[0080]

[0081] In some embodiments, the dynamic multi-metric workload balancing algorithm considers the metrics described above for CPU utilization, memory utilization (e.g., main memory, cache memory, buffers), and I / O load (e.g., IOPS and bandwidth) to trigger a workload balancing action in response to the detection of a specified imbalance. As mentioned above, such workload balancing actions may include moving NAS servers between SPs when the performance of a given SP falls below a specified threshold performance (or is trending toward and expected to fall below a specified threshold performance). This allows for better file performance and system performance to be achieved by utilizing SP resources, while avoiding undesirable problems such as OOM (Out of Memory) issues. The dynamic multi-metric workload balancing algorithm will be described in further detail below using the following notation:

[0082] C=ω CPU ·P CPU平均 +ω Mem ·P Mem平均 +ω IO ·P IO平均

[0083] Where C indicates the performance or total workload of SPi, and where P CPU P Mem and P IO The percentages of CPU utilization, memory utilization, and I / O load are displayed separately.

[0084] P CPU平均 =λ1·P CPU,1分钟 +λ2·P CPU,5分钟 +λ3·P CPU,15分钟

[0085] Where P CPU平均 Indicates the weighted average CPU utilization percentage, where P CPU,1分钟 P CPU,5分钟 and P CPU,15分钟 The percentage of CPU utilization in the last minute, the last five minutes, and the last fifteen minutes are respectively labeled, where λ1, λ2, and λ3 are respectively labeled as the weights of the percentage of CPU utilization in the last minute, the last five minutes, and the last fifteen minutes, and λ1+λ2+λ3=1.

[0086] P Mem平均 =μ1·P Mem,现在 +μ2·P Mem,15分钟 +μ3·P Mem,30分钟

[0087] Where P Mem平均Indicates the weighted average memory utilization percentage, where P Mem,现在 P Mem,15分钟 and P Mem,30分钟 The memory usage percentages for now, 15 minutes ago, and 30 minutes ago are indicated, respectively, where μ1, μ2, and μ3 indicate the weights of the memory usage percentages for now, 15 minutes ago, and 30 minutes ago, respectively, and μ1+μ2+μ3=1.

[0088] P Mem,时间i =P mainMem,时间i +P cacheMem,时间i +P buffer,时间i

[0089] Where P Mem,时间i The memory utilization rate at time i is the main memory utilization rate at time i (denoted as P). mainMem,时间i The cache utilization rate at time i (denoted as P) cacheMem,时间i ) and the buffer utilization rate at time i (denoted as P) buffer,时间i ) and.

[0090] P IO平均 =ξ1·P IO,现在 +ξ2·P IO,5分钟 +ξ3·P IO,15分钟

[0091] Where P IO平均 Indicates the weighted average IO load percentage, where P IO,现在 P IO,5分钟 and P IO,15分钟 The percentages of IO load at present, 5 minutes ago, and 15 minutes ago are indicated respectively, where ξ1, ξ2, and ξ3 indicate the weights of IO load at present, 5 minutes ago, and 15 minutes ago, respectively, and ξ1+ξ2+ξ3=1.

[0092] P IO,时间i =P IOPS,时间i +P 带宽,时间期限i

[0093] Where P IO,时间i The IO load at time i is the IOPS load index at time i (denoted as P). IOPS,时间i ) and the bandwidth load index at time i (denoted as P) 带宽,时间i ) and.

[0094]

[0095]

[0096] Where P IOPS,时间iThe IOPS load index at time i is denoted as the IOPS value at time i relative to the maximum theoretical IOPS value (denoted as IOPS). 最大 The percentage of IOPS (displayed as IOPS) 时间i ), and P 带宽,时间i The bandwidth load index at time i is represented as the value at time i relative to the maximum theoretical bandwidth (denoted as bandwidth). 最大 The percentage of bandwidth (displayed as bandwidth) 时间i ).

[0097]

[0098] Where C 平均 This indicates the average workload or performance of the storage array or system as a whole, where the storage array or system includes components denoted as C. SPA and C SPB The two SPs. For clarity, the following description assumes that only two SPs exist in the storage array or system under consideration. However, in other embodiments, the storage array or system may include any desired number of SPs, and the formulas above and below may be adjusted accordingly.

[0099]

[0100] Here, σ represents the standard deviation of the workload of SPi and is a measure of the dispersion of the SP workload or performance value. N = 2, because it is assumed that there are two SPs, labeled SPA and SPB. A low standard deviation indicates that the SP workload or performance value of SPSPA and SPB tends to be close to the mean (also known as the expected value) of the set, while a high standard deviation indicates that the value is dispersed over a wider range.

[0101]

[0102] Where φ indicates the SP performance imbalance rate of the storage array or system. Θ is used to indicate the acceptable threshold for the system workload or performance imbalance rate, and Ω indicates the acceptable threshold for SP workload. If the SP workload exceeds the threshold Ω, the algorithm can consider the SP as overloaded and initiate one or more workload balancing operations.

[0103] The dynamic multi-metric workload balancing algorithm will periodically evaluate the workload or performance of each SP. Continuing with the example above with two SPs, SPSPA and SPB, C will be determined. SPA and C SPBAnd the SP performance imbalance rate φ. Several conditions are present to trigger a workload balancing operation, including: (1) whether the imbalance rate φ exceeds the threshold Θ; and (2) whether the workload C of any SP exceeds the threshold Ω. In some embodiments, if only condition (1) is met but condition (2) is not met, then no workload balancing operation needs to be performed because the workload of each SP is below the threshold Ω and the algorithm considers that each SP can handle its own workload without negatively impacting performance or causing system problems. Therefore, unnecessary resource consumption caused by performing a workload balancing operation can be avoided. If both conditions (1) and (2) are met (where condition (2) should be met for at least one SP but not necessarily all SPs), then a workload balancing operation will be initiated.

[0104] Figure 5A and Figure 5B The process flow of a dynamic multi-index workload balancing algorithm is shown, including the performance of the workload balancing operation. For example... Figure 5A As shown, the process flow begins at step 501, and in step 503, the workloads of SPSPA and SPB are calculated. In step 505, it is determined whether the imbalance rate φ (e.g., the imbalance rate between SPA and SPB) exceeds a threshold Θ, and whether at least one of the workloads C of SP exceeds a threshold Ω. If the determination result of step 505 is "No", the process ends at step 521. If the determination result of step 505 is "Yes", the process proceeds to step 507.

[0105] In step 507, the SP with the worst performance (e.g., in this example, having a higher C value in both SPA and SPB) is selected. SP The SP is selected as the source SP, and the SP with better performance is selected as the target storage processor. In step 509, for each NAS server in the source SP that meets the SP owner change condition, the sum of the workloads of all file systems created on the NAS server is calculated, and the SP with the highest workload (worst performance) (labeled as C) is selected. Nas服务器 The target NAS server is selected from the NAS servers listed below. In step 511, all replication sessions on the target NAS server and its file system are paused.

[0106] like Figure 5B As shown, the process flow continues to step 513, where the SP owner of the target NAS server changes from the source SP to the target SP. In step 515, the workload C of SPSPA and SPB is recalculated. SPA and C SPB In step 517, the recalculated workload C of the source SP is determined. 源SP Is the recalculated workload C less than the target SP?目标SP If the determination result of step 517 is "No", the process flow returns to step 507. If the determination result of step 517 is "Yes", the process flow proceeds to step 519, where the change of the SP owner of the target NAS server is rolled back to the source SP. Then, the process flow ends in step 521.

[0107] exist Figure 5A and Figure 5B In the process flow, it is assumed that NAS servers can be moved locally between SPs within the same storage array or system (e.g., between SPs in the same "rack"). Typically, transparent local NAS server mobility between SPs in the same rack can only be manually triggered. Automatic balancing between SPs offers various advantages to avoid performance issues. In such cases, all replication sessions should be paused until the SP owner changes. In some embodiments, local NAS server mobility has two limitations: the movement is not transparent in all cases, but only in specific environments (e.g., NFSv3 and SMB3+CA client environments); and the target NAS server for file import is not allowed to be moved between SPs. These limitations can be considered when determining the target NAS server (e.g., only NAS servers meeting such conditions are eligible to be selected as target NAS servers). However, it should be understood that the above conditions and limitations are specific to one technology for transparent local NAS server mobility between SPs in the same rack and are not necessarily applicable to other types of technologies that enable NAS servers to move within SPs.

[0108] Compared to conventional methods, the illustrative embodiments offer various advantages. For example, some embodiments utilize a multi-metric workload balancing algorithm for NAS servers, which considers CPU utilization, memory utilization, and I / O load metrics to trigger workload balancing operations between SPs by moving NAS servers between SPs to achieve better file performance and system reliability. This algorithm offers various advantages over methods that perform balancing only at the network level (e.g., link aggregation using NIC bundling). NAS server failover can be used when an SP fails or goes offline. However, the algorithm described herein can automatically trigger workload balancing between SPs to avoid problems that could lead to SP failures or offline status (e.g., OOM issues, SP emergencies, etc.). In the above description, it is assumed that a file-based storage device is used instead of a block-based storage device. Block devices can utilize multipathing techniques to perform workload balancing on the host side. Furthermore, the block-side management granularity can be at the LUN level. If CPU, memory utilization, and I / O load metrics are obtained at the LUN level, the above techniques can be similarly applied to block storage devices to enhance block-side workload balancing.

[0109] It should be understood that the specific advantages described above and elsewhere herein are associated with specific illustrative embodiments and are not required to exist in other embodiments. Moreover, the specific types of information processing system features and functionalities shown in the figures and as described above are merely exemplary, and numerous other arrangements may be used in other embodiments.

[0110] Now refer to Figure 6 and Figure 7 An illustrative embodiment of a processing platform for implementing multi-metric workload balancing across storage resources is described in more detail. Although described in the context of system 100, these platforms may also be used to implement at least a portion of other information processing systems in other embodiments.

[0111] Figure 6 An exemplary processing platform including cloud infrastructure 600 is shown. Cloud infrastructure 600 includes components that can be used to implement… Figure 1 The information processing system 100 comprises at least a portion of physical and virtual processing resources. Cloud infrastructure 600 includes multiple virtual machines (VMs) and / or container sets 602-1, 602-2, ..., 602-L implemented using virtualization infrastructure 604. Virtualization infrastructure 604 runs on physical infrastructure 605 and illustratively includes one or more hypervisors and / or operating system-level virtualization infrastructures. Operating system-level virtualization infrastructure illustratively includes the kernel control group of a Linux operating system or other types of operating systems.

[0112] The cloud infrastructure 600 also includes collections of applications 610-1, 610-2, ..., 610-L running on corresponding collections of VMs / containers in collections 602-1, 602-2, ..., 602-L, under the control of the virtualization infrastructure 604. The collection of VMs / containers 602 may include a corresponding VM, a corresponding collection of one or more containers, or a corresponding collection of one or more containers running in a VM.

[0113] exist Figure 6 In some implementations of the embodiments, the VM / container set 602 includes corresponding VMs implemented using a virtualization infrastructure 604 including at least one hypervisor. A hypervisor platform can be used to implement the hypervisor within the virtualization infrastructure 604, wherein the hypervisor platform has an associated virtual infrastructure management system. The underlying physical machine may include one or more distributed processing platforms, which include one or more storage systems.

[0114] exist Figure 6 In other implementations of the embodiments, the VM / container set 602 includes corresponding containers implemented using virtualization infrastructure 604 that provides operating system-level virtualization functionality (e.g., support for Docker containers running on bare metal hosts or running on VMs). The containers are illustratively implemented using the corresponding kernel control groups of the operating system.

[0115] As is evident from the above, one or more of the processing modules or other components of system 100 may each run on a computer, server, storage device, or other processing platform element. Such an element may be considered as an example of what is more generally referred to herein as a "processing device". Figure 6 The cloud infrastructure 600 shown can represent at least a portion of a processing platform. Another example of such a processing platform is... Figure 7 The processing platform 700 shown.

[0116] In this embodiment, the processing platform 700 includes a portion of the system 100 and includes a plurality of processing devices, identified as 702-1, 702-2, 702-3, ..., 702-K, that communicate with each other via a network 704.

[0117] Network 704 can include any type of network, such as global computer networks (such as the Internet), WANs, LANs, satellite networks, telephone or wired networks, cellular networks, wireless networks (such as WiFi or WiMAX networks), or various parts or combinations of these and other types of networks.

[0118] The processing device 702-1 in the processing platform 700 includes a processor 710 coupled to a memory 712.

[0119] The processor 710 may include a microprocessor, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a video processing unit (VPU), or other types of processing circuitry, as well as portions or combinations of such circuitry elements.

[0120] Memory 712 may take any combination of forms including random access memory (RAM), read-only memory (ROM), flash memory, or other types of memory. Memory 712 and other memories disclosed herein should be considered as illustrative examples of what is more generally referred to as a “processor-readable storage medium” storing executable program code of one or more software programs.

[0121] Articles of manufacture including such processor-readable storage media are considered illustrative embodiments. Given such articles of manufacture, they may include, for example, storage arrays, storage disks, or integrated circuits comprising RAM, ROM, flash memory, or other electronic memory, or any of a variety of other types of computer program products. As used herein, the term "article of manufacture" should be understood to exclude transient propagated signals. Numerous other types of computer program products including processor-readable storage media may be used.

[0122] The processing device 702-1 also includes a network interface circuit 714 for interfacing the processing device with the network 704 and other system components, and may include a conventional transceiver.

[0123] The other processing devices 702 of the processing platform 700 are assumed to be configured in a manner similar to that shown for the processing device 702-1 in the figure.

[0124] Furthermore, the specific processing platform 700 shown in the figure is presented only by way of example, and the system 100 may include additional or alternative processing platforms, as well as a number of different processing platforms in any combination, wherein each such platform includes one or more computers, servers, storage devices or other processing devices.

[0125] For example, other processing platforms used to implement the illustrative embodiments may include converged infrastructure.

[0126] Therefore, it should be understood that in other embodiments, different arrangements of additional or alternative elements may be used. At least a subset of these elements may be implemented together on a common processing platform, or each such element may be implemented on a separate processing platform.

[0127] As previously indicated, components of the information processing system disclosed herein can be implemented, at least in part, in the form of one or more software programs stored in memory and executed by a processor of a processing device. For example, at least part of the functionality disclosed herein for performing multi-metric workload balancing among storage resources is illustratively implemented in the form of software running on one or more processing devices.

[0128] It should be emphasized again that the above embodiments are presented for illustrative purposes only. Many variations and other alternative embodiments can be used. For example, the disclosed techniques are applicable to a wide variety of other types of information processing systems, storage systems, performance metrics, etc. Moreover, the specific configurations of the system and apparatus elements illustratively shown in the drawings, and the associated processing operations, may change in other embodiments. Furthermore, the various assumptions made above in describing the illustrative embodiments should be considered exemplary and not as requirements or limitations of this disclosure. Numerous other alternative embodiments within the scope of the appended claims will be apparent to those skilled in the art.

Claims

1. An apparatus comprising: At least one processing device, the at least one processing device including a processor coupled to a memory; The at least one processing device is configured to perform the following steps: Determine the workload level of each storage processor in a set of two or more storage processors, each storage processor running a set of network-attached storage servers, wherein each network-attached storage server is associated with one or more file systems, and wherein the workload level is based at least in part on processor performance metrics, memory performance metrics, and load performance metrics. Identify the performance imbalance rate of the set of two or more storage processors; and Workload balancing is performed on the set of two or more storage processors in response to: (i) the performance imbalance rate of the set of two or more storage processors exceeds a specified imbalance rate threshold; and (ii) the workload level of at least one of the two or more storage processors in the set exceeds a specified threshold workload level; The workload balancing of the set of two or more storage processors is performed through the following operations: (i) Select a first storage processor as the source storage processor and select a second storage processor as the target storage processor; (ii) For each network-attached storage server in the set of network-attached storage servers running on the source storage processor, calculate the sum of the workloads of the one or more file systems of the network-attached storage server; (iii) Select one network-attached storage server from the set of network-attached storage service servers running on the source storage processor as the target network-attached storage server, based at least in part on the sum of the calculated workloads. (iv) Suspend the replication sessions on the target network-attached storage server and any file systems associated with the target network-attached storage server; (v) Change the owner of the storage processor of the target network attached storage service server from the source storage processor to the target storage processor; (vi) Recalculate the sum of the workloads of the network-attached storage service servers running on the source storage processor and the target storage processor; as well as (vii) In response to determining that the sum of the recalculated workloads of the set of network-attached storage service servers running on the source storage processor is greater than the sum of the recalculated workloads of the set of network-attached storage service servers running on the target storage processor, another target network-attached storage server is selected from the set of network-attached storage service servers running on the source storage processor, at least in part based on the sum of the recalculated workloads, and steps (v)-(vi) are repeated until the sum of the most recently recalculated workloads of the set of network-attached storage service servers running on the source storage processor is less than the sum of the most recently recalculated workloads of the set of network-attached storage service servers running on the target storage processor.

2. The device of claim 1, wherein the set of two or more storage processors is part of a single storage system.

3. The device of claim 1, wherein the set of two or more storage processors is part of two or more storage systems.

4. The device of claim 1, wherein the processor performance metric comprises a weighted sum of two or more processor utilization metrics associated with two or more different time intervals.

5. The device of claim 4, wherein the two or more processor utilization metrics associated with the two or more different time intervals include: The first processor utilization metric represents the current state of processor utilization for a given memory processor within a first time period relative to the current time. The second processor utilization rate represents the most recent state of the processor utilization rate of the given memory processor within a second time period relative to the current time, the second time period being longer than the first time period; as well as The third processor utilization rate represents the persistent state of the processor utilization rate of the given storage processor over a third time period relative to the current time, the third time period being longer than the second time period.

6. The device of claim 1, wherein the memory performance metric comprises a weighted sum of two or more memory utilization metrics associated with two or more different times.

7. The device of claim 6, wherein the two or more memory utilization metrics associated with the two or more different times include: The first memory utilization metric represents the percentage of memory utilization of a given memory processor at a first point in time relative to the current time. A second memory utilization metric represents the percentage of memory utilization of a given memory processor at a second time point relative to the current time, which is later than the first time point. as well as A third memory utilization metric represents the percentage of memory utilization of the given memory processor at a third point in time relative to the current time, which is later than the second point in time.

8. The device of claim 6, wherein each of the two or more memory utilization metrics comprises the sum of main memory utilization, cache memory utilization, and buffer memory utilization at one of the two or more different times.

9. The device of claim 1, wherein the load performance index comprises a weighted sum of two or more load indices associated with two or more different times.

10. The device of claim 9, wherein the two or more load indicators associated with the two or more different times comprise: The first load metric represents the percentage of load on a given storage processor at a first point in time relative to the current time. The second load metric represents the percentage of the load on the given storage processor at a second point in time relative to the current time, which is later than the first point in time. as well as The third load metric represents the percentage of the load on the given storage processor at a third point in time relative to the current time, which is later than the second point in time.

11. The device of claim 9, wherein each of the two or more load metrics comprises the sum of an input-output operation metric per second and a bandwidth metric within one of the two or more different times, wherein the input-output operation metric per second is expressed as a percentage of a specified maximum input-output operation per second, and wherein the bandwidth metric is expressed as a percentage of a specified maximum bandwidth.

12. A computer program product comprising a non-transitory processor-readable storage medium therein storing program code of one or more software programs, wherein the program code, when executed by at least one processing device, causes the at least one processing device to perform the following steps: Determine the workload level of each storage processor in a set of two or more storage processors, each storage processor running a set of network-attached storage servers, wherein each network-attached storage server is associated with one or more file systems, and wherein the workload level is based at least in part on processor performance metrics, memory performance metrics, and load performance metrics. Identify the performance imbalance rate of the set of two or more storage processors; and Workload balancing is performed on the set of two or more storage processors in response to: (i) the performance imbalance rate of the set of two or more storage processors exceeds a specified imbalance rate threshold; and (ii) the workload level of at least one of the two or more storage processors in the set exceeds a specified threshold workload level; The workload balancing of the set of two or more storage processors is performed through the following operations: (i) Select a first storage processor as the source storage processor and select a second storage processor as the target storage processor; (ii) For each network-attached storage server in the set of network-attached storage servers running on the source storage processor, calculate the sum of the workloads of the one or more file systems of the network-attached storage server; (iii) Select one network-attached storage server from the set of network-attached storage service servers running on the source storage processor as the target network-attached storage server, based at least in part on the sum of the calculated workloads. (iv) Suspend the replication sessions on the target network-attached storage server and any file systems associated with the target network-attached storage server; (v) Change the owner of the storage processor of the target network attached storage service server from the source storage processor to the target storage processor; (vi) Recalculate the sum of the workloads of the network-attached storage service servers running on the source storage processor and the target storage processor; as well as (vii) In response to determining that the sum of the recalculated workloads of the set of network-attached storage service servers running on the source storage processor is greater than the sum of the recalculated workloads of the set of network-attached storage service servers running on the target storage processor, another target network-attached storage server is selected from the set of network-attached storage service servers running on the source storage processor, at least in part based on the sum of the recalculated workloads, and steps (v)-(vi) are repeated until the sum of the most recently recalculated workloads of the set of network-attached storage service servers running on the source storage processor is less than the sum of the most recently recalculated workloads of the set of network-attached storage service servers running on the target storage processor.

13. A method comprising: Determine the workload level of each storage processor in a set of two or more storage processors, each storage processor running a set of network-attached storage servers, wherein each network-attached storage server is associated with one or more file systems, and wherein the workload level is based at least in part on processor performance metrics, memory performance metrics, and load performance metrics. Identify the performance imbalance rate of the set of two or more storage processors; and Workload balancing is performed on the set of two or more storage processors in response to: (i) the performance imbalance rate of the set of two or more storage processors exceeds a specified imbalance rate threshold; and (ii) the workload level of at least one of the two or more storage processors in the set exceeds a specified threshold workload level; The workload balancing of the set of two or more storage processors is performed through the following operations: (i) Select a first storage processor as the source storage processor and select a second storage processor as the target storage processor; (ii) For each network-attached storage server in the set of network-attached storage servers running on the source storage processor, calculate the sum of the workloads of the one or more file systems of the network-attached storage server; (iii) Select one network-attached storage server from the set of network-attached storage service servers running on the source storage processor as the target network-attached storage server, based at least in part on the sum of the calculated workloads. (iv) Suspend the replication sessions on the target network-attached storage server and any file systems associated with the target network-attached storage server; (v) Change the owner of the storage processor of the target network attached storage service server from the source storage processor to the target storage processor; (vi) Recalculate the sum of the workloads of the network-attached storage service servers running on the source storage processor and the target storage processor; as well as (vii) In response to determining that the sum of the recalculated workloads of the set of network-attached storage service servers running on the source storage processor is greater than the sum of the recalculated workloads of the set of network-attached storage service servers running on the target storage processor, another target network-attached storage server is selected from the set of network-attached storage service servers running on the source storage processor, at least in part based on the sum of the recalculated workloads, and steps (v)-(vi) are repeated until the sum of the most recently recalculated workloads of the set of network-attached storage service servers running on the source storage processor is less than the sum of the most recently recalculated workloads of the set of network-attached storage service servers running on the target storage processor; and The method is performed by at least one processing device, which includes a processor coupled to a memory.

Images