MANAGEMENT OF WORKLOAD USING A TRAINED MODEL

Abstract

A non-transitory, machine-readable storage medium containing instructions that, when executed, cause a system (400) to: Generating a training dataset (122) based on features of example workloads, wherein the training dataset (122) includes labels associated with the features of the example workloads, the labels being based on load indicators generated in a computing environment and relating to load conditions of the computing environment resulting from the execution of the example workloads; Grouping (502) selected workloads (106) into a plurality of workload clusters (118) based on features of the selected workloads (106), wherein a workload cluster (118) of the plurality of workload clusters (118) includes workloads (106) that are similar to each other according to a similarity criterion; Computation (506) of parameters (124) representing the contributions of each workload cluster (118) of the plurality of workload clusters (118) to a load condition in the computing environment, using a model (120) trained on the basis of the training data set (122), wherein the parameters (124) include a first parameter (124) representing a contribution of a first workload cluster (118) to the load condition and a second parameter (124) representing a contribution of a second workload cluster (118) to the load condition, wherein the first and second workload clusters (118) are part of the plurality of workload clusters (118); Selection of a workload cluster (118) from the multitude of workload clusters (118) based on different values ​​of the calculated parameters (124), wherein a value of a parameter (124) calculated for a selected workload cluster (118) indicates that the selected workload cluster (118) has a workload (106) that negatively affects the work performance of a workload (106) in another workload cluster (118); and Perform workload management (508) in the computer environment based on the calculated parameters (124), wherein workload management (508) includes: Determining a relative priority of the identified workload (106) over other workloads (106), and Restricting the resource utilization of the identified workload (106) in the selected workload cluster (118) in response to the determined relative priority.

Description

background

[0001] A storage system comprises storage resources and other resources (including processing and communication resources) on which various types of workloads can run. Storage resources can refer to physical or logical storage resources used to store data. Different workloads can compete for storage system resources, leading to resource conflicts that can negatively impact the performance of some workloads.

[0002] WO 2017 / 105376A1 discloses methods, systems, and computer-readable media for optimizing a system configuration. In some examples, a method includes determining whether a system configuration for running a workload in a distributed computing system is optimizable and, if so, modifying the system configuration so that at least one storage resource for storing workload data is located on a server node that runs the workload in the distributed computing system.

[0003] US 10 768 998 B2 discloses workload management with data access-oriented control in a compute cluster. After a workload is submitted for scheduling by a workload manager, a data request analysis module retrieves the necessary input data from a storage system. This module creates a ranking of cluster hosts, prioritized for workload execution based on data access criteria, and provides this list to a scheduler module. The scheduler module generates an optimized plan for distributing the workload across the hosts in the compute cluster, taking the data access criteria into account.

[0004] US Patent 2019 / 0334786 A1 discloses a system or method for predicting utilization and latency patterns in a data storage network. This system or method includes training a model in the cloud based on I / O data from the data storage network, monitoring sample I / O data in the data storage network at predefined intervals, and determining a utilization fingerprint in the trained model that corresponds to the sample I / O data. The method further includes calculating a utilization score for the sample I / O data and predicting future utilization and latency patterns using an autoregressive integrated moving average calculation based on the sample I / O time series data and the calculated utilization score for the sample I / O. Brief description of the drawings

[0005] Some embodiments of the present disclosure are described with reference to the following figures. Fig. is a block diagram of an arrangement that includes a storage system and a workload management engine, according to some examples. Fig. This is a flowchart of a machine learning-based workload management process, based on some examples. Fig. is a block diagram of the storage medium that stores machine-readable instructions, according to some examples. Fig. This is a block diagram of a system, based on some examples. Fig. This is a flowchart of a process, based on some examples.

[0006] In the drawings, identical reference numbers denote similar, but not necessarily identical, elements. The illustrations are not necessarily to scale, and the size of some parts may be exaggerated to make the example shown clearer. Furthermore, the drawings contain examples and / or embodiments that correspond to the description; however, the description is not limited to the examples and / or embodiments shown in the drawings. Detailed description

[0007] In this disclosure, the use of the term "a," "an," or "the" includes the plural forms unless the context clearly indicates otherwise. Similarly, the terms "includes," "including," "comprises," "have," or "have," when used in this disclosure, specify the presence of the elements indicated but do not exclude the presence or addition of other elements.

[0008] Multiple workloads running on a storage system can overload the system's shared resources. Shared resources can include physical storage resources such as storage devices, including disk-based storage devices, solid-state storage devices, and memory devices (e.g., DRAM (Dynamic Random Access Memory), SRAM (Static Random Access Memory), etc.). Shared resources also include processing resources (e.g., processors, processor cores, etc.) and communication resources (e.g., network interface controllers, switches, etc.).

[0009] Shared resources can also include logical resources such as virtual machines, virtual processors, virtual communication devices, etc.

[0010] A "workload" can refer to any collection of activities performed by an entity (e.g., machine-readable instructions) within a storage system. More generally, a "workload" can refer to any collection of activities performed by an entity (e.g., machine-readable instructions) within a computing environment, such as the computing environment of a storage system, a computer system, a communications system, a cloud system, a data center, a server system, and so on.

[0011] A primary workload can negatively impact the performance of a secondary workload in the storage system in scenarios where the resource consumption of the primary workload prevents the secondary workload from executing or causes it to run more slowly. Workloads that negatively impact the performance of other workloads can be referred to as "noisy neighbors."

[0012] Different types of workloads in a storage system can be characterized by a variety of I / O types and access patterns with respect to a storage volume (or multiple storage volumes). I / O types can refer to read accesses to read data from the storage medium(s), write accesses to write data to the storage medium(s), I / O accesses with different priorities, different categories of I / O accesses (e.g., I / O accesses associated with read and write operations requested by host systems, I / O accesses associated with synchronization operations, I / O accesses associated with backup operations, etc.). Different access patterns can be based on any one or a combination of the following factors: frequency of access to storage volumes, amount of data accessed in a particular access operation, frequency of I / O accesses, etc.

[0013] A "storage volume" can refer to a logical container or other construct to which data can be added, for example, through write operations. Data can also be retrieved from a storage volume.

[0014] If an overload condition is detected in a storage system, a human administrator of the storage system can be notified. The administrator can attempt to resolve the cause of the resource consumption in the storage system based on a manual analysis of the information collected for the workloads running in the storage system. Such manual analysis can be time-consuming and may yield inaccurate results.

[0015] In other examples, storage systems may be equipped with Quality of Service (QoS) mechanisms designed to ensure that workloads are delivered with predictable service levels. In some examples, QoS mechanisms may apply workload limits to restrict the resource usage of a particular workload and prevent that workload from monopolizing the storage system's resources.

[0016] However, knowing the precise workload constraints to ensure that storage system resources are appropriately distributed among competing workloads can be challenging. For example, the administrator must examine historical profiles of workloads sharing storage system resources to determine where and when resource conflicts occur and to identify which workloads are excessively consuming shared resources. Such manual analysis can be difficult and may result in incorrect constraints being applied to workloads, potentially reducing the performance of some.

[0017] In accordance with some implementations of this disclosure, workloads in a computing environment (e.g., a storage system) can be grouped into multiple workload clusters based on workload characteristics. A model (trained on a training dataset) can be used to calculate parameters that represent the contributions of each workload cluster to a load in the computing environment. Workload management in the computing environment can then be performed based on these calculated parameters.

[0018] The following discussion refers to workload management for storage systems. It should be noted that the techniques or mechanisms described in some examples can also be used for workload management in other types of data processing environments.

[0019] Fig. is a block diagram of an example arrangement comprising a storage system 102 and a machine learning-based workload management engine 104 that applies machine learning in performing workload management of workloads 106 running in the storage system 102. Although Fig. While this shows that the workload management engine 104 is separate from the storage system 102, in other examples the workload management engine 104 may be part of the storage system 102.

[0020] As used here, an “engine” can refer to a hardware processing circuit, which may include any or a combination of a microprocessor, a core of a multi-core microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, or other hardware processing circuitry. Alternatively, an “engine” can refer to a combination of a hardware processing circuit and machine-readable instructions (software and / or firmware) that can be executed on the hardware processing circuit.

[0021] The workloads 106 can be executed by programs (or more generally, machine-readable instructions) or other entities in the storage system 102. The workloads 106 can be executed in response to requests from host systems 108 received over a network 110. A "host system" can refer to any device (e.g., a server computer, a user computer, a smartphone, a vehicle, etc.) capable of requesting access to data (reading or writing data) in the storage system 102 or performing other operations on the storage system 102. The network 110 can include a Storage Area Network (SAN), a Local Area Network (LAN), etc.

[0022] Storage system 102 comprises resources 112, some of which can be shared by workloads 106 during execution. These resources 112 can include physical and / or logical resources of storage system 102.

[0023] The Workload Management Engine 104 comprises a Workload Clustering Logic 114 and a Workload Cluster Contribution Parameter Calculation Logic 116. As used here, a “logic” that is part of the Workload Management Engine 104 can refer to a part of the hardware processing circuitry of the Workload Management Engine 104 or to machine-readable instructions that can be executed by the Workload Management Engine 104.

[0024] The workload management that can be performed by the workload management engine 104 can include controlling the use of resources 112 by workloads 106. For example, the workload management engine 104 can dynamically impose restrictions on the use of resources 112 by specific workloads 106. For instance, if the workload management engine 104 determines (based on tasks performed by the workload clustering logic 114 and the workload cluster contribution parameter calculation logic 116, as explained below) that a specific workload 106 is impacting other workloads 106, the workload management engine 104 can impose a restriction on the use of resources 112 by that specific workload 106. For example, the workload management engine can impose a bandwidth limit for accessing a resource 112 (or multiple resources 112).For example, a bandwidth limit can be set for a storage volume, restricting the access rate to that volume by a given workload to less than a specific I / O throughput. This bandwidth limit could be part of a QoS rule, where a workload is limited to X% of the maximum I / O throughput used by the workload over a specific period (e.g., 4 hours or another timeframe). X% could be 75% or another percentage and can be dynamically adjusted.

[0025] In some examples, the decision of whether or not a workload is subject to a resource usage restriction may also be based on the workload's priority as well as other factors (e.g., whether the workload is short-term or long-term). A high-priority workload that is expected to run for a relatively short time may not be subject to a resource usage restriction, even if it consumes a relatively large amount of resources. However, if a workload is low-priority and / or expected to run for a relatively long time, it may be subject to a resource usage restriction if it consumes a relatively large amount of resources.

[0026] The workload clustering logic 114 groups the workloads 106 into multiple workload clusters 118 based on characteristics of the workloads 106. Each workload cluster 118 represents workloads that are similar to each other according to a similarity criterion (e.g., characteristics have values ​​that are similar to each other within a certain range or threshold). For example, workloads in a workload cluster include workloads that have similar I / O patterns, which can be based on I / O types and access patterns (as explained above).

[0027] A workload characteristic can refer to an attribute that represents a feature of the workload. For example, a characteristic might represent write throughput when accessing a storage volume. Write throughput could refer to the rate of write accesses to a number of storage devices, where "number" can be 0, 1, or more than 1. Other examples might use different characteristics, such as read throughput, which refers to read accesses to a number of storage devices; total throughput for both write and read accesses to a number of storage devices; or any other attribute that represents a feature related to I / O accesses to data in a number of storage devices.

[0028] In some examples, the characteristics of a workload can include write throughput measurements taken by a workload of a given storage volume at appropriate intervals over a (larger) overall period. For example, write throughput measurements for the given storage volume can be taken every 5-minute interval over a 4-hour period. A write throughput measurement can be captured by a monitoring agent (implemented in hardware or as machine-readable instructions) in storage system 102. In the example above, there are 48 write throughput measurements in the 48 5-minute intervals over the 4 hours. The 48 write throughput measurements for each workload yield 48 characteristics that are used for clustering. In other examples, different time intervals can be used for write throughput measurements.The total period during which the write throughput measurements are taken can also be changed.

[0029] In further examples, features for clustering workloads can be used in addition to or instead of measurements of workload write throughput.

[0030] Clustering workloads 106 into workload clusters 118 by the workload clustering logic 114 can reduce the number of workloads considered when determining which workload(s) are responsible for a load on the storage system 102 and thus potentially impact the performance of other workloads. For example, there might be hundreds or thousands of workloads 106 performing I / O accesses to corresponding storage volumes. By clustering the workloads 106, the workload management engine 104 considers a smaller number of workload clusters 118.

[0031] The number of workload clusters to be generated by the workload clustering logic can be selected using an optimization technique such as an elbow method. In other examples, a user (e.g., an expert, an administrator, etc.) can select the number of workload clusters to use.

[0032] In some examples, the clustering of workloads 106 applied by the workload clustering logic 114 is k-means clustering, which divides the workloads 106 into k clusters (k ≥ 2). In other examples, other clustering techniques may be used, such as mean-shift clustering, density-based spatial clustering, and so on.

[0033] Once the workload clusters 118 have been created by the workload clustering logic 114, the workload management engine 104 can determine which workload cluster 118 is responsible for a load on the storage system 102.

[0034] The logic 116 for calculating the parameters for the workload cluster contribution uses a model 120, trained on a training dataset 122, to calculate parameters 124 that represent the contributions of the respective workload clusters 118 to a load in the storage system 102. Based on the calculated parameters 124, the workload management engine 104 performs workload management in the storage system 102.

[0035] In some examples, the model 120 and the training dataset 122 can be stored in a repository 126. The repository 126 can be contained in one storage device or multiple storage devices. In examples where the workload management engine 104 is part of storage system 102, the repository 126 can be implemented using storage devices of storage system 102.

[0036] Model 120 can be trained during the normal operation of storage system 102. "Normal operation" refers to the operation of storage system 102 in response to requests received by storage system 102, for example, from host systems 108. The training dataset 122 can be continuously updated as measurements (such as the write throughput measurements described above) are provided. The training dataset 122 can contain features for a sample of workloads. As additional workloads run on storage system 102, features for such additional workloads are provided for inclusion in the training dataset 122, either to supplement the training dataset 122 or to (partially) replace previous features.

[0037] In response to an update of the training dataset 122, the model 120 can be updated (retrained) based on the updated training dataset 122. Updating (retraining) the model 120 can refer to updating the model 120's parameters using the updated training dataset 122. This allows the model 120 to adapt to changing workload characteristics over time.

[0038] The foregoing is part of a machine learning process in which the model 120 is continuously retrained based on historical workload pattern data available from the storage system 102, and the retrained model 120 is able to predict which workload contributes to the load on the storage system 102.

[0039] In some examples, the training dataset contains 122 data points defined by N variables (N = number of clusters). More generally, the training dataset contains 122 load indicators (e.g., values ​​of L_t), clusters of workloads, and measurements related to resource access (e.g., write throughput measurements).

[0040] The value of each variable is the collective write throughput of the workloads belonging to a specific workload cluster 118. For each workload, there are 48 characteristics (measurements of write throughput in the 48 time intervals; in this example, the write throughput measurements are taken at 5-minute intervals over a period of 4 hours).

[0041] In some examples, model 120 can be expressed by the following function: L_t=wc_1*wc_1_kbps+wc_2*wc_2_kbps+wc_3*wc_3_kbps+…

[0042] L_t represents the load on storage system 102 at time t. L_t can be set to a first value (e.g., 0) to indicate that the load on storage system 102 is low. L_t can be set to a second value (e.g., a value greater than 0) to indicate that the load on storage system 102 is high.

[0043] In some examples, the load on storage system 102 is assumed to be high during an observed time interval (and indicated by setting L_t = 1 or greater than 1) if delayed acknowledgments are reported during that interval. Conversely, the load on storage system 102 is assumed to be low (and indicated by setting L_t = 0) if no delayed acknowledgments are reported during that interval.

[0044] A delayed acknowledgment is generated by a backend subsystem of Storage System 102 when the backend subsystem is unable to keep up with incoming workloads (e.g., during a flood of writes). The delayed acknowledgment represents a delay in completing an access to a resource in Storage System 102. The backend subsystem can include any or a combination of the following: storage devices, cache memory, adapters that communicate with storage devices, etc.

[0045] The delayed acknowledgment takes the form of an indicator (e.g., a signal, a message, an information element, etc.) sent by the backend subsystem, for example, to host systems 108 or other requesters that have made requests to storage system 102. The backend subsystem's inability to keep up with incoming workloads may be due to an overload of processing resources, storage resources, communication resources, etc.

[0046] The Workload Management Engine 104 is able to detect the presence of delayed acknowledgments and can therefore adjust the value of L_t accordingly. For example, L_t can be set to 0 to indicate that the load on storage system 102 is low, and L_t can be set to a value greater than 0 (e.g., 1) to indicate that the load on storage system 102 is high.

[0047] In other examples, the utilization of storage system 102 during an observed time interval is assumed to be low if the utilization of resources 112 is below a certain threshold, and the utilization of storage system 102 during an observed time interval is assumed to be high if the utilization of resources 112 exceeds the certain threshold.

[0048] In equation 1, wc_i, i = 1, 2, ... is a coefficient corresponding to workload cluster i at time t, and the parameter wc_i_kbps represents a measured throughput of workload cluster i at time t (expressed in kilobits per second (kbps) in this example). Since workload cluster i comprises multiple workloads, the value of wc_i_kbps can be the average (or another mathematical aggregate such as mean, maximum, sum, etc.) of the measured throughputs of the workloads in workload cluster i.

[0049] Equation 1 is an example of a linear regression model that is updated using machine learning (by updating the training dataset 122 and retraining the model in response to triggers). The linear regression model models how certain parameters (e.g., wc_1, wc_2, etc.) and features (e.g., wc_1_kbps, wc_2_kbps, etc.) influence an outcome. According to Equation 1, the outcome is the load (represented as L_t in Equation 1) on the storage system 102.

[0050] In response to L_t with the second value (e.g., > 0), indicating that the load on storage system 102 is high, the model 120, expressed by Eq. 1 (as trained on the basis of features derived from the recent workloads 106 of storage system 102), can be used by the workload cluster contribution parameter calculation logic 116 to determine the values ​​of the parameters wc_1, wc_2, and so on. The parameters wc_1, wc_2, and so on are examples of the parameters 124. The throughput values ​​t measured in each time interval are supplied as inputs to the model 120, which outputs the parameters wc_1, wc_2, and so on.

[0051] The values ​​of parameters wc_1, wc_2, etc., reflect the contribution of each workload cluster 118 to the load on storage system 102. In some examples, statistical measures are used to determine whether a given workload cluster i is a significant workload cluster containing "bully" workloads that can negatively impact the performance of "victim" workloads. "Bully" workloads might be those that monopolize or consume an unfair share of storage system 102's resources 112. "Victim" workloads might be those whose performance is negatively impacted by other workloads.

[0052] An example of a statistical measure that can be used is the statistical p-value of a linear regression model (e.g., according to Equation 1). The p-value is calculated based on the regression model for each feature (wc_i_kbps) of the regression model. The coefficients (wc_i) of the regression model describe a mathematical relationship between each independent variable (e.g., L_t) and a dependent variable (e.g., wc_i_kbps). The p-values ​​for the coefficients indicate whether these relationships are statistically significant. For example, at a significance level of 95%, a p-value of less than 0.05 is not statistically significant. If the p-values ​​of all features based on the regression model are greater than 0.05, the workload clusters are identified as significant workload clusters (i.e., they contain bully workloads) associated with positive coefficients (wc_i) values.

[0053] In some examples, in addition to the p-values, a coefficient of determination R is also given. 2 Used to determine whether a workload cluster is a significant workload cluster. The coefficient of determination R 2 R² is a statistical measure of how well the data fit the regression line of the regression model. 2 lies between 0 and 1. If R 2 For example, if the χ² value for the regression model is below a threshold (e.g., 70% or another threshold), this can indicate, regardless of the p-values ​​and coefficient values, that none of the workload clusters are significant and therefore no workload cluster is identified as significant (i.e., no bully workloads are identified). If R² exceeds 100%, the χ² value is not considered significant. 2 In contrast to the threshold, p-values ​​and coefficient values ​​are used as described above to identify significant workload clusters.

[0054] In other examples, the Workload Management Engine 104 can organize the workload clusters 118 according to the values ​​of the parameters wc_1, wc_2, and so on. Assuming that a higher value of wc_i indicates that the respective workload cluster i contributes more to the load on storage system 102 than a workload cluster j assigned a lower value of wc_j, the workload clusters 118 can be ordered in descending order of the values ​​of the parameters wc_1, wc_2, etc. If the arrangement of the workload clusters 118 results in a ranking of workload clusters, then a given workload cluster (or several given workload clusters) can be at the top of the ranking of workload clusters that contribute the most to the load on storage system 102.The workloads in the given workload cluster(s) can be identified by the workload management engine 104 as the workloads responsible for the overload of the storage system 102.

[0055] Using the Workload Management Engine 104, as illustrated in some examples, the performance of the Storage System 102 can be improved by addressing noisy neighborhood scenarios where "bully" workloads can negatively impact the performance of "victim" workloads. Restrictions can then be imposed on the "bully" workloads to reduce their resource consumption, allowing other workloads to utilize the shared resources. As mentioned earlier, the decision to impose restrictions on "bully" workloads can also take into account workload priority, expected execution time, and / or other factors.

[0056] Training dataset 122 can be a tagged training dataset, where tags are assigned to the collections of workload-related data within training dataset 122. The workload-related data includes the features and parameters (e.g., 124) for example workloads (e.g., if the example workloads are represented by workload clusters in the last 4 hours). The assigned tags can be based on whether or not delayed acknowledgments were issued in the respective time intervals (e.g., the 5-minute intervals). The tags can, for example, be of the form L_t in Equation 1, where L_t is set to 0 if no delayed acknowledgment was issued in the time interval and L_t is set to 1 if a delayed acknowledgment was issued in the time interval t. Thus, training dataset 122 contains a collection of features (e.g., wc_i_kbps for each cluster i in the time interval t) and parameters (e.g.,, wc_i for each cluster i in the time interval t) with a corresponding label (e.g., L_t).

[0057] Note that the labels (e.g., L_t) are based on indicators of utilization (e.g., delayed acknowledgments) generated during the normal operation of storage system 102. Therefore, the labels of training dataset 122 are automatically generated during the normal operation of storage system 102, and a human would not need to manually assign any labels to the training data of training dataset 122.

[0058] To optimize the storage system 102 for predictable performance by handling noisy neighborhood scenarios, the machine learning-based workload management engine 104 automatically identifies "bullies" and "victims" in a heterogeneous workload environment. As described in Fig. As shown, the machine learning-based workload management engine 104 can perform a machine learning-based workload management process 200, which comprises two phases. In a first phase 202, the machine learning-based workload management engine 104 identifies workloads that monopolize the resources 112 of the storage system. In a second phase 210, the machine learning-based workload management engine 104 sets limits on such workloads in order to restore the performance of higher-priority workloads.

[0059] In the first phase 202, the machine learning-based workload management engine 104 applies a self-learning machine learning model (e.g., 120) that trains itself on the training dataset 122, which is based on historical workload pattern data available for the storage system 102 (e.g., 4 hours or another total period of historical data), to predict which workload(s) contribute to the load on the storage system 102. The load during an observed time interval (e.g., a 5-minute interval or another time interval) is assumed to be high (e.g., L_t set to 1) if a delayed acknowledgment is reported during the observed time interval, and the load in the observed interval is assumed to be low (e.g., L_t set to 0) if no delayed acknowledgment is reported during the observed time interval.

[0060] To determine which workload(s) (among several workloads running in storage system 102) is / are responsible for the load in storage system 102, the machine learning-based workload management engine 104 first groups (at 204) the workloads into a number of workload clusters, the number of which can be determined using an optimization technique such as an elbow method. For clustering, the machine learning-based workload management engine 104 defines characteristics of data points to be clustered, where a "data point" can comprise write throughput measurements at 5-minute intervals over a total period of 4 hours.

[0061] Note that Model 120 is trained on storage system 102 in the field (e.g., during operation at a customer's site) and is capable of retraining itself on the most recent I / O patterns (in workloads) observed on storage system 102. The data from these most recent I / O patterns is used to update the training dataset 122. For example, the machine learning-based workload management engine 104 can query four hours of workload data from storage system 102, starting four hours before the current time. This yields 48 measurements of write throughput for a storage volume (one for each 5-minute interval over 4 hours). This provides a data point with 48 features. A similar data point can be obtained for each storage medium.

[0062] The workload clustering logic 114 applies k-means clustering to the data points to create the workload clusters 118. Each workload cluster 118 represents workloads with similar I / O patterns on a storage volume, as measured over the last 4 hours. This helps to reduce the number of workloads to be tracked to a manageable number (from several hundred workloads to the number of workload clusters).

[0063] Model 120 (e.g., a linear regression model) is applied (at 206) to determine which workload cluster(s) is / are responsible for the load on storage system 102 in the respective time intervals. Model 120 illustrates how certain parameters and features affect an outcome (response), where the outcome is the load on storage system 102, a feature is a collective throughput of the workloads in a workload cluster as measured for each 5-minute time interval over the last 4 hours, and a parameter (e.g., the coefficient wc_i from equation 1) represents a contribution of the workload clusters to the load on storage system 102.

[0064] The training dataset 122 for the model 120 contains data points defined by variables for the set of workload clusters. The value of each variable is the collective write throughput of the workloads belonging to a respective workload cluster 118. There are 48 variables (for the 48 5-minute time intervals over four hours) for each workload cluster.

[0065] Since there is a relatively small number of training samples (including the 48 variables), the number of features is reduced by clustering the workloads to avoid the so-called "curse of dimensionality" in machine learning. The coefficients wc_i, i = 1, 2, ... of equation 1, which are assigned to the workload clusters by the trained model 120, are representative of the contribution of the respective workload clusters to the load in the storage system 102.

[0066] Each time Model 120 is retrained because more I / O pattern data is being executed for workloads in Storage System 102, the coefficients derived by Model 120 reflect the contribution of the workload clusters to the load in Storage System 102, as learned from the most recently collected information (e.g., the last 4 hours). Therefore, Model 120 is able to adapt to changing workload profiles over time. In some examples, as explained above, statistical measures are used to determine whether a given workload cluster i is a significant workload cluster containing critical workloads that could impact the performance of the victim workloads.

[0067] The second phase, performed by the machine learning-based workload management engine, involves applying a restriction to resource usage by a workload in a significant workload cluster based on workload priority.

[0068] In some examples, the decision of whether or not a workload is subject to a resource usage restriction may also be based on the workload's priority as well as other factors (e.g., whether the workload is short-term or long-term). A high-priority workload that is expected to run for a relatively short time may not be subject to a resource usage restriction, even if it consumes a relatively large amount of resources. However, if a workload is low-priority and / or expected to run for a relatively long time, it may be subject to a resource usage restriction if it consumes a relatively large amount of resources.

[0069] Resource utilization restrictions can include, for example, bandwidth limitations, which may be part of a QoS rule specifying that a workload can be limited to X% of the maximum I / O throughput used by the workload within a given time period (e.g., 4 hours or another timeframe). X% can be 75% or another percentage and can be dynamically adjusted.

[0070] In the second phase, the machine learning-based workload management engine (at 212) determines the priorities and other factors (e.g., execution time) of the respective workloads. This determination may involve accessing metadata associated with the workloads, where the metadata may contain information indicating the relative priorities and other factors of the corresponding workloads.

[0071] The machine learning-based workload management engine 104 then selects (at 214) workloads from the significant workload clusters identified in the first phase 202, applying resource utilization constraints to these selected workloads. The selected workloads are chosen based on a comparison of the relative priorities and other factors of the workloads within the significant workload clusters, such as lower priority and / or longer runtimes.

[0072] Fig. is a block diagram of a non-transitory machine-readable or computer-readable storage medium 300 that stores machine-readable instructions which, when executed, cause a system (e.g., the Workload Management Engine 104, the Storage System 102, etc.) to perform various tasks.

[0073] The machine-readable instructions include instructions for creating a training dataset (302) to train a training dataset (e.g., 122 in Fig. ) based on characteristics of example workloads. The training dataset contains labels associated with the characteristics of the example workloads, where the labels are based on load indicators (e.g., L_t in Eq. 1) generated in a computing environment (e.g., storage system 102) and relate to the load conditions of the computing environment resulting from the execution of the example workloads. The example workloads can be represented, for example, by workload clusters.

[0074] The machine-readable instructions contain instructions for workload grouping 304 (such as those of the workload cluster logic 114 from Fig. ), to group selected workloads into a variety of workload clusters based on workload characteristics. The "selected" workloads are those workloads to be subjected to machine learning-based workload management and may or may not include one or more of the example workloads.

[0075] In some examples, each workload runs on a corresponding storage volume within the data processing environment. In other examples, a workload cluster comprises workloads with similar I / O patterns regarding I / O access to a range of storage volumes.

[0076] In some examples, the characteristics used to perform workload clustering include metrics related to access to resources in the computing environment, such as write throughput measurements.

[0077] The machine-readable instructions include instructions 306 for calculating contribution parameters (such as logic 116 for calculating workload cluster contribution parameters in Fig. ), in order to use a model (e.g. 120 in Fig. The model, trained on the training dataset, calculates parameters (such as those in Equation 1) that represent the contributions of each workload cluster to a load state in a computing environment (e.g., Storage System 102 or another computing environment). Note that grouping the selected workloads into the multitude of workload clusters reduces the number of features processed when calculating the parameters using the model.

[0078] In some examples, the calculation of the parameters representing the contributions of each workload cluster to the load is triggered in response to the load condition meeting a criterion. For example, the load condition meets the criterion if more than a certain set of time intervals (e.g., the 5-minute intervals described above) exhibit an excessive load condition (e.g., a delayed acknowledgment was issued in each of the specified time intervals).

[0079] In some examples, the model is a linear regression model expressed as a function based on parameters and measurements relating to resource access in the computing environment (e.g., Equation 1).

[0080] The machine-readable instructions contain instructions for workload management 308 to manage the workload in the computer environment based on the calculated parameters.

[0081] In some examples, performing workload management in the computing environment based on the calculated parameters involves identifying a workload belonging to a workload cluster of the multitude of workload clusters indicated by a parameter as contributing to an overload state in the computing environment, and limiting resource usage by the identified workload.

[0082] In some examples, performing workload management in the computing environment based on the calculated parameters involves determining a relative priority of the identified workload over other workloads, with resource usage being restricted in response to the determined relative priority.

[0083] Fig. This is a block diagram of a System 400 (for example, the Workload Management Engine 104, the Storage System 102, or another computer or array of computers). The System 400 includes a Hardware Processor 402 (or multiple hardware processors). A hardware processor can be a microprocessor, a core of a multi-core microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, or other hardware processing circuitry.

[0084] The System 400 includes a storage medium 404 on which machine-readable instructions are stored that can be executed on the hardware processor 402 to perform various tasks. Machine-readable instructions executable on a hardware processor can refer to instructions that can be executed on a single hardware processor or on multiple hardware processors.

[0085] The machine-readable instructions in storage medium 404 contain model training instructions 406 for training a model using a training dataset (e.g., the training dataset 122 from Fig. ), which contains information about past workloads that ran in a storage environment (e.g., storage system 102). The training dataset contains labels associated with characteristics of the past workloads, where the labels are based on load indicators generated in the storage environment and relate to the load conditions of the storage environment resulting from the execution of the past workloads.

[0086] The machine-readable instructions on storage medium 304 contain instructions for grouping workloads 408 to group current workloads into a variety of workload clusters based on characteristics of the current workloads. These characteristics include measures relating to accessing storage resources in the storage environment.

[0087] In some examples, each current workload includes storage resource access activities that are performed with respect to a corresponding storage volume in the storage environment.

[0088] The machine-readable instructions in storage medium 304 contain instructions for calculating contribution parameters (410) to calculate, using the trained model, parameters that represent the contributions of the respective workload clusters of the multitude of workload clusters to an overload condition in the storage environment.

[0089] The machine-readable instructions on storage medium 404 contain workload management instructions 412 to perform workload management in the storage environment based on the calculated parameters.

[0090] Fig. This is a flowchart of a Process 500 according to some examples. Process 500 can be executed by a system with a hardware processor.

[0091] Process 500 involves grouping (at 502) workloads into a variety of workload clusters based on characteristics of the workloads, where the characteristics include measures relating to access to resources in a computing environment.

[0092] Process 500 involves training (at 504) a model based on a training dataset, wherein the training dataset contains labels associated with features of example workloads, the labels being based on load indicators generated in the computing environment and relating to load conditions of the computing environment resulting from the execution of the example workloads.

[0093] Process 500 involves the calculation (at 506) of parameters representing the contributions of each workload cluster from the multitude of workload clusters to a load in a computing environment, using the model trained on the training dataset. Grouping the workloads into the multitude of workload clusters reduces the number of features processed when calculating the parameters using the model.

[0094] Process 500 involves performing (at 508) workload management in the computing environment based on the calculated parameters.

[0095] Process 500 can update the training dataset while the computer environment is running and retrain the model using the updated training dataset.

[0096] A storage medium (e.g., 300 in Fig. in Fig.The storage device may include any or a combination of the following: a semiconductor storage device such as dynamic or static random-access memory (DRAM or SRAM), erasable and programmable read-only memory (EPROM), electrically erasable and programmable read-only memory (EEPROM), and flash memory or other type of non-volatile storage device; a magnetic disk such as a hard disk, floppy disk, and removable disk; another magnetic medium, including tape; an optical medium such as a compact disc (CD) or digital video disc (DVD); or another type of storage device. It should be noted that the instructions described above may be provided on a single computer- or machine-readable storage medium or, alternatively, on multiple computer- or machine-readable storage media distributed throughout a large system, possibly with multiple nodes.Such a computer-readable or machine-readable storage medium or media are considered part of an article (or a manufactured item). An article or manufactured item can refer to any single manufactured component or to multiple components. The storage medium or media can be located either in the machine on which the machine-readable instructions are executed or at a remote location from which machine-readable instructions can be downloaded over a network for execution.

[0097] The foregoing description includes numerous details to provide an understanding of the subject matter disclosed herein. However, implementations may be practiced without some of these details. Other implementations may include modifications and deviations from the details described above. It is intended that the accompanying claims cover such modifications and variations.

Claims

[1] A non-transitory, machine-readable storage medium containing instructions which, when executed, cause a system (400) to: Generating a training dataset (122) based on features of example workloads, wherein the training dataset (122) includes labels associated with the features of the example workloads, the labels being based on load indicators generated in a computing environment and relating to load conditions of the computing environment resulting from the execution of the example workloads; Grouping (502) selected workloads (106) into a plurality of workload clusters (118) based on features of the selected workloads (106), wherein a workload cluster (118) of the plurality of workload clusters (118) includes workloads (106) that are similar to each other according to a similarity criterion; Computation (506) of parameters (124) representing the contributions of each workload cluster (118) of the plurality of workload clusters (118) to a load condition in the computing environment, using a model (120) trained on the basis of the training data set (122), wherein the parameters (124) include a first parameter (124) representing a contribution of a first workload cluster (118) to the load condition and a second parameter (124) representing a contribution of a second workload cluster (118) to the load condition, wherein the first and second workload clusters (118) are part of the plurality of workload clusters (118); Selection of a workload cluster (118) from the multitude of workload clusters (118) based on different values ​​of the calculated parameters (124), wherein a value of a parameter (124) calculated for a selected workload cluster (118) indicates that the selected workload cluster (118) has a workload (106) that negatively affects the work performance of a workload (106) in another workload cluster (118); and Perform workload management (508) in the computer environment based on the calculated parameters (124), wherein workload management (508) includes: Determining a relative priority of the identified workload (106) over other workloads (106), and Restricting the resource utilization of the identified workload (106) in the selected workload cluster (118) in response to the determined relative priority. [2] The non-transitory machine-readable storage medium according to claim 1, wherein each workload (106) of the selected workloads (106) is executed on a corresponding storage volume in the computer environment. [3] The non-transitory machine-readable storage medium according to claim 2, wherein a workload cluster of the plurality of workload clusters contains workloads with similar input / output patterns. [4] The non-transitory machine-readable storage medium according to claim 1, wherein the features include measures relating to access to resources in the computer environment. [5] The non-transitory machine-readable storage medium according to claim 4, wherein the restriction of the resource utilization of the identified workload (106) in the selected workload cluster (118) comprises a restriction of the access rate to the resource by the workload (106) in the selected workload cluster (118). [6] The non-transitory machine-readable storage medium according to claim 4, wherein the measures relating to accessing resources include measures relating to throughput when writing to storage resources. [7] The non-transitory machine-readable storage medium according to claim 1, wherein the instructions, upon execution, cause the system to: Triggering the calculation of the parameters (124) that represent the contributions of the respective workload clusters (118) to the load condition, in response to the load condition fulfilling a criterion. [8] The non-transitory machine-readable storage medium according to claim 7, wherein the features of the selected workloads (106) comprise measures relating to access to resources of the computer environment in corresponding time intervals, and wherein the load condition satisfies the criterion if more than a certain quantity of the time intervals exhibit an excessive load condition. [9] The non-transitory machine-readable storage medium according to claim 8, wherein the state of excessive load exists in a given time interval of time intervals when an acknowledgment of the termination of an access to a resource is delayed. [10] The non-transitory machine-readable storage medium according to claim 1, wherein the different values ​​of the calculated parameters (124) each represent different contributions to the load condition. [11] The non-transitory machine-readable storage medium according to claim 10, wherein the instructions, when executed, cause the system to: to select the workload cluster (118) from the multitude of workload clusters (118) based on the identification of a workload cluster (118) that contributes more to the load condition than any other workload cluster (118) from the multitude of workload clusters (118). [12] The non-transitory machine-readable storage medium according to claim 11, wherein the resource is a resource shared by several workload clusters (118) and wherein the restriction is a restriction on the use of the shared resource by the selected workload cluster (118) from the plurality of workload clusters (118). [13] The non-transitory machine-readable storage medium according to claim 10, wherein different values ​​represent a first value of the first parameter (124) calculated for the first workload cluster (118) and a different second value of the second parameter (124) calculated for the second workload cluster (118), wherein the first value and the second value indicate that the first workload cluster (118) makes a greater contribution to the load condition by consuming more of the shared resource than the second workload cluster (118). [14] The non-transitory machine-readable storage medium according to claim 1, wherein the model (12) is a linear regression model comprising a plurality of coefficients representing the calculated parameters (124). [15] A system that includes the following: a processor (402); and a non-transient machine-readable storage medium (404) containing instructions that can be executed on the processor (402) to: Training a model using a training dataset (122) that includes information about previous workloads (106) that were executed in a storage environment, wherein the training dataset (122) includes labels that are associated with features of the previous workloads (106), the labels being based on load indicators that are generated in the storage environment and that relate to load conditions of the storage environment that result from the execution of the previous workloads (106); Grouping current workloads (106) into a plurality of workload clusters (118) based on characteristics of the current workloads (106), wherein the characteristics include measures relating to memory resource access in the memory environment, wherein a workload cluster (118) of the plurality of workload clusters (118) includes workloads (106) that are similar to each other according to a similarity criterion; Calculate, using the trained model (120), parameters (124) that represent the contributions of the respective workload clusters (118) of the plurality of workload clusters (118) to an overload state in the memory environment, wherein the parameters include a first parameter (124) that represents a contribution of a first workload cluster (118) to the overload state and a second parameter (124) that represents a contribution of a second workload cluster (118) to the overload state, wherein the first and second workload clusters (118) are part of the plurality of workload clusters (118); Selection of a workload cluster (118) from the multitude of workload clusters (118) based on different values ​​of the calculated parameters (124), wherein a value of a parameter (124) calculated for a selected workload cluster (118) indicates that the selected workload cluster (118) has a workload (106) that negatively affects the work performance of a workload (106) in another workload cluster (118); and performing the management of the workload (106) in the storage environment based on the calculated parameters (124), wherein the management of the workload (106) includes: Determining a relative priority of the identified workload (106) over other workloads (106), and Restricting the resource utilization of the identified workload (106) in the selected workload cluster (118) in response to the determined relative priority. [16] The system according to claim 15, wherein the restriction of the resource utilization of the identified workload (106) in the selected workload cluster (118) comprises a restriction of the access rate to the resource by the workload (106) in the selected workload cluster (118). [17] The system according to claim 15, wherein the instructions can be executed on the processor (402) to: a workload cluster (118) from the multitude of workload clusters (118) based on the identification of a workload cluster (118) that makes a greater contribution to the overload condition than another workload cluster (118). [18] Method for a system (400) with a hardware processor (402) comprising the following: Grouping workloads (106) into a plurality of workload clusters (118) based on characteristics of the workloads (106), wherein the characteristics include measures relating to access to resources in a computing environment, and wherein a workload cluster (118) of the plurality of workload clusters (118) comprises workloads (106) grouped together that are similar to each other according to a similarity criterion; Training a model (120) based on a training dataset (122), wherein the training dataset (122) includes labels associated with features of example workloads, the labels being based on load indicators generated in the computing environment and relating to load conditions of the computing environment resulting from the execution of the example workloads; Calculate, using the model trained on the basis of the training data set (122), parameters (124) that represent contributions of respective workload clusters (118) of the plurality of workload clusters (118) to a load in the computing environment, wherein the grouping of the workloads (106) into the plurality of workload clusters (118) reduces a set of features that are processed when calculating the parameters (124) using the model (120), and wherein the parameters (124) include a first parameter (124) that represents a contribution of a first workload cluster (118) to the load and a second parameter (124) that represents a contribution of a second workload cluster (118) to the load, wherein the first and second workload clusters (118) are part of the plurality of workload clusters (118); Selection of a workload cluster (118) from the multitude of workload clusters (118) based on different values ​​of the calculated parameters (124), wherein a value of a parameter (124) calculated for a selected workload cluster (118) indicates that the selected workload cluster (118) has a workload (106) that negatively affects the work performance of a workload (106) in another workload cluster (118); and Performing workload management in the computer environment based on the calculated parameters (124), where workload management includes: Determining a relative priority of the identified workload (106) over other workloads (106), and Restriction of resource utilization by a workload (106) in the selected workload cluster (118) in response to the determined relative priority. [19] The method according to claim 18 further comprises: Updating the training dataset (122) during operation of the computer environment; and Retraining the model (120) with the updated training dataset (122).

Images