Method, system and non-transitory machine-readable storage medium for data reduction of storage volumes

Through the combination of machine learning models and hardware processors, we can identify the contribution of resource contention and data reduction in the storage system to the storage volume, dynamically adjust the data reduction type, solving the problem of low data reduction management efficiency in the existing technology, and improving the The performance and resource utilization of the storage system.

CN117472267BActive Publication Date: 2024-11-01HEWLETT PACKARD ENTERPRISE DEV LP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211280605.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2022-07-22
Filing Date
2022-10-19
Publication Date
2024-11-01
Estimated Expiration
2042-10-19

AI Technical Summary

Technical Problem

In existing storage systems, it is difficult for administrators to effectively identify and manage the use of data reduction resources, resulting in resource waste and performance reduction. Especially when multiple workloads compete for resources, it is difficult to determine whether data reduction is effective.

Method used

Using machine learning model combined with hardware processors, we identify the contribution of resource contention and data reduction in the storage system to the storage volume. Through data compression and deduplication, we determine whether data reduction meets the storage space saving standards, and dynamically adjust the storage according to the results. The type of data reduction for the volume.

Benefits of technology

It realizes efficient management of storage system resources, avoids invalid data reduction, improves the performance and storage space utilization of storage systems, and ensures reasonable allocation of resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117472267B_ABST
    Figure CN117472267B_ABST
Patent Text Reader

Abstract

A method, system, and non-transitory machine-readable storage medium for data reduction of storage volumes are provided. In some examples, a system identifies resource contention for a resource in a storage system and determines that a set of workloads employing data reduction are consuming the resource. The system identifies relative contributions to resource consumption that can be attributed to storage volumes in the storage system, wherein the set of workloads employing data reduction is performed on data of the storage volumes. The system determines whether a storage space saving achieved due to applying data reduction to a given storage volume in the storage volumes meets a criterion, and in response to determining that the storage space saving for the given storage volume does not meet the criterion, the system indicates that data reduction is not applied to the given storage volume.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure generally relates to data reduction for storage volumes. Background Art

[0002] The storage system includes resources that allow the storage system to store data in a storage medium, which may include a collection of storage devices (a single storage device or multiple storage devices). In some examples, the storage system may include storage media and other resources (including processing resources and communication resources). In other examples, the storage system may be in the form of a control system (e.g., a storage server) that includes processing resources and communication resources and manages the storage of data on separate storage media. Summary of the invention

[0003] According to a first aspect of the present disclosure, a non-transitory machine-readable storage medium for data reduction of storage volumes is provided, comprising instructions which, when executed, cause a computer system to perform the following operations: identifying resource contention for resources in a storage system; determining that a set of workloads employing data reduction are consuming the resources; identifying contributions to resource consumption that can be attributed to storage volumes in the storage system, wherein the set of workloads employing the data reduction are performed on data of the storage volumes, wherein the identification of the contributions comprises: identifying a first contribution to resource consumption that can be attributed to a first storage volume among the storage volumes; and identifying a second contribution to resource consumption that can be attributed to a second storage volume among the storage volumes; determining whether storage space savings achieved by applying the data reduction to a given storage volume among the storage volumes meets a standard; and in response to determining that the storage space savings of the given storage volume do not meet the standard, converting the given storage volume from a first type of storage volume to which data reduction can be applied to a second type of storage volume to which data reduction is not applied.

[0004] According to a second aspect of the present disclosure, a system for data reduction of storage volumes is provided, comprising: a hardware processor; and a non-volatile storage medium storing instructions, wherein the instructions are executable on the hardware processor to perform the following operations: identifying resource contention for resources in a storage system; identifying storage volumes in the storage system to which data reduction is applied for workloads; determining whether storage space savings achieved by applying the data reduction to a given storage volume among the identified storage volumes meets a standard based on applying a machine learning model to input data, wherein the input data includes performance-related indicators related to the storage system and each storage volume; and in response to determining that the storage space savings of the given storage volume do not meet the standard, converting the given storage volume from a first type of storage volume to which data reduction can be applied to a second type of storage volume to which data reduction is not applied.

[0005] According to a third aspect of the present disclosure, a method for data reduction of a storage volume of a system including a hardware processor is provided, the method comprising: identifying a storage system in which there is resource contention for resources of the storage system from multiple storage systems; identifying a storage volume in the storage system to which data reduction is applied to a workload, wherein the data reduction comprises one or more of data compression and data deduplication; determining whether storage space savings achieved by applying the data reduction to a given storage volume among the identified storage volumes meets a standard based on applying a machine learning model to input data, wherein features of the machine learning model include resource utilization indicators, workload information of the workload, and time spent applying the data reduction, and the input data includes performance-related indicators related to the storage system and each storage volume; and based on determining that the storage space savings of the given storage volume do not meet the standard, converting the given storage volume from a first type of storage volume to a second type of storage volume by the system, wherein the data reduction is allowed for the first type of storage volume, but not for the second type of storage volume. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Some embodiments of the disclosure are described with respect to the following figures.

[0007] Figure 1 is a block diagram of an arrangement including a storage system and a data reduction management engine according to some examples.

[0008] Figure 2 is a block diagram of a machine learning model for latency identification based on some examples.

[0009] Figure 3 is a block diagram of a machine learning model recommended based on data reduction of some examples.

[0010] Figure 4 is a flowchart for recommending machine learning models based on data reduction of some examples.

[0011] Figure 5 is a block diagram of a storage medium storing machine-readable instructions according to some examples.

[0012] Figure 6 is a block diagram of a system according to some examples.

[0013] Figure 7 is a flow diagram of a process according to some examples.

[0014] Throughout the drawings, the same reference numerals refer to similar, but not necessarily identical, elements. The drawings are not necessarily drawn to scale, and the dimensions of some parts may be exaggerated to more clearly illustrate the examples shown. In addition, the drawings provide examples and / or implementations consistent with the description; however, the description is not limited to the examples and / or implementations provided in the drawings. DETAILED DESCRIPTION

[0015] In the present disclosure, unless the context clearly indicates otherwise, the use of the terms "a / an" or "the" is intended to include the plural forms as well. Similarly, when used in the present disclosure, the terms "includes / including / comprises / comprising", "have / having" indicate the presence of stated elements, but do not exclude the presence or addition of other elements.

[0016] A variety of different types of workloads can be executed on a storage system. Different workloads may compete for the resources of the storage system, resulting in resource contention, which may adversely affect the performance of some workloads.

[0017] A "workload" may refer to any collection of activities performed by entities (eg, machine-readable instructions or hardware) in a storage system.

[0018] Multiple workloads running in a storage system may overload the shared resources of the storage system. In a scenario where the resource usage of a first workload prevents the execution of a second workload in the storage system or causes the second workload to execute more slowly, the first workload may adversely affect the performance of the second workload. Shared resources may include physical resources, such as processing resources (e.g., processors, cores of processors, etc.), input / output (I / O) resources (e.g., resources such as I / O controllers that support I / O operations (e.g., read operations and write operations)), communication resources (e.g., network interface controllers, switches or routers, etc.), memory resources (e.g., memory devices such as dynamic random access memory (DRAM) devices, static random access memory (SRAM) devices, flash memory devices, etc.), and the like.

[0019] The shared resources may also include logical resources such as virtual machines, virtual processors, virtual communication devices, and / or program resources such as device drivers, operating system (OS) resources, and the like.

[0020] Data reduction can be applied by some workloads executed on storage volumes stored by the storage system. A "storage volume" can refer to a logical container or other structure to which data can be added, such as through write operations. Data can also be retrieved from a storage volume through read operations.

[0021] Examples of data reduction that may be performed on data may include data compression and / or data deduplication. Data compression may refer to encoding or otherwise transforming data so that the size of the data may be reduced, i.e., the size of the compressed data may be smaller than the size of the uncompressed data. Examples of compression techniques include Lempel-Ziv (LZ) compression, run-length encoding, dictionary encoding, and the like.

[0022] Deduplication refers to the removal of duplicate data portions that occur in data. For example, input data may be divided into data blocks. The deduplication process may identify data blocks that have been previously stored by the storage system - the identified data blocks are duplicate data blocks because the storage system has already stored these data blocks. The deduplication process may avoid storing the identified duplicate data blocks again in the storage system. Instead of storing duplicate data blocks, the deduplication process may store references to previously stored data blocks. Deduplicated data (e.g., deduplicated files, deduplicated objects, etc.) may include a collection of data blocks and references, wherein a reference is a reference to a stored data block and is generated by the deduplication process. A "reference" may be a pointer or any other indication of a storage location of a data block.

[0023] More generally, data reduction applied to data refers to any process that reduces the amount of data stored in a storage system in response to receiving input data ("raw data") to be stored in the storage system.

[0024] Performing data reduction may be resource intensive. For example, data reduction may consume processing resources. Data encoding applied by data compression may take some time to perform, especially when the input data is large. Deduplication may involve dividing the input data into data chunks, calculating signature values ​​(e.g., hash values) based on the data chunks, comparing the signature values ​​to a set of signature values ​​representing data chunks already stored by a storage system, and identifying data chunks of the input data having signature values ​​that match the set of signature values ​​as duplicate data chunks.

[0025] The amount of storage savings that can be achieved by applying data reduction can depend on the type of data to which the data reduction is applied. For example, some types of data are more easily compressed than other types of data. The effectiveness of data compression can be expressed in terms of a compression ratio, which is the ratio of the size of the uncompressed data to the size of the compressed data. The higher the data compression ratio, the more effective the data compression is. In other examples, other measures of data compression effectiveness can be used.

[0026] The effectiveness of deduplication depends on the amount of duplicate data that can be removed by applying deduplication. For example, if deduplication removes more duplicate blocks, deduplication is more effective.

[0027] When a large number of workloads are executed on storage volumes stored by a storage system, it may be difficult for an administrator to determine whether the data reduction applied to the workloads is effective. Ineffectively applying data reduction to storage volumes wastes the storage system's resources, which can slow down other workloads. An administrator may observe that the storage system is experiencing high latency ("latency" can refer to the amount of time from the start of a workload to the completion of the workload), but the administrator may not know which workloads are ineffectively using the storage system's resources, such as due to ineffectively applying data reduction.

[0028] According to some embodiments of the present disclosure, a system can identify a storage system, such as from a plurality of storage systems, in which there is contention for resources of the storage system, and can determine that a set of workloads (a single workload or multiple workloads) that employ data reduction are consuming the resources. A set of workloads including a single workload or multiple workloads may also be referred to as a "workload set". The system identifies the relative contribution to resource consumption that can be attributed to storage volumes in the storage system, wherein the set of workloads that employ data reduction is performed on data of the storage volumes. The system determines whether the storage space savings achieved by applying data reduction to a given storage volume in the storage volumes meets a criterion, and in response to determining that the storage space savings for a given storage volume do not meet the criterion, indicates that data reduction is not applied to the given storage volume.

[0029] Figure 1 is a block diagram of an example arrangement including a storage system 102 and a data reduction management engine 104 according to some embodiments of the present disclosure. Figure 1 The data reduction management engine 104 is shown as being separate from the storage system 102, but note that in other examples, the data reduction management engine 104 can be part of the storage system 102. If the data reduction management engine 104 is separate from the storage system 102, the data reduction management engine 104 can be coupled to the storage system 102 through a network or other link.

[0030] As used herein, an "engine" may refer to one or more hardware processing circuits, which may include any one or some combination of a microprocessor, a core of a multi-core microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, or other hardware processing circuits. Alternatively, an "engine" may refer to a combination of one or more hardware processing circuits and machine-readable instructions (software and / or firmware) that can be executed on the one or more hardware processing circuits.

[0031] The storage system 102 includes a processing resource 106, which may include any one or some combination of the following: a processor, a core of a multi-core processor, a microcontroller, a digital signal processor, a programmable integrated circuit device, a programmable gate array, a virtual entity such as a virtual machine, a virtual processor, etc. The processing resource 106 of the storage system 102 may be part of a single computing node, or may be included in multiple computing nodes.

[0032] Other types of resources may also be included in the storage system 102, including memory resources such as cache memory 107. Cache memory is implemented with one or more memory devices and can store data for corresponding processing resources. Other types of resources include I / O resources, program resources, etc.

[0033] In accordance with Figure 1 In the example of , the storage system 102 includes a storage subsystem 108, which may include a collection of storage devices (a single storage device or multiple storage devices). Examples of storage devices include disk-based storage devices, solid-state drives, and the like.

[0034] In other examples, storage subsystem 108 may be separate from storage system 102 , but storage subsystem 108 may be accessible by storage system 102 .

[0035] The storage subsystem 108 stores various storage volumes 110-1, 110-2, 110-3, etc., which contain data on which workloads (e.g., 112-1, 112-2, etc.) executed on the processing resources 106 are executed. A workload can be executed on data in one or more storage volumes. Conversely, one or more workloads can be applied to data in a storage volume. Workloads are initiated by requesters, which can be remote requesters communicating with the storage system 102 over a network, or requesters within the storage system 102. A "requester" can refer to a user, program, or machine that can request an operation (e.g., a read operation, a write operation, etc.) to be performed on data stored in a storage volume of the storage subsystem 108.

[0036] Despite Figure 1 A particular number of workloads 112-1, 112-2 and storage volumes 110-1, 110-2, 110-3 are depicted in FIG, but in other examples, there may be a different number of workloads and / or storage volumes.

[0037] exist Figure 1In the example of , workload 112-1 applies data reduction 114 to data in a corresponding storage volume (e.g., 110-1). Workload 112-2 does not apply data reduction to data in a corresponding storage volume (e.g., 110-3). In such an example, storage volume 110-1 is referred to as a "thick" storage volume because data reduction can be applied to achieve a beneficial data reduction ratio when reducing the amount of data stored in storage volume 110-1 (note that workloads executed on thick storage volumes may not apply data reduction). Storage volume 110-2 is referred to as a "non-thick" storage volume because data reduction is not applied to data stored in storage volume 110-2.

[0038] For example, data reduction 114 may include data compression and / or data deduplication. For write operations performed by workload 112-1, data reduction 114 may perform data reduction on the write data to generate reduced data that is written to the corresponding storage volume(s) 110. "Reduced" data may refer to compressed data and / or data that has been deduplicated.

[0039] When workload 112 - 1 reads the reduced data from storage volume 110 - 1 , data reduction may be reversed (eg, data may be decompressed and / or deduplication may be reversed, such as by replacing references to data blocks in the deduplicated data with the actual data blocks).

[0040] The storage system 102 includes a collection of monitoring engines 116 (a single monitoring engine or multiple monitoring engines) to collect various performance-related metrics related to data operations performed in the storage system 102. Examples of performance-related metrics that can be collected include usage metrics of the processing resources 106 (e.g., what percentage of the processing resources are utilized), latency metrics related to latency associated with the execution of the workload, cache-related metrics related to usage of the cache memory 107 in the storage system 102 (e.g., cache hit rate and / or cache miss rate), and the like.

[0041] The performance-related indicators collected by the set of monitoring engines 116 may include performance-related indicators related to the storage system 102 as a whole, as well as performance-related indicators related to individual storage volumes.

[0042] Performance-related indicators related to the storage system 102 as a whole may indicate the total utilization of the processing resources 106 for workloads executed on data of the storage volumes 110-1, 110-2, 110-3, the total latency associated with the workloads executed on data of the storage volumes 110-1, 110-2, 110-3, the total cache memory utilization associated with the workloads executed on data of the storage volumes 110-1, 110-2, 110-3, and the like.

[0043] The performance-related indicators associated with each storage volume 110 may indicate the utilization of the processing resources 106 of the (multiple) workloads executed on the data of the storage volume, the latency associated with the (multiple) workloads executed on the data of the each storage volume, the cache memory utilization associated with the (multiple) workloads executed on the data of the each storage volume, and the like.

[0044] The set of monitoring engines 116 may store storage system metrics 120 in the storage subsystem 108 and may store storage volume metrics 122 in the storage subsystem 108. The storage system metrics 120 include performance-related metrics related to the storage system 102 as a whole, while the storage volume metrics 122 include performance-related metrics related to each of the individual storage volumes.

[0045] In other examples, the set of monitoring engines 116 may store the storage system metrics 120 and the storage volume metrics 122 in a storage medium that is external to the storage system 102 .

[0046] The set of monitoring engines 116 may also store storage savings indicators 124 for these storage volumes 110-1, 110-2, 110-3. For each of the respective storage volumes, the storage savings indicators 124 include an indicator representing the amount of storage savings (reduction in data size) achieved by applying data reduction to the data of the respective storage volume.

[0047] The storage system index 120, the storage volume index 122, and the storage saving index 124 may be contained in corresponding data structures stored in the storage subsystem 108 or in different storage media. For example, the data structure may be in the form of a table, a file, or the like.

[0048] The data reduction management engine 104 can manage whether data reduction is used for any of the storage volumes 110-1, 110-2, 110-3. In some examples, the data reduction management engine 104 uses machine learning models, including a latency identification machine learning model 130 (for identifying increased latency in the storage system 102 to trigger the execution of a data reduction recommendation process) and a data reduction recommendation machine learning model 132 (for providing a recommendation on whether any storage volume 110-1, 110-2, or 110-3 should undergo data reduction during a data reduction recommendation process triggered by the latency identification machine learning model 130).

[0049] The delay identification machine learning model 130 and the data reduction recommendation machine learning model 132 can be stored in a memory 134, which can be implemented using a collection of memory devices (a single memory device or multiple memory devices). The memory 134 can be external to the storage system 102, or it can be part of the storage system 102.

[0050] although Figure 1 An example is shown where the delay identification machine learning model 130 and the data reduction recommendation machine learning model 132 are separate machine learning models, but in other examples, the delay identification machine learning model 130 and the data reduction recommendation machine learning model 132 can be integrated into a single machine learning model. Alternatively, the delay identification machine learning model 130 and the data reduction recommendation machine learning model 132 can be divided into more than two machine learning models.

[0051] Examples of the delay identification machine learning model 130 and the data reduction recommendation machine learning model 132 may include any one or some combination of the following: a boosted tree regression model, a distance-based clustering model, a glass box local weighted regression model, and the like.

[0052] The delay identification machine learning model 130 may be trained using a training data set 136, and the data reduction recommendation machine learning model 132 may be trained using a training data set 138. The training data sets 136 and 138 may be derived based on historical data related to performance workloads on storage volumes in the storage system 102.

[0053] Once trained, the latency-aware machine learning model 130 can be applied to the storage system metrics 120 and the storage volume metrics 122 to determine whether the storage system 102 is experiencing unacceptably high latency. If the total latency experienced by the workload exceeds a specified threshold (which can be a dynamically changing threshold or a static threshold), then the storage system 102 is experiencing unexpectedly high latency. For example, the total latency of the workload can refer to the average latency experienced by the workloads 112-1, 112-2 when operating on the data of the storage volumes 110-1, 110-2, 110-3.

[0054] Once trained, the data reduction recommendation machine learning model 132 can be used to make recommendations about whether data reduction should be applied to each of the individual storage volumes (or storage volume groups). The data reduction recommendation machine learning model 132 can be applied to the storage system indicators 120, the storage volume indicators 122, and the storage savings indicators 124.

[0055] As the delay identification machine learning model 130 and the data reduction recommendation machine learning model 132 are applied to produce results, the training data sets 136 and 138 can be continuously updated. For example, an administrator can analyze the outputs produced by the delay identification machine learning model 130 and the data reduction recommendation machine learning model 132, identify that certain outputs are inaccurate, and can update the training data sets 136 and 138 based on such identification to cause retraining of the delay identification machine learning model 130 and / or the data reduction recommendation machine learning model 132 so that future results produced by the delay identification machine learning model 130 and / or the data reduction recommendation machine learning model 132 will be more accurate.

[0056] Figure 2 is a block diagram of an example of a latency-aware machine learning model 130. The latency-aware machine learning model 130 uses various features to generate outputs based on input data. The input data includes storage system metrics 120 and storage volume metrics 122. The output generated by the latency-aware machine learning model 130 includes a high latency indication 202 to indicate that the storage system 102 is experiencing high latency.

[0057] Features used by the delay-aware machine learning model 130 include the following: delay features 204, workload features 206, workload type features 208, and configuration features 210. In other examples, the delay-aware machine learning model 130 may use additional or alternative features.

[0058] A "feature" of a machine learning model is a single attribute (or group of attributes) that can be assigned different values ​​in the input data fed to the machine learning model. The machine learning model uses the feature (or features) to perform predictions based on the input data.

[0059] The values ​​of the various features used by the latency-identifying machine learning model 130 are included in the storage system metrics 120 and the storage volume metrics 122 .

[0060] The latency characteristics 204 refer to the latency experienced by the workload(s) when operating on data of a storage volume or multiple storage volumes. The workload characteristics 206 represent an identifier (e.g., a name or another identifier) ​​of the workload. The workload type characteristics 208 represent the workload type of the corresponding workload. The workload type characteristics 208 can have a first value (first workload type) to indicate a workload that does not apply data reduction (first workload type), and a different second value (second workload type) to indicate a workload that applies data reduction (second workload type).

[0061] Configuration characteristics 210 include a set of attributes that represent the configuration of storage system 102, such as the number of processing resources 106, the storage capacity of cache 107, the number of storage devices in storage subsystem 108, which storage volumes are thick storage volumes and which are non-thick storage volumes, etc. The value of configuration characteristics 210 may be included in storage system metrics 120 or storage volume metrics 122, or in a different set of information.

[0062] exist Figure 2 In the example of , the storage system metrics 120 include latency values ​​as a function of time, as represented by graph 212. The latency values ​​represented by graph 212 represent the latency experienced by the workload in the system when operating on data in the storage subsystem 108.

[0063] The storage volume metrics 122 may similarly include latency values ​​as a function of time that represent the latency experienced by the workload(s) operating on each of the respective storage volumes.

[0064] The storage system metrics 120 and the storage volume metrics 122 may include additional or alternative metrics.

[0065] If the latency-aware machine learning model 130 determines that the storage system 102 is experiencing high latency based on the input data (including the storage system metrics 120 and the storage volume metrics 122), a high latency indication 202 is output by the latency-aware machine learning model 130 (e.g., a high latency indicator (such as a flag) is set to a first value). On the other hand, if the latency-aware machine learning model 130 determines that the storage system 102 is not experiencing high latency based on the input data, the latency-aware machine learning model 130 does not output a high latency indication 202 (e.g., the high latency indicator is set to a second value).

[0066] The input data provided to the latency-aware machine learning model 130, including the storage system metrics 120 and the storage volume metrics 122, may be specified time intervals for a predefined duration. In some examples, the latency-aware machine learning model 130 may be called at each specified time interval to determine whether the storage system 102 experiences high latency during the time interval.

[0067] Note that a computing environment (eg, a cloud environment, a data center, etc.) may include multiple storage systems. The latency-aware machine learning model 130 is applied to input data associated with each of the multiple storage systems to identify which storage system is experiencing high latency.

[0068] To train the delay-aware machine learning model 130, the training data set 136 includes multiple sets (e.g., vectors) of features 204, 206, 208, and 210, wherein each set of features 204, 206, 208, and 210 can be assigned a high-latency tag (the tag is set to a first value indicating high latency and a second value indicating non-high latency). Tags can be assigned by a human user, a program, or a machine. Using a labeled training data set to train the delay-aware machine learning model 130 is called supervised machine learning. In other examples, the delay-aware machine learning model 130 can be trained using unsupervised machine learning, which is learning by the delay-aware machine learning model 130 based on recognizing patterns in unlabeled data.

[0069] Figure 3 is a block diagram of an example of a data reduction recommendation machine learning model 132. The data reduction recommendation machine learning model 132 uses various features to generate outputs based on input data (including storage system metrics 120, storage volume metrics 122, and storage savings metrics 124).

[0070] The features used by the data reduction recommendation machine learning model 132 include the following: processor utilization features 302, workload features 304, workload type features 306, deduplication time features 308, and compression time features 310. In other examples, the data reduction recommendation machine learning model 132 may use additional or alternative features. In other examples, the data reduction recommendation machine learning model 132 may use additional or alternative features.

[0071] The values ​​of the various features used by the latency identification machine learning model 130 are included in the storage system metrics 120 , storage volume metrics 122 , and storage savings metrics 124 .

[0072] The processor utilization feature 302 refers to the utilization of the processing resource 106. In further examples, the data reduction recommendation machine learning model 132 may also use other resource-related features, such as features related to the utilization of the cache memory 107, and the like.

[0073] The deduplication time feature 308 refers to the time taken by the workload to perform deduplication on data in the storage volume (already stored in the storage volume or before being stored in the storage volume). The compression time feature 310 refers to the time taken by the workload to perform compression on the data in the storage volume. The data reduction recommendation machine learning model 132 can use the values ​​of the deduplication time feature 308 and the compression time feature 310 (as in the storage system indicator 120 and / or the storage volume indicator 122) to determine how much time is involved in performing deduplication and / or data compression, and whether the storage savings achieved by deduplication and / or data compression (as indicated in the storage savings indicator 124) can justify the continued use of the corresponding data reduction.

[0074] The output generated by the data reduction recommendation machine learning model 132 based on the input data includes a recommendation 312 as to whether to apply data reduction (e.g., data compression and / or data deduplication) to the data of each storage volume 110-1, 110-2, 110-3. For example, in response to detecting that data reduction applied to the data of a given storage volume is ineffective, the recommendation 312 provided by the data reduction recommendation machine learning model 132 may indicate that the given storage volume is to be converted from a thick storage volume (data reduction can be applied) to a non-thick storage volume (data reduction is not applied). Once the given storage volume is converted to a non-thick storage volume, any workload executed on the given storage volume will not apply data reduction to the data stored in the given storage volume.

[0075] To train the data reduction recommendation machine learning model 132, the training data set 138 includes multiple sets (e.g., vectors) of features 302, 304, 306, 308, and 310, wherein each set of features 302, 304, 306, 308, and 310 may be assigned a non-thick storage volume tag (the tag is set to a first value to indicate that the storage volume of the given feature set will be configured as a non-thick storage volume to which data reduction is not applied, and is set to a second value to indicate that the storage volume of the given feature set will be configured as a thick storage volume to which data reduction can be applied). Tags may be assigned by a human user, a program, or a machine. Training the data reduction recommendation machine learning model 132 using a labeled training data set is referred to as supervised machine learning. In other examples, the data reduction recommendation machine learning model 132 may be trained using unsupervised machine learning, which is learning performed by the data reduction recommendation machine learning model 132 based on identifying patterns in unlabeled data.

[0076] In some examples, once trained, the data reduction recommendation machine learning model 132 can be used to perform processing tasks based on Figure 4The input data (including storage system metrics 120, storage volume metrics 122, and storage savings metrics 124) may be for a specified time interval, which may be the same as the time interval for which the latency identification machine learning model 130 has identified high latency.

[0077] The data reduction recommendation machine learning model 132 determines (at 402) whether a performance degradation (e.g., high latency) of the storage system 102 is related to resource contention (e.g., contention for the processing resources 106) based on the input data (the storage system metrics 120, the storage volume metrics 122, and the storage savings metrics 124). This relationship between the performance degradation of the storage system and the resource contention may be based on the value of the processor utilization feature 302 indicating high processing resource usage.

[0078] If the performance degradation of storage system 102 is not related to resource contention, the process does not continue.

[0079] However, if the performance degradation of the storage system 102 is related to resource contention, the data reduction recommendation machine learning model 132 determines (at 404) coefficients representing the respective contributions of different activities in the storage system 102. The coefficients may include a first coefficient representing a first portion (e.g., a first percentage) of the processing resources 106 consumed by the workload (e.g., 112-1, 112-2), and a second coefficient representing a second portion (e.g., a second percentage) of the processing resources 106 consumed by background activities. "Background activities" may refer to any activities of the storage system 102 for which metrics are collected by the set of monitoring engines 116 at the storage system level without apportioning the metrics to multiple storage volumes (e.g., 110-1, 110-2, 110-3) of the storage system 102. Examples of background activities include deduplication, data compression, and any maintenance / management activities performed by the storage system 102. Although deduplication and data compression are applied to the data of the storage volumes, in some examples, the set of monitoring engines 116 do not collect performance-related metrics for deduplication and data compression at the individual storage volume level.

[0080] These coefficients are collected to determine whether the data reduction recommendation machine learning model 132 is to continue its further processing. Based on the first coefficient and the second coefficient, the data reduction recommendation machine learning model 132 determines (at 406) whether the workload consumption is greater than a specified threshold of the processing resource 106. For example, the specified threshold can be a threshold percentage of the processing resource 106 (e.g., 30%, 40%, 50%, 60%, 70%, etc.). If the workload consumption is greater than a specified threshold of the processing resource 106, the data reduction recommendation machine learning model 132 continues further processing.

[0081] However, if the workload does not consume more than the specified threshold of processing resources 106, the data reduction recommendation machine learning model 132 does not proceed with further processing because no storage volume-level metrics are available that consume a relatively large portion of the processing resources 106 for background activities. Since the processing goal of the data reduction recommendation machine learning model 132 is to make recommendations as to whether data reduction will be applied to each storage volume, performing such recommendations when a relatively large portion of the processing resources 106 is consumed by background activities will not significantly improve the performance of the storage system 102. In other words, for the data reduction recommendation machine learning model 132 to generate recommendations that will significantly improve the performance of the storage system 102 (e.g., the latency of the storage system 102 can be reduced by more than a specified amount), the workload should consume a relatively large portion of the processing resources 106.

[0082] In examples where the set of monitoring engines 116 does collect performance-related metrics for data deduplication and data compression at the storage volume level, tasks 404 and 406 may be omitted.

[0083] If the workload consumption is greater than some specified threshold of processing resources 106, the data reduction recommendation machine learning model 132 determines (at 408) the reasons for the processing resource usage of each thick storage volume in the storage system 102. Note that non-thick storage volumes need not be considered because data reduction is not applied to such storage volumes and therefore these storage volumes are not candidates for reducing the processing resource load by removing data reduction.

[0084] The data reduction recommendation machine learning model 132 uses the value of the workload type feature 306 in the storage volume metrics 122 to determine whether the (multiple) workloads on the corresponding thick storage volume are of the first workload type (data reduction is not applied) or the second workload type (data reduction is applied). If the (multiple) workloads on the corresponding thick storage volume are of the second workload type, the corresponding thick storage volume is identified (at 410) as a candidate for conversion to a non-thick storage volume. If there are multiple workloads with mixed workload types on the corresponding thick storage volume (some with data reduction applied and some without data reduction applied), the data reduction recommendation machine learning model 132 can determine whether the majority of the workloads (or a number of workloads exceeding a specified threshold) are of the second workload type, and if so, the data reduction recommendation machine learning model 132 can identify the corresponding thick storage volume as a candidate for conversion to a non-thick storage volume.

[0085] The data reduction recommendation machine learning model 132 then determines (at 412) a contribution to the usage of the processing resources 106 that can be attributed to the identified candidate thick storage volume (as identified at 410). The determination (at 412) is based on metrics included in the storage volume metrics 122, which include storage volume-level performance-related metrics (such as usage of processing resources).

[0086] Based on the determined contributions, the data reduction recommendation machine learning model 132 can rank the candidate thick storage volumes at 414. A first candidate thick storage volume associated with a greater contribution to the use of the processing resource 106 is ranked higher than a second candidate thick storage volume associated with a lesser contribution to the use of the processing resource 106.

[0087] The data reduction recommendation machine learning model 132 may then determine (at 416 ) for each of the top N (N≧1) candidate thick storage volumes whether the candidate thick storage volume should be converted to a non-thick storage volume.

[0088] In other examples, ranking of storage volumes is not performed, in which case the data reduction recommendation machine learning model 132 may apply task 416 to all storage volumes in the storage system 102 .

[0089] The determination of whether a candidate thick storage volume is to be converted to a non-thick storage volume is based on the space savings that can be achieved by applying data reduction, which can be obtained from the storage savings indicator 124. More specifically, the data reduction recommendation machine learning model 132 determines whether the storage space savings achieved by applying data reduction to the candidate thick storage volume meets the criteria, for example, determining whether the size of the data stored in the candidate thick storage volume is less than a specified threshold relative to the data size of the (multiple) workloads applied to the candidate thick storage volume (this can be expressed as, for example, a "data reduction ratio"). For example, the data reduction ratio is expressed as a 1:1.x threshold, where "1" represents the size of the data stored in the candidate thick storage volume, and "1.x" represents the data size of the (multiple) workloads applied to the candidate thick storage volume. The value of "x" can be specified by a user or another entity. In some examples, x is set to 3, 4, 5, or other values.

[0090] If 1:1.x is less than the specified threshold, it means that data reduction has reduced the amount of stored data by a sufficient amount to justify applying data reduction to the candidate thick storage volume. On the other hand, if 1:1.x exceeds (is greater than or equal to) the specified threshold, it means that data reduction has not reduced the amount of stored data by a sufficient amount to justify applying data reduction to the candidate thick storage volume.

[0091] Based on the determination (at 416), the data reduction recommendation machine learning model 132 generates (at 418) a list of identified thick storage volumes (a single identified thick storage volume or a plurality of identified thick storage volumes) to be converted to non-thick storage volumes. The list of identified thick storage volumes may be sent by the data reduction management engine 104 to the storage system 102, which may convert each thick storage volume in the list to a non-thick storage volume. Alternatively, the list of identified thick storage volumes may be sent by the data reduction management engine 104 to an administrator, who may reconfigure the identified thick storage volume(s) to non-thick storage volume(s).

[0092] Figure 5 is a block diagram of a non-transitory machine-readable or computer-readable storage medium 500 having stored thereon machine-readable instructions that, when executed, cause a computer system to perform various tasks.

[0093] The machine-readable instructions include resource contention identification instructions 502 for identifying resource contention for resources in the storage system. In some examples, the resources include any one or some combination of the following: a collection of processing resources, a collection of memory resources, a collection of communication resources, a collection of I / O resources, a collection of program resources, a collection of virtual resources, and the like.

[0094] In some examples, the first machine learning model (e.g., Figure 1 The latency identification machine learning model 130) is used to perform identification of resource contention for resources in the storage system. In some examples, the features of the machine learning model include latency indicators (e.g., Figure 2 204), workload information of a set of workloads (e.g., Figure 2 206 and 208), and storage system configuration information (e.g., Figure 2 210).

[0095] The machine-readable instructions include workload and data reduction determination instructions 504 to determine that a collection of workloads (or equivalently, a "workload collection") that employs data reduction is consuming resources. The determination may be based on workload information including an identifier of the workload and a workload type, which may be included in, for example, the storage system metrics 120 and the storage volume metrics 122.

[0096] The machine readable instructions include storage volume relative contribution identification instructions 506 for identifying relative contributions to resource consumption attributable to storage volumes in the storage system.A set of workloads employing data reduction are performed on data of the storage volumes.

[0097] The machine-readable instructions include storage space savings determination instructions 508 for determining whether storage space savings achieved due to applying data reduction to a given one of the storage volumes meets a criterion. In some examples, the determination of whether the storage space savings meets the criterion includes determining whether the size of data stored in the given storage volume is less than a specified threshold relative to the size of data of a workload applied to the given storage volume (e.g., 1:1.x or another data reduction ratio).

[0098] The machine-readable instructions include data reduction indication instructions 510, in response to determining that the storage space savings of the given storage volume does not meet the criteria, indicating that data reduction is not applied to the given storage volume. In some examples, indicating that data reduction is not applied to the given storage volume includes converting the given storage volume from a first type of storage volume to which data reduction can be applied to a second type of storage volume to which data reduction is not applied.

[0099] In some examples, a determination that a set of workloads employing data reduction are consuming resources, an identification of relative contributions to resource consumption attributable to storage volumes in the storage system, and a determination of whether storage space savings meet criteria are based on a second machine learning model (e.g., Figure 1 The features of the second machine learning model include resource utilization indicators (e.g., Figure 3 302), workload information of a set of workloads (e.g., Figure 3 304 and 306 in ), and the time it takes to apply data reduction (e.g., Figure 3 308 and 310).

[0100] In some examples, in response to determining that storage space savings achieved as a result of applying data reduction to the given storage volume meets the criteria, the machine-readable instructions enable continued application of data reduction to the given storage volume while storing data to the given storage volume.

[0101] In some examples, the machine-readable instructions rank the storage volumes according to relative contributions to resource consumption attributable to the storage volumes. Based on the ranking, the given storage volume for which the indication is provided is a higher ranked storage volume relative to another storage volume in the storage volumes.

[0102] In some examples, machine-readable instructions determine whether a workload comprising a collection of workloads consumes greater than a specified threshold amount of a resource, wherein the identification of relative contributions, the determination of whether storage space savings meets a criterion, and the indication are performed in response to determining that the workload comprising the collection of workloads consumes greater than a specified threshold amount of a resource.

[0103] Figure 6 is a block diagram of a system 600 that includes a hardware processor 602 (or multiple hardware processors). The hardware processor may include a microprocessor, a core of a multi-core microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, or another hardware processing circuit.

[0104] System 600 includes storage media 604 storing machine-readable instructions executable on hardware processor 602 to perform various tasks. Machine-readable instructions executable on a hardware processor may refer to instructions executable on a single hardware processor or instructions executable on multiple hardware processors.

[0105] The machine-readable instructions in the storage medium 604 include resource contention identification instructions 606 for identifying resource contention for a resource in the storage system.

[0106] The machine-readable instructions in the storage medium 604 include storage volume identification instructions 608 for identifying storage volumes in the storage system for workload application data reduction. In some examples, the identified storage volumes are thick storage volumes, which are different from non-thick storage volumes that do not allow data reduction.

[0107] The machine-readable instructions in the storage medium 604 include storage space savings determination instructions 610 for determining whether storage space savings achieved by applying data reduction to a given storage volume among the identified storage volumes meets criteria based on applying a machine learning model to input data including performance-related metrics related to the storage system and individual storage volumes.

[0108] The machine-readable instructions in the storage medium 604 include data reduction indication instructions 612 for indicating that data reduction is not applied to the given storage volume in response to determining that the storage space savings of the given storage volume does not meet the criteria.

[0109] In some examples, identification of storage volumes for a workload to which data reduction is applied is performed from a larger set of storage volumes that includes other storage volumes to which data reduction is not applied.

[0110] Figure 7 is a flow diagram of a process 700 that may be performed by, for example, the data reduction management engine 104 .

[0111] Process 700 includes identifying (at 702) a storage system from a plurality of storage systems for which there is resource contention for resources of the storage system.

[0112] Process 700 includes identifying (at 704) in a storage system a storage volume for which a workload applies data reduction, wherein the data reduction includes one or more of data compression and data deduplication.

[0113] Process 700 includes determining (at 706) whether storage space savings achieved due to applying data reduction to a given one of the identified storage volumes meets criteria based on applying a machine learning model to input data including performance-related metrics related to the storage system and individual storage volumes, wherein features of the machine learning model include resource utilization metrics, workload information for a collection of workloads, and time taken to apply data reduction.

[0114] Process 700 includes, in response to determining that storage space savings of a given storage volume does not meet a criterion, converting (at 708) the given storage volume from a first type of storage volume to a second type of storage volume, wherein data reduction is allowed for the first type of storage volume and not allowed for the second type of storage volume.

[0115] Storage media (e.g. Figure 5 500 or Figure 6 604) may include any one or some combination of the following: a semiconductor memory device, such as a dynamic or static random access memory (DRAM or SRAM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), and a flash memory; a magnetic disk, such as a fixed disk, a floppy disk, and a removable disk; another magnetic medium, including a tape; an optical medium, such as a compact disk (CD) or a digital video disk (DVD); or another type of storage device. Note that the instructions discussed above may be provided on one computer-readable or machine-readable storage medium, or alternatively, may be provided on multiple computer-readable or machine-readable storage media distributed in a large system with possibly multiple nodes. Such one or more computer-readable or machine-readable storage media are considered to be part of an article (or product). An article or product may refer to any manufactured single component or multiple components. One or more storage media may be located in a machine running machine-readable instructions, or at a remote site, from which machine-readable instructions may be downloaded over a network for execution.

[0116] In the foregoing description, many details are set forth to enable an understanding of the subject matter disclosed herein. However, embodiments may be practiced without some of these details. Other embodiments may include modifications and variations of the details discussed above. The appended claims are intended to cover such modifications and variations.

Claims

1. A non-transitory machine-readable storage medium for data reduction of a storage volume, comprising instructions that, when executed, cause a computer system to perform the following operations: identifying resource contention for resources in a storage system; determining that a set of workloads employing data reduction are consuming the resources; identifying contributions to resource consumption attributable to storage volumes in the storage system, wherein: The set of workloads employing the data reduction are performed on data of the storage volume, wherein the identifying of the contribution comprises: identifying a first contribution to resource consumption attributable to a first storage volume of the storage volumes; and identifying a second contribution to resource consumption attributable to a second storage volume of the storage volumes; determining whether a storage space saving achieved as a result of applying the data reduction to a given one of the storage volumes satisfies a criterion; and In response to determining that the storage space saving of the given storage volume does not meet the criterion, the given storage volume is converted from a first type of storage volume to which data reduction can be applied to a second type of storage volume to which data reduction is not applied.

2. The non-transitory machine-readable storage medium of claim 1, wherein: When the instructions are executed, the computer system performs the following operations: ranking the storage volumes according to the contributions to resource consumption attributable to the storage volumes, Wherein, according to the ranking, the given storage volume is a storage volume that ranks higher than another storage volume among the storage volumes.

3. The non-transitory machine-readable storage medium of claim 1, wherein: The data reduction includes one or more of data compression or data deduplication.

4. The non-transitory machine-readable storage medium of claim 1, wherein: When the instructions are executed, the computer system performs the following operations: determining whether a workload including one or more workloads in the set of workloads consumes greater than a specified threshold amount of the resource, Wherein, the identification of the contribution, the determination of whether the storage space savings meets the criteria, and the conversion are performed in response to determining that the workload consumption of one or more workloads in the workload set is greater than a specified threshold amount of the resource.

5. The non-transitory machine-readable storage medium of claim 1, wherein: The determination of whether the storage space saving meets the criterion includes determining whether a size of data stored in the given storage volume relative to a size of data of a workload applied to the given storage volume is less than a specified threshold.

6. The non-transitory machine-readable storage medium of claim 1, wherein: When the instructions are executed, the computer system performs the following operations: In response to determining that the storage space saving satisfies the criterion, allowing the data reduction to continue to be applied to the given storage volume when storing data to the given storage volume.

7. The non-transitory machine-readable storage medium of claim 1, wherein: Identifying resource contention for the resource in the storage system includes applying a machine learning model to input data including performance-related metrics of the storage system and the storage volume.

8. The non-transitory machine-readable storage medium of claim 7, wherein: The characteristics of the machine learning model include latency indicators, workload information of the workload set, and configuration information of the storage system.

9. The non-transitory machine-readable storage medium of claim 8, wherein: The workload information identifies each respective workload of the workload set and a type of the respective workload.

10. The non-transitory machine-readable storage medium of claim 1, wherein: The determination that the set of workloads employing data reduction are consuming the resources, the identification of contributions to resource consumption attributable to storage volumes in the storage system, and the determination of whether the storage space savings meet the criteria are performed based on the application of a machine learning model.

11. The non-transitory machine-readable storage medium of claim 10, wherein: The characteristics of the machine learning model include resource utilization metrics, workload information of the workload set, and the time taken to apply the data reduction.

12. The non-transitory machine-readable storage medium of claim 10, wherein: The machine learning model is applied to input data including performance-related metrics of the storage system, performance-related metrics of the storage volume, and storage savings metrics associated with application of the data reduction.

13. The non-transitory machine-readable storage medium of claim 1, wherein: The storage volume is part of a subset of storage volumes, and wherein the instructions, when executed, cause the computer system to: For the candidate storage volume set, determining a reason for resource usage of each candidate storage volume in the candidate storage volume set; and Based on the determined reasons for resource usage, the subset of storage volumes is identified.

14. The non-transitory machine-readable storage medium of claim 13, wherein: When the instructions are executed, the computer system performs the following operations: In response to determining that resource usage of a given candidate storage volume of the set of candidate storage volumes is due to a workload that applies the data reduction, the given candidate storage volume is included in the subset of storage volumes.

15. A system for data reduction of a storage volume, comprising: Hardware processor; as well as A non-transitory storage medium storing instructions executable on the hardware processor to: identifying resource contention for resources in a storage system; identifying, in the storage system, a storage volume for workload application data reduction; determining whether a storage space savings achieved as a result of applying the data reduction to a given one of the identified storage volumes satisfies a criterion based on applying a machine learning model to input data, wherein the input data includes performance-related metrics related to the storage system and the respective storage volumes; as well as In response to determining that the storage space saving of the given storage volume does not meet the standard, the given storage volume is converted from a first type storage volume capable of applying data reduction to a second type storage volume not applicable to data reduction.

16. The system of claim 15, wherein: The storage volume to which the data reduction is applied for the workload is identified from a larger set of storage volumes, where the larger set of storage volumes includes other storage volumes to which the data reduction is not applied.

17. The system of claim 15, wherein: Characteristics of the machine learning model include resource utilization metrics, workload information of the workload, and time taken to apply the data reduction.

18. The system of claim 15, wherein: The data reduction includes one or more of data compression or data deduplication.

19. A method for data reduction of a storage volume in a system including a hardware processor, the method comprising: identifying a storage system from a plurality of storage systems for which there is resource contention for resources of the storage system; identifying, in the storage system, a storage volume to which workload data reduction is applied, wherein the data reduction includes one or more of data compression and data deduplication; determining whether a storage space saving achieved as a result of applying the data reduction to a given storage volume among the identified storage volumes satisfies a criterion based on applying a machine learning model to input data, wherein features of the machine learning model include resource utilization metrics, workload information of the workload, and time taken to apply the data reduction, and the input data includes performance-related metrics related to the storage system and the respective storage volumes; and Based on determining that the storage space saving of the given storage volume does not meet the standard, the system converts the given storage volume from a first type of storage volume to a second type of storage volume, wherein the data reduction is allowed for the first type of storage volume but not allowed for the second type of storage volume.

20. The method of claim 19, wherein: The determination of whether the storage space saving meets the criterion includes determining whether a size of data stored in the given storage volume relative to a size of data of a workload applied to the given storage volume is less than a specified threshold.