ML Workload Provisioning Advisor for Storage Latency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Administrators face challenges in determining the performance impact of provisioning new workloads, migrating existing workloads, or increasing workload intensity in storage systems, as they cannot accurately assess the resulting latency or headroom, making it difficult to maintain service level objectives (SLOs).

Innovation Solution

A provisioning advisor device uses machine learning to generate and store workload parameters and signatures from training workloads, allowing it to estimate latency for query workloads by correlating their signatures with those in a mapping table, thereby providing administrators with accurate latency estimates and headroom information for informed provisioning decisions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If administrators manually assess workload performance impact, then provisioning decisions can be made with human judgment, but accuracy of latency estimation deteriorates due to inability to predict performance impact

Engineering Contradiction:

Improvelatency estimation accuracyVSAvoidprovisioning decision difficulty

Core Design Contradiction:

Measurement precisionVSEase of operation

Solution Approach 1:

The patent introduces a machine learning model as an intermediary between workload characteristics and latency prediction. The model takes workload intensity and characteristics as input and outputs predicted latency values, serving as a mediator that translates complex storage system behavior into actionable predictions without requiring administrators to manually assess performance impact

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent creates a virtual copy of the storage system's performance behavior through machine learning training. By training the model on historical workload data and actual latency measurements, the system creates a predictive copy that replicates the storage system's response to different workload conditions, enabling accurate latency estimation without actual workload execution

Inventive Principle:

Principle #26Copying

2Adaptability or versatility

If storage systems support many storage devices with different characteristics, then system versatility improves, but ability to determine performance impact deteriorates due to complexity

Engineering Contradiction:

Improvestorage device compatibilityVSAvoidperformance impact assessment accuracy

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent incorporates storage device characteristics (cache size, number of spindles, layout) as input parameters to the machine learning model. By including these device-specific parameters in the workload characterization, the model can adjust predictions to account for different storage device configurations, maintaining accuracy across diverse hardware environments

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent creates a universal prediction framework that works across multiple storage device types and configurations. The machine learning model is trained on data from various storage devices with different characteristics, enabling it to generalize performance predictions across diverse hardware platforms without requiring device-specific assessment methods

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Productivity

If administrators increase workload intensity to maximize utilization, then resource efficiency improves, but risk of SLO violation increases due to unknown latency impact

Engineering Contradiction:

Improveworkload utilization efficiencyVSAvoidSLO compliance

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent implements a feedback mechanism where the machine learning model predicts latency at different workload intensity levels, allowing administrators to see the impact of intensity changes before applying them. This feedback loop enables informed decisions about workload intensity adjustments while maintaining SLO compliance

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS9864749B2Methods for provisioning workloads in a storage system using machine learning and devices thereof

Publication Date: 2018.01.09 NETAPP INC

AI summary

A method, non-transitory computer readable medium, and provisioning advisor device that obtains an intensity and characteristics for each of a plurality of training workloads from storage device volumes. For each of the training workloads, at least first and second training workload parameters are generated, based on the training workload intensity, and an associated training workload signature is generated, based on the training workload characteristics. The first and second training workload parameters and associated training workload signatures are stored in a mapping table. A signature and an intensity for a query workload are obtained. First and second query workload parameters are determined based on a correlation of the query workload signature with the training workload signatures of the mapping table. An estimated latency for the query workload is determined, based on the first and second query workload parameters and the query workload intensity, and the estimated query workload latency is output.