Trained Model Workload Clustering for Noisy-Neighbor Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face challenges in managing resource contention among different workloads, leading to performance degradation due to noisy neighbors, where manual analysis is time-consuming and often inaccurate, and quality-of-service mechanisms struggle to apply precise workload constraints.
Innovation Solution
A machine learning-based workload management engine groups workloads into clusters based on features like I/O patterns and computes parameters to identify bully workloads, using a self-training model to dynamically apply constraints and optimize resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual analysis is used to identify noisy neighbors, then accuracy can be achieved, but time consumption increases significantly
Solution Approach 1:
The patent replaces manual analysis (mechanical human effort) with an automated machine learning model that uses clustering algorithms and parameter computation to identify bully workloads. This substitution maintains identification accuracy while dramatically reducing time consumption by automating the entire analysis process.
Solution Approach 2:
The system implements self-service through automated workload analysis where the machine learning model independently identifies noisy neighbors without requiring manual intervention. The model continuously monitors workload parameters, performs clustering analysis, and automatically detects bully workloads, enabling the system to serve itself in terms of performance optimization.
2Reliability
If quality-of-service mechanisms apply workload constraints, then resource contention is reduced, but precision in constraint application is insufficient
Solution Approach 1:
The patent applies local quality by computing specific parameters for each workload cluster that capture their unique characteristics and resource usage patterns. Instead of applying generic constraints, the system tailors constraints to each identified bully workload based on its specific behavior, achieving precise and targeted resource management.
Solution Approach 2:
The system changes parameters by computing workload cluster contribution parameters that quantify the impact of each workload on resource contention. These parameters are used to dynamically adjust constraints, transforming the constraint application from static and imprecise to dynamic and precise based on actual workload behavior.
3Loss of information
If workloads are analyzed individually, then detailed information is obtained, but system complexity increases
Solution Approach 1:
The patent merges individual workload analyses by grouping workloads into clusters based on similar characteristics and resource usage patterns. This clustering approach preserves the detailed information of individual workloads while reducing analysis complexity by treating similar workloads as a unified group for constraint application.
Solution Approach 2:
The system implements universality by creating workload clusters that represent multiple workloads with similar behaviors. A single analysis process can handle multiple workloads through their cluster representation, making the system multi-functional in analyzing different workload types while maintaining a unified and manageable complexity level.
Data Source
AI summary
In some examples, a system creates a training data set based on features of sample workloads, the training data set comprising labels associated with the features of the sample workloads, where the labels are based on load indicators generated in a computing environment relating to load conditions of the computing environment resulting from execution of the sample workloads. The system groups selected workloads into a plurality of workload clusters based on features of the selected workloads, and computes, using a model trained based on the training data set, parameters representing contributions of respective workload clusters of the plurality of workload clusters to a load in the computing environment. The system performs workload management in the computing environment based on the computed parameters.