Adaptable Response Time Prediction for Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current approaches for predicting response times in computing systems, such as storage arrays, face challenges due to inadequate data quality, accuracy issues, and inability to handle unknown and variable workloads, leading to inefficient resource allocation and potential failure in meeting user requirements.
Innovation Solution
A method combining open-set classification with cluster membership calculation techniques to build a robust ML model that can identify and classify new workloads, allowing for the instantiation of specialized predictors to address unseen data types, thereby improving prediction accuracy and adaptability to variable workloads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional ML approaches are used for response time prediction, then the model can handle known workload types, but it fails to accommodate unknown and variable workloads
Solution Approach 1:
The patent implements dynamic workload classification by continuously monitoring workload characteristics and adapting cluster assignments in real-time. The system uses online learning techniques to update cluster centroids and reassign workloads to appropriate clusters as they evolve, enabling the model to adapt to unknown workload types while maintaining prediction accuracy through dynamic adjustment rather than static classification
Solution Approach 2:
The patent segments the workload space into multiple dynamic clusters based on similarity metrics, allowing unknown workloads to be grouped with similar known workloads. This segmentation approach enables the system to handle variable workloads by creating flexible clusters that can accommodate new workload types while leveraging patterns from existing clusters to maintain reliable predictions
2Adaptability or versatility
If clustering techniques are used to classify workloads, then the model can handle known workload types, but it fails when new workloads belong to unseen clusters
Solution Approach 1:
The patent implements a universal clustering framework where a single clustering algorithm serves multiple functions: initial workload classification, detection of new workload types, and dynamic reassignment. The system uses a universal similarity metric and cluster representation that can accommodate any workload type, eliminating the need for separate specialized models for different workload categories and reducing overall system complexity
Solution Approach 2:
The system employs self-organizing clustering mechanisms where the model automatically adapts to new workload types without external intervention. When new workloads are detected, the system automatically creates new clusters or reassigns existing ones based on similarity metrics, enabling self-service adaptation that reduces complexity by eliminating manual model updates and retraining requirements
3Measurement precision
If extensive data filtering and feature selection are performed, then data quality improves, but the processing time and complexity increase
Solution Approach 1:
The patent performs preliminary feature engineering and selection during the offline training phase, pre-computing relevant workload characteristics and storing them in optimized formats. This preliminary action ensures high data quality for online prediction while minimizing processing time during runtime, as the expensive feature selection and transformation operations are completed beforehand rather than in real-time
Solution Approach 2:
The system extracts and focuses on the most critical workload features that have the highest impact on response time prediction, removing less informative attributes from the processing pipeline. This selective extraction of key features maintains data quality by concentrating on relevant information while reducing processing time and complexity by eliminating unnecessary data handling steps
Data Source
AI summary
One example method includes running a workload through a trained open-set classification model, recovering, as a result of the running, a class and an open-setness score corresponding to the workload, determining, based on the class and the open-setness score, whether the workload is new, and when the workload is determined to be new, starting a new cluster that includes the workload. A response time predictor model may be used to predict a response time associated with the new workload.