ML-Based Resource Contention Identification in Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional time series analysis approaches face challenges in accurately identifying workload and resource imbalances in storage systems, leading to performance degradations due to their limitations in accuracy and scalability.
Innovation Solution
The implementation of machine learning techniques to calculate similarity measurements between primary and candidate time series, assigning weights based on similarity, and generating similarity scores to identify resources in contention within the storage system, enabling automated actions to address imbalances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional time series analysis approaches are used, then the system can identify workload and resource imbalances, but the accuracy and scalability are insufficient
Solution Approach 1:
The patent replaces conventional time series analysis approaches with machine learning techniques. Specifically, it uses supervised learning algorithms (such as random forests, support vector machines, or neural networks) to classify resources in contention based on time series data from storage systems. This substitution enables the system to achieve both high accuracy in identifying resources in contention and scalability to handle large volumes of data from multiple storage systems and workloads simultaneously.
2Measurement precision
If machine learning techniques are implemented to calculate multiple similarity measurements and assign weights, then accuracy of identifying resources in contention is improved, but computational complexity increases
Solution Approach 1:
The patent performs preliminary actions by pre-processing time series data and pre-training machine learning models offline. It calculates baseline statistics (mean, standard deviation) from historical data and pre-trains classification models using labeled datasets. This preliminary preparation reduces the computational complexity during real-time operation, as the system only needs to perform inference rather than full model training, thereby achieving high accuracy without excessive real-time computational burden.
Solution Approach 2:
The patent segments the analysis process into distinct phases: data collection, pre-processing, model training (offline), and inference (online). It also segments the feature extraction into multiple independent similarity measurements that can be calculated and weighted separately. This segmentation allows each component to be optimized independently, reducing overall computational complexity while maintaining high accuracy in identifying resources in contention.
Data Source
AI summary
Methods, apparatus, and processor-readable storage media for automatic identification of resources in contention in storage systems using machine learning techniques are provided herein. An example computer-implemented method includes obtaining a primary time series and multiple candidate time series; calculating, using machine learning techniques, similarity measurements between the primary time series and each candidate time series; for each similarity measurement, assigning weights to the candidate time series based on similarity to the primary time series relative to the other candidate time series; generating, for each candidate time series, a similarity score based on the weights assigned across the similarity measurements; identifying, based on the similarity scores, at least one of the candidate time series as representative of at least one resource in contention with respect to latency data represented by the primary time series; and outputting identification of the identified candidate time series for use in automated actions.