Storage Tier Distribution Optimization via PCA Clustering

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current data storage systems face challenges in optimizing storage tier configurations, leading to sub-optimal I/O performance and increased costs due to misconfigured tier distributions, which can result in reduced performance and higher support costs.

Innovation Solution

The method involves applying Principal Component Analysis (PCA) to reduce dimensionality of tier distribution data, clustering similar configurations, and selecting optimal tier distributions based on storage capacity requirements and expected I/O workloads to determine a recommended drive configuration that meets specified I/O workload requirements.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If manual configuration of storage tier distributions is performed, then flexibility in customization is improved, but configuration accuracy deteriorates leading to misconfigurations

Engineering Contradiction:

Improveconfiguration flexibilityVSAvoidconfiguration accuracy

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

The system automatically evaluates storage tier distributions and determines optimal configurations without requiring manual expert intervention. The evaluation system self-services by collecting performance data, analyzing it through multiple metrics, and generating recommended configurations autonomously, eliminating the trade-off between manual flexibility and accuracy.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system implements continuous feedback loops where storage performance data is collected, analyzed, and used to adjust and optimize tier distributions. Performance metrics from the storage system feed back into the evaluation engine, which refines configurations based on actual observed behavior, ensuring both adaptability and precision.

Inventive Principle:

Principle #23Feedback

2Manufacturing precision

If comprehensive performance evaluation is conducted, then configuration optimization is improved, but evaluation time deteriorates

Engineering Contradiction:

Improveconfiguration optimizationVSAvoidevaluation time

Core Design Contradiction:

Manufacturing precisionVSLoss of time

Solution Approach 1:

The system pre-calculates and stores performance baseline data for various storage configurations during system setup and initial operation. When evaluation is needed, it retrieves and compares against these pre-established benchmarks rather than conducting full-performance tests from scratch, significantly reducing evaluation time while maintaining optimization quality.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The evaluation system selectively analyzes only the most relevant performance metrics and configuration parameters based on the specific storage workload and system state. Rather than evaluating all possible parameters comprehensively, it focuses on the critical subset that has the greatest impact on optimization, reducing evaluation time while maintaining effective configuration improvement.

Inventive Principle:

Principle #16Partial or excessive action

3Productivity

If multiple storage tiers are implemented, then I/O performance is improved, but system complexity deteriorates

Engineering Contradiction:

ImproveI/O performanceVSAvoidsystem complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The system manages multi-tier complexity by dynamically adjusting configuration parameters such as tier capacity allocations, performance thresholds, and data placement policies based on observed workloads. Rather than requiring complex manual setup, the system adapts parameters automatically, maintaining high I/O performance while reducing operational complexity through parameter-driven management.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS11372561B1Techniques for identifying misconfigurations and evaluating and determining storage tier distributions

Publication Date: 2022.06.28 EMC IP HLDG CO LLC

AI summary

Determining drive configurations may include: receiving a data set including tier distributions for data storage systems; applying principal component analysis to the data set to generate a resulting data set having number of dimension in comparison to the data set; determining clusters using the resulting data set, wherein each cluster includes a portion of the tier distributions, wherein each cluster has an associated cluster tier distribution determined in accordance with the portion of the tier distributions in the cluster; selecting one of the clusters; and performing first processing that determines, in accordance with a storage capacity requirement and in accordance with a corresponding cluster tier distribution of the selected one cluster, a drive configuration.