Storage IO Feature Calibration for Unified ML Across Heterogeneous Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Heterogeneous storage systems with varying performance characteristics pose challenges for building unified machine learning models due to differences in raw performance data and feature values, making it difficult to combine training sets and perform inference across different systems.
Innovation Solution
A method for calibrating time-dependent IO features using coefficients extracted from multiple storage systems, applying a standard benchmark to normalize and align feature values across systems, enabling a single machine learning model to be used across diverse configurations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traces are collected from heterogeneous storage systems with different hardware and software configurations, then the quantity of training data increases, but the feature values become statistically different and cannot be combined into a unified training set
Solution Approach 1:
The patent applies parameter changes by introducing calibration coefficients that transform feature values from different storage systems into a unified statistical framework. Each storage system's time-dependent features are adjusted using system-specific coefficients derived from performance characteristics, enabling heterogeneous data to be combined while maintaining statistical consistency across systems
Solution Approach 2:
The patent uses calibration coefficients as an intermediary mechanism between heterogeneous storage systems and the unified machine learning model. These coefficients act as translators that normalize feature values from different systems, allowing the model to process data from diverse sources without requiring system-specific models
2Adaptability or versatility
If a unified machine learning model is built to work across multiple storage systems, then the model's versatility improves, but the precision of feature representation deteriorates due to statistical differences in feature values
Solution Approach 1:
The patent transforms the precision problem by changing the parameter representation through calibration. Instead of using raw feature values that vary statistically across systems, the patent applies calibration coefficients to transform these values into a standardized representation that maintains both universality and precision. The calibration process preserves the relative relationships between features while adjusting absolute values to match the training distribution
Solution Approach 2:
The patent applies preliminary calibration action before feeding features into the machine learning model. By pre-adjusting feature values using calibration coefficients derived from system performance characteristics, the patent ensures that features from any storage system are already aligned with the statistical expectations of the unified model, eliminating the need for system-specific preprocessing during inference
Data Source
AI summary
A method, computer program product, and computing system for processing a plurality of input/output (IO) requests associated with a storage object in a storage system. A plurality of IO features are generated using the plurality of IO requests associated with the storage object. A time dependent IO feature is identified from the plurality of IO features. A coefficient for the time dependent IO feature for the storage system is extracted. The time dependent IO feature is calibrated using the coefficient for the time dependent IO feature for the storage system relative to the time dependent IO feature from at least one other storage system.