Storage Operation Time Prediction Using Hybrid API Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face challenges in accurately predicting the processing time for configuration operations due to varying network conditions and insufficient resource allocation, leading to unpredictable and inefficient management operations.
Innovation Solution
A management system that utilizes both analytical and statistical models to predict processing times by integrating an analytical model based on quantitative relations and a statistical model using historical data, adjusting weights based on variance to enhance accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If only a statistical model using historical data is used for prediction, then the model can be constructed from actual values, but prediction accuracy is insufficient when there are few actual values or for infrequently executed APIs
Solution Approach 1:
The patent combines an analytical model and a statistical model into a hybrid prediction system. The analytical model provides theoretical predictions based on quantitative relations, while the statistical model uses historical data. By merging these two models and using weighted combination, the system achieves accurate predictions even when historical data is limited, as the analytical model compensates for the statistical model's data scarcity issues.
Solution Approach 2:
The patent dynamically adjusts the weights of the analytical model and statistical model based on the availability and quality of historical data. When few actual values are available, the weight of the analytical model is increased; when sufficient historical data exists, the statistical model's weight is increased. This parameter change approach optimizes prediction accuracy across different data conditions.
2Adaptability or versatility
If configuration operations are executed asynchronously to handle complex resource changes, then more complex operations can be performed, but the processing time becomes difficult to predict due to varying network conditions and server congestion
Solution Approach 1:
The patent performs preliminary analysis by calculating the analytical prediction value before executing the configuration operation. This pre-calculation uses the analytical model to estimate processing time based on the operation's quantitative characteristics, providing an early reference point that helps manage expectations and plan subsequent actions despite the asynchronous nature of the operation.
Solution Approach 2:
The system continuously monitors actual processing times of configuration operations and feeds this information back to update the statistical model. This feedback mechanism allows the system to learn from past performance under various network and server conditions, improving the accuracy of future predictions for asynchronous operations.
3Speed
If storage systems prioritize I/O operations to ensure data access performance, then I/O response time is improved, but management operations such as configuration operations suffer from insufficient performance due to limited resource allocation
Solution Approach 1:
The management system performs self-analysis by automatically calculating prediction values for management operations without requiring external intervention. The system uses its own historical data and analytical models to assess and optimize management operation performance, enabling it to self-regulate resource allocation and identify performance bottlenecks independently.
Solution Approach 2:
The patent introduces a prediction calculation mechanism as an intermediary between management operation requests and execution. This intermediary analyzes the operation characteristics, predicts processing requirements, and provides guidance for resource allocation, thereby improving management operation performance without directly interfering with the I/O operation prioritization mechanism.
Data Source
AI summary
The prediction accuracy of required lengths of time for management operations to be performed in a storage system in response to operation requests made via a communication network from an API for management operations is improved. A management system of the storage system calculates a prediction value of a required length of time by an analytical model, which is a model of an ideal operation performed in response to an operation request for an API, calculates a prediction value of the required length of time by a statistical model, which is a model constructed on the basis of statistics of history of an operation performed in response to the operation request for the API, and determines a prediction value of the required length of time from those prediction values and weights of those models.