Storage Array Data Migration via ROI-Based Pool Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current distributed data storage systems face challenges in efficiently managing storage devices to maintain high IO rates under varying user workloads, as they lack effective mechanisms to dynamically migrate data between fast and slow devices based on host interest and cost-benefit analysis.
Innovation Solution
The system dynamically forms a fast pool using solid-state drives (SSDs) and a slow pool using hard disc drives (HDDs), with a controller migrating data sets between them based on a return on investment (ROI) determination, ensuring improved data transfer capacity over a predetermined payback period, thereby optimizing storage array performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored on fast devices (SSDs), then data transfer rate is improved, but storage cost increases
Solution Approach 1:
The storage array is segmented into multiple device types (fast devices like SSDs and slow devices like HDDs), allowing the system to distribute data across different performance tiers. This segmentation enables the system to achieve high data transfer rates for active data while using cost-effective storage for less frequently accessed data, thereby resolving the contradiction between speed and storage cost.
Solution Approach 2:
The system dynamically migrates data between fast and slow devices based on host interest levels and ROI calculations. Data that generates high host interest is automatically moved to fast devices to improve data transfer rates, while data with lower host interest is moved to slow devices to reduce storage costs. This dynamic adaptation allows the system to optimize the balance between speed and cost continuously.
2Adaptability or versatility
If data is migrated frequently between fast and slow devices, then system adaptability is improved, but resource consumption increases
Solution Approach 1:
The system implements a feedback mechanism that continuously monitors host interest levels and calculates ROI for data migration decisions. By using feedback from actual host access patterns and performance metrics, the system makes informed migration decisions that improve adaptability while avoiding unnecessary migrations that would consume excessive resources. The ROI calculation acts as a gatekeeper to ensure migrations are only performed when beneficial.
Solution Approach 2:
The system changes the parameter of host interest level thresholds to control migration behavior. By adjusting these parameters, the system can optimize the balance between adaptability and resource consumption. When host interest changes significantly, the system adapts by migrating data; when changes are minor, the system maintains the current state to conserve resources.
3Productivity
If ROI calculation is performed for every migration decision, then migration efficiency is improved, but processing time increases
Solution Approach 1:
The system performs preliminary actions by pre-calculating and caching host interest metrics and ROI values before migration decisions are required. This allows the system to have migration readiness information prepared in advance, so when a migration decision is needed, the ROI calculation can be performed quickly using pre-computed data rather than gathering all information from scratch at the moment of decision.
Data Source
AI summary
Method and apparatus for managing data in a multi-device data storage array. In accordance with various embodiments, a storage array of independent data storage devices are arranged to form a fast pool and a slow pool of said devices. A controller is adapted to migrate a distributed data set stored across a first plurality of said devices in the slow pool to a second plurality of said devices in the fast pool. The migration is carried out responsive to a return on investment (ROI) determination by the controller that an estimated cost of said migration will be outweighed by an overall improved data transfer capacity of the storage array over a predetermined minimum payback period of time. In some embodiments, the fast pool is formed from a plurality of solid-state drives (SSDs) and the slow pool is formed from a plurality of hard disc drives (HDD).