Time-Series Replication Period Prediction for Storage Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional storage systems face a tradeoff between efficient resource usage and data loss vulnerability due to fixed replication cycles, which are either too long and waste resources or too short and increase data loss risk.
Innovation Solution
A dynamic replication cycle adjustment technique using machine learning models to predict optimal replication periods based on write-on-write distributions and utilization metrics, adjusting the cycle length according to data usage patterns.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a fixed replication cycle is used in storage systems, then resource usage efficiency can be improved by extending the cycle length, but data loss vulnerability increases due to longer exposure periods between replications
Solution Approach 1:
The patent implements dynamic replication cycle adjustment by transitioning from a fixed replication cycle to a variable one that adapts to actual data usage patterns. The system continuously monitors write-on-write time metrics and adjusts the replication cycle length in real-time, making the replication process flexible and responsive to changing data activity levels rather than operating on a static schedule
Solution Approach 2:
The system changes the replication cycle parameter based on observed data patterns. By analyzing write-on-write time distributions and identifying whether the system is in a bursty or steady state, the patent dynamically modifies the replication cycle duration to optimize both resource efficiency and data protection, adjusting this critical parameter according to actual workload characteristics
2Reliability
If a short replication cycle is used, then data protection is improved by reducing exposure time to data loss, but resource efficiency deteriorates due to unnecessary replications during high-activity periods
Solution Approach 1:
The system dynamically adjusts the replication cycle parameter based on the identified data activity state. During steady states with predictable write patterns, the system extends the replication cycle to improve resource efficiency. During bursty states with unpredictable writes, the system shortens the cycle to enhance data protection, thereby optimizing the parameter according to actual conditions rather than using a fixed conservative setting
Solution Approach 2:
The patent implements a dynamic replication strategy that responds to real-time data usage patterns. The system continuously monitors write-on-write time metrics and adapts the replication cycle length accordingly, making the replication process flexible and responsive to changing data activity levels rather than operating on a static schedule
Data Source
AI summary
A method comprising: generating a write-on-write distribution for a plurality of data units in a data entity, the write-on-write distribution including a plurality of buckets, each bucket corresponding to a different write-on-write time range, each bucket having a respective bucket size that is based on a count of ones of the plurality of data units whose respective predicted write-on-write times fall within the bucket's write-on-write time range; identifying one or more utilization metrics of a storage system where the data entity is hosted; classifying a representative bucket size of the write-on-write distribution and the one or more utilization metrics with a machine learning model to predict a replication period for the data entity, the representative bucket size being a bucket size within a predetermined distance from a mean bucket size of the write-on-write distribution; and configuring the storage system to replicate the data entity in accordance with the replication period.