ML-Assisted Cloud Data Replication Timing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cloud storage services, such as Amazon S3, lack automated support for archiving existing data objects across regions, leading to insufficient duplicity and resilience against disasters, particularly in scenarios where manual setup is required for Cross Region Replication (CRR).
Innovation Solution
A computer-implemented method using a trained machine learning model to identify existing objects in a bucket for replication, determine a commencing time based on predicted replication failures, capture snapshots including object metadata and ACLs, and replicate these objects to a destination cloud across regions, enhancing timeliness, redundancy, efficiency, accuracy, comprehensiveness, and authenticity for robust data archiving and disaster recovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If manual setup is required for Cross Region Replication, then configuration flexibility is improved, but ease of operation deteriorates
Solution Approach 1:
The system performs self-service by automatically detecting objects for replication and determining optimal replication timing without requiring manual configuration. The machine learning model autonomously identifies existing objects and schedules replication based on predicted failures, eliminating the need for manual setup while maintaining operational flexibility.
2Productivity
If automated replication is implemented, then productivity is improved, but device complexity increases
Solution Approach 1:
A machine learning model serves as an intermediary between the cloud storage service and the replication system. This intermediary automatically detects objects for replication and determines optimal timing by analyzing predicted failures, thereby automating the replication process without requiring complex manual configuration or intervention.
Solution Approach 2:
The system incorporates feedback mechanisms where the machine learning model continuously learns from replication outcomes and predicted failures. This feedback loop enables the system to automatically adjust replication strategies, improving productivity while managing complexity through data-driven decision-making rather than hard-coded rules.
3Reliability
If replication timing is determined based on predicted failures, then reliability is improved, but loss of time increases due to prediction processing
Solution Approach 1:
The machine learning model performs preliminary analysis by predicting potential replication failures before they occur. By proactively identifying objects at risk and determining optimal replication timing in advance, the system ensures reliable data protection while minimizing the impact of prediction processing time on overall replication performance.
Data Source
AI summary
Systems and methods of data replication via machine learning techniques are disclosed. In one embodiment, an exemplary computer-implemented method may comprise: utilizing a trained replication machine learning model to identify an existing object in the bucket for replication, and a commencing time to replicate the existing object, the commencing time determined based on replication failure predicted by the replication machine learning model; capturing, in response to identifying the existing object for replication, a snapshot of the bucket, the snapshot comprising information related to at least one of: the existing object, metadata of the existing object, and/or an access control list (ACL) of the existing object; and replicating the existing object to a destination cloud according to the determined commencing time, the destination cloud being hosted at a cross-region storage.