Event-Driven Replication Mechanism for Scalable Multi-Cloud Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object storage replication systems face scalability issues as the number of objects grows, and they require tight coupling between source and target systems, which limits flexibility and efficiency, especially in multi-cloud environments.
Innovation Solution
A scalable and system-agnostic replication mechanism that uses event notifications to track changes and decouples the replication source from the target system, allowing for replication across different storage types and enabling flexible replication policies, including streaming processing to handle high-time constraint scenarios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional replication systems are used, then replication functionality is provided, but scalability deteriorates as the number of objects grows
Solution Approach 1:
The patent extracts the replication logic from the storage systems and implements it through a separate replication service that consumes events from a messaging system. This extraction allows the replication mechanism to scale independently from the storage systems, resolving the contradiction between quantity of objects and replication efficiency.
Solution Approach 2:
The patent introduces a messaging system as an intermediary between source and target storage systems. This mediator decouples the replication process from direct system coupling, enabling scalable event-driven replication that maintains efficiency regardless of the number of objects.
2Reliability
If tight coupling between source and target systems is used, then replication is achieved, but flexibility and adaptability deteriorate
Solution Approach 1:
The messaging system serves as a universal intermediary that enables replication between different storage systems without direct coupling. This allows the system to maintain replication reliability while achieving flexibility in multi-cloud environments by simply changing the target storage system without modifying the source system.
Solution Approach 2:
The replication service is designed with universal interfaces that can communicate with different types of storage systems through standardized event protocols. This universality enables the same replication mechanism to work across multiple cloud providers and storage types, enhancing adaptability while maintaining reliability.
3Productivity
If event notification-based replication is implemented, then scalability is improved, but system complexity increases
Solution Approach 1:
By extracting replication logic into a separate service that consumes events from a messaging system, the patent simplifies the overall architecture despite introducing new components. The event notification mechanism handles scalability automatically, and the separated architecture makes the system more maintainable and easier to implement.
Data Source
AI summary
A system can register, by a replication component and with a notification component, for notifications to changes in a group of data in data storage, wherein the notification component is configured to write respective changes in the group of data to a replication stream. The system can retrieve, by the replication component, a change of the changes in the group of data from the replication stream. The system can, in response to determining that the change corresponds to a replication policy, replicate, by the replication component, data of the group of data that corresponds to the change to a target system, wherein the replication component is configured to perform a replication on target systems having respective different storage types.