Instance Data Replication via Logging Infrastructure
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In cloud computing, there is a need for efficiently replicating data across multiple instances while ensuring reliability, scalability, and security, particularly for overlapping data such as user data, customer data, and product catalog data, which is currently not effectively addressed.
Innovation Solution
A method for replicating instance data involves detecting data modification events in a producer instance, generating replication event data based on predefined criteria, and transporting it to a logging infrastructure, where it is read and loaded onto a consumer instance, using a publish/subscribe-model messaging platform for secure and scalable data replication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is replicated across multiple cloud instances, then data availability and consistency are improved, but system complexity and security management become worsened
Solution Approach 1:
The patent introduces a logging infrastructure as an intermediary component that sits between producer instances and consumer instances. This mediator receives replication event data from producers, standardizes the data format, and distributes it to consumers, thereby simplifying the overall system architecture and reducing the complexity of direct peer-to-peer replication management while maintaining data availability across instances
Solution Approach 2:
The patent implements data copying through replication event data that captures changes at the producer instance and transmits them to consumer instances. This copying mechanism ensures data consistency across multiple instances without requiring complex synchronization protocols, as each consumer independently receives and applies copies of the replication events
2Reliability
If real-time data replication is implemented, then data consistency is improved, but processing overhead and resource consumption become worsened
Solution Approach 1:
The patent extracts only the necessary replication event data from the producer instance, specifically capturing change data in a standardized format. This extraction approach avoids copying entire datasets or unnecessary information, reducing processing overhead while maintaining data consistency. The system takes out only the incremental changes that need to be replicated rather than processing complete data sets
Solution Approach 2:
The patent changes the parameter of data representation by using a standardized replication event data format with specific fields (producer instance ID, consumer instance ID, replication set name, change data). This parameter standardization enables efficient processing and reduces computational overhead compared to unstructured or instance-specific data formats, allowing real-time replication with lower resource consumption
3Loss of substance
If selective data replication is implemented, then network bandwidth usage is reduced, but data selection complexity becomes worsened
Solution Approach 1:
The patent implements self-service data selection through subscriber criteria that are configured once and automatically applied. The logging infrastructure autonomously filters and selects replication event data based on predefined subscriber criteria without requiring manual intervention or complex selection logic. This self-service mechanism reduces network bandwidth usage by automatically transmitting only relevant data while avoiding the complexity of manual data selection management
Data Source
AI summary
A method for replicating instance data includes: detecting, based on producer instance replication configuration data, a data modification event associated with a record of a replication target table included in a replication set of the producer instance; generating replication event data of the data modification event when the data modification event is determined to be eligible for replication based on predetermined criteria included in the producer replication configuration data; placing the encrypted replication event data on an outbound replication queue to transport the replication event data to a logging infrastructure for replication; reading, at a consumer instance subscribing to the replication set of the producer instance, the replication event data from the logging infrastructure based on consumer instance replication configuration data; and loading a record onto a replication destination table of the consumer instance based on the read replication event data.