Real-time Heterogeneous Database Replication System
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database replication solutions do not support real-time replication of heterogeneous databases, leading to inefficiencies in data management and increased load on database servers and networks, and lack real-time verification and correction capabilities.
Innovation Solution
An integrated replication system that includes a reader for collecting data changes, a storage module for storing this information, a replication module for replicating data using two methods (one for each transaction and one for each SQL), and a selection module that dynamically switches between these methods based on predetermined criteria to ensure real-time data consistency and minimize network load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If real-time replication of heterogeneous databases is implemented, then data consistency and business continuity are improved, but system complexity and resource consumption increase
Solution Approach 1:
The patent introduces a replication management system as an intermediary between source and target databases. This mediator handles the complexity of heterogeneous database replication by providing standardized interfaces, automatic schema mapping, and centralized control mechanisms, thereby achieving real-time data consistency without requiring direct complex integration between disparate database systems.
Solution Approach 2:
The replication management system is designed with multi-functional capabilities including automatic schema detection, data type mapping, conflict resolution, and support for multiple replication strategies. This universal approach allows a single system to handle various heterogeneous database combinations (Oracle, SQL Server, MySQL, etc.) without requiring separate specialized solutions for each database pair.
2Reliability
If real-time replication is implemented, then data availability is improved, but network load and server resource consumption increase
Solution Approach 1:
The system implements incremental replication where only changed data portions are transmitted in real-time, rather than replicating entire datasets. The replication management system tracks data changes using change data capture (CDC) mechanisms and selectively replicates only the modified records, significantly reducing network bandwidth consumption while maintaining real-time data availability.
Solution Approach 2:
The patent employs periodic batch replication combined with real-time change propagation. The system uses committed transaction logs to identify changes and replicates them in periodic batches, optimizing network utilization by grouping small changes together while still maintaining near-real-time data availability at the target system.
3Manufacturing precision
If data verification and correction capabilities are added, then data quality is improved, but processing time and system complexity increase
Solution Approach 1:
The replication management system performs preliminary data validation and schema verification during the replication setup phase and before data transmission. By pre-defining data type mappings, constraint rules, and validation schemas, the system ensures data quality is maintained without requiring extensive post-replication verification, thus reducing processing time while improving data quality.
Data Source
AI summary
Disclosed is an integrated replication system. The integrated replication system may include a reader configured to collect information including a transaction or QSL information indicative of data changes of a source system, a storage module configured to store the information collected by the reader, a replication module configured to replicate change data of the source system to a target system according to a first replication method or a second replication method, a selection module configured to control a replication module in order to selectively apply any one of the first replication method and the second replication method according to a predetermined criterion, and a deletion module configured to delete data after a specific point of time when a change of the replication method is generated by the replication module.