Heterogeneous Database Replication via Shared Memory Intermediary
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for replicating data among heterogeneous databases face challenges such as low efficiency, poor performance in real-time data processing, and insufficient support for mass data transfer and synchronization, particularly due to high storage overhead and additional load on source databases.
Innovation Solution
A system utilizing shared memory, message queues, and semaphores to enable concurrent execution of data reading, converting, and replicating, where data is loaded into shared memory segments and replicated according to message queue notifications, optimizing the use of memory and reducing interference between database processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is replicated by exporting to files with specific format through database management system, then data can be transferred between heterogeneous databases, but storage overhead is large and real-time performance is poor
Solution Approach 1:
The patent introduces a shared memory space as an intermediary between source and target databases. Instead of direct file-based export/import or manufacturer-specific replication tools, the system uses shared memory segments that can be accessed by multiple processes simultaneously, enabling efficient data transfer without relying on file I/O overhead or proprietary replication assemblies.
Solution Approach 2:
The shared memory mechanism provides universal data transfer capability across different database types without requiring manufacturer-specific replication tools. The shared memory space serves as a common ground that any database system can access, eliminating the need for specialized replication assemblies for each database vendor.
2Productivity
If changes in data are captured using triggers, logs, or broadcasting, then real-time synchronization is achieved, but additional heavy load is produced on source database
Solution Approach 1:
The system performs preliminary data loading into shared memory segments before replication to target databases. By pre-loading data in bulk operations rather than processing changes incrementally through triggers or logs, the system reduces the ongoing load on the source database while maintaining real-time synchronization capability.
Solution Approach 2:
The shared memory segments enable continuous data replication operations without interrupting source database operations. Multiple processes can concurrently read from and write to shared memory segments, allowing data synchronization to continue smoothly without the heavy periodic loads associated with change capture methods.
3Ease of manufacture
If manufacturer-specific replication products are used, then replication functionality is provided, but support for heterogeneous databases is insufficient
Solution Approach 1:
The shared memory space acts as a universal intermediary that transcends database vendor limitations. Instead of manufacturer-specific replication tools that work only with their own databases, the shared memory mechanism provides a common interface that can be accessed by different database systems, enabling heterogeneous database replication.
Solution Approach 2:
The system creates a copy of data in shared memory segments that can be read by multiple processes and databases simultaneously. This copying mechanism eliminates the need for proprietary replication assemblies and provides universal access to data across heterogeneous database systems.
4Quantity of substance
If mass data transfer is performed using traditional methods, then data replication is achieved, but storage overhead is high and efficiency is low
Solution Approach 1:
The patent merges multiple data transfer operations into a single shared memory space. Instead of transferring data through multiple file I/O operations or sequential replication steps, the system combines all transfers into concurrent shared memory accesses, significantly improving efficiency for mass data transfer.
Solution Approach 2:
Data is loaded into shared memory segments in advance before replication to target databases. This preliminary loading allows subsequent replication operations to proceed more efficiently by working with data already in memory rather than performing repeated disk I/O operations during transfer.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
A system and a method for replicating data among heterogeneous databases are disclosed. The system comprises: an initialization module for creating and initializing at least one shared memory, at least one message queue and at least one semaphore in a computer; a source data processing module including a data reader, a data adapter and a data loader; a target data replication module for replicating the data in the shared memory to at least one target database according to the messages in the message queue; and a cleaning module for releasing the shared memory after the replication has been completed. The method comprises the steps of: creating and initializing the shared memory, the message queue and the semaphore in the computer; reading data from at least one source database, then converting the data into target data to be stored in at least one target database, and loading the target data in the shared memory; replicating the target data in the shared memory to the at least one target database according to the messages in the message queue; and releasing the shared memory after the replication has been completed.