Database Replication Server Pulling Transaction Logs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database replication methods are resource-intensive and inefficient, leading to performance degradation and high latency due to synchronous data transfer and reliance on shared resources, which limits the number of replication servers that can be used without affecting primary database efficiency.
Innovation Solution
The method involves using database requests initiated by replication servers with metadata to specify the replication data needed, allowing the replication server to 'pull' data from the database server, reducing the workload on the primary database and enabling efficient data transfer of redo log data in portions rather than whole files, thus minimizing network bandwidth and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If synchronous mirroring of transaction log is used, then data consistency between primary and replication database is improved, but data throughput is significantly slowed down due to additional I/O activity
Solution Approach 1:
Instead of the primary database pushing transaction logs to replication databases (which causes I/O bottlenecks), the replication database pulls transaction log data by initiating database requests to the primary database. This inversion of the data flow direction eliminates the I/O burden on the primary database while maintaining data consistency through the use of metadata to track and retrieve only the necessary transaction log portions.
2Ease of operation
If whole transaction log files are copied to replication database, then replication is simplified, but latency increases due to the time needed to process entire files
Solution Approach 1:
The transaction log replication process is segmented into manageable portions using metadata that tracks the current position and scope of replicated data. Instead of copying entire transaction log files, the system divides the replication into smaller units (individual transactions or transaction blocks) that can be selectively retrieved and applied, significantly reducing latency while maintaining replication integrity.
Solution Approach 2:
The system performs partial replication by retrieving only the specific portions of transaction logs that are necessary to maintain data consistency, rather than copying entire files. The metadata mechanism enables the replication database to request only the minimal required data segments, reducing unnecessary data transfer and processing time.
3Reliability
If more replication servers are added, then load balancing and availability are improved, but performance degradation occurs due to increased workload on primary database
Solution Approach 1:
Each replication server independently manages its own data retrieval by initiating database requests to the primary database based on its own metadata state. This self-service approach eliminates the need for the primary database to coordinate or manage multiple replication connections, allowing unlimited replication servers to be added without increasing the primary database's workload, thus maintaining performance while improving availability.
Data Source
AI summary
The present invention concerns a method for replicating data of a database server (10) by at least one replication server (20), the method comprising the following steps performed by the at least one replication server (20):a. sending a database request (40) to the database server (10), wherein the database request (40) comprises metadata (421c, 421d) indicating replication data requested by the at least one replication server (20); andb. receiving a database response (40′) from the database server (10), wherein the database response (40′) comprises the requested replication data indicated by the metadata (421c, 421d).