Database Replication via Symbolic Links and Delta Logs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database replication mechanisms result in high memory consumption and performance issues due to the need to write large tables into recovery logs multiple times during recovery operations, leading to excessively large files.
Innovation Solution
A replication mechanism that uses symbolic links to create a replica of a source database, where only the symbolic link information is initially written to the recovery log, reducing the data volume and maintaining a manageable recovery log size by tracking changes through delta logs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all copies of the original table are written into the recovery log for replication, then replication functionality is achieved, but the recovery log becomes extremely large
Solution Approach 1:
The patent segments the replication information into two parts: (1) metadata about the replication relationship stored in the recovery log, and (2) actual data copies stored separately in delta logs. This segmentation allows the recovery log to remain small while still enabling full replication functionality.
Solution Approach 2:
The patent introduces symbolic links as intermediaries between the recovery log and the actual data copies in delta logs. The symbolic links store minimal reference information in the recovery log, while the actual data is accessed through these links, acting as a mediator that reduces the burden on the recovery log.
2Productivity
If tables are replicated across multiple hosts for local access, then system performance improves, but memory consumption increases
Solution Approach 1:
The patent segments data storage across multiple locations (source database and replica databases), allowing each host to access local replicas for improved performance while the segmentation of data and metadata reduces overall memory consumption in any single location.
Solution Approach 2:
The patent adds a temporal dimension to data storage by using delta logs that record changes over time. This allows replicas to be created and updated efficiently by applying delta logs rather than copying entire tables, reducing memory consumption while maintaining performance.
3Adaptability or versatility
If the original table is copied multiple times for replication, then local replicas are created on all servers, but the copying process becomes complex
Solution Approach 1:
The patent implements self-service replication where the system automatically manages replica creation and synchronization through delta logs and symbolic links. The copying process is automated and does not require manual intervention, reducing complexity while maintaining adaptability.
Solution Approach 2:
The patent performs preliminary actions by establishing symbolic links and metadata in the recovery log before actual data copying occurs. This preliminary setup simplifies the subsequent copying process by pre-defining the replication structure and relationships.
Data Source
AI summary
Data replication in a database includes identifying a source database system. The source database includes a main index file and a delta log file. To create a replica, one or more symbolic links to the source database system are generated. The symbolic links identify a path to a physical location of the source database. A replica of the source database is generated based on the symbolic links. The replica includes a copy of the main index file and delta log file. Information associated with the replica and the symbolic links is stored in a recovery log. Replica are provided transparently to most database engine components by re-using partitioning infrastructure. Components “see” replica as tables with a single partition; that partition is a local replica.