Database Caching via Asynchronous Log-Based Replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional database caching methods using synchronous replication impose usability restrictions and data availability issues due to granular locking, hindering application performance and scalability.
Innovation Solution
Implementing asynchronous log-based replication to manage database transactions and caching, where a caching engine dynamically routes queries based on table synchronization state using metadata, avoiding locking scenarios and enabling seamless caching and routing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If synchronous replication with granular locking is used to keep cache synchronized with database, then data consistency is improved, but usability restrictions and data availability deteriorate
Solution Approach 1:
The patent segments the locking mechanism from the replication process. Instead of using granular locking on individual tables or rows during replication, the system divides replication into discrete log-based events that can be processed independently. This allows the cache to be updated in small, manageable units without requiring locks on the entire database or cache structure, thereby maintaining data consistency while reducing usability restrictions.
Solution Approach 2:
The patent introduces an intermediary log table that sits between the database and the cache. Database transactions are first logged to this intermediate log table, which then serves as the source for cache replication. This intermediary layer decouples the database from the cache, allowing the cache to be updated asynchronously without requiring locks on the database tables, thus improving data availability while maintaining consistency through the log-based replication process.
2Reliability
If synchronous replication with granular locking is used to keep cache synchronized with database, then data consistency is improved, but scalability and performance deteriorate
Solution Approach 1:
The patent performs preliminary action by logging database transactions to the log table before they are applied to the database tables. This preliminary logging step captures all data changes in a structured format that can be efficiently replicated to the cache later. By separating the logging phase from the replication phase, the system can maintain data consistency without requiring synchronous locking during cache updates, thereby improving scalability and performance.
Solution Approach 2:
The patent implements dynamic replication where the cache synchronization process can adapt to system conditions. The replication manager can dynamically adjust the replication rate, batch size, and timing based on database load, cache state, and system resources. This dynamic approach allows the system to maintain data consistency while optimizing for scalability and performance under varying workloads, avoiding the rigid constraints of synchronous replication.
3Productivity
If asynchronous log-based replication is used to improve scalability and performance, then scalability and performance are improved, but implementation complexity increases
Solution Approach 1:
The patent implements a universal log table structure that can serve multiple functions: it acts as a transaction log, a replication source, and a synchronization checkpoint mechanism. The same log table infrastructure supports both synchronous and asynchronous replication modes, as well as providing audit trails and rollback capabilities. This multi-functionality reduces implementation complexity by consolidating what could be separate complex systems into a single unified log-based approach.
Solution Approach 2:
The patent uses copying by creating a replicated version of the database table structure in the cache based on the log entries. Rather than implementing complex real-time synchronization protocols, the system simply copies data from the database through the log mechanism to the cache. This copying approach, combined with the structured log format, simplifies the implementation of asynchronous replication while maintaining scalability and performance benefits.
Data Source
AI summary
A database table within a database to persist within a cache as a cached table can be identified. The database can be a relational database management system (RDBMS) or an object oriented database management system (OODBMS). The cache can be a database cache. Database transactions within can be logged within a log table and the cached table within the cache can be flagged as not cached during runtime. An asynchronous replication of the database table to the cached table can be performed. The replication can execute the database transactions within the log table upon the cached table. The cached table can be flagged as cached when the replication is completed.