Database Cluster Page Pre-fetching via Replication Write-set
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Buffer pool page replacement in database clusters is inefficient due to higher storage device access latencies and lower throughput compared to memory access, leading to performance lag in slave servers during data replication, as they need to fetch required database pages from storage devices after the master server has completed computation-intensive work.
Innovation Solution
A method for proactive page pre-fetching, where a master node generates a page pre-fetch request for database pages referenced by a transaction and adds it to a replication write-set, which is then communicated to a slave node, allowing the slave node to pre-fetch these pages into its buffer pool before the replication write-set is applied.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the buffer pool is made as large as possible to reduce page replacements, then the need for buffer pool page replacement is reduced, but the available memory size is exhausted
Solution Approach 1:
The patent implements pre-fetching of database pages into the buffer pool before they are actually needed during transaction processing. The system proactively identifies pages that will be required soon and loads them into memory in advance, reducing the need for expensive on-demand page replacements while not requiring the buffer pool to be oversized.
2Productivity
If the slave server waits until it needs to apply a transaction to fetch required database pages, then the replication process is simple, but the slave server lags behind the master server
Solution Approach 1:
The patent enables slave servers to proactively pre-fetch database pages into their buffer pools before the replication write-set is applied. The slave receives a list of required pages from the master and loads them in advance, so when the transaction needs to be applied, the pages are already in memory, eliminating replication lag.
Solution Approach 2:
The system uses feedback mechanisms where the master server provides information about required database pages to the slave server, allowing the slave to adjust its pre-fetching behavior accordingly. This feedback loop ensures that the slave pre-loads the correct pages at the right time.
3Reliability
If the slave server fetches database pages from storage device during replication, then data consistency is maintained, but storage device access latency significantly reduces performance
Solution Approach 1:
The patent moves the page fetching operation from the moment of need (during transaction application) to a preliminary phase before the transaction is applied. By pre-loading pages into the buffer pool in advance, the system eliminates the latency of on-demand storage device access while maintaining data consistency through proper coordination between master and slave servers.
Data Source
AI summary
A method for use in a database cluster is provided. A first transaction is received at a first node of the database cluster. A page pre-fetch request corresponding to the first transaction is generated while executing the first transaction at the first node. The page pre-fetch request is indicative of at least one database page that is referenced by the first transaction. Moreover, the page pre-fetch request is added to a replication write-set of a second transaction at the first node. Subsequently, the replication write-set of the second transaction, along with the page pre-fetch request, is communicated to a second node of the database cluster.