Extended Cache for Database Object Store Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud-native database systems face higher latencies during read and write operations on object stores, and the high cost of RAM makes it impractical to rely solely on a buffer manager to alleviate these latencies, especially in cloud computing environments where RAM is expensive.
Innovation Solution
Implementing an extended cache system that uses a unique physical layout, hybrid caching policies, and redirecting a varying percentage of I/O operations to the object store to optimize bandwidth utilization and reduce latency, while maintaining transactional consistency and minimizing costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If a buffer manager relies purely on RAM to alleviate higher latencies, then read and write operation latency is reduced, but storage and compute costs increase due to RAM being more expensive on the cloud
Solution Approach 1:
The buffer manager is segmented into two distinct components: a RAM-based buffer cache for high-speed access and an extended cache layer for capacity extension. This segmentation allows the system to use inexpensive storage for bulk capacity while maintaining expensive RAM for performance-critical operations, resolving the contradiction between latency reduction and cost control.
Solution Approach 2:
The extended cache acts as an intermediary layer between the RAM buffer cache and the object store. It mediates the trade-off by providing additional capacity at lower cost while maintaining performance through its integration with the buffer cache hierarchy, allowing the system to achieve both cost efficiency and performance without pure RAM reliance.
2Productivity
If an extended cache system is implemented to increase buffer cache capacity, then object store read operations are reduced, but device complexity increases
Solution Approach 1:
The extended cache is merged with the existing buffer cache manager infrastructure, sharing common data structures, replacement policies, and management routines. This merging approach increases buffer cache capacity while minimizing the addition of complexity, as the extended cache integrates seamlessly with the existing RAM buffer cache rather than operating as a separate system.
Data Source
AI summary
Disclosed herein are system, method, and computer program product embodiments for utilizing an extended cache to access an object store efficiently. An embodiment operates by executing a database transaction, thereby causing pages to be written from a buffer cache to an extended cache and to an object store. The embodiment determines a transaction type of the database transaction. The transaction type can a read-only transaction or an update transaction. The embodiment determines a phase of the database transaction based on the determined transaction type. The phase can be an execution phase or a commit phase. The embodiment then applies a caching policy to the extended cache for the evicted pages based on the determined transaction type of the database transaction and the determined phase of the database transaction.