Database Buffer Pool Extension via Secondary Memory Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large databases often face performance bottlenecks due to the limitations of rotational media, which are expensive, power-intensive, and cannot keep the working set in main memory, leading to inefficiencies in data access and storage.
Innovation Solution
Implementing a buffer pool that utilizes secondary memory, such as solid state storage, to classify pages as 'hot', 'warm', or 'cold' based on access history, allowing for efficient eviction of pages from main memory to secondary storage when needed, thereby extending the buffer pool and optimizing memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If rotational media is used to store database pages, then storage capacity is sufficient, but access performance becomes a bottleneck and power consumption increases
Solution Approach 1:
The patent segments the storage system into multiple layers: a fast buffer pool in main memory for frequently accessed pages, and slower rotational media for less frequently accessed pages. This segmentation allows the system to maintain high performance for hot data while preserving adequate storage capacity on rotational media.
Solution Approach 2:
The patent introduces a new dimension to the storage hierarchy by utilizing available memory space beyond the traditional buffer pool. This extends the buffer pool into previously unused memory regions, creating an expanded buffer pool that can hold more working set pages in fast memory, thereby improving access performance without adding rotational media.
2Productivity
If rotational media subsystem is expanded to keep working set in memory, then access performance improves, but cost and power consumption increase
Solution Approach 1:
The system uses available memory resources that would otherwise be unused or underutilized to extend the buffer pool. This self-service approach leverages existing memory capacity to improve performance without requiring additional power-intensive rotational media subsystems.
3Quantity of substance
If rotational media subsystem is expanded to increase storage capacity, then storage capacity improves, but cost increases
Solution Approach 1:
The patent makes existing memory resources serve multiple functions: the extended buffer pool serves both as cache for performance and as additional storage capacity. This multi-functionality eliminates the need for expensive rotational media expansion while maintaining adequate storage capacity for the working set.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Aspects of the subject matter described herein relate to a buffer pool for a database system. In aspects, secondary memory such as solid state storage is used to extend the buffer pool of a database system. Thresholds such as hot, warm, and cold for classifying pages based on access history of the pages may be determined via a sampling algorithm. When a database system needs to free space in a buffer pool in main memory, a page may be evicted to the buffer pool in secondary memory or other storage based on how the page is classified and conditions of the secondary memory or other storage.