Cascaded PID Control of Metadata Page Eviction and Free Pool Size
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As data sets grow in enterprise storage systems, the increasing size of metadata requires substantial volatile memory, leading to high costs, and metadata paging helps reduce this by storing metadata in non-volatile storage and copying it into volatile memory in fixed-size pages, but managing this efficiently remains a challenge.
Innovation Solution
A metadata paging controller adjusts the target size of the free pool in memory based on setpoints and actual usage, using cascaded proportional-integral-derivative (PID) controllers to manage page evictions and eviction rates, ensuring efficient use of memory resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If metadata is maintained in volatile memory to ensure low-latency access, then system performance is improved, but the cost of volatile memory becomes prohibitively high as data set size increases
Solution Approach 1:
The patent segments metadata into fixed-size pages that can be independently managed. Instead of loading all metadata into volatile memory, only necessary pages are paged in, allowing the system to maintain low-latency access for active metadata while avoiding the cost of holding entire metadata sets in expensive volatile memory.
Solution Approach 2:
The patent extracts frequently accessed metadata pages from non-volatile storage into volatile memory on-demand. This selective extraction ensures that only the necessary portion of metadata resides in fast memory at any given time, reducing overall volatile memory requirements while maintaining performance for accessed data.
2Ease of operation
If a fixed-size free pool is maintained in memory for metadata paging, then memory management simplicity is improved, but the ability to adapt to varying workload demands deteriorates
Solution Approach 1:
The patent implements dynamic adjustment of the free pool target size using a cascaded PID controller. The controller continuously monitors the actual free pool relative minimum and adjusts the target size accordingly, allowing the system to adapt to varying workload demands while maintaining manageable memory control through structured control loops.
Solution Approach 2:
The patent employs feedback control mechanisms where the actual free pool relative minimum is measured and fed back to the PID controller. This feedback loop enables the system to automatically adjust the free pool target size based on actual conditions, improving adaptability while maintaining systematic control.
3Quantity of substance
If aggressive page eviction is performed to maintain free pool size, then memory resource efficiency is improved, but access latency increases due to frequent page-in operations
Solution Approach 1:
The patent implements periodic background page evictions that occur at controlled intervals rather than aggressively on-demand. This periodic approach allows the system to maintain free pool size while spreading out eviction operations, reducing their impact on access latency compared to aggressive immediate evictions.
Solution Approach 2:
The patent performs background page evictions in advance during periods of lower activity, preparing free space before it is needed. This preliminary action ensures that free pool size is maintained without causing latency during critical access operations, as evictions are performed proactively rather than reactively.
Data Source
AI summary
In a storage system that implements metadata paging, the page free pool is replenished in the background to reduce foreground evictions and associated latency on page-in. A two-level page eviction controller with cascaded proportional, integral, derivative (PID) controllers optimizes the size of the free page pool and optimizes the rate at which pages are freed in the background. By optimizing these two parameters the page eviction controller dynamically maximizes used pages (minimizing free pages) to increase the metadata cache hit ratio. Optimizing the parameters also reduces the chances of foreground page evictions, thereby reducing IO latency, during both steady state and burst page-in requests.