Persistent Memory Reference Count Recovery via Shadow Scanning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In persistent memory file systems, maintaining accurate reference counts is challenging due to the unpredictable order of operation persistence, leading to potential memory leaks or data corruption after system failures.
Innovation Solution
Implementing a mechanism to maintain primary reference counts by incrementing before linking and decrementing after unlinking, and using a scanner to traverse the file system post-failure to update shadow reference counts, ensuring correct counts are determined after recovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If reference counts are updated during normal operation in persistent memory file systems, then data management functionality is maintained, but accuracy of reference counts cannot be guaranteed after system failures due to unpredictable persistence order
Solution Approach 1:
The reference count management is segmented into two independent components: primary reference counts updated during normal operations, and shadow reference counts updated by a post-failure scanner. This segmentation allows each component to have a specific, simplified function while together they provide robust accuracy guarantees.
Solution Approach 2:
Shadow reference counts are prepared in advance by scanning the file system structure and counting references before failure occurs. This preliminary action ensures that even if primary reference count updates are lost or corrupted during failure, the shadow counts provide a reliable baseline for recovery.
2Reliability
If a scanner traverses the file system to recompute reference counts after failure, then accurate reference counts are restored, but system latency and downtime increase
Solution Approach 1:
The scanner performs reference count computation as a preliminary recovery action immediately after failure detection. By systematically traversing the file system structure and counting references, it establishes accurate baseline counts before normal operations resume, ensuring reliability without requiring complex real-time validation.
Solution Approach 2:
The scanner creates a copy of the reference count information by traversing the file system structure and counting references independently of the primary reference count mechanism. This copied information serves as a verification and correction source, allowing the system to restore accuracy without relying on potentially corrupted in-memory state.
3Reliability
If primary reference counts are incremented before linking and decremented after unlinking, then memory leaks are prevented, but the risk of data corruption increases if failures occur during these operations
Solution Approach 1:
The system applies beforehand cushioning by maintaining shadow reference counts that serve as a safety buffer against corruption. If a failure occurs during primary reference count updates, the shadow counts provide a fallback mechanism to detect and correct inconsistencies, cushioning the system against the harmful effects of potential corruption.
Solution Approach 2:
The scanner provides feedback by traversing the file system structure and comparing computed reference counts against stored primary reference counts. This feedback mechanism identifies discrepancies caused by failures during update operations and triggers correction, ensuring memory management correctness while mitigating data corruption risks.
Data Source
AI summary
Techniques are provided for maintaining and recomputing reference counts in a persistent memory file system of a node. Primary reference counts are maintained for pages within persistent memory of the node. In response to receiving a first operation to link a page into a persistent memory file system of the persistent memory, a primary reference count of the page is incremented before linking the page into the persistent memory file system. In response to receiving a second operation to unlink the page from the persistent memory file system, the page is unlinked from the persistent memory file system before the primary reference count is decremented. Upon the node recovering from a crash, the persistent memory file system is traversed in order to update shadow reference counts for the pages with correct reference count values, which are used to overwrite the primary reference counts with the correct reference count values.