Storage System Decrement Protection for Reference Counts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In information processing systems, small write requests can lead to data page deletion before associated inflight write requests are completed, resulting in data loss and inefficiencies due to premature decrementing of reference counts.
Innovation Solution
Implementing decrement protection logic that postpones the decrement request for data pages with inflight small write requests, ensuring data pages are not deleted until all related write transactions are completed, using a decref hash table and journal to manage inflight write counts and postponed flags.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If reference count is decremented immediately when a write request is received, then storage space can be released sooner, but data pages may be deleted before inflight write requests are completed causing data loss
Solution Approach 1:
The system performs preliminary actions by checking whether a write request is inflight before decrementing the reference count. The storage controller determines the status of write requests in advance and only decrements the reference count when it is confirmed that the write request is not inflight, preventing premature deletion of data pages while still enabling timely space release when safe.
2Reliability
If reference count decrement is postponed for all write requests, then data integrity is maintained, but processing latency increases and system efficiency decreases
Solution Approach 1:
The system applies different quality treatment to different write requests based on their status. Instead of uniformly postponing all decrement operations, the storage controller selectively decrements reference counts only for write requests that are not inflight, while maintaining postponed decrement for inflight requests. This localized differentiation maintains data integrity for critical operations while avoiding unnecessary delays for safe operations.
3Productivity
If reference count is decremented without checking write request status, then processing is faster, but data pages may be incorrectly marked for removal leading to processing waste
Solution Approach 1:
The storage controller implements a feedback mechanism by continuously monitoring the status of write requests before decrementing reference counts. The controller receives feedback about whether a write request is inflight or completed, and uses this feedback to determine whether to decrement the reference count. This feedback loop prevents incorrect removal of data pages that are still being processed, eliminating processing waste while maintaining reasonable operational speed.
Data Source
AI summary
A storage system in one embodiment comprises a plurality of storage devices storing data pages. Each data page has a content-based signature derived from that data page. The content-based signatures are associated with physical locations storing the data pages. In response to receipt of a write input/output (IO) request that includes a data segment that is smaller than a page granularity of the storage devices, a content-based signature associated with the data segment is determined which also corresponds to a target data page stored at one of the physical locations. In response to determining the content-based signature, an inflight write count corresponding to the content-based signature is incremented. In response to a decrement request to decrement a reference count of the physical location corresponding to the content-based signature, a decrement flag corresponding to the content-based signature is set in the data structure and the decrement request is postponed.