Bypass Destaging Decrement Reference Count Metadata Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In data storage systems, the decref tier often becomes full due to high system workloads or bursts of decref MD updates, leading to storage issues and potential data unavailability, despite overprovisioning, as existing methods struggle with efficient destaging and aggregation of decref MD updates.
Innovation Solution
The bypass destage technique directly applies in-memory decref MDL updates to MD pages in the MD page store, bypassing the decref tier and omitting the first phase of the two-phase destage process, allowing for direct storage without aggregating across multiple in-memory decref MDL instances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If the two-phase destage process is used to aggregate decref MD updates, then manufacturing precision of metadata consistency is improved, but the decref tier becomes full leading to storage issues and data unavailability
Solution Approach 1:
The patent extracts the problematic aggregation step from the two-phase destage process. Instead of aggregating decref MD updates across multiple in-memory decref MDL instances in the decref tier, the system applies decref updates directly to MD pages in the MD page store, bypassing the decref tier entirely. This extraction eliminates the storage capacity problem while maintaining metadata consistency through direct application of updates.
Solution Approach 2:
The patent introduces drop delta indicators as intermediary markers in the active decref MDL to track which decref updates have been applied to MD pages. This intermediary mechanism enables recovery processing to identify and skip already-applied updates, ensuring metadata consistency without requiring aggregation in the decref tier, thus resolving the storage capacity issue.
2Reliability
If overprovisioning is used to prevent decref tier overflow, then reliability of data availability is improved, but device complexity and cost increase
Solution Approach 1:
The patent removes the need for overprovisioning by extracting the aggregation function from the decref tier. By applying decref updates directly to the MD page store, the system eliminates the decref tier as a bottleneck, ensuring data availability without requiring additional storage capacity or complex overprovisioning strategies.
Solution Approach 2:
Instead of expanding storage capacity to handle aggregation (the conventional approach), the patent inverts the approach by eliminating the aggregation step entirely. The system goes from accumulating updates in the decref tier to directly applying them to the MD page store, thereby ensuring reliability without increasing device complexity or storage requirements.
3Manufacturing precision
If decref MD updates are aggregated across multiple in-memory decref MDL instances, then manufacturing precision of metadata consistency is improved, but loss of time due to aggregation processing increases
Solution Approach 1:
The patent extracts the time-consuming aggregation operation from the destage process. By directly applying decref updates from individual in-memory decref MDL instances to MD pages without aggregation, the system eliminates the time loss associated with collecting and processing updates across multiple instances, while still ensuring metadata consistency through direct application.
Solution Approach 2:
The patent skips the aggregation step entirely in the destage process. By rushing through the update application process directly from in-memory decref MDL instances to the MD page store, the system avoids the time penalty of aggregation while maintaining consistency through the use of drop delta indicators for tracking applied updates.
4Productivity
If the decref tier is made larger to handle high workloads, then productivity of decref processing is improved, but device complexity and storage cost increase
Solution Approach 1:
The patent extracts the processing bottleneck from the decref tier by eliminating the aggregation function. This allows the system to handle high workloads with existing storage capacity, improving decref processing throughput without requiring a larger or more complex storage infrastructure.
Solution Approach 2:
Instead of scaling up the decref tier to improve productivity, the patent inverts the approach by removing the aggregation step that causes the bottleneck. This enables high throughput decref processing with the existing storage capacity, avoiding the need for expanded or more complex storage systems.
Data Source
AI summary
In at least one embodiment, processing can include: determining that an inactive decref (decrement reference count) MDL (metadata log) of decref MD (metadata) updates stored in memory is unable to be destaged to a decref tier of non-volatile storage in a first destage phase of a two phase destage process; and responsive to determining that the inactive decref MDL is unable to be destaged to the decref tier, performing bypass destage processing of the inactive decref MDL, wherein said bypass destage processing includes directly applying MD updates of the inactive decref MDL to corresponding MD pages stored persistently in a MD page store and wherein said bypass destage processing omits storing the inactive decref MDL on the decref tier.