Storage Scrubbing Scheduling via Conformal Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large data storage systems face performance degradation due to frequent scrubbing of physical storage devices, which increases wear and load, while infrequent scrubbing may delay detection of media errors. Existing methods do not differentiate between storage devices based on health, leading to inefficient resource allocation.
Innovation Solution
Implement a method to determine the relative eligibility of physical storage devices for scrubbing using conformal prediction analysis, ranking them based on health scores, and scheduling scrubbing frequency according to predicted workload and health categories, thereby reducing unnecessary scrubbing and optimizing resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If frequent scrubbing is performed on all physical storage devices, then media errors are detected timely, but system performance degrades and device wear increases
Solution Approach 1:
The patent applies local quality by differentiating scrubbing frequency based on individual device health status. Instead of uniform scrubbing across all devices, the system assigns different scrubbing frequencies to different physical storage devices according to their specific health conditions, thereby optimizing the balance between error detection and performance impact.
Solution Approach 2:
The system dynamically changes the scrubbing frequency parameter based on device health metrics. By monitoring health indicators and adjusting the scrubbing interval parameter accordingly, the system reduces scrubbing frequency for healthy devices while maintaining high frequency for deteriorating devices, thus resolving the contradiction between reliability and productivity.
2Reliability
If frequent scrubbing is performed on all physical storage devices, then media errors are detected timely, but device wear increases
Solution Approach 1:
The patent applies local quality by differentiating scrubbing frequency based on individual device health status. Instead of uniform scrubbing across all devices, the system assigns different scrubbing frequencies to different physical storage devices according to their specific health conditions, thereby optimizing the balance between error detection and performance impact.
Solution Approach 2:
The system performs partial scrubbing action by selectively applying full scrubbing only to devices that need it most (those with deteriorating health), while applying reduced or no scrubbing to healthy devices. This partial action approach minimizes unnecessary wear on healthy devices while maintaining adequate monitoring for at-risk devices.
3Productivity
If infrequent scrubbing is performed, then system performance is maintained, but detection of media errors is delayed
Solution Approach 1:
The system dynamically changes the scrubbing frequency parameter based on device health metrics. By monitoring health indicators and adjusting the scrubbing interval parameter accordingly, the system reduces scrubbing frequency for healthy devices while maintaining high frequency for deteriorating devices, thus resolving the contradiction between reliability and productivity.
4Ease of operation
If uniform scrubbing frequency is applied to all devices, then management is simplified, but resource allocation is inefficient
Solution Approach 1:
The patent applies local quality by differentiating scrubbing frequency based on individual device health status. Instead of uniform scrubbing across all devices, the system assigns different scrubbing frequencies to different physical storage devices according to their specific health conditions, thereby optimizing the balance between error detection and performance impact.
Data Source
AI summary
From among physical storage devices (PSDs) of a storage system, a set of two or more of the PSDs that are eligible for scrubbing may be determined; and from among the set, a relative eligibility of the PSDs may be determined. Conformance prediction analysis may be applied to determine the set and the relative eligibility of PSDs of the set. The conformance prediction analysis may determine a scrubbing eligibility classification (e.g., label), and a confidence value for the classification, which may serve as the relative eligibility of the PSD. The eligible PSDs may be ranked in an order according to determined confidence values, and may be further classified according to such order. The future workload of the storage system may be forecasted, and the scrubbing of PSDs may be scheduled based on the forecasted workload of the system and the relative eligibilities of the set of PSDs.