Redundant Secondary Storage Cache Rebuild via Metadata Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Rebuilding a redundant secondary cache in storage systems is complex and time-consuming, especially in mission-critical environments, due to the need to flush large amounts of dirty data from SSDs, which disrupts performance and requires keeping the IO system offline.
Innovation Solution
A system and method for rebuilding a redundant secondary storage cache by using metadata to selectively rebuild only valid data regions, allowing IO operations to continue while rebuilding, and maintaining a watermark to differentiate between online and degraded cache operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If dirty data is flushed from SSD cache to primary storage to enable rebuild, then data redundancy can be restored, but performance deteriorates and rebuild time increases significantly
Solution Approach 1:
The patent divides the cache into multiple regions (first cache region and second cache region) and processes them separately. The first region containing dirty data is excluded from the rebuild operation, while only the second region is rebuilt. This segmentation allows the rebuild to proceed without requiring flushing of the entire cache, thereby reducing rebuild time while maintaining data redundancy for the processed regions.
Solution Approach 2:
The patent extracts and excludes the first cache region containing dirty data from the rebuild operation. By identifying and separating the dirty data region from the clean cache region, the system can perform rebuild operations on only the necessary portions of the cache without being constrained by the presence of dirty data elsewhere, thus avoiding the time-consuming flush operation.
2Reliability
If dirty data is flushed from SSD cache to primary storage, then cache can be rebuilt, but system performance deteriorates during the process
Solution Approach 1:
The cache is segmented into a first region with dirty data and a second region eligible for rebuild. By processing only the second region and excluding the first region from rebuild operations, the system maintains productivity for cached I/O operations while ensuring cache validity for the rebuilt portions. This avoids the performance deterioration associated with flushing entire caches.
Solution Approach 2:
The patent enables continuous I/O operations to proceed against the cache during the rebuild process. By using metadata markers to identify valid cache regions and allowing I/O to continue against those regions, the system maintains productivity without interruption while the rebuild operation progresses in the background on eligible regions.
3Reliability
If the entire dirty cache is flushed every time a server is rebooted, then data consistency is maintained, but performance suffers in configurations with large storage capacities
Solution Approach 1:
The patent divides the cache into regions based on data validity, using metadata markers to identify which regions contain dirty data versus clean data. After a server reboot or drive absence, only the regions that are eligible for rebuild (second region) are processed, while regions containing dirty data (first region) are excluded. This selective approach maintains data consistency for rebuilt regions without requiring performance-costly flush operations on entire caches.
Solution Approach 2:
Instead of flushing or rebuilding the entire cache, the patent performs partial actions only on the necessary regions. The rebuild operation is applied selectively to the second cache region that does not contain dirty data, while the first region with dirty data is left untouched. This partial action approach maintains data consistency where needed while preserving I/O performance by avoiding unnecessary operations on the entire cache.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
System and techniques for rebuilding a redundant secondary storage cache including a first storage device and a second storage device are described. A metadata entry indicative of a validity of a portion of information stored by a first storage cache device and associated with a region of a primary storage device is received. When the validity of the portion of information associated with the region of the primary storage device is established, a region lock is requested on the region of the primary storage device associated with the portion of information stored by the first storage cache device. Then, the portion of information and the corresponding metadata entry associated with the region of the primary storage device is copied from the first cache storage device to a second storage cache device to rebuild the second storage cache device.