Storage Controller Change Log for Selective Data Rebuild
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed storage systems, when a storage device becomes inoperable, existing techniques require a full rebuild of the dataset, which is time-consuming and burdens the system, and there is no guarantee that changes are written to the inaccessible device upon its return, leading to potential data loss and synchronization issues.
Innovation Solution
Implementing a method where a storage controller tracks write transactions using a change log, allowing for a selective rebuild of only modified data extents when a device returns online, and storing the change log in a controller cache for reduced burden and improved redundancy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a full rebuild of the dataset is performed when a storage device returns online, then data integrity is ensured, but rebuild time increases significantly and system performance deteriorates
Solution Approach 1:
The system performs preliminary actions by tracking and logging all data changes (writes, deletes, updates) in a change log while the storage device is inaccessible. This preliminary tracking enables a selective rebuild later, avoiding the need to reconstruct the entire dataset and significantly reducing rebuild time while maintaining data integrity.
Solution Approach 2:
The invention extracts only the necessary information (change log entries) from the overall dataset during the device's inaccessibility period. When the device returns online, only the specific changed data portions identified in the change log are rebuilt, rather than rebuilding the entire dataset, thus reducing rebuild time and system burden.
2Reliability
If a full rebuild of the dataset is performed when a storage device returns online, then data synchronization is achieved, but system resources and processing power are excessively consumed
Solution Approach 1:
The system performs preliminary tracking of all data changes in the change log during the device's inaccessibility. This preliminary action captures exactly what needs to be rebuilt, allowing the system to process only the necessary data portions when the device returns, thereby reducing processing power consumption while achieving complete synchronization.
Solution Approach 2:
Instead of performing a complete rebuild (excessive action), the system performs only the partial action necessary to rebuild the specific changed data portions identified in the change log. This partial rebuild approach achieves full synchronization without consuming excessive system resources.
3Reliability
If the entire dataset is tracked and stored to ensure data recovery, then complete data recovery is possible, but storage space and system complexity increase
Solution Approach 1:
The system extracts and stores only the essential recovery information in the change log (change identifiers, data portions, metadata) rather than tracking and storing the entire dataset. This extraction approach enables complete data recovery while minimizing storage space requirements and system complexity.
Solution Approach 2:
The system creates a simplified copy (change log) that contains only the necessary information to reconstruct changed data portions, rather than maintaining copies of the entire dataset. This copying strategy enables data recovery with reduced storage overhead and system complexity.
Data Source
AI summary
A system and method for maintaining operation of a storage array with one or more failed storage devices and for quickly recovering when failing devices are replaced are provided. In some embodiments, the method includes receiving a data transaction directed to a volume and determining that a storage device associated with the volume is inoperable. In response to determining that the storage device is inoperable, a data extent is recorded in a change log in a storage controller cache. The data extent is associated with the data transaction and allocated to the storage device that is inoperable. The data transaction is performed using at least one other storage device associated with the volume, and data allocated to the storage device is subsequently reconstructed using the recorded data extent.