Extent Map for Accelerated Clustered Storage Data Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In clustered storage systems, data recovery performance is hindered by random I/O read performance issues due to fragmented backup images across multiple disks, limiting the number of parallel recovery streams and decreasing throughput when multiple clients request data simultaneously.
Innovation Solution
The system generates an extent map associating the original order of data with its stored order across multiple storage units, allowing for optimized data retrieval by either static or dynamic optimization methods, which include generating a second map for stored data fragments or sending qualifying identifiers to rearrange data in its original order using write-back caching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is stored sequentially on disks for backup, then single-client restore performance is high, but multi-client parallel restore capability is limited due to random I/O reads
Solution Approach 1:
The backup image is segmented into multiple data fragments that are distributed across different storage units. The system creates an extent map that tracks the location of each fragment, enabling parallel retrieval operations. Multiple client restore requests can be served simultaneously by retrieving different fragments in parallel from different storage units, then reassembling the complete data sets.
Solution Approach 2:
The system transitions from sequential single-stream restore to parallel multi-stream restore by adding the dimension of concurrent operations. The extent map enables the system to manage multiple restore streams simultaneously by tracking fragment locations across storage units, allowing parallel I/O operations that were previously impossible with sequential restore methods.
2Quantity of substance
If multiple clients backup data simultaneously to the same server, then backup capacity increases, but data fragmentation increases and restore performance degrades
Solution Approach 1:
Data from multiple clients is segmented into distinct fragments and distributed across different storage units. The extent map maintains precise tracking of which fragments belong to which clients and their original positions. This segmentation allows the system to handle large volumes of backup data from multiple clients while maintaining efficient retrieval by accessing only the specific fragments needed for each restore operation.
Solution Approach 2:
The extent map acts as an intermediary data structure that mediates between the fragmented storage layout and the logical data organization. It enables the system to manage multiple client backups simultaneously by translating fragment locations back to their original contextual positions, allowing efficient data retrieval despite the fragmented physical storage arrangement.
3Adaptability or versatility
If disk heads search for scattered data fragments during restore, then data can be retrieved from fragmented backups, but I/O performance deteriorates due to random reads
Solution Approach 1:
The extent map is pre-computed and stored during the backup process, containing all information about fragment locations and their original positions. When a restore operation begins, the system uses this pre-computed map to immediately determine the optimal retrieval sequence without needing to search for scattered fragments. This preliminary action eliminates the time loss associated with random I/O searches during restore operations.
Solution Approach 2:
The extent map provides feedback information about fragment locations and retrieval status. The system uses this feedback to optimize the restore process by determining the most efficient sequence for reading fragments from storage units, minimizing disk head movement and random I/O operations while maintaining the ability to retrieve data from fragmented backups.
Data Source
AI summary
A system and method having accelerated data recovery in a clustered storage system is provided. The method may include generating an extent map of data having an original order, wherein the extent map associates the original order with a stored order of the data in a plurality of storage units coupled to a server node. After receipt of a request for data recovery of a subset of data, the method may further include optimizing data retrieval of the data to a minimum recovery time using the extent map. For static optimization, the server node can generate a second map indicating the stored order of each fragment of data to be sent prior to the data. Alternatively in dynamic optimization, the server node can send a qualifying identifier with each fragment indicating the storage location. Thereby, the client node is enabled to rearrange the data upon receipt to its original order.