Segmented Storage Tier Logging for Backup Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems face inefficiencies in backup and restore processes due to the lack of optimized temperature-based storage techniques, leading to high storage costs and I/O penalties, especially when dealing with diverse performance characteristics across storage tiers.
Innovation Solution
A method is introduced that divides storage tiers into subsets based on performance characteristics, creating separate log files for each subset to maintain data change history, allowing for localized backup and recovery, and utilizing a global log sequence number for seamless restoration across tiers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single log file is used for all storage tiers, then the logging process is simple to manage, but backup and restore operations incur high I/O penalties and take longer to complete
Solution Approach 1:
The patent divides the storage tiers into multiple subsets based on performance characteristics (hot, warm, cold tiers) and creates separate log files for each subset. This segmentation allows backup and restore operations to target only specific log files relevant to the requested storage tier, rather than scanning through a single large log file containing all tiers' data, thereby reducing I/O penalties and accelerating recovery operations.
2Productivity
If separate log files are created for each storage tier subset, then backup and restore operations become more efficient, but the logging process complexity increases
Solution Approach 1:
The patent introduces a mapping table that acts as an intermediary between storage tiers and their corresponding log files. This mapping table stores the relationships between storage tier identifiers and log file identifiers, allowing the system to efficiently translate restore requests into targeted log file operations without requiring complex direct mapping logic throughout the entire logging process.
Solution Approach 2:
The patent changes the parameter of log file organization from a single consolidated file to multiple segmented files based on storage tier performance characteristics. By categorizing storage tiers into hot, warm, and cold subsets and assigning separate log files to each, the system optimizes backup and restore operations by reading only the relevant log files needed for specific tier recoveries, thereby improving efficiency despite increased structural complexity.
3Reliability
If all storage tiers are restored together, then data consistency across tiers is maintained, but recovery time increases significantly
Solution Approach 1:
The patent segments the restore operation into independent tasks corresponding to different storage tier subsets. Each subset (hot, warm, cold tiers) has its own dedicated log file, allowing parallel restoration of multiple tiers simultaneously. The system can restore frequently accessed hot tiers quickly using their specific log files while concurrently restoring less critical cold tiers, thereby reducing overall recovery time while maintaining data consistency through coordinated transaction management.
Data Source
AI summary
A logging process in a data storage system having a set of storage tiers, each storage tier of the set of storage tiers having different performance characteristics, wherein the set of storage tiers is divided into a plurality of subsets of storage tiers using the performance characteristics, may include initiating the logging process for creating a separate log file for each of the plurality of subsets of storage tiers for maintaining a history of data changes in the subset of storage tiers, thereby creating a plurality of log files. In response to a change in data stored in at least one storage tier of a subset of storage tiers of the plurality of subsets of storage tiers, one or more log records including information about the change may be generated and written into respective log files.