Log Replay Buffer Preloading for Storage Server Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional networked storage systems experience slow log replay during disaster recovery due to serial execution of operations and high latency in fetching data from multiple storage devices.
Innovation Solution
Preloading a buffer with data based on a catalog in response to a storage server failure, allowing simultaneous replay of logged operations using the preloaded data, thereby reducing the need for data fetching during log replay.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If operations are replayed serially in the same order as stored in the log, then data consistency is maintained, but log replay speed deteriorates due to high latency in fetching data from multiple mass storage devices
Solution Approach 1:
The patent preloads data from mass storage devices into a buffer before log replay begins. By anticipating and fetching required data in advance (prior to the actual replay operations), the system eliminates waiting time during replay while maintaining the ability to replay operations in the correct sequence, thus resolving the contradiction between speed and data consistency.
2Productivity
If loads of transactions are issued in parallel for a large number of replay operations, then spindle resource utilization improves, but time wasted in fetching and loading data deteriorates
Solution Approach 1:
The system performs preliminary data loading into a buffer before replay operations begin. This pre-fetching approach ensures that when parallel replay operations are issued, the data is already available in memory, eliminating the time wasted during the actual replay phase while maintaining high spindle utilization during the preliminary loading phase.
3Speed
If parallelization of load operations is implemented to speed up log replay, then log replay speed improves, but system complexity deteriorates due to dependencies amongst transactions
Solution Approach 1:
The patent resolves transaction dependencies by preloading all required data into a buffer before replay begins. This eliminates the need for complex dependency checking and coordination during replay, as all data is already available. The system can then replay operations in parallel or serially without complexity, as the preliminary action has satisfied all data requirements.
Data Source
AI summary
A method and an apparatus to perform fast log replay have been disclosed. In one embodiment, the method includes preloading a buffer with data based on a catalog in response to a failure of a storage server and replaying a log as the buffer is being preloaded, wherein replaying the log comprises re-executing a set of modifying operations previously logged in a local non-volatile storage device in the storage server using the data preloaded.