Log Replication State Checks Using Compaction and Override Indices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing log compaction schemes in leader-follower server configurations fail to maintain consistency in replicated logs, leading to temporary inconsistencies in follower servers when using log cleaning approaches, which is critical for low-latency state externalization in telecommunications systems.
Innovation Solution
A method where the leader server performs log compaction by removing obsolete entries and shares compaction and override indices with follower servers, enabling them to determine consistency of their replica states during log replay.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If log compaction is performed to remove obsolete entries, then log size is reduced and storage efficiency is improved, but temporary inconsistencies may occur in follower servers during log replay
Solution Approach 1:
The leader server performs log compaction in advance before sending the compacted log to follower servers. By pre-identifying and removing obsolete log entries that will never be needed, the system reduces the log size that followers must process, while maintaining consistency through the index sharing mechanism
Solution Approach 2:
The system implements a feedback mechanism where the leader server shares compaction indices and override indices with follower servers. These indices provide feedback information that allows followers to verify their log replay progress and detect any inconsistencies, ensuring reliability during the compaction process
2Ease of manufacture
If log cleaning is used to remove overridden entries, then storage efficiency is improved, but follower servers may experience temporary state inconsistencies
Solution Approach 1:
The compaction indices and override indices act as intermediary elements between the leader server and follower servers. These indices mediate the log cleaning process by providing reference points that allow followers to reconstruct the correct state even when log entries are removed, preventing state inconsistencies
Solution Approach 2:
The system replaces the traditional mechanical approach of maintaining complete log history with a substituted mechanism using indices. Instead of physically retaining all log entries, the system uses compaction indices and override indices to represent and verify the logical state, achieving storage efficiency without sacrificing consistency
3Productivity
If the leader server sends compacted log to follower servers, then transmission efficiency is improved, but followers must verify consistency during log replay
Solution Approach 1:
The system changes the parameters of log transmission by including compaction indices and override indices alongside the compacted log entries. These additional parameters enable followers to efficiently verify consistency during replay without requiring complex verification logic, as the indices provide direct reference points for validation
Data Source
AI summary
According to an aspect there is provided a computer-implemented method of operating a leader server in a leader server-follower server log replication configuration. The method comprises (i) storing (801) a current master state of a system and maintaining a master log relating to the current master state, wherein the master log comprises a plurality of log entries each having a respective log index, wherein each log entry indicates a respective change to the master state of the system; (ii) performing (803) a log compaction procedure to remove from the master log any log entry for which the respective change to the master state of the system has been overridden by a change to the master state in a subsequent log entry; (iii) storing (805) a compaction index that is the highest log index of the log entries removed in the log compaction procedure; (iv) storing (807) an override index that is the highest log index of the log entries for which the respective change to the master state overrode a change to the master state by an earlier log entry; and (v) sending (809) the compaction index and the override index to one or more follower servers that are to store a replica of the master log relating to the current master state.