Standby Storage Detecting Cluster Health via Encoded Log Status
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-server data storage systems, standby data storage systems face challenges in detecting failures of primary data storage servers and preventing split-brain conditions, leading to data stalls and potential corruption due to delayed log merging and uncoordinated updates.
Innovation Solution
A method and apparatus for detecting operating conditions in a multi-server data storage system, where status information is encoded into log data and transmitted to a standby data storage system, allowing it to identify disabled servers and detect split-brain conditions, enabling timely log merging and preventing data corruption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the standby data storage system waits for log streams from all primary data storage servers before merging, then data integrity is maintained, but the system stalls and experiences delays when servers fail
Solution Approach 1:
The system performs preliminary actions by encoding status information into log data before transmission, allowing the standby to proactively identify disabled servers and proceed with merging logs from surviving servers without stalling, while maintaining data integrity through the status verification mechanism
Solution Approach 2:
The system implements feedback by encoding status information about server enablement into the log streams themselves. The standby data storage system uses this feedback to determine which log streams to merge and which to skip, resolving the contradiction between maintaining integrity and avoiding stalls
2Reliability
If hardware and software mechanisms are added to prevent split-brain conditions, then data corruption is reduced, but system complexity increases
Solution Approach 1:
The log stream serves multiple functions: it carries both data changes and status information about server enablement. This multi-functionality allows the system to detect split-brain conditions using the existing log infrastructure without adding separate complex detection mechanisms
Solution Approach 2:
The log data itself carries the information needed to detect split-brain conditions through encoded status information. The system uses its own operational data (status encoded in logs) to monitor its health, eliminating the need for external monitoring systems
3Productivity
If the standby data storage system merges logs in real-time, then data availability is improved, but the system may corrupt data if it merges logs from failed servers
Solution Approach 1:
The system encodes status information into log data before transmission, allowing the standby to identify and skip logs from disabled servers before attempting to merge them, thus preventing corruption while maintaining real-time availability from surviving servers
Solution Approach 2:
The encoded status information acts as an intermediary marker within the log stream that enables the standby to distinguish between valid and invalid log entries, allowing safe real-time merging without corruption risk
Data Source
AI summary
Detecting an operating condition in a multi-server data storage system. One or more of the data storage servers are designated with a status within the cluster. The designating may be performed by at least one of the data storage servers. The status is encoded into log data having records that report changes to the primary data storage performed by the cluster of data storage servers. The log data is transmitted from the primary data storage to the standby data storage system. The standby data storage system determines an operating condition of the cluster of data storage servers based, at least in part, on the status.