Storage System Availability via Forbidden Operation Blocking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In storage systems with multiple compute nodes, kernel panics can lead to secondary failures when a compute node attempts to retry operations that caused the initial failure, disrupting system availability.
Innovation Solution
A storage system and method that manage forbidden storage operations by identifying and publishing problematic operations in a data structure, allowing compute nodes to avoid executing them, and reallocating load based on forbidden operations to maintain system availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a compute node retries execution of operations that caused initial failure, then system productivity is improved through operation completion, but system reliability deteriorates due to secondary kernel panics
Solution Approach 1:
The system applies preliminary anti-action by maintaining a forbidden operations data structure that pre-identifies and blocks operations known to cause failures. Before a compute node attempts to execute a problematic operation, the system checks the forbidden operations data structure and prevents execution, thereby counteracting the potential secondary kernel panic before it can occur.
Solution Approach 2:
The forbidden operations data structure serves as an intermediary between the compute nodes and the storage operations. This intermediary component filters out dangerous operations by checking against the data structure, allowing safe operations to proceed while blocking those that have previously caused kernel panics, thus resolving the contradiction between productivity and reliability.
2Reliability
If compute nodes avoid executing forbidden operations, then system reliability is improved through prevention of secondary failures, but productivity deteriorates due to operation blocking
Solution Approach 1:
The system applies dynamics by making the forbidden operations data structure dynamic rather than static. When operations are successfully executed or conditions change, the system updates the data structure to remove restrictions. This allows the system to adapt its blocking behavior, maintaining high reliability by blocking dangerous operations while preserving productivity by allowing previously forbidden operations when safe to do so.
Solution Approach 2:
The system changes parameters by dynamically updating the forbidden operations data structure based on system state. When a compute node successfully completes operations or when conditions indicate safety, the system modifies the data structure parameters (adding or removing forbidden operations), thereby adjusting the balance between reliability and productivity based on current system conditions.
Data Source
AI summary
A method for maintaining an availability of a storage system, the method may include obtaining, by a control module of the storage system, problem related information generated by one or more compute nodes of the storage system, the problem related information is indicative of one or more problems associated with an execution of one or more storage operations; determining, by the control module and based on the problem related information, whether to forbid an execution of a storage operation of the one of more storage operations; and updating, by the control module, and based on the determining, a forbidden storage operation data structure that is accessible to the compute nodes of the storage system.