Survivor Process for Smooth Storage Cluster Restart

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional storage cluster systems face challenges in smoothly restarting crashed applications, leading to loss of connectivity and context data due to automatic closure of open handles by the kernel, resulting in disconnection of clients and loss of application context.

Innovation Solution

Implementing a survivor process that manages open handles associated with the crashed storage cluster application, allowing for a smooth restart by copying and maintaining handle references in shared data structures, thereby keeping handles open and restoring context data from shared memory objects.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If the kernel automatically closes open handles when a storage cluster application crashes, then system stability is improved, but connectivity is lost and application context data is lost

Engineering Contradiction:

Improvesystem stabilityVSAvoidapplication context data

Core Design Contradiction:

ReliabilityVSLoss of information

Solution Approach 1:

The patent applies preliminary action by creating a survivor process that is pre-configured to take over handle management before the crash occurs. The survivor process is started alongside the main storage cluster application and is prepared to assume control of open handles immediately upon crash detection, preventing the kernel from automatically closing them and preserving application context data.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The survivor process acts as an intermediary between the crashed storage cluster application and the kernel. It intercepts the handle closure action that would normally be triggered by the kernel upon crash detection, and instead maintains the handles open while transferring control from the crashed application to the survivor process, thereby preserving connectivity and context data.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If a storage cluster application crashes and is restarted conventionally, then the application can resume operation, but clients are disconnected and the restart process is visible to the kernel

Engineering Contradiction:

Improveapplication resumptionVSAvoidclient connectivity

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The survivor process serves as an intermediary that maintains client connectivity during the application restart. It continues to handle client requests and maintain open handles throughout the restart process, making the crash recovery invisible to both the kernel and clients, thereby ensuring continuous operation without disconnection.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent ensures continuity of useful action by having the survivor process maintain handle references and continue processing client requests uninterrupted during the application restart. The survivor process preserves the operational state and seamlessly transitions control back to the restarted application, ensuring continuous service without interruption to clients.

Inventive Principle:

Principle #20Continuity of useful action

3Device complexity

If open handles are closed by the kernel upon crash detection, then system resource management is improved, but application state is lost and reconnection is required

Engineering Contradiction:

Improveresource managementVSAvoidapplication state

Core Design Contradiction:

Device complexityVSLoss of information

Solution Approach 1:

The survivor process creates and maintains copies of handle references that point to the same kernel resources. These duplicate handle references allow the survivor process to manage and preserve application state independently of the crashed application, preventing the kernel from closing the original handles and enabling state recovery without reconnection.

Inventive Principle:

Principle #26Copying

Data Source

PatentUS9983937B1Smooth restart of storage clusters in a storage system

Publication Date: 2018.05.29 EMC IP HLDG CO LLC

AI summary

Described embodiments provide methods and systems for smoothly restarting a crashed storage cluster application associated with a storage cluster of a storage system. A survivor process associated with each storage cluster application is started. If a crash of at least one of the storage cluster applications is detected, the storage system performs a smooth restart of the at least one crashed storage cluster application. A smooth restart of the at least one crashed storage cluster application is performed according to a new storage cluster application that is configured to replace the crashed storage cluster application by managing open handles associated with the crashed storage cluster application.