Storage Cluster Session Multiplexing via Channel Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current storage system clusters face challenges in maintaining high throughput due to long-running or lost requests, which can block sessions and impact overall performance, especially in scenarios with multiple connections.
Innovation Solution
The implementation of a multi-layered network protocol that integrates a session infrastructure and application operation set into a session layer, allowing for multiplexing channels over multiple connections, enabling asynchronous request processing and priority-based request routing, thereby preventing long-running or lost requests from stalling the session.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If requests are processed sequentially over a single connection, then request processing simplicity is maintained, but throughput is reduced due to blocking by long-running or lost requests
Solution Approach 1:
The session is segmented into multiple independent channels, each capable of processing requests concurrently. This segmentation allows the session to maintain high throughput by distributing requests across multiple channels, preventing any single long-running or lost request from blocking the entire session. Each channel operates independently with its own connection, resolving the contradiction between maintaining simplicity and achieving high throughput.
2Productivity
If multiple connections are used for parallel request processing, then throughput is improved, but complexity of managing multiple connections increases
Solution Approach 1:
The channel acts as an intermediary layer between the session and multiple connections. The session layer manages channels, which in turn manage connections, providing a hierarchical abstraction that simplifies connection management. This intermediary structure allows the session to leverage multiple connections for parallel processing while maintaining manageable complexity through the channel abstraction layer.
3Ease of operation
If a single connection is used for a session, then connection management is simplified, but the session is blocked by long-running or lost requests
Solution Approach 1:
The session is divided into multiple channels, each with its own connection, allowing parallel request processing. This segmentation enables the session to maintain ease of operation through unified session-level management while achieving high throughput by distributing requests across multiple independent channels, preventing blocking by long-running or lost requests in any single channel.
4Ease of operation
If channels are bound to specific connections, then request routing is simplified, but flexibility to handle connection failures is reduced
Solution Approach 1:
The channel-connection binding is made dynamic rather than static. Channels can be dynamically associated with different connections based on availability and load conditions. This dynamic binding maintains routing simplicity through automated management while improving reliability by allowing the system to adapt to connection failures and redistribute requests across available connections.
Data Source
AI summary
A system and method multiplexes channels over multiple connections between one or more nodes. Each node includes a cluster fabric interface module adapted to implement a novel network protocol that enables intra-cluster communication among the nodes. The network protocol is a multi-layered protocol that integrates a session infrastructure and an application operation set into a session layer. The network protocol is illustratively a request/response protocol wherein a node (requester) receiving a data access request from a client redirects that request to another node (responder) that services the request and, upon completion, returns a response.