Common Storage Communication Channel Using Distributed Event Topics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face challenges in efficiently managing inter-node, inter-application, and intra-application communication due to the use of multiple communication channels and protocols, leading to overhead and potential conflicts, especially in high-availability environments.
Innovation Solution
A common communication channel is implemented using a distributed event store, such as a Kafka server, with an intelligent log driver to manage topics, enabling applications to subscribe and publish messages, and arbitrate between nodes, simplifying communication and resolving conflicts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple communication channels and protocols are used for inter-node, inter-application, and intra-application communication, then communication flexibility and adaptability are improved, but system complexity and overhead increase
Solution Approach 1:
The patent merges multiple communication channels and protocols into a single unified communication channel. The storage system uses one common channel for all types of communication (inter-node, inter-application, intra-application) instead of maintaining separate channels for each communication type, thereby reducing system complexity while preserving communication flexibility through topic-based message routing.
Solution Approach 2:
The unified communication channel is designed to handle multiple communication functions simultaneously. By implementing a universal channel that can carry different types of messages (events, commands, notifications) for different communication scenarios, the system achieves multi-functionality without requiring separate dedicated channels for each purpose.
2Adaptability or versatility
If multiple communication channels are used to manage different communication types, then communication specificity is improved, but overhead and potential conflicts increase
Solution Approach 1:
The patent segments the communication channel into different topics, allowing specific types of messages to be routed to appropriate destinations. This segmentation enables communication specificity through topic-based organization while avoiding the overhead of maintaining separate channels, as all topics share a single underlying communication infrastructure.
3Reliability
If multiple communication channels are used for high-availability environments, then communication reliability is improved, but conflict resolution complexity increases
Solution Approach 1:
The patent introduces an intermediary mechanism (the unified communication channel with topic-based routing) that mediates between different communication needs. This intermediary structure enables reliable communication in high-availability environments by providing a centralized coordination point that manages message routing and conflict resolution without requiring complex direct coordination between multiple channels.
Data Source
AI summary
A common communication channel, implemented as a distributed event store, is provided to implement all inter-node, inter-application, and intra-application communication within a storage system. The distributed event store may be implemented as a Kafka server/cluster that hosts topics that applications subscribe to as producers, consumers, or both. An Intelligent Log Driver controls creation of the topics on the Kafka server and also is a subscriber and producer on select topics. The Intelligent Log Driver also interacts with the storage system API to implement configuration changes of the storage system based on commands received on the topics and based on application activity observed on the topics. By providing a common communication channel for all inter-node, inter-application, and intra-application communication within a storage system, it is possible to simplify communication between applications, such as between a storage system management application, software defined network attached storage applications, and the underlying storage system.