Event Routing Across Heterogeneous Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern enterprise systems face challenges in processing and storing large volumes of machine-generated data, including failures in data transmission and retrieval inefficiencies due to the scalability limitations of single storage systems.
Innovation Solution
A system that routes events to multiple storage systems with individual rules for event storage, partitioning, and failure detection, using a flexible directory structure and health monitoring indicators to ensure efficient data processing and storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a single storage system is used to store all events, then the system is simple to manage, but it cannot scale effectively when querying large volumes of events
Solution Approach 1:
The patent divides the storage system into multiple independent storage systems, each capable of storing events independently. This segmentation allows the overall system to scale by adding more storage systems without increasing the complexity of individual components, directly resolving the contradiction between scalability and device complexity.
2Productivity
If events are transmitted to multiple storage systems, then storage capacity and retrieval efficiency improve, but the complexity of routing and managing events across systems increases
Solution Approach 1:
The patent applies partitioning rules beforehand to determine which storage system should receive each event based on event properties. This preliminary classification simplifies the routing process by pre-establishing clear criteria for event distribution, reducing the complexity of real-time routing decisions while maintaining high data processing efficiency.
3Reliability
If health monitoring is implemented across multiple storage systems, then failure detection capability improves, but the system requires additional monitoring infrastructure and complexity
Solution Approach 1:
The patent implements health indicators that continuously monitor the status of each storage system and provide feedback when failures or performance degradation occur. This feedback mechanism enables reliable failure detection across distributed storage systems without requiring complex monitoring infrastructure, as each storage system self-reports its health status through standardized indicators.
Data Source
AI summary
Multiple storage system event handling includes obtaining multiple events for storage on multiple storage systems. For each of the multiple events, field values from each event are extracted. The field values are matched to configurations of the storage systems to identify a subset of the storage system having a matching configuration. The event is transmitted to the subset. The multiple events are transmitted to heterogeneous subsets.