Metrics Aggregation Using Rotating Compute Entity Managers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for metrics aggregation in large-scale storage systems are inefficient, particularly when numerous modules are involved, as they require substantial compute resources and suffer from lock wait times due to the use of mutual exclusion mechanisms for updating shared counters.
Innovation Solution
A storage system architecture where metrics are aggregated across multiple compute entities using a shared metric store, with each compute entity taking turns as the manager to update metrics, reducing communication overhead and balancing workload through predefined cycles and shared memory access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a central management entity collects measurements from various modules, then metrics can be aggregated, but the process becomes lengthy and consumes substantial compute resources
Solution Approach 1:
The patent divides the metrics aggregation task among multiple compute entities rather than concentrating it in a single central management entity. Each compute entity is responsible for collecting measurements from its own modules and updating shared counters, distributing the workload across the system and reducing the compute resource burden on any single entity.
Solution Approach 2:
Compute entities perform self-service by autonomously collecting measurements from their respective modules and updating shared counters without requiring continuous intervention from a central management entity. This self-service approach reduces the computational overhead and resource consumption associated with centralized collection.
2Reliability
If mutual exclusion mechanism is used for updating shared counter, then data consistency is maintained, but lock wait times increase and modules waste time
Solution Approach 1:
The patent implements periodic action by having compute entities update shared counters in a coordinated sequence rather than continuously. Each compute entity updates its designated counter at specific intervals, allowing other entities to proceed without waiting for lock releases. This periodic updating maintains data consistency while minimizing lock wait times.
Solution Approach 2:
The patent segments the shared counter updates into distinct, non-overlapping operations. Each compute entity is assigned specific counters to update, preventing simultaneous access conflicts. This segmentation allows multiple entities to operate in parallel without requiring mutual exclusion locks, thereby reducing lock wait times while maintaining consistency.
3Adaptability or versatility
If multitude of modules are involved in metrics collection, then comprehensive monitoring is achieved, but system complexity increases
Solution Approach 1:
The patent implements universality by designing compute entities that can perform multiple functions: collecting measurements from various modules, updating shared counters, and participating in metrics aggregation. This multi-functional design allows the same compute entities to handle diverse monitoring tasks without increasing overall system complexity.
Solution Approach 2:
The patent segments the monitoring function across multiple compute entities, each responsible for specific modules or metric types. This segmentation allows comprehensive monitoring of numerous modules while keeping individual entity responsibilities manageable and simplifying the overall system architecture through modular organization.
Data Source
AI summary
A method for monitoring a storage system, the method may include (a) generating a compute entity (CE) storage metric by each CE of a group of CEs to provide multiple CE storage metrics, wherein the multiple CE metrics are related to a monitoring period; and (b) calculating, during a calculation period, a group metric based on the multiple CE storage metrices; wherein the calculating includes performing multiple calculations iterations, wherein each calculation iteration includes (a) selecting an updating CE that belongs to the group of CEs and was not previously selected during the calculation period, (b) accessing, by the updating CE, a shared data structure that stores the group storage metric, and (c) updating the group storage metric using the CE storage metric of the updating CE.