Parallelized Performance Data Collection in Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional storage systems face increasing overhead in performance data collection as the number of tracked objects grows, leading to reduced system performance due to resource utilization in monitoring and reporting, making it difficult to handle storage operations efficiently.
Innovation Solution
A technique that prioritizes the collection of high-priority data, such as performance metrics, while deprioritizing lower-priority data, allowing for asynchronous and parallel collection and storage, using separate memory structures and virtual address spaces to reduce latency and resource contention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If conventional storage systems monitor and report performance data from all tracked objects, then performance monitoring completeness is improved, but system overhead increases
Solution Approach 1:
The patent segments performance data collection into separate threads: a first thread collects high-priority performance data while a second thread collects low-priority data. This segmentation allows the system to monitor all required objects without requiring a single monolithic collection process, thereby maintaining monitoring completeness while distributing the overhead across specialized, concurrent operations.
Solution Approach 2:
The patent implements preliminary action by having the first thread collect high-priority performance data before the second thread collects low-priority data. This preliminary collection of critical metrics ensures that essential performance information is captured first, allowing the system to maintain monitoring completeness while optimizing the sequence of data collection to minimize overall overhead impact.
2Adaptability or versatility
If the number of tracked objects increases, then monitoring coverage is improved, but data collection overhead increases
Solution Approach 1:
The patent applies dynamics by implementing an asynchronous data collection mechanism where the first and second threads operate independently at different speeds and priorities. This dynamic approach allows the system to adapt to varying numbers of tracked objects by adjusting the collection frequency and priority of different data types, thereby maintaining comprehensive monitoring coverage while optimizing data collection efficiency according to system conditions.
Solution Approach 2:
The patent changes parameters by assigning different priorities and collection frequencies to high-priority and low-priority performance data. This parameter differentiation allows the system to maintain comprehensive monitoring of all tracked objects while adjusting the overhead characteristics of data collection based on the criticality of each data type, enabling scalable monitoring as the number of objects increases.
3Measurement precision
If performance data is collected at preconfigured intervals, then data accuracy is improved, but resource availability for storage operations decreases
Solution Approach 1:
The patent implements periodic action through scheduled performance data collection at preconfigured intervals while maintaining storage operation efficiency. By using separate threads for high-priority and low-priority data collection, the system can accurately capture performance metrics at appropriate intervals without blocking storage operations, as the data collection occurs in dedicated time slices that do not interfere with primary storage functions.
Solution Approach 2:
The patent uses an intermediary approach by introducing a buffer or memory structure that decouples the performance data collection process from storage operations. The first and second threads can collect data at their required intervals and store it in intermediate structures, allowing storage operations to proceed independently without being blocked by data collection activities, thereby maintaining both measurement precision and operational productivity.
Data Source
AI summary
A system and method for of prioritizing accumulation of time-dependent data is disclosed. In an embodiment, a plurality of data elements are identified to be retrieved. The data elements include a high-priority data element and a low-priority data element. A first data retrieval operation is performed to retrieve the high-priority data element, to store a copy of the high-priority data element in a memory structure, and to reserve a memory space in the memory structure for the low-priority data element based on the low-priority data element corresponding to the high-priority data element. In parallel with the first data retrieval operation, a second data retrieval operation is performed to analyze the memory structure to detect the reserved memory space, upon detecting the reserved memory space, to retrieve the low-priority data element, and to store a copy of the low-priority data element in the reserved memory space.