Telemetry Data Priority Segmentation for Cluster QoS
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current telemetry systems face challenges in prioritizing and managing different types of data streams, leading to data unavailability and inefficiencies, particularly when critical data like alerts and metering information are not separated and transported effectively, resulting in potential data loss and unavailability issues.
Innovation Solution
A system and method that categorize telemetry data into high, medium, and low priority levels based on data type and tags, storing and transporting them separately with distinct quality-of-service (QoS) allocations, ensuring higher priority data receives better bandwidth and processing, thereby reducing latency and ensuring critical data is processed efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all telemetry data is treated equally without prioritization, then the system structure remains simple, but critical data may be lost or delayed due to resource contention
Solution Approach 1:
The patent segments telemetry data into different priority levels (high, medium, low) based on data type and tags. Critical data such as alerts and metering information are classified as high priority, while less critical data is classified as low priority. This segmentation allows the system to allocate resources differently for different data streams, ensuring reliable transmission of critical data while maintaining manageable system complexity through structured classification.
2Reliability
If critical data is prioritized with higher bandwidth allocation, then data transmission reliability improves, but system complexity increases due to separate storage and transport mechanisms
Solution Approach 1:
The patent implements separate storage and transport queues for different priority levels of telemetry data. High priority data is stored in dedicated storage regions and transported through prioritized queues, ensuring reliable and timely delivery. This segmentation of storage and transport mechanisms provides the necessary reliability for critical data while maintaining clear organizational structure that manages system complexity.
Solution Approach 2:
The patent applies local quality by providing differentiated quality of service (QoS) to different data streams based on their priority classification. High priority data receives higher bandwidth allocation, lower latency guarantees, and more reliable transport mechanisms, while low priority data receives standard service levels. This localized quality enhancement ensures critical data transmission reliability without unnecessarily complicating the handling of non-critical data.
3Speed
If telemetry data is collected and processed without prioritization, then the collection process remains simple, but latency increases for critical data during high traffic periods
Solution Approach 1:
The patent performs preliminary classification of telemetry data into priority levels at the point of collection, before the data enters the processing and transport pipeline. By determining the priority level based on data type and tags upfront, the system can immediately route data through appropriate processing channels, reducing latency for critical data during high traffic periods. This preliminary action maintains relatively simple collection processes while enabling fast processing of urgent data.
Solution Approach 2:
The patent implements dynamic resource allocation and processing based on data priority levels. During high traffic periods, the system dynamically adjusts processing speed, bandwidth allocation, and queue prioritization to ensure critical data is processed and transmitted with minimal latency. This dynamic approach enables fast processing of important data while managing overall system load, with the added benefit of automated priority-based management that controls complexity.
Data Source
AI summary
Various embodiments disclosed herein are related to a non-transitory computer readable storage medium. In some embodiments, the medium includes instructions stored thereon that, when executed by a processor, cause the processor to determine, in a cluster of host machines, a priority level of telemetry data collected in the cluster, at least based on a data type of the telemetry data and a tag and store the telemetry data in a storage in the cluster. In some embodiments, a quality-of-service (QoS) is associated with the priority level. In some embodiments, the storage is associated with the priority level. In some embodiments, the medium includes the instructions stored thereon that, when executed by the processor, cause the processor to send the telemetry data from the storage to a server in accordance with the QoS, wherein the server is separate from the cluster.