Cloud Metric Selection Using Probabilistic Frequency Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud-based computing environments face challenges in accurately determining which metrics to present to subscribers due to limited computational resources and potential inaccuracies in selecting or processing data, leading to suboptimal performance assessments.
Innovation Solution
A system implements a model to process metric data over time periods, comparing probabilistic data structures and hash values to identify frequent metrics, and transmits a subset of these metrics to client devices, optimizing resource usage and network efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a monitoring service collects and processes large amounts of metric data to accurately assess service performance, then measurement precision is improved, but device complexity and computational resource consumption increase
Solution Approach 1:
The patent segments the large volume of metric data into distinct time periods (e.g., 15-minute intervals) and processes each segment separately. The metric determination component divides incoming metric data streams into manageable time-based segments, applying probabilistic data structures to each segment independently. This segmentation reduces the computational burden on individual processing instances while maintaining overall assessment accuracy through aggregated results across multiple segments.
Solution Approach 2:
The patent extracts only the most relevant and frequent metrics from the complete dataset for transmission to client devices. By using probabilistic data structures to identify and extract the top-k most frequent metrics, the system removes unnecessary data elements, transmitting only those metrics that provide the most value for performance assessment. This extraction process reduces network bandwidth consumption and client-side processing requirements.
2Loss of information
If all collected metric data is transmitted to client devices, then information completeness is improved, but network bandwidth consumption and data transmission volume increase
Solution Approach 1:
The patent applies partial action by transmitting only a subset of the most significant metrics rather than the complete dataset. The system determines the top-k most frequent metrics based on probabilistic analysis and transmits only this partial set to client devices. This approach provides sufficient information for effective performance monitoring while dramatically reducing network bandwidth consumption compared to transmitting all collected metrics.
Solution Approach 2:
The patent changes the parameter of data representation by using probabilistic data structures (such as Bloom filters or Count-Min sketches) to summarize metric frequencies. Instead of transmitting raw metric values, the system transforms the data into compressed probabilistic representations that capture the essential frequency information. This parameter transformation enables efficient transmission while preserving the ability to determine the most frequent metrics at the client side.
3Device complexity
If the monitoring service processes and selects specific portions of data, then device complexity is reduced, but measurement precision and reliability of performance information decrease
Solution Approach 1:
The patent introduces probabilistic data structures as intermediary representations between the raw metric data and the final performance assessment. These intermediary structures (such as Count-Min sketches or Bloom filters) serve as a bridge that compresses the data while preserving the essential frequency information needed for accurate metric determination. The intermediary representation enables reduced processing complexity while maintaining reliability through mathematically guaranteed bounds on estimation accuracy.
Data Source
AI summary
Techniques for a service provider network to determine a subset of available metric data associated with an entity in a cloud computing environment are discussed herein. A system can receive metrics associated with a service and determine a subset of the metrics that represents service performance. In some examples, the system can identify a frequency of unique application activity over time in a dynamic data stream, and report metric data to a client device based on the frequency.