Self-service Metrics Publication Service for Data Center Monitoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Managing and provisioning physical computing resources in large-scale data centers has become increasingly complex due to the scale and scope of operations, and existing technologies lack efficient solutions for users to define, publish, and manage metrics related to computer systems and applications across diverse networks.
Innovation Solution
A provider network offers metrics publication services that allow users to define and publish metrics by making data sets available, retrieving data, performing computations, and publishing results to specified destinations, implemented as lightweight software services within the network to reduce latency and scale efficiently for multiple users.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional manual methods are used for provisioning and managing physical computing resources in large-scale data centers, then users can maintain control and visibility over resources, but the complexity of management increases significantly and efficiency decreases
Solution Approach 1:
The patent implements self-service capabilities through automated resource provisioning systems that allow users to independently define, publish, and manage their own metrics without manual intervention. The system automatically collects data from diverse sources, processes metrics according to user-defined configurations, and distributes them to subscribed destinations, eliminating the need for complex manual provisioning while maintaining user control.
Solution Approach 2:
The patent segments the complex resource management task into distinct modular components: metric definition, data collection from multiple sources, metric computation, and distribution to various destinations. Each component operates independently with well-defined interfaces, allowing the system to handle large-scale data center operations without overwhelming complexity.
2Loss of information
If comprehensive metrics are collected and published across diverse data sources in large-scale networks, then monitoring capabilities and decision-making quality improve, but system latency increases and scalability becomes challenging
Solution Approach 1:
The patent implements preliminary action by pre-configuring metric definitions, data source connections, computation logic, and destination subscriptions before actual metric publication. Users define their metric requirements in advance, and the system prepares the data collection and processing pipelines beforehand, enabling rapid metric publication when needed without real-time configuration delays.
Solution Approach 2:
The patent implements dynamic adaptation by allowing the metrics publication system to automatically adjust to varying data source availability, metric computation requirements, and destination subscription changes. The system dynamically optimizes data collection frequencies and processing priorities based on current system conditions, maintaining low latency while ensuring comprehensive metrics coverage.
3Ease of operation
If a centralized metrics publication system is implemented to manage diverse data sources, then centralized control and consistency are achieved, but the system cannot efficiently scale to serve multiple users with diverse needs
Solution Approach 1:
The patent implements universality by designing a metrics publication system that serves multiple users with diverse requirements through a single unified platform. The system universally handles various data sources (logs, databases, APIs, sensors), multiple metric types, and diverse destination formats, allowing any user to define and publish their specific metrics using the same core infrastructure without requiring user-specific system instances.
Data Source
AI summary
Techniques for self-service publication of metrics are described. A provider network includes metrics publications services. Aspects of the metrics publication services allow customers to generate customer-defined metrics from customer-defined queries of data sets. Such data sets may contain data related to software and/or hardware performance. Other aspects of the metrics publication services allow customers to make data sets available to metrics publications services, to identify available data sets, and to identify available metrics.