Proxy Service Integrating Cloud and On-Premise Heat Map Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Integrating performance counter data from cloud-based nodes into on-premise clusters is challenging due to higher latency and unreliability in cloud communication, making it difficult to maintain a unified heat map that accurately represents the state of both on-premise and cloud nodes.
Innovation Solution
Deploying a cloud-based computing service that configures cloud nodes to write performance counters to a cloud storage location, maps these counters into a compatible format, and integrates them with on-premise counters using a proxy service to render a unified heat map, minimizing latency impacts and optimizing data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If cloud nodes are incorporated into the hybrid environment, then the computing service capability is improved, but the heat map integration becomes difficult due to higher latency and unreliability in cloud communication
Solution Approach 1:
A gateway service is introduced as an intermediary component that bridges cloud-based nodes and on-premise heat map services. The gateway collects performance counters from cloud nodes, formats them appropriately, and transmits them to the on-premise heat map service, thereby enabling reliable integration despite network latency and unreliability issues.
2Measurement precision
If performance counters are collected from cloud nodes, then the monitoring coverage is improved, but the data transmission reliability deteriorates due to higher-latency and lower-speed cloud communication
Solution Approach 1:
The gateway service performs preliminary actions by collecting and buffering performance counter data from cloud nodes before transmission to the on-premise heat map service. This buffering mechanism allows the system to accumulate data during periods of network instability and transmit it when conditions improve, ensuring data completeness despite transmission reliability issues.
3Productivity
If cloud based nodes are deployed, then the computing resource scalability is improved, but the network communication stability worsens due to higher-latency and unreliability
Solution Approach 1:
The gateway service acts as a stable intermediary that absorbs network communication instability. It maintains reliable data collection from cloud nodes and ensures consistent data delivery to the on-premise heat map service, isolating the stable monitoring infrastructure from cloud network fluctuations.
4Measurement precision
If performance counters from multiple sources are integrated, then the monitoring comprehensiveness is improved, but the data format compatibility becomes challenging
Solution Approach 1:
The gateway service transforms performance counter data by changing its format parameters. It converts performance counters from various sources (cloud nodes, on-premise nodes) into a unified format that the on-premise heat map service can process, handling differences in data structures, naming conventions, and measurement units.
Data Source
Figure 1
Figure 2
AI summary
The present invention extends to methods, systems, and computer program products for integrating external and cluster heat map data. Embodiments of the invention include a proxy service that manages (e.g., asynchronous) communication with cloud nodes. The proxy service simulates packets to on-premise services to simplify the integration with an existing heat map infrastructure. The proxy maintains a cache of performance counter mappings and timestamps on the on-premise head node to minimize the impact of latency into heat map infrastructure. In addition, data transfer is minimized by mapping a fixed set of resource based performance counters into a variable set of performance counters compatible with the on premise heat map infrastructure.