Topic-Based Routing for APM Compute Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Application Performance Management (APM) software faces scalability limitations due to the need for numerous servers to handle raw metric data, which restricts the capacity of host machines and inefficiencies in data processing and analysis.
Innovation Solution
Assigning topics to compute nodes based on labeled performance metrics, allowing each node to listen for and analyze metrics associated with its assigned topics, and using a Workload Mapper to distribute calculations efficiently, ensuring that necessary metric data is routed locally for processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If calculators are located so that all collectors have a well-known destination for their raw metric data, then intermediate results are guaranteed to be produced local to their consuming calculators, but this places an absolute limit on the scalability of the APM software
Solution Approach 1:
The system segments the monolithic calculator location model into distributed compute nodes, where each node is assigned specific topics and calculations. This allows the system to scale by adding more segmented nodes while maintaining local result production through topic-based routing.
Solution Approach 2:
The patent introduces a new dimension of topic-based assignment alongside the traditional collector-calculator relationship. Instead of solely relying on physical location, calculations are assigned to nodes based on topic subscriptions, enabling scalability without sacrificing locality of intermediate results.
2Quantity of substance
If many servers are provisioned to receive raw metric data from agents, then the system can handle larger deployments, but the capacity of host machines running the calculations is restricted
Solution Approach 1:
Compute nodes are designed to perform multiple functions: they can act as collectors receiving raw metric data, perform calculations on assigned topics, and route results to consuming calculators. This multi-functionality eliminates the need for separate dedicated collector servers, increasing overall processing capacity.
Solution Approach 2:
The patent merges the roles of collectors and calculators into unified compute nodes. By combining these functions, the system reduces the total number of servers needed while maintaining or increasing processing capacity through efficient resource utilization.
3Adaptability or versatility
If calculators consume data from any or all collectors, then data availability is improved, but data transfer efficiency decreases
Solution Approach 1:
Topic-based routing acts as an intermediary mechanism between collectors and calculators. Instead of direct point-to-point data transfer, the topic subscription model routes data efficiently to only those compute nodes that need it, maintaining data availability while reducing unnecessary data transfer.
Data Source
AI summary
A method, system and computer program product are disclosed for routing performance data to compute nodes. According to one aspect of the present disclosure each of a plurality of compute nodes are assigned a topic. Each topic may be associated with a set of calculations. Labeled performance metrics for an application are received. Each performance metric is labeled with a context under which the performance metric was collected. A topic is associated with each of the performance metrics based on the labeled context. Each respective node listens for a topic assigned to it in order to access the performance metrics associated with the assigned topic. Each respective node analyzes the performance metrics associated with the topic assigned to it.