Distributed Trace Collectors for Lower-Bandwidth Monitoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional tracing procedures in multi-network applications face challenges due to high bandwidth requirements and data transfer costs, leading to increased network latencies and a single point of failure in data management.
Innovation Solution
Distribute collectors across a multi-network environment to minimize data transfer by sending trace events to proximate collectors, selectively retain relevant data, and use probabilistic sets of trace IDs to reduce data volume.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If trace events are collected across multiple networks using conventional procedures, then complete trace data is obtained for analysis, but network bandwidth is excessively consumed and data transfer costs increase
Solution Approach 1:
The system segments trace event collection by deploying multiple distributed collectors across different networks. Each collector handles trace events locally within its network segment, preventing the need to transfer all trace events across network boundaries. This segmentation reduces overall network bandwidth consumption while maintaining complete trace data collection through the distributed architecture.
Solution Approach 2:
The patent implements local quality by allowing each distributed collector to retain and process trace events locally according to network-specific requirements. Collectors can selectively retain trace events based on local criteria, ensuring that each network segment maintains the necessary trace data for local analysis without requiring constant cross-network data transfer.
2Measurement precision
If all sampled trace events are transferred across networks for centralized analysis, then comprehensive performance monitoring is achieved, but data transfer time and network latency increase
Solution Approach 1:
The system performs preliminary action by having distributed collectors sample and pre-process trace events locally before any cross-network transfer. Collectors can immediately analyze and filter trace events at the source, performing initial performance monitoring tasks without waiting for centralized collection. This preliminary local processing reduces the volume of data requiring network transfer while maintaining monitoring comprehensiveness.
3Device complexity
If a single collector is used for centralized trace event collection, then data management is simplified, but the system creates a single point of failure
Solution Approach 1:
The patent applies segmentation by dividing the single centralized collector into multiple distributed collectors across different networks. Each collector independently handles trace events in its local network segment, eliminating the single point of failure. The distributed architecture maintains data management simplicity through standardized collector interfaces while significantly improving system availability through redundancy.
Solution Approach 2:
The system merges the functionality of multiple distributed collectors into a coordinated network where each collector operates semi-autonomously but contributes to overall system objectives. This merging approach maintains the reliability benefits of distribution while preserving simplified data management through unified collection standards and protocols.
4Loss of information
If full trace event data is retained and transferred, then complete diagnostic information is available, but computing resource usage and storage costs increase
Solution Approach 1:
The patent implements extraction by having distributed collectors selectively extract and retain only the most relevant trace events based on local criteria such as error conditions, performance thresholds, or business logic. Instead of transferring and storing all trace events, collectors extract only the diagnostically valuable subset, reducing computing resource usage and storage costs while maintaining sufficient diagnostic information completeness.
Data Source
AI summary
Systems and methods for application performance management across one or more networks are disclosed. A system includes a plurality of geographically distributed computing devices executing one or more applications. A plurality of collectors are distributed across the one or more networks, each collector being positioned proximate a respective computing device. The collectors may sample, by each of the plurality of collectors, a plurality of trace events received from the applications executing on the computing devices proximate the collector. The collectors may retain in memory sampled trace events that meet a configurable attribute. The collectors may use probabilistic sets and tail-based sampling to determine root events and policies for identifying relevant traces. Benefits can be achieved in bandwidth savings, network cost and cyber security.