Correlating Microservice Logs and Traces for Distributed App Debugging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Troubleshooting performance issues in distributed applications is challenging due to the large volumes of log messages generated by each microservice, requiring significant time and resources to identify root causes, which can lead to prolonged downtime and reputational damage.
Innovation Solution
A system that collects, stores, and correlates metrics, log messages, and traces from microservices using an open telemetry collector and forwarder, forming a common identifier for correlated data, and displaying interactive user interfaces to visualize the request flow and identify failures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If distributed applications use microservices architecture, then adaptability and ease of deployment are improved, but device complexity and difficulty of troubleshooting increase
Solution Approach 1:
The patent segments the distributed application into independent microservices, each generating its own log messages. This segmentation enables independent deployment and adaptation of individual services while maintaining overall system functionality. The system collects and correlates log messages from multiple segmented sources to manage the complexity arising from this division.
Solution Approach 2:
The patent introduces an intermediary system that collects, aggregates, and correlates log messages from multiple microservices. This intermediary layer manages the complexity of distributed logging by providing a unified interface for troubleshooting while allowing microservices to remain independent and adaptable.
2Measurement precision
If distributed applications generate separate log files for each microservice, then measurement precision and troubleshooting detail are improved, but loss of time and productivity deteriorate
Solution Approach 1:
The patent merges log messages from multiple microservices into a unified correlated view. By combining logs with common identifiers while preserving the precision of individual service logging, the system eliminates the time-consuming process of manually correlating separate log files, thus maintaining measurement precision while reducing troubleshooting time.
Solution Approach 2:
The system performs preliminary correlation of log messages by attaching common identifiers to logs from different microservices as they are generated. This preliminary action prepares the data for efficient troubleshooting, so when issues arise, the correlated information is already available without requiring time-consuming manual correlation during incident response.
3Difficulty of detecting and measuring
If distributed applications process large volumes of log messages, then measurement precision and detection capability are improved, but use of energy and productivity worsen
Solution Approach 1:
The patent extracts and correlates only the relevant log messages that share common identifiers across microservices, rather than processing all log messages uniformly. This extraction approach maintains high detection capability for troubleshooting while reducing the overall processing load and energy consumption by focusing computational resources on correlated error traces.
Data Source
AI summary
The present disclosure relates to a system for monitoring execution of a distributed application running in a data center. The system includes a forwarder that runs in a tenant environment of the data center. The forwarder collects metrics, log messages, and spans output from microservices of the distributed application. The system includes an ingestion engine that receives the metrics, the log messages, and the spans output from the forwarder. The ingestion engine forms a trace from the spans, and stores the metrics, the log messages, and the trace with a common identifier in a database. The includes a user interface (“UT”) that displays the metrics, the log messages, and the trace in response to a user selection to the metrics, the log messages, and the trace via the UI.