Distributed Transaction Trace Correlation via Server ID Routing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Centralized correlation servers in monitoring systems for distributed transactions face bottlenecks due to high processing and memory requirements, leading to excessive network communication and bandwidth usage among correlation servers when correlating transaction trace data from independent agents.

Innovation Solution

A system where agents randomly select a correlation server for each transaction and attach the server ID to communication data, allowing all transaction trace data to be forwarded to a single server, eliminating the need for inter-server communication and using intermediate nodes for routing and backup server management.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If a centralized correlation server is used to process all transaction trace data, then correlation accuracy is improved, but processing capacity and memory requirements become unmanageable at scale

Engineering Contradiction:

Improvecorrelation accuracyVSAvoidprocessing capacity

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent divides the centralized correlation server into multiple distributed correlation servers that process trace data in parallel. Each server handles a subset of trace data fragments, distributing the processing load while maintaining correlation accuracy through coordinated processing across the cluster.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent transitions from a single-dimensional centralized processing model to a multi-dimensional distributed cluster architecture. Trace data flows through multiple dimensions including data reception, correlation processing, and result aggregation across several servers simultaneously, enabling scalable processing capacity.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Productivity

If multiple correlation servers are deployed to distribute processing load, then processing capacity is improved, but network communication and bandwidth usage increase exponentially

Engineering Contradiction:

Improveprocessing capacityVSAvoidnetwork bandwidth usage

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The patent performs preliminary routing of trace data fragments to the appropriate correlation server based on transaction identifiers before distribution. This pre-routing mechanism ensures that each fragment is sent directly to the correct server without requiring inter-server communication or redistribution, eliminating exponential network traffic growth.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces a data reception module as an intermediary that receives trace data fragments and routes them to appropriate correlation servers. This mediator coordinates data flow across the cluster, ensuring efficient distribution without requiring direct communication between correlation servers.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Adaptability or versatility

If trace data fragments are distributed across multiple correlation servers, then processing scalability is improved, but inter-server communication is required to complete transaction correlation

Engineering Contradiction:

ImprovescalabilityVSAvoidsystem complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent extracts the communication coordination function from the correlation servers themselves and places it in a dedicated data reception module. This separation allows correlation servers to focus solely on processing their assigned fragments without the complexity of inter-server communication protocols.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent enables each correlation server to independently process its assigned trace data fragments and generate local correlation results without requiring service calls to other servers. Each server is self-sufficient for its designated processing tasks, reducing system complexity.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS10924326B2Method and system for clustered real-time correlation of trace data fragments describing distributed transaction executions

Publication Date: 2021.02.16 DYNATRACE LLC

AI summary

A system and method is disclosed that provides fail-save, scalable and clustered correlation of transaction trace data. The transaction trace data is provided by a set of agents injected into application processes and processed by a set of clustered correlation servers. Each agent monitors parts of transaction executions performed by the application process into which it is injected. The agents generate transaction trace data fragments describing the transaction execution part and report those transaction trace data fragments to the correlation cluster. Routing of transaction trace data from agents to correlation servers is performed in a way that transaction data fragments describing a specific transaction are received by one correlation server regardless of the agent sending it. Intermediate nodes may be used to server as proxy between agents and the correlation server cluster to further improve the availability and failure tolerance of the monitoring system.