Dynamic API Tracing for Distributed System Message Flows
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Obtaining precise request traces in distributed systems is challenging due to insufficient information in traditional log data, with system call logs being too low-layered and not useful for debugging, and existing dynamic instrumentation tools having high overhead or requiring source code access.
Innovation Solution
Implementing dynamic borderline tracing at the API level using a black-box approach with library interposition technology, such as LD_PRELOAD, to capture and analyze API function calls and return values, allowing for dynamic instrumentation without modifying source code or incurring high overhead, and reconstructing end-to-end transaction flows.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional log data and system call logs are used for tracing, then the tracing can be implemented without modifying source code, but the tracing precision and usefulness for debugging are insufficient
Solution Approach 1:
The patent introduces an intermediary instrumentation layer that intercepts API calls at the boundary between components. This mediator captures detailed information about inter-module communications without requiring modifications to the actual application code, thus achieving high tracing precision while maintaining ease of implementation through a separate instrumentation framework.
Solution Approach 2:
The tracing system is segmented into independent instrumentation points at API boundaries rather than requiring comprehensive source code modification. This segmentation allows precise tracking of message flows between modules without modifying the core application logic, resolving the contradiction between precision and ease of implementation.
2Measurement precision
If dynamic instrumentation tools are used to capture API function calls, then tracing precision improves, but the overhead increases significantly
Solution Approach 1:
Instead of instrumenting the entire application globally, the patent applies instrumentation locally only at API boundary points where inter-module communication occurs. This localized instrumentation captures necessary tracing information without the excessive overhead of comprehensive dynamic instrumentation tools that monitor all system calls and operations.
Solution Approach 2:
The patent extracts and focuses instrumentation only on the essential API call interfaces between modules, separating the tracing function from the core application logic. This extraction allows precise tracking of message flows while minimizing overhead by instrumenting only the necessary boundary points rather than the entire execution path.
3Measurement precision
If existing dynamic instrumentation tools are used, then tracing capability is provided, but source code access is required which reduces ease of operation
Solution Approach 1:
The patent employs an intermediary instrumentation framework that operates at API boundaries without requiring access to or modification of the application's source code. This mediator captures tracing information by intercepting calls to standardized interfaces, enabling precise tracing capability while maintaining ease of operation through a black-box approach that works with compiled binaries.
Solution Approach 2:
The instrumentation system creates a copy of the API interface layer with instrumentation logic, allowing tracing without modifying the original application code. This copying approach enables tracing capability while preserving ease of operation, as the instrumentation can be applied to existing binaries without requiring source code access or compilation.
4Measurement precision
If comprehensive instrumentation is applied to track all execution flows, then complete transaction flow tracking is achieved, but the overhead and complexity increase
Solution Approach 1:
The instrumentation system segments the complex task of complete transaction flow tracking into discrete, manageable pieces at API boundary points. By focusing on these segmentation points where modules communicate, the system achieves complete transaction flow tracking without the complexity of instrumenting every internal operation and execution path within each module.
Solution Approach 2:
The patent implements a universal instrumentation framework that handles multiple tracing functions through a unified approach at API boundaries. This multi-functional instrumentation captures call parameters, return values, and timing information simultaneously, achieving complete transaction flow tracking without proportionally increasing complexity through a single cohesive framework.
Data Source
AI summary
The present invention enables capturing API level calls using a combination of dynamic instrumentation and library overriding. The invention allows event level tracing of API function calls and returns, and is able to generate an execution trace. The instrumentation is lightweight and relies on dynamic library/shared library linking mechanisms in most operating systems. Hence we need no source code modification or binary injection. The tool can be used to capture parameter values, and return values, which can be used to correlate traces across API function calls to generate transaction flow logic.