Distributed Link Tracing with Local Cache Splicing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data tracing methods in distributed systems require significant resource usage and can be inefficient, affecting the timeliness of data analysis due to the need for extensive data transmission and reliance on centralized log services.
Innovation Solution
A distributed link tracing data processing method that caches tracing data locally at each service node, constructs sub-requests with tracing identifications, and splices data from multiple nodes to generate full link tracing data for analysis, reducing the need for centralized uploads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If centralized log services are used to collect and analyze tracing data from all service nodes, then comprehensive tracing analysis can be achieved, but resource consumption increases and data transmission overhead increases
Solution Approach 1:
The patent divides the centralized tracing system into distributed segments where each service node independently caches and processes its own tracing data. The tracing data is segmented by service node and stored locally, eliminating the need for continuous centralized collection and reducing overall system resource consumption while maintaining comprehensive tracing capability.
Solution Approach 2:
The patent extracts the tracing data caching and initial processing function from the centralized log service and places it at each service node. By taking out the data collection overhead from the centralized system, the patent reduces resource consumption and transmission overhead while the centralized service retains its analytical function.
2Loss of information
If all tracing data is transmitted to centralized log services for analysis, then complete tracing information is available, but data transmission overhead increases and timeliness is affected
Solution Approach 1:
The patent performs preliminary caching of tracing data at each service node before centralized collection. By preparing and organizing tracing data locally in advance, the system eliminates the need for real-time data transmission during analysis, improving timeliness while ensuring complete tracing information is available when needed.
Solution Approach 2:
The patent extracts tracing data from the centralized transmission pipeline and stores it locally at each service node. This extraction eliminates continuous data transmission overhead and delays, allowing the centralized log service to access only the necessary data on-demand, thereby improving analysis timeliness without sacrificing information completeness.
3Productivity
If tracing data is cached locally at each service node, then data transmission overhead is reduced and timeliness is improved, but dependence on centralized log services decreases
Solution Approach 1:
The patent enables each service node to self-serve by caching and managing its own tracing data locally. Each node independently performs initial data processing and maintains its tracing information, reducing reliance on centralized services for data collection and improving overall tracing efficiency while adding minimal architectural complexity through standardized local caching mechanisms.
Data Source
AI summary
Embodiments of the present disclosure relate to a distributed link tracing data processing method, a device, and a storage medium. The method includes: receiving a service request, processing the service request, caching first tracing data generated in a local cache, constructing a sub-request of the service request, delivering the sub-request to a next-hop service node, receiving second tracing data generated by the next-hop service node through processing the sub-request of the service request and fed back through a response message, caching the second tracing data in the local cache, processing the response message, caching third tracing data generated in the local cache, obtaining the first tracing data, the second tracing data, and the third tracing data, splicing the above tracing data to obtain full link tracing data generated by processing the service request, and storing the full link tracing data in a target storage device.