A distributed link tracking and analysis method, server, and global scheduler

Abstract

A distributed link tracking and analysis method, server, and global scheduler. During link tracking, the servers in the distributed cluster generate events during the process of processing user requests, and the events are sequenced according to the time when the events are generated They are stored in the files in the local event storage directory in sequence, and the next file is saved after one file reaches the set maximum size. During link analysis, the global scheduler in the distributed cluster sends a link query request to the server, and the server returns information about events related to the tracking identifier after searching locally; the global scheduler then based on the received event information Link analysis is performed on the user request. This application adopts the method of local storage and on-demand analysis, which solves the problems of high cost and scalability of data storage, analysis and network bandwidth in traditional methods.

Description

technical field [0001] The present invention relates to computer technology, and more particularly, to a distributed link tracking and analysis method, server and global scheduler. Background technique [0002] Tracing is a technique used by computer systems to track the running status of the system, and is mostly used for debugging, monitoring and tracing purposes. It is similar to logging, but with better performance, less interference to the target system, wider coverage, and is often used to record higher-frequency, lower-level events. [0003] In the tracing field of computer science, "event" means that when the processor runs to a specified code point, a series of operations are triggered because the conditions set in the code point in advance are met, which is identifiable. For example, a user sets a Boolean variable somewhere in the program code in advance. When the processor runs here and detects that the Boolean variable is true, it will trigger an event and perfo...

AI Technical Summary

Problems solved by technology

[0012] One problem with this approach is the high cost of data storage and network bandwidth

The original information data on each server must be transmitted to the central warehouse for storage through the network, so when there are many servers in the cluster and the original information data volume of a single server is also large, the cost of storage and network bandwidth is very high. High (such as daily TB level), causing great pressure on the central warehouse

These network transmissions will also have some additional overhead on the cluster production server (overhead)

In addition, scalability is also a problem. For example, the central warehouse must increase with the number of servers in the cluster and the overall QPS ((how many requests the server processes per second)), so the central warehouse may become a bottleneck point

After massive data is stored in the central warehouse, a large database of TB level is required to meet the efficient query of these data, which in turn introduces a large amount of CPU resources

Finally, people are generally more concerned about the data of abnormal requests that account for a small proportion of them, and usually only check the link status of these abnormal requests, which makes this solution less cost-effective

Images