A distributed link tracking, analysis method and server, global scheduler

Abstract

A distributed link tracking, analysis method and server, global scheduler, in the process of processing user request, when producing detail event, the meta event for detail event search is produced, the meta event contains tracking identification and event production time information, and the data quantity is less than the data quantity of corresponding detail event; The server saves the detail event and the meta event produced separately in local. The global scheduler in the distributed cluster sends a link query request to the server in the distributed cluster, receives the event information related to the tracking identification returned by the server after local search; and based on the received event information, the user request is analyzed by link. The application adopts local storage, on-demand analysis, removes the central warehouse and large database, solves the problems of high cost of data storage, analysis and network bandwidth, and scalability.

Description

TECHNICAL FIELD

[0001] The present application relates to computer technology, more particularly, to a distributed link tracking, analysis method and server, global scheduler. BACKGROUND

[0002] Tracing is a technique for computer systems to track system running status, mostly used for debugging, monitoring and tracing purposes. It is similar to logging, but has better performance, lower interference to target system, wider coverage, and is often used to record higher frequency and lower layer events.

[0003] In the field of computer science Tracing, an "event" (hereinafter referred to as a detail event) refers to a series of operations triggered when a processor runs to a specified code point, due to meeting a condition previously set at the code point. For example, a user sets a Boolean variable at a certain point in the program code, and when the processor runs to this point, the event is triggered if the Boolean variable is true, and a series of previously arranged operations are performed. These operations are usually recording relevant information of the event in a log, such as event name, event occurrence time, thread ID, and other data carried by the event.

[0004] "Code instrumentation" (instrumentation) is a term mentioned in the "event" that refers to a user implanting a piece of logic (set by the user) at a certain point in the code of an application program (selected by the user), which usually records various information contained in the event. It has dynamic and static parts. Dynamic recording implantation refers to not needing to add tracing code in the program code in advance, but adding a special instruction set at the user-specified code location (such as the head and tail of a function) by the operating system kernel or privileged process during the running of the target program, to achieve the purpose of running the user-set code logic. Static implantation is to modify the source code of the application program to add code logic for tracing. The implanted program is called an instrumented binary.

[0005] Some events are not independent and have relationships with each other. For example, a user application sends a data packet using the TCP protocol (event A), the TCP data segment is transmitted through the local network card (event B), and TCP detects that the data segment is lost and then retransmits the data segment (event C). In the entire retransmission segment process, if events (A, B, C) containing timestamps and carrying useful information are recorded at the above-mentioned key points, and these causally related events are selected from the total set of recorded events and arranged in order (such as time order), a quantitative explanation of the delay of the user's data packet sending can be given. The process of selecting events related to each other and sorting them is called "event correlation".

[0006] Contemporary Internet services are usually implemented with large-scale distributed clusters, and a user's request can be dispatched to one or more clusters for processing. The servers in the clusters can assume different roles in the processing of the request and eventually work together to complete the request and return the result to the user. In order to better understand the entire service / system, locate faults, and optimize performance, developers / administrators usually employ a tracing technique to record information about the operations on the servers, and then aggregate and analyze the recorded information. For example, people are interested in the time and resources spent by a single user request at each processing stage in its life cycle. The process of generating and recording relevant information when a server processes a user request is called "link tracking", and the process of analyzing the information data is called "link analysis".

[0007] There are two solutions in the industry for correlating events - black-box tracing and white-box tracing. The advantage of the black-box solution is that it is lightweight and does not modify the application code, and is completely transparent to the application side. It either uses a statistical method to obtain information from communication message packets - thus being inaccurate, or accurately traces according to the inherent causal relationship between messages based on the interaction messages collected between functional modules. Therefore, its disadvantage is that it can be inaccurate to some extent, and has greater resource consumption when inferring correlations. Tracing systems that adopt the black-box solution include vPath, BoarderPartrol, PreciseTracer, etc. The white-box solution requires code instrumentation, which is its disadvantage. The advantage is that it is relatively simple to infer correlations, and the event correlation is accurate. Tracing systems that adopt the white-box solution include X-Trace, PinPoint, Magpie, Dapper, and Eagle Eye, etc.

[0008] The trace ID is a global identifier used in a simple event correlation method adopted by most white-box solutions. The tracing system generates a trace ID for each request sent by a user, and the trace ID accompanies the request in all processing stages and is recorded with event information when each event is triggered. In this way, all events related to a request can be linked by the trace ID.

[0009] User requests can be divided into type I user requests and type II user requests. Type II user requests only involve a small number of machines in the cluster. For example, a write request for distributed storage is usually written to three servers; and a read request for distributed storage only needs to read one server. User requests that do not meet this characteristic are type I requests, such as search requests. A search request can involve almost all servers in the cluster (if most servers have documents containing the search keyword).

[0010] Distributed link tracking and analysis systems need to record information of all operations completed by the system in a user request service process. For example, Figure 1 A distributed request service process involving five servers is shown. The servers involved include a front-end server A, middle-tier servers B and C, and back-end servers D and E. When a user initiates a request, the request first arrives at the front-end server A, which sends two remote procedure call (RPC) messages to servers B and C; server B responds immediately, but server C needs to interact with back-end servers D and E before returning a response to server A, and finally server A responds to the initial user request. For such a request service process, the implementation of distributed link tracking and analysis in the related art is that server A generates a trace ID for the user request, and records event information including the trace ID and a timestamp at each operation on each server. With the trace ID, events related to the user request can be associated in subsequent data analysis.

[0011] The implementation of distributed link tracking and analysis systems in the related art, such as Google's Dapper and Taobao's Eagle Eye, all adopt a three-stage process. The first stage is that each server records event information related to the request and the code path point when the request service process arrives at a pre-buried code path point, which is generally written to a local file; the second stage is that a daemon or log collection program pulls the information data from each server to a central repository through network transmission; and the third stage is to take data from the central repository for subsequent analysis, query, association and calculation. This stage also introduces a large database between the central repository and analysis and calculation, such as the link tracking systems of Jingdong, Google and Taobao, which use mysql, HBase, Infobright, HiStore and other databases in the third stage, respectively.

[0012] A problem caused by this method is the high cost of data storage and network bandwidth. The raw information data on each server is transmitted to the central repository for storage through the network, so when the number of servers in the cluster is large and the amount of raw information data on a single server is also large, the cost of storage and network bandwidth is very high (such as TB level per day), which puts a lot of pressure on the central repository. These network transmissions also have some overhead for the cluster production servers. In addition, scalability is also a problem, such as the central repository must increase with the number of servers in the cluster and the overall QPS (the number of requests processed per second by the server), so the central repository can become a bottleneck point. After mass data storage to the central repository, a large database of TB level is needed to meet the efficient query of these data, which introduces a large amount of CPU resources. Finally, people are generally more concerned about the data of the small percentage of abnormal requests, and usually only check the link conditions of these abnormal requests, which makes this solution less cost-effective. SUMMARY

[0013] Therefore, the present application provides the following solutions:

[0014] A distributed link tracking method, comprising:

[0015] In the process of processing user requests, a server in a distributed cluster generates a meta event for searching a detail event when generating the detail event, the meta event contains information of a tracking identifier and a time of event generation, and the data volume of the meta event is less than that of the corresponding detail event;

[0016] The server separately saves the generated detail event and meta event locally.

[0017] A server in a distributed cluster, comprising a link tracking module, the link tracking module comprising:

[0018] An event generation unit is configured to generate a meta event for searching a detail event when generating the detail event in the process of processing user requests; the meta event contains information of a tracking identifier and a time of event generation, and the data volume of the meta event is less than that of the corresponding detail event;

[0019] An event storage unit is configured to separately save the generated detail event and meta event locally.

[0020] A server in a distributed cluster, comprising a processor and a memory, wherein:

[0021] The memory is configured to save program codes;

[0022] The processor is configured to read the program codes and perform the following link tracking processing:

[0023] In the process of processing the user request, a meta event for searching the detail event is generated when the detail event is generated, the meta event contains tracking identification and event generation time information, and the data amount is less than that of the corresponding detail event;

[0024] The generated detail event and meta event are separately saved locally.

[0025] The above-mentioned distributed link tracking method and server generate a meta event for searching the detail event when the detail event is generated, and save the event locally, so that in the link analysis process, the detail event can be quickly searched locally through the meta event, without saving the event information to the data warehouse, and without establishing a large amount of data.

[0026] Therefore, the present application further provides the following solutions:

[0027] A distributed link analysis method, comprising:

[0028] When a global scheduler in a distributed cluster needs to perform link analysis on a user request, the global scheduler sends a link query request to a server in the distributed cluster, carrying a tracking identification of the user request;

[0029] The global scheduler receives event information related to the tracking identification returned by the server after local search;

[0030] The global scheduler performs link analysis on the user request based on the received event information related to the tracking identification.

[0031] A global scheduler in a distributed cluster, comprising:

[0032] A link query module is configured to: when link analysis on a user request is needed, send a link query request to a server in the distributed cluster, carrying a tracking identification of the user request; and receive event information related to the tracking identification returned by the server after local search;

[0033] A link analysis module is configured to: perform link analysis on the user request based on the received event information related to the tracking identification.

[0034] A global scheduler in a distributed cluster, comprising a processor and a memory, wherein:

[0035] The memory is configured to save program code;

[0036] The processor is configured to read the program code and perform the following processing:

[0037] When link analysis of a user request is needed, a link query request is sent to a server in the distributed cluster, carrying a tracking identifier of the user request;

[0038] Event information related to the tracking identifier returned by the server after local search is received;

[0039] Based on the received event information related to the tracking identifier, link analysis is performed on the user request.

[0040] The above-mentioned distributed link analysis method and global scheduler adopt a local storage and on-demand analysis manner, eliminate the central repository and large database, and solve the problems of high cost of data storage, analysis and network bandwidth and scalability in the traditional link tracking analysis system.

[0041] Therefore, the application further provides the following solutions.

[0042] A distributed link analysis method, comprising:

[0043] A server in a distributed cluster receives a link query request sent by a global scheduler, and the link query request carries a tracking identifier of a user request;

[0044] The server locally searches meta-events related to the tracking identifier, determines a first search time window for detail events according to event generation time information contained in the searched meta-events, and locally searches detail events related to the tracking identifier according to the first search time window.

[0045] The server returns information of the searched detail events to the global scheduler.

[0046] A server in a distributed cluster, comprising a link analysis module, wherein the link analysis module comprises:

[0047] A first query interface unit is configured to receive a link query request sent by a global scheduler, and the link query request carries a tracking identifier of a user request; and return information of searched detail events to the global scheduler;

[0048] A meta-event searching unit is configured to locally search meta-events related to the tracking identifier;

[0049] A detail event searching unit is configured to determine a first search time window for detail events according to event generation time information contained in the meta-events searched by the meta-event searching unit, and locally search detail events related to the tracking identifier according to the first search time window.

[0050] A server in a distributed cluster, comprising a processor and a memory, wherein:

[0051] The memory is configured to save program code.

[0052] The processor is configured to read the program code and perform the following processing:

[0053] Receiving a link query request sent by a global scheduler, the link query request carrying a tracking identifier requested by a user;

[0054] Locally searching meta events related to the tracking identifier, determining a first search time window of detail events according to event generation time information contained in the searched meta events, and locally searching detail events related to the tracking identifier according to the first search time window;

[0055] Returning information of the searched detail events to the global scheduler.

[0056] The link analysis method and the server described above can greatly reduce the search range of detail events and accelerate the link analysis process by searching meta events to obtain a search time window of detail events. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 is an example diagram of a service process of a distributed request in a distributed cluster;

[0058] Figure 2 is a flowchart of a distributed link tracking method of an embodiment of the present application;

[0059] Figure 3 is a unit structure diagram of a link tracking module in a server of an embodiment of the present application;

[0060] Figure 4 is a flowchart of a distributed link analysis method of a global scheduler side of an embodiment of the present application;

[0061] Figure 5 is a module diagram of a global scheduler of an embodiment of the present application;

[0062] Figure 6 is a flowchart of a distributed link analysis method of a server side of an embodiment of the present application;

[0063] Figure 7 is a unit structure diagram of a link analysis module in a server of an embodiment of the present application;

[0064] Figure 8 is a whole schematic diagram of a link analysis process of an I-type request of example two of the present application;

[0065] Figure 9 is a schematic diagram of two-stage processing of locally searching events by a server of example two of the present application;

[0066] Figure 10 is a schematic diagram of a first optimization scheme of the first stage processing of Example Two of the present application;

[0067] Figure 11 is a schematic diagram of a second optimization scheme of the first stage processing of Example Two of the present application;

[0068] Figure 12 is a schematic diagram of the second stage processing of Example Two of the present application;

[0069] Figure 13 is a schematic diagram of the overall link analysis process for Type II requests of Example Three of the present application;

[0070] Figure 14 is a schematic diagram of the two-stage processing and downstream tracking processing of Example Three of the present application;

[0071] Figure 15 is a schematic diagram of the downstream tracking processing of Example Three of the present application. DETAILED DESCRIPTION

[0072] In order to make the objects, technical solutions and advantages of the present application clearer and more comprehensible, the embodiments of the present application will be described in detail below with reference to the accompanying drawings. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other in any manner without conflict.

[0073] In this document, for the convenience of expression, when referring to the content contained in event information and the saving of event information, the event information of the detailed event record is also referred to as "detailed event" and the event information of the meta event record is referred to as "meta event".

[0074] In this document, the server that generates a tracking identifier for a user request in the process of the user request service is referred to as the source server of the user request.

[0075] The embodiments of the present application adopt the local storage and on-demand calculation mode to solve the problems of high cost of data storage, analysis and network bandwidth and scalability in the traditional distributed link tracking analysis system.

[0076] Embodiment One

[0077] The present embodiment relates to a white-box distributed link tracking analysis system based on tracking identifiers, which includes a global scheduler and servers in a distributed cluster. The events generated by the servers in the process of processing user requests are saved locally in the servers. When performing link analysis, the global scheduler collects the corresponding event information from the servers to perform link analysis, and no data warehouse and large database for link tracking analysis are established.

[0078] The embodiment relates to a distributed link tracking method and a corresponding server, and the distributed link tracking mainly involves generation and storage of events and is the basis of distributed link analysis.

[0079] The distributed link tracking method of the embodiment, as shown in Figure 2 , comprises the following steps.

[0080] In step 110, a server in the distributed cluster generates a meta event for searching a detail event when generating the detail event in the process of processing a user request, the meta event contains information of a tracking identifier and a time of event generation, and the data amount of the meta event is less than that of the corresponding detail event.

[0081] In order to distinguish the two types of events, the present application refers to the "event" in the existing tracking technology as a "detail event", and refers to the event generated along with the detail event as a "meta event". The meta event is used for searching the detail event, and does not need to contain information of specific operations, so that the data amount of the meta event is much less than that of the corresponding detail event, and the searching speed is faster than that of the detail event.

[0082] In the present application, a server generates a meta event corresponding to a detail event along with the generation of the detail event, so that the meta event is generated along with the detail event. It should be noted that "at the same time" herein does not require that the time of generation of the two events is absolutely equal, and a small difference is allowed. For example, through program code design, the difference between the actual generation time of the detail event and the meta event generated along with the detail event is always less than 10 ms, and only time values of 100 ms or more are recorded when recording the time of event generation. When recording the time of event generation of the detail event and the meta event, the small difference between the actual generation time of the two events can be ignored and recorded as the same value. When performing link analysis, the time of event generation of the meta event can be used to determine the search time window of the corresponding detail event.

[0083] In order to avoid searching events in the whole storage directory, it is necessary for the server to determine a search time window to narrow the search range when performing link analysis on a user request. The start time of the search time window can be set as the time when the tracking identifier of the user request is generated. In this embodiment, the source server encodes the information of the time when the tracking identifier is generated into the generated tracking identifier, and the tracking identifier will be sent to the server in the link analysis. The server can obtain the start time of the search time window from the tracking identifier very conveniently. Of course, in other embodiments, the tracking identifier can not contain the information of the time when the tracking identifier is generated. For example, the global scheduler can first query the source server to obtain the information of the time when the tracking identifier is generated, and then carry it in the link query request sent to the server. That is, the information of the time when the tracking identifier is generated can be contained in the tracking identifier carried by the link query request, or can be contained in other information units carried by the link query request.

[0084] The length of the search time window used in link analysis (length = end time - start time) can be set as the maximum life cycle of the user request, such as 10 minutes, and the calling duration of all user requests will not exceed this length. In order to further narrow the search range, the length of the search time window can be set as the calling duration of the user request to be analyzed. In this embodiment, the source server generates an application programming interface (API: Application Programming Interface) event when the user request is completed and saves it. The API event contains the tracking identifier and the calling duration information of the user request. The API event can be a meta event generated by a special accompanying detail event, but is not limited thereto.

[0085] In particular, the source server of this embodiment saves the timeout API event in the special directory of the timeout API event. The timeout API event refers to the API event whose calling duration exceeds the corresponding timeout duration. The timeout duration corresponding to different user requests can be different, but is less than the maximum life cycle set by the system for the user request. Generally, the number of timeout API events is at least one order of magnitude less than that of API events, and link analysis is usually initiated for abnormal requests such as timeout requests. Therefore, searching in the special directory of the timeout API event can greatly speed up the search speed of the timeout request. When the timeout API event cannot be searched, it means that the calling duration of the user request does not time out, and the timeout duration corresponding to the user request can be returned to the global scheduler as the calling duration of the user request.

[0086] In another embodiment, the API events (including the timeout API event) are all stored in an API event storage directory, and the source server can directly search the API events in the API event storage directory to obtain the invocation duration of the user request. In yet another embodiment, the API events other than the timeout API event are stored in the API event storage directory, and the timeout API event is stored in a timeout API event specific directory (which can be a subdirectory of the API event storage directory or an independent directory). When the timeout API event cannot be searched, the source server searches the API events in the API event storage directory to obtain the invocation duration of the user request. The API event storage directory can be a subdirectory of the meta event storage directory or independent of the meta event storage directory.

[0087] By storing the API events, in subsequent link analysis, the global scheduler can query the source server for the invocation duration of the user request stored by the source server, and use the invocation duration to accurately determine the search time window of the meta event.

[0088] In order to realize the query of the global scheduler to the source server, the source server can encode the address information of the server into the tracking identifier when generating the tracking identifier. The address information of the server can be direct address information such as the IP address of the server, or indirect address information such as the server ID and the like which can be used to find the IP address of the server. However, in other embodiments, the source server can also report its address information to the global scheduler through other ways such as messages.

[0089] In the embodiment, if the user request is a distributed read request or a distributed write request, the processing of the user request only involves a small number of servers in the distributed cluster, and the subsequent link analysis can only query the links of the small number of servers. In order to identify the small number of servers, the server generates a communication event when performing network communication processing of sending the user request to the next hop server, and the communication event contains the tracking identifier and the address information of the next hop server to which the user request is sent. In link analysis, the global scheduler can only initiate a query to the source server (such as the front-end server), and the source server initiates a query to the next hop server processing the user request according to the address information of the next hop server saved in the communication event. Each server on the user request processing path processes in this way, and the query of the links along the user request processing path can be completed.

[0090] In the embodiment, the meta event further contains the identification information of the processor generating the meta event, that is, only contains the tracking identifier, the event generation time, and the identification of the processor generating the meta event. The meta event can also only contain the tracking identifier and the event generation time.

[0091] The server separately saves the generated detail events and meta events in the local.

[0092] In this embodiment, the server sequentially saves the detail events and meta events in files in respective directories according to the time sequence of event generation, and switches to a next file for saving after a file reaches a set maximum size. Because of the sequential storage manner, the time period in which the event generation time of the meta events (or detail events) saved in the file is located can be determined according to the file creation time and / or last modification time.

[0093] In this embodiment, when saving the detail events in the local, the server saves the detail events generated by the same processor in a file group corresponding to the processor, and different processors correspond to different file groups. In this way, when performing link analysis, the server can determine the file group in which the corresponding detail events are located according to the identification information of the processor saved in the meta events (i.e., the file group corresponding to the processor with the identification information), and further narrows down the search range of the detail events to the range of these file groups, which can greatly reduce the search range and accelerate the search speed.

[0094] This embodiment also provides a server in a distributed cluster, including a link tracking module, as shown in Figure 3 The link tracking module includes:

[0095] The event generation unit 10 is configured to: in the process of processing a user request, generate a meta event for searching a detail event when generating the detail event; the meta event contains tracking identification and information of event generation time, and the data amount is less than that of the corresponding detail event;

[0096] The event storage unit 20 is configured to: separately save the generated detail events and meta events in the local.

[0097] In this embodiment, the link tracking module can further include an identification generation unit configured to: generate tracking identification for a user request, encode information of tracking identification generation time into the tracking identification, or encode information of tracking identification generation time and address information of the server into the tracking identification.

[0098] In this embodiment, the event storage unit separately saves the detail events and meta events in the local, and can include: sequentially saving the detail events and meta events in files in respective directories according to the time sequence of event generation, and switching to a next file for saving after a file reaches a set maximum size.

[0099] In the embodiment, the meta-event generated by the event generation unit can contain identification information of the processor generating the meta-event; and the event storage unit can save the detail events locally, which can include saving the detail events generated by the same processor in a file group corresponding to the processor, and saving the detail events generated by different processors in different file groups.

[0100] In the embodiment, the event generation unit can also be configured to generate an application programming interface (API) event when the user request is completed, the API event containing the tracking identifier and call duration information of the user request; and the event storage unit can also be configured to save the API event.

[0101] In the embodiment, the event storage unit can save the API event locally, which can include saving the timeout API event in a special directory of the timeout API event, the timeout API event being an API event whose call duration exceeds a timeout duration.

[0102] In the embodiment, the event generation unit can also be configured to generate a communication event when performing network communication processing of sending the user request to a next-hop server, the communication event containing the tracking identifier and address information of the next-hop server to which the user request is sent; and the user request is a user request for distributed reading or distributed writing.

[0103] The functions performed by the units of the link tracking module in the embodiment server can also be referred to the specific description in the link tracking method.

[0104] The embodiment also provides a server in a distributed cluster, including a processor and a memory, wherein,

[0105] The memory is configured to save program code.

[0106] The processor is configured to read the program code and perform the following link tracking processing:

[0107] In the process of processing the user request, a meta-event for searching the detail event is generated when the detail event is generated, the meta-event containing tracking identifier and information of the event generation time, and the data quantity of the meta-event being less than that of the corresponding detail event;

[0108] The generated detail event and meta-event are saved separately locally.

[0109] In the embodiment, the link tracking processing performed by the processor can include all the processing in the link tracking method of the embodiment, which will not be repeated here.

[0110] The distributed link tracking method and server of the embodiment generates meta events for detail event search when generating detail events, and saves the events locally, so that in the link analysis process, the detail events can be quickly searched locally through the meta events, without saving the event information to the data warehouse, and without establishing a large data.

[0111] Embodiment two

[0112] In embodiment one, the link tracking method of the server in the distributed cluster has been described, and the server saves the generated events locally, without uploading to the data warehouse. In the distributed link analysis method of the embodiment, the global scheduler sends a link query request to the server, and after obtaining the relevant event information, the global scheduler analyzes and summarizes the event information.

[0113] The distributed link analysis method of the embodiment is used for a global scheduler, as shown in Figure 4 The distributed link analysis method comprises the following steps:

[0114] In step 210, when the global scheduler in the distributed cluster needs to perform link analysis on a user request, the global scheduler sends a link query request to the server in the distributed cluster, and carries the tracking identifier of the user request.

[0115] In the embodiment, the global scheduler can trigger the link analysis on the user request according to the user instruction, or automatically trigger the link analysis according to the configured condition, and the embodiment is not limited in this regard.

[0116] As described above, when the server searches for the events related to a user request, the server needs to first determine a search time window, the start time of the search time window can be set as the generation time of the tracking identifier of the user request, and the time length can be set as the maximum lifetime information of the user request or the calling time length of the user request. The information of the generation time of the tracking identifier can be included in the tracking identifier. If the time length is set as the maximum lifetime of the user request, the maximum lifetime information of the user request can be carried in the link query request. Of course, the maximum lifetime information of the user request can also be configured as the server configuration information, without being carried in the link query request. If the time length is set as the calling time length of the user request, the global scheduler can query the calling time length information of the user request from the source server, and carry the calling time length information in the link query request.

[0117] In the embodiment, the global scheduler adopts different link query methods when the type of the user request is different: if the user request is a search request, the global scheduler sends the link query request to all servers in the distributed cluster; if the user request is a distributed read request or a distributed write request, the global scheduler sends the link query request to the source server of the user request.

[0118] The distributed cluster can provide different services, such as some distributed clusters providing read-write services and some distributed clusters providing search services. Thus, the distributed cluster itself can default a link query mode. For a distributed cluster that can provide multiple types of services simultaneously, the global scheduler can adopt a corresponding link query mode according to a user request type or adopt a corresponding link query mode according to an indication processing mode flag, for example, using 1 bit as a flag in a tracking identifier, where the flag is "0" to indicate that a link query request should be sent to all servers in the distributed cluster and is "1" to indicate that only a link query request needs to be sent to a source server in the distributed cluster.

[0119] In step 220, the global scheduler receives event information related to the tracking identifier returned by the server after local search;

[0120] In step 230, the global scheduler performs link analysis on the user request based on the received event information related to the tracking identifier.

[0121] In the distributed tracking analysis method of the embodiment, each user request has a unique tracking identifier, and thus the event information related to the tracking identifier is event information generated in a user request processing process. The embodiment mainly focuses on an event information collection process, and does not make any limitation on an analysis and processing method after receiving the event information.

[0122] The embodiment also provides a global scheduler in a distributed cluster, as shown in Figure 5 The global scheduler comprises:

[0123] The link query module 30 is configured to: when link analysis on a user request is needed, send a link query request to a server in the distributed cluster, where the link query request carries a tracking identifier of the user request; and receive event information related to the tracking identifier returned by the server after local search.

[0124] The link analysis module 40 is configured to: perform link analysis on the user request based on the received event information related to the tracking identifier.

[0125] In the embodiment, the link query request sent by the link query module can also carry maximum lifetime information of the user request, and the tracking identifier contains information about a time when the tracking identifier is generated; or the link query request sent by the link query module can also carry calling duration information of the user request obtained by the global scheduler from a source server, and the tracking identifier contains information about a time when the tracking identifier is generated.

[0126] In the embodiment,

[0127] The user request is a search request, and the link query module sends a link query request to the servers in the distributed cluster, including: sending the link query request to all servers in the distributed cluster; or

[0128] The user request is a distributed read request or a distributed write request, and the link query module sends a link query request to the servers in the distributed cluster, including: sending the link query request to the source server of the user request in the distributed cluster.

[0129] The embodiment also provides a global scheduler in a distributed cluster, including a processor and a memory, wherein:

[0130] The memory is configured to save program code.

[0131] The processor is configured to read the program code and perform the following link analysis processing:

[0132] When link analysis of a user request is needed, a link query request is sent to the servers in the distributed cluster, carrying a tracking identifier of the user request;

[0133] Event information related to the tracking identifier returned by the servers after local search is received;

[0134] Based on the received event information related to the tracking identifier, link analysis is performed on the user request.

[0135] In the embodiment, the link analysis processing performed by the processor in the global scheduler can include all the processes in the link analysis method of the embodiment, which will not be repeated here.

[0136] The distributed link analysis method and the global scheduler in the embodiment adopt the local storage and on-demand analysis mode, remove the central repository and large database, and solve the problems of high cost of data storage, analysis and network bandwidth and scalability in the traditional link tracking analysis system.

[0137] Embodiment three

[0138] The embodiment relates to a link analysis method performed by a server in a distributed cluster. After receiving a link query request sent by a global scheduler, the server searches related events according to a tracking identifier, and returns the searched event information to the global scheduler. In order to speed up the search process, the search range can be narrowed from two aspects of a search time window and a file group, and the search time window and the file group are determined according to event generation time information in a meta event and identifier information of a processor.

[0139] The distributed link analysis method in the embodiment is as follows: Figure 6As shown, comprising:

[0140] In step 310, the server in the distributed cluster receives a link query request sent by the global scheduler, the link query request carrying a trace identifier of a user request;

[0141] In step 320, the server locally searches meta events related to the trace identifier, determines a first search time window of detail events according to event generation time information contained in the searched meta events, and locally searches detail events related to the trace identifier according to the first search time window.

[0142] In this embodiment, the server locally searches meta events related to the trace identifier, including: searching, under a storage directory of meta events, a second target file in which meta events fall into a second search time window in terms of event generation time, and then searching, in the second target file, meta events related to the trace identifier; wherein the second search time window has a start time of a generation time of the trace identifier and a time length of a calling time length of the user request or a maximum life cycle of the user request, and the information of the generation time of the trace identifier is carried in the link query request, and can be contained in a trace identifier carried in the link query request or other information units different from the trace identifier carried in the link query request.

[0143] Wherein,

[0144] The server searches the second target file, including: determining a time period in which event generation times of meta events saved in a file fall, according to a creation time and / or a last modification time of the file under the meta event storage directory; and searching a file in which the time period falls into the second search time window, and the searched file is the second target file; wherein the meta events are saved in the file in a sequence of event generation time, and a next file is used to save the meta events after a set maximum size of the file is reached. In an example, the time period in which event generation times of meta events saved in a file fall is determined according to a last modification time of the file, for example, a last modification time of a file is t4, and a last modification time of a previous file of the file is t3, and the time period in which event generation times of the meta events saved in the file fall can be determined as [t3, t4]; for another example, a creation time of a file is t5, and a last modification time of the file is t6, and the time period in which event generation times of the meta events saved in the file fall can be determined as [t5, t6]; for another example, a creation time of a file is t7, and a creation time of a next file of the file is t8, and the time period in which event generation times of the meta events saved in the file fall can be determined as [t7, t8].

[0145] Wherein,

[0146] If the invocation duration of the user request is taken as the duration of the second search time window, the global dispatcher can query the source server for information of the invocation duration of the user request and carry the information in the link query request sent by the global dispatcher.

[0147] In the query process of the invocation duration, the processing performed by the source server includes:

[0148] The source server receiving the invocation duration query request of the global dispatcher, the invocation duration query request carrying the trace identifier;

[0149] In this embodiment, the source server locally searches for a timeout API event related to the trace identifier, and obtains the information of the invocation duration from the searched timeout API event and returns the information to the global dispatcher. The timeout API event refers to an API event whose invocation duration of the user request exceeds a corresponding timeout duration, and the API event contains information of the trace identifier of the user request and the invocation duration.

[0150] In particular, if the source server does not locally search for a timeout API event related to the trace identifier, the source server can continue to search in a storage directory of other API events, or return information of a timeout duration corresponding to the user request to the global dispatcher as the information of the invocation duration. Taking the timeout duration corresponding to the user request as the invocation duration can save the time for continuing to search for the API event, and the invocation duration of the user request returned to the global dispatcher is also less than the maximum lifetime of the user request, so that the first search time window is small.

[0151] In another embodiment, the source server saves the timeout API event and other API events in the same API event storage directory, and after receiving the invocation duration query request, searches for an API event related to the trace identifier in the API event storage directory, and obtains the information of the invocation duration from the searched API event and returns the information to the global dispatcher.

[0152] In particular, the search for the API event or the timeout API event can be accelerated in the following manner: searching for a third target file in which the API event or the timeout API event falls within a third search time window, and then searching for the API event or the timeout API event related to the trace identifier in the third target file, wherein the start time of the third search time window is the trace identifier generation time contained in the trace identifier, and the time length is the maximum life cycle requested by the user. The search for the third target file is similar to the search for the first target file, and is also determined according to the creation time and / or the last modification time of the file under the API event storage directory or the timeout API event storage directory, to determine the time period in which the event generation time of the API event saved in the file falls. Then, the file in which the time period falls within the third search time window is searched, and the searched file is the third target file.

[0153] In the embodiment, the server determines a first search time window for the detail event, and searches for the detail event related to the trace identifier locally according to the first search time window, including:

[0154] The server determines [t1, t2] as the first search time window, wherein t1 and t2 are the earliest time and the latest time in the event generation time contained in the searched meta event, respectively.

[0155] The server searches for a first target file in which the detail event falls within the first search time window under the detail event storage directory, and then searches for the detail event related to the trace identifier in the first target file.

[0156] Wherein,

[0157] In link tracking, the server saves the detail events in the files in the order of the event generation time, and switches to the next file when a file reaches the set maximum size. Thus, the server can determine the time period in which the event generation time of the detail event saved in the file falls according to the creation time and / or the last modification time of the file under the detail event storage directory, and then search for the file in which the time period falls within the first search time window, and the searched file is the first target file.

[0158] In the embodiment, the search is also greatly accelerated by dividing different file groups for different processors. In link tracking, the meta event contains the identification information of the processor generating the meta event, and the detail events generated by the same processor are saved in the file group corresponding to the processor, and different processors correspond to different file groups. In link analysis, the server limits the search range to the file group corresponding to the processor in the following manner:

[0159] The server determines a file group corresponding to the processor according to the identification information of the processor contained in the searched meta event, searches the first target file in the corresponding file group, and then searches the detail event related to the tracking identification in the first target file.

[0160] The server determines a file group corresponding to the processor according to the identification information of the processor contained in the searched meta event, searches the first target file in the corresponding file group, and then searches the detail event related to the tracking identification in the first target file.

[0161] Some servers have tens or hundreds of processors, and the processor recorded in the meta event may only involve a few of them, so the search range can be reduced by 10 times or less. The improvement of search speed is very significant.

[0162] The server searches the detail event related to the tracking identification in the file group corresponding to the processor according to the identification information of the processor contained in the searched meta event.

[0163] When the server generates a detail event and generates a meta event at the same time, the processor may be switched (although the possibility is very small), that is, the processor generating the detail event is different from the processor generating the corresponding meta event, and at this time, the identification information of the processor contained in the meta event and the identification information of the processor generating the corresponding detail event are different, so that the detail event will be missed when searching in the file group. In order to avoid this situation, after the server searches the detail event related to the tracking identification, it further includes: counting the number M1 of the searched detail event related to the tracking identification, if M1 is less than the number M2 of the meta event related to the tracking identification searched locally, searching the detail event related to the tracking identification in other file groups under the detail event storage directory.

[0164] In step 330, the server returns the information of the searched detail event to the global scheduler.

[0165] If the link query request is a link query request for a search request, the global scheduler can directly send a link query request to all servers in the distributed cluster, and the server can return the query result to the global scheduler after local query.

[0166] If the link query request is a link query request for a user's distributed read request or distributed write request, the server needs to perform the following processing:

[0167] The server queries the local storage of the communication event related to the tracking identification, and the communication event contains the address information of the next hop server to which the user request is sent.

[0168] The server sends a link query request to the next hop server according to address information in the communication event, carrying the tracking identifier of the user request, if the server finds the communication event;

[0169] The server receives information of the detail event related to the tracking identifier returned by the next hop server and returns the information to the global scheduler.

[0170] The embodiment also provides a server in a distributed cluster, including a link analysis module, as shown in Figure 7 The link analysis module includes:

[0171] The first query interface unit 50 is configured to receive a link query request sent by the global scheduler, the link query request carrying a tracking identifier of a user request, and return information of a searched detail event to the global scheduler;

[0172] The meta event searching unit 60 is configured to search a meta event related to the tracking identifier locally.

[0173] The detail event searching unit 70 is configured to determine a first search time window of a detail event according to event generation time information contained in the meta event searched by the meta event searching unit, and search a detail event related to the tracking identifier locally according to the first search time window.

[0174] In the embodiment, the meta event searching unit searches a meta event related to the tracking identifier locally, which can include searching a second target file in which an event generation time of a meta event falls into a second search time window under a storage directory of the meta event, and searching a meta event related to the tracking identifier in the second target file; wherein a start time of the second search time window is a generation time of the tracking identifier, and a time length is a calling time length of the user request or a maximum life cycle of the user request, and information of the generation time of the tracking identifier is carried in the link query request.

[0175] In the embodiment, the meta event searching unit searches the second target file, which can include determining a time period in which an event generation time of a meta event saved in a file falls according to a creation time and / or a last modification time of the file under the storage directory of the meta event, and searching a file in which a time period falls into the second search time window, i.e. the second target file; wherein the meta events are saved in the files in a sequence of event generation time in turn, and a next file is used to save after a file reaches a set maximum size.

[0176] In the embodiment, the link analysis module can also include:

[0177] The call duration storage unit is configured to store an API event or a timeout API event, the API event containing information of a tracking identifier and a call duration of a user request, and the timeout API event referring to an API event whose call duration exceeds a corresponding timeout duration;

[0178] The call duration searching unit is configured to receive a call duration query request sent by the global dispatcher, search locally for an API event or a timeout API event related to a tracking identifier carried in the call duration query request, and return information of a call duration of a user request to the global dispatcher from the searched API event or timeout API event.

[0179] In this embodiment, after searching locally for a timeout API event related to the tracking identifier, the call duration searching unit can further include: if no timeout API event is searched, returning information of a timeout duration corresponding to the user request to the global dispatcher as the information of the call duration.

[0180] In this embodiment, the call duration searching unit can search locally for an API event or a timeout API event related to the tracking identifier, including: searching for a third target file in which an API event or a timeout API event whose event generation time falls within a third search time window is located, and then searching for an API event or a timeout API event related to the tracking identifier in the third target file; wherein the start time of the third search time window is the generation time of the tracking identifier, and the duration is the maximum life cycle of the user request.

[0181] In this embodiment, the detail event searching unit can include:

[0182] The time window subunit is configured to determine [t1, t2] as the first search time window, wherein t1 and t2 are the earliest time and the latest time, respectively, in the event generation times contained in the searched meta event.

[0183] The searching subunit is configured to search, under the detail event storage directory, a first target file in which a detail event whose event generation time falls within the first search time window is located, and then search for a detail event related to the tracking identifier in the first target file.

[0184] In the embodiment, the searching subunit searches the first target file under the detail event storage directory, which can include: determining a time period in which the event generation time of the detail event saved in the file is located according to the creation time and / or the last modification time of the file under the detail event storage directory, and then searching the file in which the time period falls into the first search time window, i.e., the first target file; wherein the detail events are saved in the file in the order of the event generation time, and a next file is used to save the detail events after a file reaches a set maximum size.

[0185] In the embodiment, the meta event contains identification information of the processor generating the meta event, the detail events generated by the same processor are saved in the file group corresponding to the processor, and different processors correspond to different file groups; the searching subunit searches the first target file under the detail event storage directory, and then searches the detail events related to the tracking identification in the first target file, which can include:

[0186] According to the identification information of the processor contained in the searched meta event, the file group corresponding to the processor is determined, the first target file is searched in the corresponding file group, and then the detail events related to the tracking identification are searched in the first target file; or

[0187] According to the identification information of the processor contained in the searched meta event, the file group corresponding to the processor is determined, and after the first target file is searched, files belonging to the corresponding file group are selected from the first target file, and then the detail events related to the tracking identification are searched in the selected files.

[0188] In the embodiment, the searching subunit can also be configured to: after searching the detail events related to the tracking identification, the number M1 of the searched detail events related to the tracking identification is counted, and if M1 is less than the number M2 of the meta events related to the tracking identification searched by the meta event searching unit, the detail events related to the tracking identification are searched in other file groups under the detail event storage directory.

[0189] In the embodiment, the link query request is a link query request for a user's distributed read request or distributed write request; the link analysis module can further include a second query interface unit, and the second query interface unit includes:

[0190] An address acquisition subunit is configured to: after receiving the link query request, locally query the communication event related to the tracking identification, and if the query is successful, acquire address information of a next hop server to which the user request is sent.

[0191] The link query subunit is configured to send a link query request to the next hop server according to address information of the next hop server, and carry a tracking identifier of the user request;

[0192] The information transmission subunit is configured to receive information of a detail event related to the tracking identifier returned by the next hop server and return the information to the global scheduler.

[0193] The embodiment also provides a server in a distributed cluster, including a processor and a memory, wherein,

[0194] The memory is configured to save program code.

[0195] The processor is configured to read the program code and perform the following processing:

[0196] receive a link query request sent by a global scheduler, the link query request carrying a tracking identifier of a user request;

[0197] search a meta event related to the tracking identifier locally, determine a first search time window of a detail event according to event generation time information contained in the searched meta event, and search the detail event related to the tracking identifier according to the first search time window;

[0198] return information of the searched detail event to the global scheduler.

[0199] In the embodiment, the link analysis processing performed by the processor in the server can include all processing in the link analysis method of the embodiment, which will not be repeated here.

[0200] The link analysis method and the server of the embodiment obtain a search time window of a detail event by searching a meta event, and propose multiple technical means for accelerating the search of the meta event and the search of the detail event, which can greatly reduce the search range of the detail event and accelerate the link analysis process.

[0201] The distributed link tracking and analysis method adopted in the above embodiment abandons the central warehouse and large database, and the original information data is always saved on the local server. Only when a user really needs to query the link of a user request, the related events of the user request are queried and analyzed. Therefore, most of the time, the data such as detail events of most user requests are only stored on the server and do not participate in network transmission. In addition, the above embodiment designs some optimization schemes around the feature that the events themselves are time-ordered, and the link analysis of a request can be completed in seconds under the pressure of 10,000 QPS (i.e. 10000 requests per second) of a single machine, and the time does not increase with the increase of the size of the target cluster, which is suitable for online query analysis. From the storage and computing perspectives, scalability is not a problem.

[0202] The application will be further described below through several examples in actual applications.

[0203] Example 1

[0204] This example relates to link tracking, and mainly focuses on the generation and saving of events.

[0205] The distributed link tracking and analysis system of this example includes a global scheduler (which can be distributed on one or several machines) and a server (in this application, it refers to a production server that provides services for users).

[0206] This example relates to the link tracking method common to type I requests (such as search requests) and type II requests (such as distributed read requests and distributed write requests).

[0207] In this example, a small meta event is generated every time the server generates a detail event. In this example, the meta event only contains a trace ID, event generation time information, and the identifier of the processor generating the event, such as CPU number (referred to as CPU number). A meta event is about 20 bytes, which is about one tenth of a detail event. Although the meta event is generated along with the detail event, the two types of events are stored in two local directories.

[0208] In this example, in the link tracking process, the events (meta events, detail events, communication events, and API events) are saved in local files, and the following rules are followed:

[0209] a) The detail events generated by the same processor are saved in a file group corresponding to the processor, and different processors correspond to different file groups.

[0210] Each processor in the server corresponds to a file group. In this example, the CPU number of the CPU that generates each file is written directly into the file name of the file, so the file group corresponding to the CPU can be determined easily, but the application can also bind the file group to the processor in other ways, such as establishing a correspondence table between the processor identifier and the file attributes (including but not limited to the file name). When the server saves an event, it will select a file in the corresponding file group according to the CPU that generated the event. The maximum size of each file can be fixed, such as 2M bytes. When the file size exceeds this limit, the next file in the same file group will be used for saving. For example, the files can be generated in a rotate manner, and contain a serial number in the file name, and the serial number is incremented sequentially. When the number of generated files reaches the maximum number, the initial file is overwritten.

[0211] According to the above rules, the local storage directory of the server that saves events contains a batch of files, and the names of the files can indicate which CPU generated the event that the file stores.

[0212] b) Each event generation is time-ordered, and the server will store the events according to the order of the event generation time.

[0213] The events are saved in the files in the order of the event generation time, and the attribute "last modified time" of a file can reflect the event generation time of the last event saved in the file (some files only record a modify time, which is the last modified time of the file). According to rule a), the last modified time of the previous file can be used as the creation time of the file. Similarly, the creation time of a file can reflect the event generation time of the first event saved in the file, and the creation time of the next file can be used as the last modified time of the file.

[0214] In this example, the source server needs to perform the following processing:

[0215] a) When the source server (such as the front-end server A of Figure 1 ) generates a traceID, it encodes the traceID generation time into the traceID. In this way, when link analysis is performed, the server can decode the generation time of the traceID from the traceID.

[0216] b) The source server generates an API event when the user request is completed, the API event records information including the trace ID and the user request call duration. The API event can be a special meta event generated along with the detail event generated when the user request is completed, but can also be generated separately. In this example, the source server finds out the timeout API event in the API event and saves it in a special directory (i.e. only the timeout API event is saved in the storage directory), the timeout API event refers to the API event whose user request call duration exceeds the corresponding timeout duration (the timeout duration set for the user request). For the API event, it can also be saved in the special directory of the API event to facilitate searching (the API event storage directory can or can not include the timeout API event).

[0217] c) If the user request call duration is used to narrow down the search time window, the source server encodes the IP address of the server into the trace ID when generating the trace ID, and the global scheduler can decode the IP address of the source server from the trace ID when performing link analysis, and initiate a query to the source server for the user request call duration.

[0218] The special requirements of the link tracking method for the type II request in this example are as follows:

[0219] a) Whether or not the user request call duration is used to narrow down the search time window, the source server encodes the IP address of the server into the trace ID when generating the trace ID, and the global scheduler can decode the IP address of the server from the trace ID when performing link analysis, and initiate a query to the source server for the user request call duration.

[0220] b) The server generates a communication event when processing the network communication of the user request, the communication event includes the IP address of the next hop server to which the user request is sent, and such events can be stored separately in a communication event storage directory.

[0221] In this example, the communication event is saved in a special directory of the communication event, and the timeout API event is saved in a special directory of the timeout API event, which is independent of the storage directory of other events (but can be a subdirectory of the storage directory of other events), so as to speed up the search of these two special events. However, the present application is not limited thereto.

[0222] Example Two

[0223] This example relates to the link analysis process of type I request such as search request, and the related link tracking process adopts the scheme of example one, and this example mainly focuses on the collection of event information in the link analysis process.

[0224] The overall process of the link analysis process of this example can be seen in Figure 8,include:

[0225] (1) If a user needs to view the link status of a user request (hereinafter referred to as the request), he / she sends a link viewing request to the global scheduler, carrying the traceID of the request;

[0226] (2) The global scheduler sends a link query request to all production servers in the distributed cluster, carrying the traceID; after receiving it, the analyzer on each server (equivalent to the link analysis module in the embodiment) searches for all detailed events related to the traceID in the local storage directory. If found, it sends the information of the searched detailed events back to the global scheduler.

[0227] (3) The global scheduler gathers information on all detailed events related to the traceID, performs link analysis on it, and sends the final processing result back to the user.

[0228] Figure 8 In the cluster production server, the "embedded server program" is used to complete the event generation and storage functions, which is equivalent to the link tracing module in the embodiment.

[0229] Specifically, the analyzer in the server executes a two-stage processing procedure, such as... Figure 9 As shown, the first-stage analyzer searches the local meta-event storage directory to see if the server has generated any meta-events related to the traceID. If found, the second stage searches the local detail event storage directory for all detail events related to the traceID and returns the search results to the global controller. If the first stage does not find any meta-events related to the traceID, it exits directly and does not go through the second stage. This is because searching in a smaller local meta-event storage directory first is much less costly and time-consuming than searching a detail event storage directory that is tens of times larger.

[0230] This example employs two optimization schemes for the first stage.

[0231] The first optimization scheme in the first phase

[0232] The present scheme does not search the entire meta-event storage directory when searching traceID. The entire storage directory of each server can store data for several days, and user requests usually have a timeout, which is usually not large (the timeout of various requests is not the same, but is less than or equal to the maximum lifetime of the user request, such as 10 minutes). The server can decode the traceID generation time from the traceID, so the first stage can search for data in a 10-minute time range from the traceID generation time. The 10-minute time range from the traceID generation time is the search time window used in the first stage, which is referred to as the second search time window.

[0233] Because the events (including meta-events and detail events) are stored in the link tracking process by first selecting a file group according to the CPU number of the CPU generating the event, and then stored in the file of the file group in chronological order, when the size of a file reaches the maximum size (such as the size limited by the user), the new event is transferred to the next file in the file group for storage, so the attribute "last modified time" of each file can reflect the event generation time of the last event in the file. In other words, each file can infer the time period in which all events saved in the file are located from the last modified time of itself and the previous file. In this way, the first stage does not need to search the entire directory, but only searches the file set in which the traceID falls within the second search time window. This optimization makes the maximum time consumption of the first stage not increase with the increase of the number of meta-events in the meta-event storage directory, as long as the single machine QPS is fixed.

[0234] The processing flow of the analyzer (equivalent to the link analysis module) in the first stage server (see Figure 10 ) includes:

[0235] The analyzer parses the time stamp t of the traceID generation from the encoding of the traceID.

[0236] Traverse the local meta-event storage directory to find a set S composed of all file groups whose modify time is in the range [t, t+10 minutes].

[0237] If the set S is not empty, search the meta-events related to the traceID in the files of the set, and the meta-event set is denoted as E; otherwise, return an empty result directly.

[0238] Record the CPU number of each meta-event in the set E, which can be used to optimize the second stage.

[0239] Second optimization scheme of the first stage

[0240] The optimization scheme can accelerate the link analysis of timeout requests. Timeout requests are a kind of requests that users are interested in. The first optimization scheme takes the maximum lifetime of 10 minutes of user requests as the length of the second search time window, that is, the first stage searches the meta-event storage directory of the server to see whether there is a meta-event containing the traceID in the data within 10 minutes from the time when the traceID occurs. But not all kinds of requests (write requests, read requests, etc.) are 10 minutes timeout, and the optimization scheme is to find the real timeout time of user requests, which is usually much smaller than 10 minutes. The source server only needs to generate a timeout API event and store it in another special directory after the calling time of the API event contained exceeds the corresponding timeout time (which can be set by the user). Normally, the number of timeout requests is much smaller than the number of normal requests, so the timeout API event storage directory is at least an order of magnitude smaller than the normal meta-event storage directory.

[0241] The scheme first searches the local timeout API event set of the source server for a timeout API event containing the traceID, obtains the calling time of the request from the searched timeout API event, and if no timeout API event is found, searches the normal API event set or takes the timeout time corresponding to the request as the calling time of the request; then, the calling time of the request can be used instead of the original 10 minutes as the length of the second search time window of the meta-event search of each server.

[0242] The process of the first stage optimized for timeout requests (its interaction with the global scheduler and the source server can be seen from Figure 11 ), includes:

[0243] The global scheduler parses the traceID to obtain the IP address of the source server that generates it, sends the traceID of the request to the server to query the calling time of the request, and then waits for the analyzer to return the calling time t1 of the request;

[0244] The analyzer on the source server parses the timestamp t of the generation of the traceID from the encoding of the received traceID;

[0245] The analyzer on the source server searches in the set composed of all files in the local timeout API event storage directory whose creation to last modification time period is in the range of [t, t+10 minutes] to see whether there is a file containing the timeout API event of the traceID. The time period from the creation to the last modification of the file, that is, the time period in which the events saved in the file occur, can be determined according to the last modification time and / or creation time of the file (or the file and its adjacent files);

[0246] If found, the profiler on the source server takes the call duration t1 of the request and sends it to the global dispatcher. If not found, the timeout duration of the request is taken as t1 and sent to the global dispatcher.

[0247] The global dispatcher tries to wait for t1 from the profiler on the source server. If t1 is received within a reasonable waiting time, x is set as t1. Otherwise, x is set as 10 minutes. x is taken as y in the profiler program and sent to the profilers on all servers together with traceID.

[0248] The profilers on all servers receive traceID and corresponding y from the global dispatcher. Then, they find all files from the local meta event storage directory whose creation time and last modification time are within the time range [t, t+y] and form a set S.

[0249] If S is not empty, search for meta events related to traceID from S. The searched meta event set is denoted as E. Otherwise, return an empty result directly.

[0250] Record the CPU numbers contained in each meta event in set E. This information can be used to optimize the second stage.

[0251] If the profiler on a server finds that the target traceID (i.e. the traceID of the user request that needs to be analyzed) is found in the meta event storage directory of the server, the second stage processing process is entered. Otherwise, the search process is ended directly. The second stage needs to search all events related to the target traceID from the local larger detail event storage directory and send the result to the global dispatcher.

[0252] The second stage also has two optimizations. First, because each detail event is accompanied by a meta event, the time of all meta events on the server searched in the first stage can be used to know the first search time window of the detail event on the server, and the entire detail storage directory does not need to be searched. The second optimization is related to the CPU number. Because the event is generated in which CPU, it is stored in the corresponding file group file. Knowing which CPU generates a detail event can directly search the corresponding file group file in the detail event storage directory. The server usually has 24 or 32 CPUs, and now only one CPU generated file needs to be searched to find the detail event, which can undoubtedly greatly reduce the search amount and improve the search efficiency. Because the meta event and the detail event are accompanied by generation, in most cases, the two events are generated by the same CPU. Therefore, the CPU number contained in each meta event on the server searched in the first stage can be used to search the detail event in the corresponding file group file in the detail event storage directory. In rare cases, CPU scheduling occurs between the meta event and the corresponding detail event, which can cause search failure, so that the number of searched meta events and detail events is inconsistent. At this time, search can be performed in other files in the detail event storage directory. With these optimizations, as long as the single machine QPS is fixed, the search time of the second stage will not increase with the increase of the detail event storage directory.

[0253] The specific process of the second stage of the present example (see Figure 12 , which includes:

[0254] (1) The analyzer on the server obtains the event generation time and CPU number information of each meta event associated with the target traceID from the first stage, determines the first search time window [t1, t2] and the CPU set X to be searched. In addition, the number of meta events searched in the first stage can also be calculated: n

[0255] (2) Find the file in the local detail event storage directory whose creation to last modification time period falls within [t1, t2], and then select the file under the file group corresponding to the CPU set X from it to form set S;

[0256] (3) If set S is empty, go to (5); otherwise, perform (4);

[0257] (4) Search all detail events associated with the traceID from files in set S, form set E, and determine if the number of events m in E is less than n. If m equals n, it means that the files in set S contain all the detail events, set E2 to empty, and go to (6). If m is less than n, it means that there are detail events not found, form set E2, and go to (5);

[0258] (5) From the local detail event storage directory, find files created and last modified in the time period [t1, t2] (excluding files already searched), search for the detail events not found, form set E2. Note that the files to be searched now contain all the files generated by the CPU;

[0259] (6) Combine set E and set E2 as the search result set F, return it to the global scheduler, and the second phase is over.

[0260] Example Three

[0261] This example is about the link analysis process for type II requests such as distributed write requests or distributed read requests. The relevant link tracking process uses the scheme described in Example One in relation to type II requests. This example focuses on the collection of event information in the link analysis.

[0262] Overall Process

[0263] Because the traceID contains the IP address of the source server that generated it, the global scheduler can decode the IP address of the corresponding source server for a given traceID, and send the traceID to the source server. The analyzer on the source server searches for local events associated with the traceID while performing the two-phase process described in Example One for type I requests, and finds out which servers (next-hop servers) the traceID was sent to by itself, and sends a link query request to these servers, carrying the traceID. The analyzers on these servers also operate in this way, until the last-hop server.

[0264] The difference between type II requests and type I requests is that type II requests involve fewer servers, so this kind of chain tracking scheme can make only the few servers that actually processed the relevant request participate in the link analysis process, leaving the vast majority of servers in the cluster undisturbed, which minimizes the disturbance of the link analysis to the production servers.

[0265] The link analysis process for type II requests in this example is shown in Figure 13 , which includes:

[0266] (1) User wants to check the link of a request, sends a link check request to the global dispatcher, carrying the traceID of the request;

[0267] (2) The global dispatcher parses the IP address of the source server that generates the traceID, and sends a link check request to the source server, carrying the traceID.

[0268] (3) The analyzer on the source server receives the traceID, searches for events associated with the traceID according to the two-stage processing process of I-type requests, and finds the IP address of the next-hop server to which the traceID is sent from the communication event, and sends a link check request to the next-hop server, carrying the traceID. In this way, the chain processing is performed until the last-hop server.

[0269] (4) The analyzers on all servers that receive the link check request send the information of the detailed events associated with the traceID to the global dispatcher;

[0270] (5) The global dispatcher collects all events, analyzes and processes them, and sends the results to the user.

[0271] In addition to implementing the two-stage processing process of I-type requests in Example 1, the analyzer on the server in this example also performs a trace process to find downstream servers and send traceIDs to them, as shown in Figure 14 The two-stage processing and downstream tracking processing of the analyzer for II-type requests include:

[0272] Two-stage processing process: same as Example 2;

[0273] Downstream tracking process: while performing the two-stage processing, the analyzer checks whether the communication event storage directory contains communication events related to the traceID. If it does, it finds the IP addresses of all next-hop servers and sends a link check request to them, carrying the traceID; otherwise, it returns directly.

[0274] The first stage of the two-stage process of II-type requests has the same advantages as I-type requests. Because only a few servers are involved, only the analyzers on these servers will run the second stage, and the analyzers on most servers in the cluster will not go through the second stage, so the CPU and disk IO resources of the cluster can be greatly reduced. Therefore, this solution has greater advantages for such requests.

[0275] Because the two-stage processing process of II-type requests is the same as the two-stage processing process of I-type requests, only the downstream tracking process needs to be specifically looked at:

[0276] The downstream tracking process searches the communication event storage directory instead of the meta event storage directory, so the optimized solution and effect are the same as example two, as shown in Figure 15 includes:

[0277] (1) Parse the timestamp t generated by the traceID from the encoding of the traceID

[0278] (2) Traverse the local communication event storage directory to find all file sets whose creation time and last modification time fall within the period [t, t+10 minutes]: S

[0279] (3) If the set S is not empty, search for the communication event related to the traceID in the files in the set, and the communication event set is denoted as E; otherwise, return an empty result directly

[0280] (4) If the set E is not empty, record the IP address of the next hop server contained in each communication event in the set E, and send a link query request to it carrying the traceID; otherwise, return an empty result directly

[0281] (5) End the downstream tracking process.

[0282] The above embodiments and examples realize completely localized storage and calculation, eliminating the central repository and large database in the classic distributed link tracking analysis system. In addition, at least the following features are provided:

[0283] Each detail event is accompanied by a meta event, which is used to optimize the process of searching the traceID locally, including using the time of generating the meta event, the generating CPU number, etc.

[0284] Generate a timeout API event to optimize the process of searching for a timeout request

[0285] Add a communication event containing the target service IP to optimize the search process of type II requests. Only a few servers that have actually processed the request participate in the link analysis process, and most servers in the cluster are not disturbed. The interference of link analysis on production servers is minimized

[0286] The above-mentioned embodiment numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments. Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and the necessary general hardware platform, of course, they can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, or network device, etc.) execute the methods described in various embodiments of the present application.

[0287] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A distributed link tracking method, comprising: a server in a distributed cluster generates a meta event for searching a detail event when generating the detail event in processing a user request, the meta event containing information of a tracking identifier and a time of generating the event, and a data volume of the meta event being less than that of the corresponding detail event; the server separately saves the generated detail event and meta event locally; wherein the meta event further contains identification information of a processor generating the meta event; the server saves the detail event locally, including saving detail events generated by a same processor in a file group corresponding to the processor, and different processors corresponding to different file groups. 2.The method of claim 1, wherein: the tracking identifier contains information of a time of generating the tracking identifier; or the tracking identifier contains information of a time of generating the tracking identifier and address information of a source server. 3.The method of claim 1, wherein: the server separately saves the detail event and meta event locally, including sequentially saving the detail event and meta event in files under respective directories according to a time sequence of generating the events, and switching to a next file for saving after a file reaches a set maximum size. 4.The method of claim 1 or 2 or 3, further comprising: a source server of the user request generates and saves an application programming interface (API) event when the user request is completed, the API event containing the tracking identifier and information of a calling duration of the user request. 5.The method of claim 4, wherein: after the source server generates the API event, the method further includes saving a timeout API event in a special directory of timeout API events, the timeout API event referring to an API event whose calling duration of the user request exceeds a corresponding timeout duration. 6.The method of claim 1 or 2 or 3 or 5, wherein: the user request is a distributed read request or a distributed write request; the method further includes that the server generates a communication event when processing network communication of sending the user request to a next hop server, the communication event containing the tracking identifier and address information of the next hop server to which the user request is sent. The link tracking module comprises:

7. A server in a distributed cluster comprising a link tracking module, characterized in that, an event generating unit configured to generate a meta event for searching a detail event when generating the detail event in processing a user request, the meta event containing information of a tracking identifier and a time of generating the event, and a data volume of the meta event being less than that of the corresponding detail event; an event storage unit configured to separately save the generated detail event and meta event locally; wherein the event generating unit generates a meta event containing identification information of a processor generating the meta event; the event storage unit saves the detail event locally, including saving detail events generated by a same processor in a file group corresponding to the processor, and different processors corresponding to different file groups. ​ 8. The server of claim 7, wherein: the link tracking module further comprises an identification generating unit configured to generate a tracking identification for the user request, encode information of a time when the tracking identification is generated into the tracking identification, or encode information of the time when the tracking identification is generated and address information of the server into the tracking identification.

9. The server of claim 7, wherein: the event storage unit separately saves the detail events and the meta events locally, including: sequentially saving the detail events and the meta events in files under respective directories according to the order of the times when the events are generated, and switching to a next file to save after a file reaches a set maximum size.

10. The server of claim 7 or 8 or 9, wherein: the event generating unit is further configured to generate an application programming interface (API) event when the user request is completed, the API event containing the tracking identification and information of a calling duration of the user request; the event storage unit is further configured to save the API event.

11. The server of claim 10, wherein: after the event storage unit generates the API event, the event storage unit further comprises: saving a timeout API event in a dedicated directory of timeout API events, the timeout API event being an API event whose calling duration of the user request exceeds a timeout duration.

12. The server of claim 10, wherein: the event generating unit is further configured to generate a communication event when a network communication process of sending the user request to a next-hop server is performed, the communication event containing the tracking identification and address information of the next-hop server to which the user request is sent; the user request is a user request for distributed reading or distributed writing.

13. A server in a distributed cluster, comprising a processor and a memory, wherein: the memory is configured to save program code; the processor is configured to read the program code and perform the following link tracking processing: in a process of processing a user request, generating a meta event for searching a detail event when the detail event is generated, the meta event containing a tracking identification and information of a time when the event is generated, and a data amount of the meta event being less than a data amount of the corresponding detail event; and separately saving the generated detail event and meta event locally, the meta event further containing identification information of a processor that generates the meta event; wherein the server is configured to save detail events generated by a same processor in a file group corresponding to the processor, and different processors correspond to different file groups.

14. A distributed link analysis method, comprising: when a global scheduler in a distributed cluster needs to perform link analysis on a user request, sending a link query request to servers in the distributed cluster, the link query request carrying a tracking identification of the user request; the global scheduler receiving event information related to the tracking identification returned by the servers after the servers search locally; The global scheduler performs link analysis on the user request based on the received event information related to the tracking identifier; The tracking identifier contains information about the time when the tracking identifier is generated, and the link query request also carries maximum lifetime information of the user request; Or The tracking identifier contains information about the time when the tracking identifier is generated, and the link query request also carries calling duration information of the user request obtained by the global scheduler from a source server.

15. The method of claim 14, wherein: The user request is a search request, and the global scheduler sends the link query request to the servers in the distributed cluster, including sending the link query request to all servers in the distributed cluster; or The user request is a distributed read request or a distributed write request, and the global scheduler sends the link query request to the servers in the distributed cluster, including sending the link query request to a source server of the user request.

16. A global scheduler in a distributed cluster, the global scheduler comprising: Including: The link query module is configured to send a link query request to the servers in the distributed cluster, carrying a tracking identifier of the user request, when link analysis of the user request is needed; And, receive the event information related to the tracking identifier returned by the server after local search; The link analysis module is configured to perform link analysis on the user request based on the received event information related to the tracking identifier; The link query request sent by the link query module also carries maximum lifetime information of the user request, and the tracking identifier contains information about the time when the tracking identifier is generated; Or The link query request sent by the link query module also carries calling duration information of the user request obtained by the global scheduler from a source server, and the tracking identifier contains information about the time when the tracking identifier is generated.

17. The global scheduler of claim 16, wherein: The user request is a search request, and the link query module sends the link query request to the servers in the distributed cluster, including sending the link query request to all servers in the distributed cluster; or The user request is a distributed read request or a distributed write request, and the link query module sends the link query request to the servers in the distributed cluster, including sending the link query request to a source server of the user request in the distributed cluster.

18. A global scheduler in a distributed cluster, comprising a processor and a memory, characterized in that: The memory is configured to store program code; The processor is configured to read the program code and perform the following link analysis processing: When link analysis of a user request is needed, send a link query request to the servers in the distributed cluster, carrying a tracking identifier of the user request; Receive the event information related to the tracking identifier returned by the server after local search; Perform link analysis on the user request based on the received event information related to the tracking identifier; The tracking identifier contains information of a time when the tracking identifier is generated, and the link query request further carries maximum life cycle information of the user request. Or The tracking identifier contains information of a time when the tracking identifier is generated, and the link query request further carries calling duration information of the user request obtained by the global scheduler from a source server.

19. A distributed link analysis method, comprising: A server in a distributed cluster receives a link query request sent by a global scheduler, and the link query request carries a tracking identifier of a user request; The server locally searches meta events related to the tracking identifier, determines a first search time window of detail events according to event generation time information contained in the searched meta events, and locally searches detail events related to the tracking identifier according to the first search time window; The server returns information of the searched detail events to the global scheduler; The meta events are generated when the server generates detail events in the process of processing the user request; the meta events contain the tracking identifier and event generation time information, and have a data volume smaller than that of the corresponding detail events.

20. The method of claim 19, wherein: The meta events are sequentially saved in files according to the event generation time, and a next file is used to save meta events after a file reaches a set maximum size; The server locally searches meta events related to the tracking identifier, including searching a second target file in which meta events with event generation time falling within a second search time window are located, and searching meta events related to the tracking identifier in the second target file; The second search time window starts at the time when the tracking identifier is generated, and has a duration of the calling duration of the user request or the maximum life cycle of the user request, and the information of the time when the tracking identifier is generated is carried in the link query request.

21. The method of claim 20, wherein: The server searches the second target file, including determining a time period in which event generation time of meta events saved in a file falls, according to creation time and / or last modification time of the file under the meta event storage directory, and searching a file in which the time period falls within the second search time window, and the searched file is the second target file.

22. The method of claim 20, wherein: The information of the calling duration is carried in the link query request sent by the global scheduler; The method further comprises: A source server of the user request receives a calling duration query request of the global scheduler, and the calling duration query request carries the tracking identifier; The source server locally searches application programming interface (API) events or timeout API events related to the tracking identifier, obtains the information of the calling duration from the searched API events or timeout API events, and returns the information to the global scheduler. The API event includes tracking identification and calling duration information requested by a user, and the timeout API event refers to an API event whose calling duration exceeds a corresponding timeout duration.

23. The method of claim 22, wherein: After the source server searches the timeout API event related to the tracking identification locally, the source server further returns the information of the timeout duration of the user request to the global dispatcher as the calling duration information if no timeout API event related to the tracking identification is searched.

24. The method of claim 23, wherein: The source server searches the API event or the timeout API event related to the tracking identification locally, including searching a third target file in which the API event or the timeout API event generated at a time falling within a third search time window is located, and then searching the API event or the timeout API event related to the tracking identification in the third target file, wherein the third search time window starts at the generation time of the tracking identification and has a duration of the maximum life cycle of the user request.

25. The method of any one of claims 19-24, wherein: The detail events are sequentially saved in files according to the generation time of the events, and a next file is used to save the detail events after a file reaches a set maximum size; The server determines a first search time window for the detail events, and searches the detail events related to the tracking identification locally according to the first search time window, including: The server determines [t1, t2] as the first search time window, wherein t1 and t2 are the earliest time and the latest time in the generation time of the events included in the searched meta event, respectively; The server searches a first target file in which the detail event generated at a time falling within the first search time window is located under a detail event storage directory, and then searches the detail event related to the tracking identification in the first target file.

26. The method of claim 25, wherein: The server searches the first target file under the detail event storage directory, including determining a time period in which the generation time of the detail events saved in the file falls, according to the creation time and / or the last modification time of the file under the detail event storage directory, and then searching the file in which the time period falls within the first search time window, and the searched file is the first target file.

27. The method of claim 26, wherein: The meta event includes identification information of a processor generating the meta event, and the detail events generated by the same processor are saved in a file group corresponding to the processor, and the detail events generated by different processors are saved in different file groups; The server searches the first target file under the detail event storage directory, and then searches the detail event related to the tracking identification in the first target file, including: The server determines a file group corresponding to the processor according to the identification information of the processor contained in the searched meta-event, searches the first target file in the corresponding file group, and searches the detail event related to the tracking identification in the first target file. Or The server determines a file group corresponding to the processor according to the identification information of the processor contained in the searched meta-event, and selects a file belonging to the corresponding file group from the first target file after searching the first target file, and searches the detail event related to the tracking identification in the selected file.

28. The method of claim 27, wherein: After the server searches the detail event related to the tracking identification, the method further comprises: counting the number M1 of the searched detail event related to the tracking identification, and searching the detail event related to the tracking identification in other file groups under the detail event storage directory if M1 is less than the number M2 of the searched meta-event related to the tracking identification.

29. The method of any one of claims 19-23, 26-28, wherein: The link query request is a link query request of a distributed read request or a distributed write request of a user; After the server receives the link query request, the method further comprises: The server queries the communication event related to the tracking identification stored locally, and the communication event contains address information of a next hop server to which the user request is sent; If the server queries the communication event, the server sends a link query request to the next hop server according to the address information in the communication event, and carries the tracking identification of the user request; The server receives the information of the detail event related to the tracking identification returned by the next hop server and returns the information to the global scheduler.

30. A server in a distributed cluster comprising a link analysis module, characterized in that, The link analysis module comprises: A first query interface unit configured to receive a link query request sent by the global scheduler, the link query request carrying a tracking identification of a user request, and return information of a searched detail event to the global scheduler; A meta-event searching unit configured to search a meta-event related to the tracking identification locally; A detail event searching unit configured to determine a first search time window of a detail event according to event generation time information contained in a meta-event searched by the meta-event searching unit, and search a detail event related to the tracking identification locally according to the first search time window. The meta-event is generated when a server generates a detail event in the process of processing a user request, contains information of a tracking identification and an event generation time, and has a data volume less than that of a corresponding detail event.

31. The server of claim 30, wherein: The meta-event searching unit searches the second target file, including: determining a time period in which the event generation time of the meta-event saved in the file falls according to the creation time and / or last modification time of the file under the meta-event storage directory; and searching the file in which the time period falls in the second search time window, that is, the second target file; wherein the meta-events are saved in the file in the order of event generation time, and a next file is used to save the meta-events after a file reaches a set maximum size.

32. The server of claim 31, wherein: The meta-event searching unit searches the second target file, including: determining a time period in which the event generation time of the meta-event saved in the file falls according to the creation time and / or last modification time of the file under the meta-event storage directory; and searching the file in which the time period falls in the second search time window, that is, the second target file; wherein the meta-events are saved in the file in the order of event generation time, and a next file is used to save the meta-events after a file reaches a set maximum size.

33. The server of claim 31, wherein: The link analysis module further includes: The call duration storage unit is configured to save an application programming interface (API) event or a timeout API event, the API event containing information of a tracking identifier and a call duration of a user request, and the timeout API event referring to an API event whose call duration of the user request exceeds a corresponding timeout duration; The call duration searching unit is configured to receive a call duration query request sent by the global scheduler, search locally for an API event or a timeout API event related to a tracking identifier carried in the call duration query request, and return information of the call duration of the user request to the global scheduler from the searched API event or timeout API event.

34. The server of claim 33, wherein: After searching locally for the timeout API event related to the tracking identifier, the call duration searching unit further includes: returning information of a timeout duration corresponding to the user request to the global scheduler as the information of the call duration if no timeout API event is searched.

35. The server of claim 32 or 34, wherein: The call duration searching unit searches locally for the API event or the timeout API event related to the tracking identifier, including: searching a third target file in which an API event or a timeout API event falls in a third search time window, and searching the third target file for the API event or the timeout API event related to the tracking identifier; wherein the start time of the third search time window is the generation time of the tracking identifier, and the duration is the maximum life cycle of the user request.

36. The server of any one of claims 30-34, wherein: The detail event searching unit includes: a time window subunit configured to determine [t1, t2] as the first search time window, where t1 and t2 are the earliest time and the latest time respectively in the event generation time of the searched meta event; a search subunit configured to search, under the detailed event storage directory, a first target file in which the detailed event with the event generation time falling into the first search time window is located, and then search the detailed event related to the tracking identifier in the first target file.

37. The server of claim 36, wherein: the search subunit searches the first target file under the detailed event storage directory, comprising: determining a time period in which the event generation time of the detailed event saved in the file falls according to the creation time and / or the last modification time of the file under the detailed event storage directory, and then searching the file in which the time period falls into the first search time window, i.e. the first target file; wherein the detailed events are saved in the file in the order of the event generation time, and a next file is used to save the detailed events after a set maximum size of the file is reached.

38. The server of claim 37, wherein: the meta event contains the identifier information of the processor generating the meta event, and the detailed events generated by the same processor are saved in the file group corresponding to the processor, and the detailed events generated by different processors are saved in different file groups; the search subunit searches the first target file under the detailed event storage directory, and then searches the detailed event related to the tracking identifier in the first target file, comprising: determining the file group corresponding to the processor according to the identifier information of the processor contained in the searched meta event, searching the first target file in the corresponding file group, and then searching the detailed event related to the tracking identifier in the first target file; or determining the file group corresponding to the processor according to the identifier information of the processor contained in the searched meta event, and after the first target file is searched, selecting the file belonging to the corresponding file group from the first target file, and then searching the detailed event related to the tracking identifier in the selected file.

39. The server of claim 38, wherein: the search subunit is further configured to, after searching the detailed event related to the tracking identifier, count the number M1 of the searched detailed event related to the tracking identifier, and if M1 is less than the number M2 of the meta event related to the tracking identifier searched by the meta event search subunit, search the detailed event related to the tracking identifier in other file groups under the detailed event storage directory.

40. The server of any one of claims 30-34, 37-39, wherein: the link query request is a link query request for a user's distributed read request or distributed write request; the link analysis module further comprises a second query interface unit, and the second query interface unit comprises: ​ The address obtaining subunit is configured to: after receiving the link query request, locally query the communication event related to the tracking identifier, and obtain the address information of the next hop server to which the user request is sent from the query result; The link query subunit is configured to: send a link query request to the next hop server according to the address information of the next hop server, and carry the tracking identifier of the user request; The information transmission subunit is configured to: receive the information of the detail event related to the tracking identifier returned by the next hop server and return the information to the global scheduler. 41.A server in a distributed cluster, comprising a processor and a memory, characterized in that, The memory is configured to: save program code; The processor is configured to: read the program code and perform the following processing: receive a link query request sent by a global scheduler, the link query request carrying a tracking identifier of a user request; locally search a meta event related to the tracking identifier, determine a first search time window of a detail event according to event generation time information contained in the searched meta event, and locally search the detail event related to the tracking identifier according to the first search time window; return the information of the searched detail event to the global scheduler; The meta event is an event that is accompanied by the generation of a detail event in the process of the server processing a user request; the meta event contains information of a tracking identifier and an event generation time, and the data amount of the meta event is less than that of the corresponding detail event.

Images