Request link tracking method and device and server

Abstract

The invention relates to the technical field of cloud computing, and discloses a request link tracking method and device and a server. The method comprises the following steps: acquiring incremental logs generated when a plurality of micro-service instances execute a target request; generating a tracking link based on the attribute information of the plurality of incremental logs, the tracking link being used for representing a calling hierarchical relationship among the plurality of micro-service instances; determining state information of a tracking link based on request state information corresponding to an incremental log included in the attribute information; and sampling the tracking link based on the state information to obtain a link tracking result. By implementing the technical scheme of the invention, the calling relationship between the micro-services can be automatically constructed based on the incremental log, and state monitoring and sampling are carried out on the request link, so that faults in a distributed system are efficiently positioned and diagnosed.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud computing, and in particular to a method, device and server for tracking a request link. Background Art

[0002] As the scale of distributed systems continues to expand, the cost and potential losses of fault diagnosis are also increasing. Even minor errors may cause multiple services to be unavailable, and the increase in system complexity requires developers to invest a lot of time to locate problems, resulting in a waste of human resources. In this context, distributed link tracking systems came into being, which can record end-to-end information of requests to help quickly locate faults. However, many early designed systems lack support for link tracking, and face problems such as difficulty in manual tracking maintenance, insufficient support for asynchronous programming models, inflexible sampling strategies, and lack of persistence in link data uploads during integration. Summary of the invention

[0003] In view of this, the present invention provides a method, device and server for tracing a request link to solve the problems of fault diagnosis complexity and resource waste in a distributed system.

[0004] In a first aspect, the present invention provides a method for tracing a request link, comprising: obtaining incremental logs generated when multiple microservice instances execute target requests; generating a tracing link based on attribute information of the multiple incremental logs, the tracing link being used to characterize the call hierarchy relationship between the multiple microservice instances; determining status information of the tracing link based on request status information corresponding to the incremental log included in the attribute information; sampling the tracing link based on the status information to obtain a link tracing result.

[0005] The request link tracing method provided by the embodiment of the present invention can comprehensively and accurately characterize the call relationship and hierarchy between microservices by obtaining the incremental logs generated by multiple microservice instances and generating tracing links based on their attribute information, thereby helping the system to effectively monitor and analyze the execution status of the request link. By combining the status information of the request, the link can be finely tracked and sampled, thereby quickly locating problems, optimizing performance, and improving the maintainability and reliability of the microservice architecture.

[0006] In an optional implementation, the attribute information also includes identification information corresponding to the incremental log; based on the attribute information of multiple incremental logs, a tracing link is generated, including: constructing a log structure corresponding to the target request according to the request status information and the identification information, the log structure including multiple parent nodes and child nodes; based on the node relationship between the parent nodes and the child nodes, multiple target span objects are generated; based on the hierarchical relationship between the multiple target span objects, a tracing link is generated.

[0007] The request link tracing method provided by the embodiment of the present invention further optimizes the generation process of the tracing link by generating multiple target span objects and utilizing their hierarchical relationships, making the link tracing clearer and more detailed. Through this precise structured approach, the call hierarchy and dependency relationships between microservices can be better revealed, the visual monitoring and troubleshooting capabilities of the system can be improved, developers can be helped to quickly identify and locate potential problems, and the performance and stability of the system can be improved.

[0008] In an optional embodiment, a log structure corresponding to the target request is constructed according to the request status information and the identification information, including: based on the request status information, identifying the request status of each incremental log; based on the identification information, determining the target incremental log associated with the target request in multiple incremental logs; matching the multiple target incremental logs according to the request status to obtain a log matching result; if it is determined based on the log matching result that the target request has ended or a timeout has occurred, generating a log structure using the request status information.

[0009] The request link tracking method provided by the embodiment of the present invention can effectively identify the incremental logs related to the target request by accurately utilizing the request status information and identification information, and match and analyze the logs according to the request status, thereby ensuring that the life cycle of the target request can be accurately tracked and whether the request has ended or timed out can be identified in a timely manner, thereby helping the system to handle abnormal situations in the request process more efficiently. With the assistance of the log matching results, it is possible to quickly locate and generate a complete log structure when a problem occurs, providing the system with stronger fault diagnosis capabilities and performance optimization support, thereby improving overall reliability and response speed.

[0010] In an optional implementation, a log structure is generated using request status information, including: obtaining a first timestamp corresponding to each target incremental log; determining a node relationship between each target incremental log based on each first timestamp and request status information; and generating a log structure based on multiple node relationships.

[0011] The request link tracking method provided by the embodiment of the present invention can clearly reflect the execution process and dependency order of requests between different microservices through node relationship judgment based on timestamps, avoiding time confusion or mismatch problems that may occur in traditional methods. This refined log structure helps to better track the state changes of requests, quickly locate performance bottlenecks or faulty nodes, improve the efficiency of system monitoring, debugging and optimization, and enhance the reliability and maintainability of the system.

[0012] In an optional embodiment, based on the node relationship between the parent node and the child node, multiple target span objects are generated, including: identifying the first log corresponding to each parent node and the second log corresponding to each child node in the log structure; mapping each first log to a first span object, and mapping each second log to a second span object; and constructing a parent-child relationship between the first span object and the first span object based on the node relationship.

[0013] The request link tracing method provided by the embodiment of the present invention can accurately reflect the call dependency and execution order between microservices by mapping the logs of the parent node and the child node to different span objects respectively. Constructing a hierarchical structure of span objects based on the node relationship between the parent and child nodes helps to accurately track and analyze the execution trajectory of each request in the system, thereby improving the accuracy and efficiency of problem location. Therefore, this method helps to improve the visualization capability of link tracing, allowing developers to more intuitively understand and optimize the service call process in the system, thereby enhancing the maintainability, reliability and performance of the system.

[0014] In an optional implementation, a tracing link is generated based on a hierarchical relationship between multiple target span objects, including: obtaining a second timestamp corresponding to each first log and a third timestamp corresponding to each second log; arranging the first span object and the second span object according to the parent-child relationship, the second timestamp and the third timestamp to generate a tracing link.

[0015] The request link tracking method provided by the embodiment of the present invention can clearly show the execution order and time dependency of each service call by obtaining the timestamps of the first log and the second log and arranging them in combination with the parent-child relationship. Such a tracking link not only reflects the transmission path of the request in the system, but also provides a detailed time perspective to help developers analyze the delay and performance bottleneck of the request. Through accurate time sorting and hierarchical structure, troubleshooting and performance optimization can be carried out more effectively, improving the monitorability, stability and response efficiency of the system.

[0016] In an optional implementation, the tracking link is sampled based on the status information to obtain the link tracking result, including: if the tracking link is determined to be a normal link based on the status information, probability sampling is performed on the normal link to obtain the link tracking result of the normal link; if the tracking link is determined to be an abnormal link based on the status information, all samples are performed on the abnormal link to obtain the link tracking result of the abnormal link.

[0017] The request link tracking method provided by the embodiment of the present invention adopts a probabilistic sampling method when the link is judged to be normal, which can effectively reduce the amount of data processing and improve the sampling efficiency; while for abnormal links, comprehensive sampling is performed to ensure that each abnormal situation is fully analyzed and recorded. Through this flexible sampling strategy, the system can optimize performance and resource consumption while ensuring comprehensive data coverage, thereby improving the accuracy and response speed of troubleshooting.

[0018] In an optional embodiment, when it is determined that the tracking link is an abnormal link, it is detected whether the abnormal link has dependent services and upper-layer services; when it is determined that the abnormal link has dependent services, the dependent services are notified so that the dependent services determine whether to report the corresponding first link data; when it is determined that the abnormal link has an upper-layer service, the upper-layer service is notified so that the upper-layer service determines whether to report the corresponding second link data.

[0019] The request link tracking method provided by the embodiment of the present invention can realize accurate fault tracing and data reporting by dynamically detecting and notifying the dependent services and upper-layer services of the abnormal link. By identifying and notifying the dependent services and upper-layer services, it can ensure that the relevant services obtain abnormal information in a timely manner and decide whether to report the link data. This collaborative processing method improves the diagnostic efficiency of abnormal links and ensures that all potential influencing factors can be tracked and reported, thereby comprehensively and quickly locating the source of the problem.

[0020] In a second aspect, the present invention provides a request link tracking device, comprising: an acquisition module for acquiring incremental logs generated when multiple microservice instances execute target requests; a generation module for generating a tracking link based on attribute information of the multiple incremental logs, the tracking link being used to characterize the call hierarchy relationship between the multiple microservice instances; a determination module for determining status information of the tracking link based on request status information corresponding to the incremental log included in the attribute information; and a tracking module for sampling the tracking link based on the status information to obtain a link tracking result.

[0021] In a third aspect, the present invention provides a server, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the request link tracking method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0022] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the request link tracking method of the first aspect or any corresponding embodiment thereof.

[0023] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions, wherein the computer instructions are used to enable a computer to execute the request link tracking method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 is a flow chart of a method for tracing a request link according to an embodiment of the present invention;

[0026] Figure 2 is a flowchart of another request link tracking method according to an embodiment of the present invention;

[0027] Figure 3 is a schematic diagram of a log tree structure according to an embodiment of the present invention;

[0028] Figure 4 is a flowchart of another request link tracking method according to an embodiment of the present invention;

[0029] Figure 5 is a structural block diagram of a tracking device for a request link according to an embodiment of the present invention;

[0030] Figure 6 It is a schematic diagram of the hardware structure of the server according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0032] As the scale of distributed systems (such as cloud management platforms) continues to expand, the cost and potential losses of fault diagnosis are also increasing. First, even a small error may cause the unavailability of multiple services in the production environment. Secondly, as the complexity of the system continues to increase, developers and operation and maintenance personnel need to spend a lot of time and resources to accurately locate the problem, resulting in the waste of valuable human resources. Against this background, the distributed link tracking system came into being. It can record the end-to-end details of the request execution, thus providing an effective solution for troubleshooting faults in distributed systems.

[0033] However, in some earlier designed distributed systems, despite being equipped with powerful logging systems, local link tracking systems have not been integrated. When trying to integrate link tracking systems into these systems, the following problems may be encountered:

[0034] First, in order to perform distributed tracing, developers need to manually add tracing points in the code, including creating and closing tracing points, setting labels, associating contexts, etc. This process is time-consuming, labor-intensive, error-prone, and very difficult to maintain.

[0035] Secondly, distributed systems usually use asynchronous programming models (such as coroutines and asynchronous task libraries such as Celery) to improve concurrency and response speed. However, existing link tracking clients often do not support these asynchronous programming models well, making the configuration and deployment of the original system more complicated. For example, for projects based on Python 3.6 and earlier, Jaeger-client only provides AsyncioScopeManager as an asynchronous scope manager, but when using it, the function needs to be modified and the upper-level span needs to be manually passed in, which increases the workload of tracking points and the difficulty of deployment.

[0036] Thirdly, common distributed link tracing systems use two strategies: head sampling and tail sampling. Head sampling decides whether to trace the entire request chain at the beginning of the request, which may result in missing fault tracing data. Tail sampling sends all requests to the tracing system and then decides whether to trace, which may cause problems such as excessive network bandwidth and memory consumption.

[0037] Finally, link tracking clients usually upload tracking data based on gRPC or HTTP protocols, but these protocols themselves do not provide a persistent storage mechanism. If an interruption occurs during the upload process, the tracking data that was not successfully transmitted will be lost, affecting the integrity and accuracy of the data.

[0038] In view of this, the technical solution of the present invention collects incremental logs of multiple microservice instances, builds a tracking link, analyzes the call relationship and request status between services, and samples the link based on the status information to obtain the link tracking result, thereby being able to effectively track the interaction and call hierarchy between multiple microservice instances, help analyze the flow status, performance bottlenecks or fault points of requests, and improve the observability and troubleshooting efficiency of the system.

[0039] According to an embodiment of the present invention, an embodiment of a method for tracing a request link is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0040] In this embodiment, a request link tracking method is provided, which can be used for a server. Figure 1 is a flow chart of a method for tracking a request link according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0041] Step S101, obtaining incremental logs generated when multiple microservice instances execute target requests.

[0042] Multiple microservice instances refer to the situation in a microservice architecture where requests for the same business or functional target may be processed jointly by different microservice instances (i.e., service instances running on different servers or containers). For example, a user request may call multiple microservice instances in sequence, each responsible for a different functional module. Incremental logs refer to log records generated by microservice instances during request processing. These logs reflect the execution process, timestamp, request status, and other information of the request, and are incremental information recorded step by step in chronological order. Specifically, relevant incremental logs can be extracted from each microservice instance through log collection and aggregation systems (such as ELK, Prometheus, etc.) to ensure that the execution path of the same request in multiple service instances can be traced, and these log data can be aggregated for subsequent analysis.

[0043] Step S102: Generate a tracing link based on the attribute information of the multiple incremental logs, where the tracing link is used to characterize the call hierarchical relationship between the multiple microservice instances.

[0044] The attribute information of multiple incremental logs refers to the key data contained in the log entries generated by each microservice instance, such as request ID (uniquely identifies each request), timestamp (records the time point of request processing), service name (identifies the microservice instance through which the request passes), call status (such as success, failure), and parent-child relationship identifier (marks the hierarchical relationship in the request call chain). The tracking link refers to a call chain constructed through the attribute information in these logs that represents the flow of requests between multiple microservice instances, showing the call relationship and hierarchy between each microservice instance. Specifically, based on the attribute information of multiple incremental logs, a tracking link can be constructed by tracking the ID and timestamp of the same request and combining the log records of each microservice instance, that is, through the request flow path and hierarchical relationship, the call order and dependency relationship of the request between different microservice instances are represented, thereby forming a complete service call chain.

[0045] Step S103: Determine the status information of the tracking link based on the request status information corresponding to the incremental log included in the attribute information.

[0046] The request status information corresponding to the incremental log refers to the status information recorded when each microservice instance processes the request, usually including the processing result of the request (such as success, failure, timeout), error code, exception information, etc., reflecting the processing status of each microservice instance for the request. The status information of the tracking link refers to the overall status of the entire request link determined according to the status of each microservice instance, which can be divided into normal links and abnormal links. If all the microservice instances involved successfully process the request, the link is a normal link; if a microservice instance fails to process or an exception occurs, the link status is an abnormal link. Specifically, by analyzing the request status information in the incremental log, the processing result of each microservice instance can be determined, and the status of the entire tracking link can be determined based on this. If any service processing fails in the link, it is marked as an abnormal link, otherwise it is a normal link.

[0047] Step S104: sampling the tracking link based on the state information to obtain a link tracking result.

[0048] Link tracking results refer to the sampling and analysis results based on the status information of the tracked link (such as normal or abnormal). Specifically, different sampling strategies can be used to sample according to the different states of the tracked link to obtain sampling results. The sampling results can reflect the key characteristics and potential problems of the requested link, thereby generating accurate link tracking results and providing a basis for subsequent performance optimization and troubleshooting.

[0049] The request link tracing method provided by the embodiment of the present invention can comprehensively and accurately characterize the call relationship and hierarchy between microservices by obtaining the incremental logs generated by multiple microservice instances and generating tracing links based on their attribute information, thereby helping the system to effectively monitor and analyze the execution status of the request link. By combining the status information of the request, the link can be finely tracked and sampled, thereby quickly locating problems, optimizing performance, and improving the maintainability and reliability of the microservice architecture.

[0050] In this embodiment, a request link tracking method is provided, which can be used for a server. Figure 2 is a flow chart of a method for tracking a request link according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0051] Step S201, obtain the incremental logs generated when multiple microservice instances execute target requests. For details, see Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0052] Step S202: Generate a tracing link based on the attribute information of the multiple incremental logs, where the tracing link is used to characterize the call hierarchical relationship between the multiple microservice instances.

[0053] Specifically, the attribute information also includes identification information corresponding to the incremental log. The above step S202 includes:

[0054] Step S2021: construct a log structure corresponding to the target request according to the request status information and the identification information, where the log structure includes a plurality of parent nodes and child nodes.

[0055] The identification information corresponding to the incremental log refers to the field in the incremental log that is used to uniquely identify a request or transaction. The log structure corresponding to the target request is a structure that records all log information related to the target request, including the request status information, identification information, and the hierarchical relationship between multiple parent nodes and child nodes. For example, it can be a log tree structure (such as Figure 3 ). Multiple parent nodes and child nodes refer to the call hierarchy of the target request between multiple microservice instances, where the parent node represents the microservice instance that initiates the request and the child node represents the called microservice instance. Specifically, based on the request status information and identification information, the log structure is constructed by identifying the parent-child node relationship, thereby reflecting the call path of the request between different services.

[0056] In some optional implementations, the above step S2021 includes:

[0057] Step a1: Identify the request status of each incremental log based on the request status information.

[0058] The request status of each incremental log refers to the different stages or status information related to the request processing process, such as the initiation, processing, completion, failure or timeout of the request. These states can reflect the progress and status of the request flowing between microservices and help track the execution process of the request. Specifically, the specific stage of the request represented by the log is determined by analyzing the request status field in the log. For example, the state of the request corresponding to the log during the processing can be determined by looking at the status identifier (tag) of each log record. Among them, the state of the request during the processing process includes three categories: request start (start), processing (processing) and request end, and the request end includes two types: request completion (end) and request error (error).

[0059] Step a2: determining a target incremental log associated with the target request from among the multiple incremental logs based on the identification information.

[0060] Target incremental logs refer to log records directly related to the target request. These logs contain detailed information about the execution of the request in different microservice instances, and match them with the target request through identification information. Specifically, by analyzing the globally unique identifier (traceId) in each incremental log, log records belonging to the same target request are associated. The target incremental log can be matched with the unique identifier of the target request to filter out log items related to the request, thereby determining which logs are incremental logs generated by the target request in different microservice instances.

[0061] Step a3, matching multiple target incremental logs according to the request status to obtain log matching results.

[0062] The log matching result refers to determining which logs belong to different processing stages of the same target request by comparing the request status information in multiple target incremental logs, and judging the flow and end of the request based on the status identifier in the log. Specifically, according to the request status information recorded in each incremental log, determine which processing stage the request represented by the log is in (for example, start, processing, end, timeout, etc.). Based on the identification information of each incremental log, find the logs associated with the target request, which belong to the processing records of the same link or multiple microservice instances. According to the different states of the request, match and sort each incremental log to form a complete request processing link to ensure the consistency of the log's timing and state. Through these matching processes, a log matching result is finally formed, indicating whether the processing flow of the target request between different microservice instances is completed normally or an exception occurs.

[0063] Step a4: If it is determined based on the log matching result that the target request ends or times out, a log structure is generated using the request status information.

[0064] Check the request status information of multiple incremental logs to determine whether the target request has been processed or whether it has not been completed due to timeout. If the target request has ended or timed out, a log structure will be built based on this status information, including the parent and child node relationships of the log, indicating the call hierarchy and processing flow of the request between multiple microservice instances.

[0065] Specifically, if the incremental log obtained is a request start log, the traceId and generation time of the log are added to the queue to be matched, waiting for the end log associated with the same traceId to be matched. In the matching queue, the log monitoring device will scan the log in a loop to determine whether the log exceeds the set timeout. If a timeout occurs, the log monitoring device will remove the log from the matching queue, add a timeout tag to the relevant log, and then trigger the log tree construction task. If no timeout occurs, the log will continue to remain in the queue waiting for matching.

[0066] If the incremental log obtained is a request end log, the traceId of the log is matched with the records in the queue to be matched. If the match is successful, it means that the request has ended and the log tree construction task will be triggered. If the match fails, it means that the request has timed out and the corresponding request start log has triggered the timeout process.

[0067] If the acquired incremental log is a log in progress, no processing is performed.

[0068] In the above implementation, by accurately using the request status information and identification information, the incremental logs related to the target request can be effectively identified, and the logs can be matched and analyzed according to the request status, ensuring that the life cycle of the target request can be accurately tracked and whether the request has ended or timed out can be identified in a timely manner, thereby helping the system to handle abnormal situations in the request process more efficiently. With the assistance of log matching results, when problems occur, the complete log structure can be quickly located and generated, providing the system with stronger fault diagnosis capabilities and performance optimization support, thereby improving overall reliability and response speed.

[0069] In some optional implementations, the above step a4 includes:

[0070] Step a41: If it is determined based on the log matching result that the target request ends or times out, the first timestamp corresponding to each target incremental log is obtained.

[0071] The first timestamp refers to the timestamp when each incremental log record is generated, which is used to identify the specific time point when the log is generated. The timestamp field associated with the incremental log is extracted from the target incremental log, which records the moment when the log is generated. Specifically, when processing the target request, the incremental log generated by each microservice instance will contain a timestamp, indicating the start of the request processing or the occurrence time of the key event. Through these timestamps, the order of the logs and the time relationship of the processing can be accurately reflected, thereby helping to determine the node relationship between each target incremental log and further generate an accurate log structure.

[0072] Step a42, determining the node relationship between each target incremental log based on each first timestamp and request status information.

[0073] The node relationship between each target incremental log refers to the call order and dependency relationship between different microservice instances in the request link, which reflects the hierarchy and front-end relationship between each incremental log. Specifically, the generation time of each log record is obtained through the timestamp (i.e., the first timestamp) in the incremental log, and these logs are arranged in chronological order. Combined with the request status information (such as whether the request has been completed, timed out, etc.), the parent-child relationship and call order between logs are determined according to the order of timestamps and status changes. For example, if the timestamp of a log is earlier than that of another log and the status indicates that the request has not ended, then it may be the parent node of the latter.

[0074] Step a43, generating a log structure based on multiple node relationships.

[0075] Using the timestamp and request status information of each incremental log, we determine the hierarchy and dependencies between them (for example, which logs are parent nodes and which are child nodes). Based on these node relationships, we build a tree or graph log structure, where each node represents an incremental log, and the connection between parent and child nodes represents the call sequence and hierarchy between different microservice instances. Ultimately, the generated log structure can accurately reflect the call chain of requests between multiple microservice instances.

[0076] In the above implementation, by judging the node relationship based on timestamps, the execution process of requests between different microservices and their dependency order can be clearly reflected, avoiding the time confusion or mismatch problems that may occur in traditional methods. This refined log structure helps to better track the status changes of requests, quickly locate performance bottlenecks or faulty nodes, improve the efficiency of system monitoring, debugging and optimization, and enhance the reliability and maintainability of the system.

[0077] Step S2022: Generate multiple target span objects based on the node relationship between the parent node and the child node.

[0078] The node relationship between the parent node and the child node refers to the calling hierarchical relationship of the request between different microservice instances. The target span object (span) refers to the object that represents the time span and execution status of the target request propagating between microservices. It encapsulates the life cycle of the target request, the calling information of the upstream and downstream services and their related performance data (such as response time, etc.). Specifically, through the hierarchical calling relationship between the parent node and the child node, multiple span objects are constructed. Each span object represents an independent process of a service call, and then the calling relationship and time series of the entire request link are established through these span objects.

[0079] Optionally, when generating multiple target span objects based on the node relationship between the parent node and the child node in the log structure, a hook-based coroutine enhancement method can be used to further improve the accuracy and detail of the log. Specifically, the hook method is used to generate logs of coroutine calls and return information, especially in projects that use CPython as the Python compiler, by calling the PyEval_SetProfile method to execute the callback function, the context information of the coroutine is captured and mapped to the span, thereby providing more fine-grained context data for building the log structure and the parent-child relationship of the span object.

[0080] In some optional implementations, the above step S2022 includes:

[0081] Step b1, identifying the first log corresponding to each parent node and the second log corresponding to each child node in the log structure.

[0082] The first log corresponding to each parent node refers to the log at a higher level (upstream) in the request link. These logs represent the previous microservice instances or operations that initiated calls to downstream microservices. The second log corresponding to each child node refers to the log at a lower level (downstream), indicating that the microservice instance received the parent node request, performed the relevant operations and returned the results. Specifically, the first log corresponding to each parent node and the second log corresponding to each child node in the log structure are identified by analyzing the request status information and identification information in the incremental log.

[0083] Step b2: Map each first log into a first span object, and map each second log into a second span object.

[0084] The target span object includes a first span object and a second span object. Among them, the first span object refers to the operation span represented by the log corresponding to the parent node in the request link, which represents the time span of the microservice instance or operation that initiated the request. The second span object corresponds to the log of the child node, which represents the operation span of the microservice instance that receives the parent node request and processes the request. Specifically, each first log is mapped to a first span object by extracting information such as the start time, end time and request status of the parent node log to form a span object. Similarly, mapping the second log to the second span object is to create a corresponding span object based on the relevant information of the child node log (such as timestamp and request status).

[0085] For example, according to the definition of span in OpenTracing semantic standard, each span should contain the following status: operation name, start time, finish time, span tag, span log, and span relationship. The span relationship includes the parent-child (ChildOf) and follow (FollowsFrom) relationship.

[0086] Step b3: construct a parent-child relationship between the first span objects and the first span objects based on the node relationship.

[0087] The parent-child relationship between the first span object and the first span object means that in the log structure, after some span objects (such as parent nodes) initiate a request, other span objects (such as child nodes) respond and process the request, so there is a parent-child hierarchical relationship between them. When constructing these parent-child relationships, the order and hierarchical structure of the parent node and the child node can be determined based on the node relationship. Specifically, the log of the parent node contains the time information of the initiation request, and the log of the child node contains the time information of the response request. By matching these timestamps and request status, the call relationship between the parent span object and the child span object can be clearly mapped to obtain the parent-child relationship between the first span object and the first span object.

[0088] For example, when the log structure is a log tree structure, each edge in the log tree is mapped to a span, and the logs at both ends of the edge (parent node log and child node log) are converted as input parameters, and the following conversion relationship can be obtained: operation name (operation name), start time (start timestamp), end time (finish timestamp), Span label, Span log (log), span relationship (references), TraceId. Among them, the operation name (operationname) refers to the name of the parent node log execution method; the start time (start timestamp) refers to the parent node log timestamp; the end time (finish timestamp) refers to the child node log timestamp; the span label refers to the default addition of the span status label, and supports rewriting the conversion method to obtain from the log. If an exception or error occurs in the parent node log and the child node log, the span status is an error; if a timeout occurs in the parent node log and the child node log, the span status is a timeout; in other cases, the span status is normal; Span log (log) means that it is not provided by default, and supports overriding the conversion method to obtain it from the log; the relationship between spans (references) means that the span corresponding to the parent node of the parent node log of the current span is the parent-child relationship of the current span. Generally, the sibling nodes of the parent node log of the current span are follow-up relationships, and the timestamp of the parent node log determines which span comes first; TraceId refers to the one obtained in the request start log and passed to all spans generated by this conversion. In other words, it supports re-conversion rules for each edge and can generate span objects in a customized manner.

[0089] In the above implementation, by mapping the logs of the parent node and the child node to different span objects respectively, the call dependency and execution order between microservices can be accurately reflected. Building a hierarchical structure of span objects based on the node relationship between parent and child nodes helps to accurately track and analyze the execution trajectory of each request in the system, thereby improving the accuracy and efficiency of problem location. Therefore, this method helps to improve the visualization capability of link tracking, allowing developers to more intuitively understand and optimize the service call process in the system, thereby enhancing the maintainability, reliability and performance of the system.

[0090] Step S2023: Generate a tracking link based on the hierarchical relationship between the multiple target span objects.

[0091] The hierarchical relationship between multiple target span objects refers to the calling order and upstream and downstream relationship between these span objects, reflecting the execution path and time sequence of the target request between different microservices. Specifically, through the hierarchical structure of these span objects, they are combined according to the calling relationship and execution order to form a complete request link, which can reflect the entire flow process of the request from initiation to completion, including the execution status and latency of each service.

[0092] In some optional implementations, the above step S2023 includes:

[0093] Step c1, obtaining a second timestamp corresponding to each first log and a third timestamp corresponding to each second log.

[0094] The second timestamp refers to a time node associated with the first log, and the third timestamp refers to a time node related to the second log. Specifically, by parsing the time information recorded in the incremental log, a corresponding timestamp is extracted from each log entry.

[0095] Step c2: Arrange the first span object and the second span object according to the parent-child relationship, the second timestamp and the third timestamp to generate a tracking link.

[0096] By analyzing the relationship between the parent node and the child node in the log, the hierarchical structure of each span object is clarified, and based on the second and third timestamps of each span object, it is sorted in chronological order to generate a tracing link. Specifically, the parent span object and the child span object are associated by timestamp, and then the sorting order is adjusted according to the parent-child relationship to ensure that the time sequence and call hierarchy of each span object are correct. Finally, all span objects are arranged according to time and hierarchical relationships to generate a complete tracing link.

[0097] In the above implementation, by obtaining the timestamps of the first log and the second log and arranging them in combination with the parent-child relationship, the execution order and time dependency of each service call can be clearly shown. Such a tracking link not only reflects the transmission path of the request in the system, but also provides a detailed time perspective to help developers analyze the delay and performance bottleneck of the request. Through precise time sorting and hierarchical structure, troubleshooting and performance optimization can be carried out more effectively, improving the monitorability, stability and response efficiency of the system.

[0098] Step S203: Determine the status information of the tracking link based on the request status information corresponding to the incremental log included in the attribute information. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0099] Step S204: Sample the traced link based on the state information to obtain the link trace result. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0100] The request link tracing method provided by the embodiment of the present invention further optimizes the generation process of the tracing link by generating multiple target span objects and utilizing their hierarchical relationships, making the link tracing clearer and more detailed. Through this precise structured approach, the call hierarchy and dependency relationships between microservices can be better revealed, the visual monitoring and troubleshooting capabilities of the system can be improved, developers can be helped to quickly identify and locate potential problems, and the performance and stability of the system can be improved.

[0101] In this embodiment, a request link tracking method is provided, which can be used for a server. Figure 4 is a flow chart of a method for tracking a request link according to an embodiment of the present invention. Figure 4 As shown, the process includes the following steps:

[0102] Step S301, obtain the incremental logs generated when multiple microservice instances execute target requests. For details, see Figure 2 Step S201 of the illustrated embodiment will not be described in detail here.

[0103] Step S302: Generate a tracing link based on the attribute information of multiple incremental logs. The tracing link is used to characterize the call hierarchy relationship between multiple microservice instances. Figure 2 Step S202 of the illustrated embodiment will not be described in detail here.

[0104] Step S303: Determine the status information of the tracking link based on the request status information corresponding to the incremental log included in the attribute information. Figure 2 Step S203 of the illustrated embodiment will not be described in detail here.

[0105] Step S304: sampling the tracking link based on the state information to obtain a link tracking result.

[0106] Specifically, the above step S304 includes:

[0107] Step S3041: If it is determined based on the state information that the traced link is a normal link, probability sampling is performed on the normal link to obtain a link trace result of the normal link.

[0108] If the tracking link is determined to be a normal link based on the state information, some normal links can be selected for collection and analysis by probabilistic sampling of the normal links according to preset probability rules, thereby obtaining the link tracking result of the normal link.

[0109] Step S3042: If the traced link is determined to be an abnormal link based on the state information, all samples are taken for the abnormal link to obtain a link trace result of the abnormal link.

[0110] If the tracking link is determined to be an abnormal link based on the status information, all samples of the abnormal link should be taken, that is, each link in the link and the processing process of each request should be recorded and analyzed in detail to obtain a complete link tracking result.

[0111] Step S3043: When it is determined that the traced link is an abnormal link, it is detected whether the abnormal link has dependent services and upper layer services.

[0112] When the trace link is determined to be an abnormal link, the link's attribute information, especially the dependency relationship and service call hierarchy in the request, can be checked to detect whether there are dependent services and upper-layer services. Specifically, each microservice node in the abnormal link is analyzed to find out whether there are other services that are dependent on the link (i.e., downstream services), and whether the link is initiated or dependent on an upstream service.

[0113] Step S3044: When it is determined that the abnormal link has a dependent service, the dependent service is notified so that the dependent service determines whether to report the corresponding first link data.

[0114] If the abnormal link involves dependent services, notifications will be sent to these downstream services to inform them whether they need to report their own link data (first link data). Dependent services need to determine whether to report this data to the link tracking system based on their own status in order to fully track the flow of abnormal requests.

[0115] Step S3045: When it is determined that there is an upper layer service for the abnormal link, the upper layer service is notified so that the upper layer service determines whether to report the corresponding second link data.

[0116] If the abnormal link involves the upper-layer service, these upstream services will be notified to remind them whether they need to report their own link data (second link data). By reporting the link data of the upper-layer service, it can help to fully track the entire process of the request from initiation to final processing.

[0117] For example, if the tracking link is an abnormal link, the backtracking reporting mechanism is triggered, the local tracking link is uploaded to the collector, and then a reporting request is sent to all dependent services and to the upper-layer service. Then, determine whether the tracking link has dependent subsequent services. If not, it means that the request has ended. At this time, obtain the sampling rate from the configuration center to determine whether the tracking link needs to be reported, and send the result to all dependent services and upper-layer services. If there is a dependent subsequent service, wait for the subsequent service to notify whether to report, and pass the result to all dependent services and upper-layer services.

[0118] The request link tracking method provided by the embodiment of the present invention adopts a probabilistic sampling method when the link is judged to be normal, which can effectively reduce the amount of data processing and improve the sampling efficiency; and for abnormal links, comprehensive sampling is performed to ensure that each abnormal situation is fully analyzed and recorded. Through this flexible sampling strategy, the system can optimize performance and resource consumption while ensuring the comprehensiveness of data coverage, thereby improving the accuracy and response speed of troubleshooting. Accurate fault tracing and data reporting can be achieved by dynamically detecting and notifying the dependent services and upper-layer services of the abnormal link. By identifying and notifying the dependent services and upper-layer services, it can ensure that the relevant services obtain abnormal information in a timely manner and decide whether to report the link data. This collaborative processing method improves the diagnostic efficiency of abnormal links and ensures that all potential influencing factors can be tracked and reported, so as to comprehensively and quickly locate the source of the problem.

[0119] In this embodiment, a tracking device for requesting a link is also provided, and the device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0120] This embodiment provides a tracking device for requesting a link, such as Figure 5 As shown, including:

[0121] The acquisition module 401 is used to obtain the incremental logs generated when multiple microservice instances execute target requests;

[0122] A generation module 402 is used to generate a tracing link based on the attribute information of the plurality of incremental logs, where the tracing link is used to characterize the call hierarchical relationship between the plurality of microservice instances;

[0123] A determination module 403, configured to determine the state information of the tracking link based on the request state information corresponding to the incremental log included in the attribute information;

[0124] The tracking module 404 is used to sample the tracking link based on the state information to obtain the link tracking result.

[0125] In some optional implementations, the generating module 402 includes:

[0126] A construction submodule is used to construct a log structure corresponding to the target request according to the request status information and the identification information, and the log structure includes multiple parent nodes and child nodes;

[0127] A first generating submodule, used for generating a plurality of target span objects based on a node relationship between a parent node and a child node;

[0128] The second generating submodule is used to generate a tracking link based on the hierarchical relationship between multiple target span objects.

[0129] In some optional embodiments, the construction submodule includes:

[0130] A first identification unit, configured to identify a request status of each incremental log based on the request status information;

[0131] A first determining unit, configured to determine, based on the identification information, a target incremental log associated with the target request from among the plurality of incremental logs;

[0132] A matching unit, used for matching multiple target incremental logs according to the request status to obtain a log matching result;

[0133] The first generating unit is configured to generate a log structure using the request status information if it is determined based on the log matching result that the target request is terminated or a timeout occurs.

[0134] In some optional implementations, the generating unit includes:

[0135] A first acquisition unit, used to acquire a first timestamp corresponding to each target incremental log;

[0136] A second determining unit, configured to determine a node relationship between each target incremental log based on each first timestamp and the request status information;

[0137] The second generating unit is used to generate a log structure based on multiple node relationships.

[0138] In some optional implementations, the first generation submodule includes:

[0139] A second identification unit, used to identify a first log corresponding to each parent node and a second log corresponding to each child node in the log structure;

[0140] A mapping unit, used to map each first log into a first span object, and map each second log into a second span object;

[0141] A construction unit is used to construct a parent-child relationship between the first span object and the second span object based on the node relationship.

[0142] In some optional implementations, the second generation submodule includes:

[0143] A second acquisition unit, used to acquire a second timestamp corresponding to each first log and a third timestamp corresponding to each second log;

[0144] The third generating unit is used to arrange the first span object and the second span object according to the parent-child relationship, the second timestamp and the third timestamp to generate a tracking link.

[0145] In some optional implementations, the tracking module 404 includes:

[0146] A first tracking submodule is used for, if it is determined based on the state information that the tracked link is a normal link, to perform probability sampling on the normal link to obtain a link tracking result of the normal link;

[0147] The second tracking submodule is used to perform sampling on all abnormal links to obtain link tracking results of the abnormal links if it is determined that the tracked link is an abnormal link based on the state information.

[0148] In some optional implementations, the tracking module 404 further includes:

[0149] A detection submodule, for detecting whether the abnormal link has dependent services and upper layer services when the traced link is determined to be an abnormal link;

[0150] A first notification submodule is used to notify the dependent service when it is determined that the abnormal link has a dependent service, so that the dependent service determines whether to report the corresponding first link data;

[0151] The second notification submodule is used to notify the upper layer service when it is determined that there is an upper layer service in the abnormal link, so that the upper layer service determines whether to report the corresponding second link data.

[0152] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0153] The tracking device of the request link in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0154] The request link tracking device provided by the embodiment of the present invention can comprehensively and accurately characterize the call relationship and hierarchy between microservices by obtaining the incremental logs generated by multiple microservice instances and generating tracking links based on their attribute information, thereby helping the system to effectively monitor and analyze the execution status of the request link. By combining the status information of the request, the link can be tracked and sampled in a refined manner, thereby quickly locating problems, optimizing performance, and improving the maintainability and reliability of the microservice architecture.

[0155] The embodiment of the present invention also provides a server having the above Figure 5 The tracking device of the request link is shown.

[0156] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a server provided by an optional embodiment of the present invention, such as Figure 6 As shown, the server includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the server, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple servers can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0157] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0158] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0159] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created according to the use of the server, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the server via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0160] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0161] The server further comprises a communication interface 30 for the server to communicate with other devices or a communication network.

[0162] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0163] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0164] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for tracking a request link, characterized in that: The method comprises: Get the incremental logs generated when multiple microservice instances execute target requests; Based on the attribute information of the plurality of incremental logs, a tracing link is generated, where the tracing link is used to characterize the call hierarchical relationship between the plurality of microservice instances; Determine the status information of the tracking link based on the request status information corresponding to the incremental log included in the attribute information; The tracking link is sampled based on the state information to obtain a link tracking result.

2. The method according to claim 1, characterized in that: The attribute information also includes identification information corresponding to the incremental log; and the generating of a tracking link based on the attribute information of the plurality of incremental logs includes: Constructing a log structure corresponding to the target request according to the request status information and the identification information, wherein the log structure includes a plurality of parent nodes and child nodes; Based on the node relationship between the parent node and the child node, generate a plurality of target span objects; The tracking link is generated based on the hierarchical relationship between the multiple target span objects.

3. The method according to claim 2, characterized in that The step of constructing a log structure corresponding to the target request according to the request status information and the identification information includes: Based on the request status information, identifying the request status of each of the incremental logs; Based on the identification information, determining a target incremental log associated with the target request among the multiple incremental logs; Matching the plurality of target incremental logs according to the request status to obtain a log matching result; If it is determined based on the log matching result that the target request is ended or times out, the log structure is generated using the request status information.

4. The method according to claim 3, characterized in that: The step of generating the log structure by using the request status information includes: Obtaining a first timestamp corresponding to each of the target incremental logs; Determine, based on each of the first timestamps and the request status information, a node relationship between each of the target incremental logs; The log structure is generated based on a plurality of the node relationships.

5. The method according to claim 2, characterized in that: Based on the node relationship between the parent node and the child node, a plurality of target span objects are generated, including: Identify a first log corresponding to each of the parent nodes and a second log corresponding to each of the child nodes in the log structure; Mapping each of the first logs into a first span object, and mapping each of the second logs into a second span object; A parent-child relationship between the first span object and the first span object is constructed based on the node relationship.

6. The method according to claim 5, characterized in that Generating the tracking link based on the hierarchical relationship between the multiple target span objects includes: Obtaining a second timestamp corresponding to each of the first logs and a third timestamp corresponding to each of the second logs; The first span object and the second span object are arranged according to the parent-child relationship, the second timestamp and the third timestamp to generate the tracking link.

7. The method according to claim 1, characterized in that The sampling of the tracking link based on the state information to obtain a link tracking result includes: If it is determined based on the state information that the traced link is a normal link, probability sampling is performed on the normal link to obtain a link trace result of the normal link; If it is determined based on the state information that the traced link is an abnormal link, all samples are taken for the abnormal link to obtain a link trace result of the abnormal link.

8. The method according to claim 7, characterized in that Also includes: When it is determined that the traced link is an abnormal link, detecting whether the abnormal link has dependent services and upper layer services; When it is determined that the abnormal link has the dependent service, notifying the dependent service so that the dependent service determines whether to report the corresponding first link data; When it is determined that the abnormal link exists in the upper layer service, the upper layer service is notified so that the upper layer service determines whether to report the corresponding second link data.

9. A tracking device for a request link, characterized in that: The device comprises: The acquisition module is used to obtain the incremental logs generated when multiple microservice instances execute target requests; A generation module, used to generate a tracing link based on the attribute information of the plurality of incremental logs, wherein the tracing link is used to characterize the call hierarchical relationship between the plurality of microservice instances; A determination module, configured to determine the status information of the tracking link based on the request status information corresponding to the incremental log included in the attribute information; The tracking module is used to sample the tracking link based on the state information to obtain a link tracking result.

10. A server, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the request link tracking method according to any one of claims 1 to 8 by executing the computer instructions.

Images