Method for tracing business call chain based on Opentracing link

Abstract

The present invention discloses a method for tracing a business call chain based on an Opentracing link, comprising: S1: deploying a microservice Opentrace probe and a log collection probe; S2: initializing the Opentrace probe and the log collection probe; S3: the microservice processes a business request, and the Opentrace probe collects call chain information; S4: the microservice prints a business processing log containing call chain information, and the log collection probe collects the business processing log and forwards it to a log analysis system; S5: the log analysis system performs standardized processing on unstructured business data; S6: generating a link topology by concatenating log information according to the call chain information, and correlating business information in the log according to the link topology. The present invention adds call chain information that complies with the Opentracing specification to each service log of the microservice architecture, concatenates discrete log events therein, and realizes the businessization of the call chain information.

Description

Technical Field

[0001] The present invention relates to a method for tracing a service call chain, and in particular to a method for tracing a service call chain based on an Opentracing link. Background Art

[0002] Java is a widely used computer programming language that boasts cross-platform, object-oriented, and generic programming features, making it widely used in enterprise-level web application development and mobile application development. The Java programming language's style is very similar to that of the C++ language. Inheriting the core object-oriented technology of the C++ language, Java discards the error-prone pointers in C++ and replaces them with references. It also removes the original C++ and operator overloading, as well as the multiple inheritance feature, replacing it with interfaces, and adds garbage collector functionality. Java differs from typical compiled or interpreted languages. It first compiles the source code into bytecode, then relies on virtual machines on various platforms to interpret and execute the bytecode, thus achieving the cross-platform "write once, run anywhere" feature. The Java programming language is simple, object-oriented, distributed, interpreted, robust, secure, system-independent, portable, high-performance, multi-threaded, and dynamic.

[0003] Ivory is a big data log analysis system that provides a one-stop solution for the collection, cleaning, conversion, storage, search, analysis, alerting, and visualization of metrics and log data. It uses machine learning to quickly predict and locate faults, monitor business health in real time, and improve the level of intelligent business operations.

[0004] Opentracing is an open, distributed link tracing framework. OpenTracing provides a platform-independent and vendor-independent API, allowing developers to easily add (or replace) tracing system implementations.

[0005] When troubleshooting multi-service systems in a microservices architecture, it's difficult to correlate specific business data and determine the scope of business impact, creating challenges in business analysis. Therefore, it's necessary to provide a method for tracing business call chains based on Opentracing. This method can easily search and collect statistics on various business data within the call chain, providing data support for business function usage analysis and optimization, thereby optimizing business process orchestration. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a method for tracing business call chains based on Opentracing links, so as to solve the problem that existing microservices cannot be associated with specific business data.

[0007] The technical solution adopted by the present invention to solve the above technical problems is to provide a method for tracing business call chains based on Opentracing links, including the following steps: S1: deploying microservice Opentrace probes and deploying log collection probes; S2: starting microservices, initializing Opentrace probes and log collection probes; S3: microservices processing business requests, and Opentrace probes collecting call chain information; S4: microservices printing business processing logs containing call chain information, and log collection probes collecting business processing logs and forwarding them to log analysis systems; S5: the log analysis system standardizes unstructured business data to obtain structured business data; S6: generating a link topology by concatenating log information according to the call chain information, and correlating business information in the logs according to the link topology to form a single business call chain.

[0008] Furthermore, the method further includes step S7: performing a global business profile based on multiple single business call chain data to obtain a global business call chain; counting the branches of the global business call chain and calculating the critical path.

[0009] Furthermore, step S3 includes: the microservice component receives the user business request, and the Opentrace probe writes the call chain information that complies with the Opentracing specification into the log as necessary information of the microservice log, and the call chain information includes at least link information traceId, tracing segment information segmentId and span information spanId.

[0010] Furthermore, the deployment of the microservice Opentrace probe in step S1 includes: implanting a Java script in the microservice component, non-invasively implanting the Opentrace probe through the Java proxy mechanism, and when the Opentrace probe detects the business operation of the microservice component, adding call chain information through the extended Java log framework in accordance with the Opentracing specification.

[0011] Furthermore, the Java logging framework includes log4j, logback or log4j2, and extending the Java logging framework includes formulating the necessary log information specified in the microservice log specification. The necessary log information includes: log output timestamp timestamp, tracing segment information segmentId, parent span information parent_spanId, span information spanId, thread name thread, log level level, link information traceId, microservice hosting host name / IP hostName, microservice IP and port port, microservice name serverName, processing start timestamp startTime, method name method, method input parameter input, method output parameter output, method time times and custom business log message.

[0012] Furthermore, step S4 includes: the log collection probe cuts the business processing log into complete discrete log events and sends them to the log analysis system; the log collection probe detects cross-service RPC remote calls, message queues, caches, and database operation interfaces, and adds the log call chain ID, parent span ID, current span ID and business field information by default, so that the discrete log events recorded during the microservice operation process contain cross-service link information, thereby forming a link-level log across multiple microservices; the log analysis system collects the link-level logs recorded by the microservice as the original analysis data for tracing the business call chain.

[0013] Furthermore, the step S5 includes: the log analysis system performs regularization processing on the unstructured discrete log event data, extracts key information and structures the business data.

[0014] Furthermore, the step S6 specifically includes: reassembling the structured log data according to the link information traceId, the tracking segment information segmentId and the span information spanId in the log to construct a link topology, and by searching and analyzing the log data within the link topology range, associating the business data with the link topology to form a single business call chain.

[0015] Furthermore, the step S7 includes: S71: based on the call chain information, superimpose multiple single business call chain data, perform a global business portrait and construct a global business call chain; S72: assign a weight to each node in the global business call chain; S73: starting from the root span at the beginning of the business, repeatedly depict the call chain of the business according to the call situation, count the number of times each link in the global business call chain passes, and calculate the key call path of the business based on the weight of the node and the number of times the link passes.

[0016] Furthermore, step S72 assigns a static weight and a dynamic weight to each node in the global business call chain. The static weight is a node weight for determining whether the node is available, which is calculated based on the asset information recorded and stored on the node through a health algorithm based on asset monitoring indicators; the dynamic weight is a traffic weight for determining the direction of business traffic flow, which is calculated based on a smoothing algorithm based on the number of inter-node calls within a specified time window and the in-degree and out-degree information of the business topology according to link-level log statistics.

[0017] Compared with the existing technology, the present invention has the following beneficial effects: the method of tracing the business call chain based on Opentracing link provided by the present invention extends the standard protocol of OpenTracing, adds the link information (traceId), tracing segment information (segmentId) and span information (spanId) to the service log through a probe and outputs it to the log analysis system. The log information is reassembled according to the link information and the link call process is displayed; statistics are aggregated from a large number of single call chain information to perform a global portrait of the business, obtain business branches and key path information in various scenarios, and provide a basis for business optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of a method for tracing a service call chain based on the Opentracing link in an embodiment of the present invention;

[0019] Figure 2 This is a data flow diagram of a method for tracing a service call chain based on Opentracing links in an embodiment of the present invention;

[0020] Figure 3 This is a schematic diagram of a single call chain in an embodiment of the present invention;

[0021] Figure 4 This is a schematic diagram of a global service call chain in an embodiment of the present invention; DETAILED DESCRIPTION

[0022] The present invention will be further described below with reference to the accompanying drawings and examples.

[0023] Basic concept description:

[0024] Link trace: The execution workflow of a complete request, transaction, or process in a distributed system.

[0025] A tracing segment is a collection of all spans in a link trace within a process. It is used to mark all operations performed by a single application in a distributed application architecture. It is an innovative concept based on Opentracing.

[0026] Span: An operation named by its execution method that forms part of a workflow. Spans have key:value tags and fine-grained, timestamped structured logs (call chain logs) attached to specific span instances. Spans can have the following logical relationships: Parent-child (ChildOf) logical relationship: For example, if spanB and spanC are children of spanA, spanB and spanC can be executed sequentially or in parallel. FollowsFrom logical relationship: The follow-from relationship is used to identify the execution order between sibling spans.

[0027] Span context: Carries tracing information for the workflow, including information as it passes from service to service over a network or message bus. The span context contains the traceId, the spanId or parent-spanId / child-spanId, and any other data the tracing system needs to propagate to downstream services.

[0028] The link trace in OpenTracing is implicitly defined by span. The link trace can be considered as a directed acyclic graph of spans, where the edges between spans are called references.

[0029] Figure 1 This is a flow chart of a method for tracing a service call chain based on the Opentracing link in an embodiment of the present invention; Figure 2 This is a data flow diagram of the method for tracing the service call chain based on the Opentracing link in an embodiment of the present invention.

[0030] See Figure 1 and Figure 2 The method for tracing a service call chain based on the Opentracing link in an embodiment of the present invention includes the following steps:

[0031] Step S1: Deploy the microservice Opentrace probe and the log collection probe;

[0032] Deploying a microservice Opentrace probe involves embedding a Java script in the microservice component and non-invasively embedding the Opentrace probe through the Java proxy mechanism. When the Opentrace probe detects a business operation in the microservice component, it adds call chain information through an extended Java logging framework in accordance with the Opentracing specification. Java logging frameworks include log4j, logback, or log4j2. Extending the Java logging framework involves developing the necessary log information specified by the microservice logging specification. This necessary log information includes: log output timestamp, tracing segment information (segmentId), parent span information (parent_spanId), span information (spanId), thread name (thread), log level (level), link information (traceId), microservice hosting host name / IP (hostName), microservice IP and port (port), microservice name (serverName), processing start timestamp (startTime), method name (method), method input parameters (input), method output parameters (output), method duration (times), and a custom business log message. Details are shown in Table 1 below.

[0033] Table 1 Log information table

[0034]

[0035] Step S2: Start the microservice and initialize the Opentrace probe and log collection probe;

[0036] Step S3: The microservice processes the business request, and the Opentrace probe collects the call chain information; the microservice component receives the user business request, and the Opentrace probe writes the call chain information that complies with the Opentracing specification into the log as necessary information of the microservice log. The call chain information includes at least the link information traceId, the tracing segment information segmentId, and the span information spanId.

[0037] Step S4: The microservice prints the business processing log containing the call chain information. The log collection probe collects the business processing log and forwards it to the log analysis system. The log collection probe cuts the business processing log into complete discrete log events and sends them to the log analysis system.

[0038] Log collection probes use bytecode enhancement technology to detect cross-service RPC remote calls and common middleware operation interfaces such as message queues, caches, and databases. By default, they add log call chain IDs, parent span IDs, current span IDs, business fields, and other related information. This includes cross-service link information in discrete log events recorded during microservice operation, forming link-level logs across multiple microservices (e.g., when a microservice performed an operation, which upstream event triggered the operation, and which downstream operations were affected). The log analysis system collects link-level logs recorded by microservices as raw analysis data for business call chain backtracing.

[0039] Step S5: The log analysis system normalizes the unstructured business data to generate structured business data. The log analysis system performs regularization on the unstructured, discrete log event data, extracting key information and structuring the business data. The collection, cleaning, conversion, storage, search, and analysis of log data are the basic functions of the log analysis system and will not be discussed in detail here.

[0040] Step S6: Generate a link topology based on the call chain information and the log information, and associate the business information in the log based on the link topology to form a single business call chain.

[0041] According to the traceId, segmentId, and spanId in the log, the structured log data is reassembled to build a link topology. By searching and analyzing the log data within the link topology, the business data is associated with the link topology to form a single business call chain. Figure 3 shown.

[0042] It also includes step S7: performing a business global profile based on multiple single business call chain data to obtain a business global call chain; counting the branches of the business global call chain and calculating the critical path.

[0043] S71: Based on the call chain information, multiple single business call chain data are superimposed to create a global business profile and build a global business call chain;

[0044] S72: Assign the following static weight and dynamic weight to each node in the global service call chain:

[0045] 1) Static weight (node ​​weight). Nodes refer to asset information such as various business microservices, message queues, caches, and databases that are recorded in the CMDB asset library and have comprehensive monitoring indicators. Asset monitoring indicators are divided into life and death indicators, gold indicators, and silver indicators. The weight of a single node (a positive number within 100, used to determine whether the node is available) is calculated using a health algorithm based on these three indicators.

[0046] 2) Dynamic weight (traffic weight), which is mainly based on link-level log statistics of the number of inter-node calls within a specified time window and the in-degree and out-degree information of the business topology. A smoothing algorithm is used to generate a positive number within 100 (to determine the direction of business traffic flow);

[0047] S73: Starting from the root span at the beginning of the business, repeatedly depict the business call chain according to the call situation, count the number of times each link of the global business call chain passes through, and calculate the key call path of the business based on the weight of the node and the number of times the link passes through. Figure 4 As shown, it can be seen from the specific embodiment that the nodes of the critical call path are the nodes through which most links pass.

[0048] To sum up, the method for tracing the business call chain based on the Opentracing link in an embodiment of the present invention extends the standard protocol of OpenTracing, adds the link information (traceId), tracing segment information (segmentId) and span information (spanId) to the service log through a probe and outputs it to the log analysis system. The system reassembles the log information according to the link information and displays the link call process; statistics are collected from a large number of single call chain information to perform a global portrait of the business, obtain business branches and key path information in various scenarios, and provide a basis for business optimization.

[0049] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art may make some modifications and improvements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be based on the definition of the claims.

Claims

1. A method for tracing a service call chain based on Opentracing, characterized in that: The steps include: S1: Deploy microservice Opentrace probes and log collection probes; S2: Start the microservice and initialize the Opentrace probe and log collection probe; S3: Microservices process business requests, and Opentrace probes collect call chain information; S4: The microservice prints the business processing log containing call chain information. The log collection probe collects the business processing log and forwards it to the log analysis system. S5: The log analysis system standardizes the unstructured business data to obtain structured business data; S6: Generate a link topology based on the call chain information and the log information. Then, associate the business information in the log with the link topology to form a single business call chain. Deploying the microservice Opentrace probe in step S1 includes: implanting a Java script in the microservice component, implanting the Opentrace probe non-invasively through the Java proxy mechanism, and adding call chain information through the extended Java log framework in accordance with the Opentracing specification when the Opentrace probe detects the business operation of the microservice component; The step S3 includes: the microservice component receives the user business request, and the Opentrace probe writes the call chain information that complies with the Opentracing specification into the log as necessary information of the microservice log, wherein the call chain information includes at least link information traceId, tracing segment information segmentId, and span information spanId; The step S4 includes: the log collection probe cuts the business processing log into complete discrete log events and sends them to the log analysis system; The log collection probe detects cross-service RPC remote calls, message queues, caches, and database operation interfaces, and adds the log call chain ID, parent span ID, current span ID, and business field information by default, so that the discrete log events recorded during the operation of the microservice contain cross-service link information, thereby forming a link-level log spanning multiple microservices; the log analysis system collects the link-level logs recorded by the microservice as the original analysis data for tracing the business call chain.

2. The method for tracing a service call chain based on Opentracing as claimed in claim 1, characterized in that: It also includes step S7: performing a business global profile based on multiple single business call chain data to obtain a business global call chain; counting the branches of the business global call chain and calculating the critical path.

3. The method for tracing a service call chain based on Opentracing as claimed in claim 1, characterized in that: The Java log framework includes log4j, logback or log4j2. Extending the Java log framework includes formulating the necessary log information specified in the microservice log specification. The necessary log information includes: log output timestamp timestamp, tracking segment information segmentId, parent span information parent_spanId, span information spanId, thread name thread, log level level, link information traceId, microservice hosting host name / IP hostName, microservice IP and port port, microservice name serverName, processing start timestamp startTime, method name method, method input parameter input, method output parameter output, method time times and custom business log message.

4. The method for tracing a service call chain based on Opentracing as claimed in claim 1, characterized in that: The step S5 includes: the log analysis system performs regularization processing on the unstructured discrete log event data, extracts key information and structures the business data.

5. The method for tracing a service call chain based on Opentracing as claimed in claim 4, characterized in that: The step S6 specifically includes: reassembling the structured log data according to the link information traceId, the tracking segment information segmentId and the span information spanId in the log to build a link topology, and by searching and analyzing the log data within the link topology range, associating the business data with the link topology to form a single business call chain.

6. The method for tracing a service call chain based on Opentracing as claimed in claim 2, characterized in that: The step S7 comprises: S71: Based on the call chain information, multiple single business call chain data are superimposed to create a global business profile and build a global business call chain; S72: Assign a weight to each node in the global service call chain; S73: Starting from the root span at the beginning of the business, repeatedly depict the call chain of the business according to the call situation, count the number of times each link of the global business call chain passes through, and calculate the key call path of the business based on the weight of the node and the number of times the link passes through.

7. The method for tracing a service call chain based on Opentracing as claimed in claim 6, characterized in that: Step S72 assigns a static weight and a dynamic weight to each node in the global service call chain. The static weight is a node weight used to determine whether a node is available. The static weight is calculated based on the asset information stored on the node using a health algorithm based on asset monitoring indicators. The dynamic weight is a traffic weight used to determine the direction of business traffic flow, which is calculated based on the number of inter-node calls within a specified time window and the in-degree and out-degree information of the business topology based on link-level log statistics, and is obtained using a smoothing algorithm.

Images