A link tracking method, device, system and storage medium

Abstract

The application provides a link tracking method, device, system and storage medium. The method comprises the following steps: in the case that a distributed system is started and a proxy plug-in module is preloaded, if it is detected that a user initiates a service request, a service call session is received by a micro-service node and an executable function of the service call session is determined; wherein the service call session comprises a service call request and a service call response, and the service call session carries a tracking identifier; the executable function is dynamically plugged by the proxy plug-in module to obtain session content in the service call session; and based on the tracking identifier and the session content, a behavior transaction corresponding to the service request of the user is tracked.

Description

Technical Field

[0001] This application relates to the field of data processing technology in financial technology (Fintech), and includes, but is not limited to, a link tracing method, apparatus, system, and storage medium. Background Technology

[0002] With the development of computer computing, more and more technologies are being applied in the financial field, and the traditional financial industry is gradually transforming into financial technology (Fintech). However, due to the security and real-time requirements of the financial industry, Fintech also places higher demands on technology.

[0003] In the fintech and cloud computing environments, an increasing number of business systems, such as financial systems, are adopting distributed microservice architectures. Business processes are completed collaboratively by multiple microservices. For example, a user's purchase of a wealth management product requires assistance from account services, product services, and transaction services. Complete data for a single action, such as... Figure 1 As shown, it contains a complete dataset of behaviors in both the request and response phases.

[0004] In related technologies, solutions based on dynamic instrumentation can obtain data sets of {request headers / response headers, timestamps, request-side feature information / response-side feature information} for both the request and response phases. However, since the request body and response body data are stored in byte stream data structures (e.g., the request body data is stored in an InputStream data structure, and the response body data is stored in an OutputStream data structure), and these byte stream data structures have a `pos` cursor pointer, the `pos` pointer is synchronously shifted while the data interface is reading data. After a complete data read, the `pos` pointer will also move to the end of the data content, and resetting the `pos` pointer position is not supported. Therefore, the code block added to the dynamic instrumentation in related technologies to implement data capture, once it has read the request or response body data, will prevent the application from reading it again, leading to request failure and the inability to obtain complete transaction data. Summary of the Invention

[0005] This application provides a link tracing method, apparatus, system, and storage medium to solve the problem of inability to obtain specific behavioral data, i.e., lack of content integrity, when performing link tracing in related technologies.

[0006] The technical solution of this application embodiment is implemented as follows:

[0007] This application provides a method comprising:

[0008] When a distributed system is started and a proxy instrumentation module is preloaded, if a user initiates a business request, the microservice node receives the service call session and determines the executable function of the service call session; wherein, the service call session includes a service call request and a service call response, and the service call session carries a tracking identifier;

[0009] The executable function is dynamically instrumented using the proxy instrumentation module to obtain the session content in the service call session;

[0010] Based on the tracking identifier and the session content, the behavioral transactions corresponding to the user's business request are tracked.

[0011] This application provides an apparatus, including:

[0012] The receiving module, when the distributed system is started and the proxy instrumentation module is preloaded, receives the service call session through the microservice node if it detects that a user has initiated a business request.

[0013] A determination module is used to determine the executable functions of the service call session; wherein the service call session includes a service call request and a service call response, and the service call session carries a tracking identifier;

[0014] The processing module is used to dynamically instrument the executable function through the proxy instrumentation module to obtain the session content in the service call session;

[0015] The processing module is also used to track the behavioral transactions corresponding to the user's business request based on the tracking identifier and the session content.

[0016] This application provides a distributed system, including:

[0017] Memory, used to store executable instructions;

[0018] The processor implements the above method when executing executable instructions stored in memory.

[0019] This application provides a storage medium storing executable instructions for inducing a processor to execute the above-described method.

[0020] The embodiments of this application have the following beneficial effects:

[0021] When a distributed system is started and a proxy instrumentation module is preloaded, if a user initiates a business request, the microservice node receives the service call session and determines the executable function of the service call session. The service call session includes a service call request and a service call response, and carries a tracing identifier. The proxy instrumentation module dynamically instrumentes the executable function to obtain the session content within the service call session. Based on the tracing identifier and session content, the behavioral transaction corresponding to the user's business request is traced. In other words, this application obtains the executable functions of the service call request and service call response, and dynamically instrumentes them to obtain the request content in the service call request and the response content in the service call response. This solves the problem of non-repeatable reading of request and response content. Furthermore, based on the request content, response content, and the tracing identifier carried in the service call request, for the access data of individual microservices in a discrete record distributed system, complete acquisition of user behavioral data can be achieved. This ensures the integrity of the behavioral data content of behavioral transactions and allows determination of the call order of each microservice, thereby determining the access chain sequence and tracing the behavioral transaction of the user's specific behavioral data. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the complete set of behavioral data for the request and response phases provided in related technologies;

[0023] Figure 2 It is a flowchart provided in related technologies for recording user behavior transaction data;

[0024] Figure 3 This is an optional architecture diagram of the distributed system provided in the embodiments of this application;

[0025] Figure 4 This is an optional flowchart illustrating the link tracing method provided in an embodiment of this application;

[0026] Figure 5 This is an optional flowchart illustrating the link tracing method provided in an embodiment of this application;

[0027] Figure 6 This is an optional flowchart illustrating the link tracing method provided in an embodiment of this application;

[0028] Figure 7 This is an optional flowchart illustrating the link tracing method provided in an embodiment of this application;

[0029] Figure 8 This is an optional flowchart illustrating the link tracing method provided in an embodiment of this application;

[0030] Figure 9 This is a schematic diagram of obtaining the tracking segment marker provided in an embodiment of this application;

[0031] Figure 10 This is an optional flowchart illustrating the link tracing method provided in an embodiment of this application;

[0032] Figure 11 This is an optional flowchart illustrating the link tracing method provided in an embodiment of this application;

[0033] Figure 12 This is a timing diagram for tracing user service requests provided in an embodiment of this application;

[0034] Figure 13 This is a system block diagram of the distributed system provided in the embodiments of this application;

[0035] Figure 14 This is a schematic diagram of the microservice node integrated proxy instrumentation module of the distributed system provided in the embodiments of this application. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0037] In the following description, references to "some embodiments" refer to a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict. Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiments of this application pertain. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit the application.

[0038] To better understand the link tracing method provided in the embodiments of this application, the link tracing methods in related technologies are first described:

[0039] In the fintech and cloud computing environments, an increasing number of business systems, such as financial systems, are adopting distributed microservice architectures. Business processes are completed collaboratively by multiple microservices. For example, a user's purchase of a financial product requires assistance from account services, product services, and transaction services. Microservice architectures are often very complex, involving numerous remote service calls between services, such as via Hypertext Transfer Protocol (HTTP) or Remote Procedure Call (RPC), calling message queues, accessing in-memory databases, and relational databases. It's important to note that detailed user transaction data needs to be recorded during service processing, such as transactions involving user account login, product browsing, and product purchase.

[0040] In related technologies, refer to Figure 2 As shown, user behavior transaction data is recorded primarily through the following methods: first, by mirroring network traffic using network hardware devices such as switches to obtain traffic data; second, by backing up traffic data through a microservice gateway; and third, by printing local logs for each service to record user behavior. However, while these methods can record discrete events—that is, the interaction data of users accessing services A, B, and C separately—they cannot chain them together to identify a complete transaction. For example, if user A purchases two identical products for the same amount, the transaction service logs might record them as [User ID A - Product P - Amount M - Transaction Transaction B1] and [User ID A - Product P - Amount M - Transaction Transaction B2]. For the two transactions initiated by user A, the discrete behavior data cannot trace the corresponding complete transaction (the transaction transaction is generated in the transaction service, but the behavior data generated by the account service and product service does not include the transaction transaction, making it impossible to record a complete transaction from entry to exit). In microservice systems, handling a large number of user requests simultaneously results in multiple discrete event records, making it impossible to trace the transaction behavior of different users. Furthermore, the complete data for a single transaction behavior includes a complete set of behavior data from both the request and response phases.

[0041] In related technologies, solutions based on dynamic instrumentation can obtain data sets of {request headers / response headers, timestamps, request-side feature information / response-side feature information} for both the request and response phases. However, because the request body and response body data are stored in byte stream data structures (e.g., the request body data is stored in an input byte stream (InputStream) data structure, and the response body data is stored in an output byte stream (OutputStream) data structure), each byte stream data structure has a pos cursor pointer. While the provided data interface is reading data, the pos pointer is synchronously shifted. After a complete data read, the pos pointer will also move to the end of the data content, and resetting the pos pointer position is not supported. Therefore, the code block added in the dynamic instrumentation of related technologies to implement data capture, once it has read the data content of the request or response body, will prevent the application from reading it again, leading to request failures such as failure to retrieve product lists or billing information, and inability to obtain complete transaction data.

[0042] The following describes exemplary applications of the distributed system provided in this application. The distributed system provided in this application can be implemented as any terminal with screen display capabilities, such as a laptop, tablet, desktop computer, or intelligent robot, or as a server. Exemplary applications of the distributed system will be described below.

[0043] See Figure 3 , Figure 3 This is a schematic diagram of the structure of the distributed system 100 provided in the embodiments of this application. Figure 3 The distributed system 100 shown includes at least one processor 110, at least one network interface 120, a user interface 130, and a memory 150. The various components in the distributed system 100 are coupled together via a bus system 140. It is understood that the bus system 140 is used to implement communication between these components. In addition to a data bus, the bus system 140 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 3 The general labeled all buses as Bus System 140.

[0044] The processor 110 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0045] User interface 130 includes one or more output devices 131 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 130 also includes one or more input devices 132, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0046] Memory 150 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. Memory 150 may optionally include one or more storage devices physically located remote from processor 110. Memory 150 may include volatile memory or non-volatile memory, or both. Non-volatile memory may be read-only memory (ROM), and volatile memory may be random access memory (RAM). The memory 150 described in this application embodiment is intended to include any suitable type of memory. In some embodiments, memory 150 is capable of storing data to support various operations, examples of which include programs, modules, and data structures, or subsets or supersets thereof, as exemplarily described below.

[0047] Operating system 151 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0048] The network communication module 152 is used to reach other computing devices via one or more (wired or wireless) network interfaces 120, such as Bluetooth, WiFi, and Universal Serial Bus (USB).

[0049] The input processing module 153 is used to detect and translate one or more user inputs or interactions from one or more input devices 132.

[0050] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 3A link tracing device 154 stored in memory 150 is shown. This link tracing device 154 can be a link tracing device in a distributed system 100. It can be software in the form of programs and plug-ins, including the following software modules: a receiving module 1541, a determining module 1542, and a processing module 1543. These modules are logically linked and can therefore be arbitrarily combined or further divided according to their implemented functions. The functions of each module will be described below.

[0051] In other embodiments, the apparatus provided in this application can be implemented in hardware. As an example, the apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the link tracing method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0052] The link tracing method provided in this application will be described below with reference to exemplary applications and implementations of the distributed system 100 provided in the embodiments of this application. See also Figure 4 , Figure 4 This is an optional flowchart illustrating the link tracing method provided in this application embodiment, which will be combined with... Figure 4 The steps shown are explained below.

[0053] Step 201: When the distributed system is started and the proxy instrumentation module is preloaded, if a user initiates a business request, the microservice node receives the service call session and determines the executable function of the service call session.

[0054] The service call session includes the service call request and the service call response, and the service call session carries a tracking identifier.

[0055] In this embodiment, the proxy instrumentation module is applied to dynamically instrument the startup service. It preloads the pluggable injection module before startup and injects the injection code into the microservice via inter-process communication. Specifically, the proxy instrumentation module is used to inject a piece of logic code into an independent application that dynamically runs within any microservice in a distributed system, dynamically changing the execution flow or function of the service's internal code. It should be noted that the proxy instrumentation module is preloaded when the distributed system starts; when the logic code of the proxy instrumentation module embedded in the microservice is triggered, the application within the proxy instrumentation module is executed, thereby changing the execution flow or function of the microservice's internal code.

[0056] In this embodiment, when a business transaction involves service calls to multiple microservices, the distributed system can record the order in which service calls occur between different services through a protocol, thereby forming a service call chain for a single business transaction and enabling tracking of the performance and status of each service call node in a single business transaction. Here, the distributed system employs distributed tracing technology, which may include the Open Tracing protocol.

[0057] In this embodiment of the application, the business request can be a request based on the HTTP protocol.

[0058] In this embodiment, the service call session includes a service call request and a service call response. In a distributed system, upon receiving a business request initiated by a user, when executing the processing corresponding to that business request, any node containing any microservice other than the first microservice in the microservice executing the business request will receive a service call request sent by the upstream microservice node. Here, the service call request includes a request header, a request body (also known as request content), a timestamp, and request-side characteristic information. The service call request carries a trace identity document (traceID); of course, the service call request also includes a span identifier (spanID), where the trace identity document is used to indicate the unique identifier of the business request, and the span identifier is used to indicate the span identifier of the node where the called microservice resides.

[0059] Furthermore, the node containing the microservice that receives the service call request will respond to the request, obtaining a service call response, and then return the response to the microservice that sent the request. This service call response includes a response header, a response body (also known as the response content), a timestamp, and response endpoint characteristics.

[0060] In this embodiment of the application, a business request refers to a transaction triggered by a user operation. For example, when a user performs a subscription operation, the business request triggers a transaction. First, the business request enters the microservice gateway. The microservice gateway routes the request to microservice node A, such as the account service for authentication, then to microservice node B, such as the product service for obtaining the product list, and finally to microservice node C, such as the transaction service for completing the subscription.

[0061] In this embodiment, the executable function refers to the function used to execute the query action of the aforementioned service call session. The executable function can be determined in two ways: first, by searching among the functions executing the service call session for functions with a preset keyword such as `doexecute`, which is the executable function; second, since the executable functions of the service call session can also be annotated with annotations, the executable function used to execute the service call session can be determined by searching for functions annotated with annotations. This application does not impose specific limitations on this. Furthermore, after determining the executable function, logic is re-inserted into the code area of ​​the executable function.

[0062] Here, if the microservice uses the Apache HttpClient component, and the executable function for the service call session is determined to be the `doexecute` function, then the service must call the `doexecute` function to execute an HTTP request. This function includes the request's Uniform Resource Locator (URL) and the request parameters. After the function executes, it also returns the returned object from the request interface, including the response headers, response body, timestamp, and response endpoint characteristics; these can be obtained by intercepting the `doexecute` function.

[0063] In this embodiment of the application, when the distributed system starts, the proxy instrumentation module is preloaded, and when the user initiates a business request, the service call session is received through the microservice node in the distributed system, and the executable function of the service call session is determined.

[0064] Step 202: Dynamically instrument the executable function through the proxy instrumentation module to obtain the session content in the service call session.

[0065] The session content includes, but is not limited to, the request body in the service call request and the response body in the service call response.

[0066] In this embodiment, when the distributed system starts and the proxy instrumentation module is preloaded, and a user initiates a business request, after the microservice node receives the service call session and determines the executable function of the service call session, the proxy instrumentation module dynamically instrumentes the executable function to obtain the session content in the service call session. Thus, by dynamically instrumenting the executable function, the request body in the service call request and the response body in the service call response can be read.

[0067] Step 203: Based on the tracking identifier and session content, track the behavioral transactions corresponding to the user's business requests.

[0068] In this embodiment of the application, behavioral transactions can be understood as a set of time-sequential behavioral data triggered by a single user business request operation in various microservices in a distributed system.

[0069] In this embodiment of the application, link tracing can be understood as a set of behavioral data triggered by a single behavioral transaction in a distributed system, and the distributed service call chain is connected according to the tracing identifier.

[0070] In this embodiment of the application, each microservice in the distributed system performs dynamic instrumentation on the executable function through a proxy instrumentation module. After obtaining the session content in the service call session, it obtains the behavioral data triggered by a single user business request in each microservice. By using tracking identifiers, it concatenates the behavioral data of the microservices involved in the business request to form a distributed call chain. This not only enables the tracking of the behavioral transactions corresponding to the user's business request, but also obtains the user's specific behavioral data in each microservice.

[0071] This application provides a link tracing method. When a distributed system is started and a proxy instrumentation module is preloaded, if a user initiates a business request, the microservice node receives the service call session and determines the executable function of the service call session. The service call session includes a service call request and a service call response, and carries a tracing identifier. The proxy instrumentation module dynamically instrumentes the executable function to obtain the session content within the service call session. Based on the tracing identifier and session content, the behavioral transaction corresponding to the user's business request is traced. In other words, this application obtains the executable functions of the service call request and service call response, and dynamically instrumentes them to obtain the request content in the service call request and the response content in the service call response. This solves the problem of non-repeatable reading of request and response content. Furthermore, based on the request content, response content, and the tracing identifier carried in the service call request, for the access data of individual microservices in a discrete record distributed system, complete acquisition of user behavioral data can be achieved. This ensures the integrity of the behavioral data content of behavioral transactions and determines the call order of each microservice, thereby determining the access link sequence and tracing the behavioral transaction of the user's specific behavioral data.

[0072] Here, in a microservice node that supports decorator objects replacing specific components, combined with... Figure 5 This section further explains step 202, which involves dynamically instrumenting the executable function using a proxy instrumentation module to obtain the session content from the service call session.

[0073] Step 301: When the microservice node supports the replacement of specific components by decorator objects, generate a decorator instance with data caching function using byte data stream.

[0074] In this embodiment of the application, the decorator is used to add new functionality to the function without changing the existing function code. For example, the decorator can be a Wrapper decorator.

[0075] In this application embodiment, the specific component is a component used to provide an efficient, up-to-date, and feature-rich client programming toolkit that supports the HTTP protocol. The specific component includes, but is not limited to, the Apache-httpclient component.

[0076] In this embodiment, the byte data stream is used to cache data; that is, the byte data stream writes data to an internal buffer, the buffer automatically grows, and data can be retrieved from it when the writing is complete. Here, the byte data stream is a ByteArrayOutputStream, which is used to store data for a single write operation.

[0077] In this embodiment of the application, different microservices may use different technical frameworks, including but not limited to Java technical frameworks and Python technical frameworks. Since each microservice may use a different technical framework, some microservices support decorator objects to replace specific components, while others do not support decorator objects to replace specific components.

[0078] For example, when a microservice node supports decorators such as Wrapper objects to replace specific components such as the Apache-httpclient component, the microservice node in the distributed system uses a byte data stream ByteArrayOutputStream to write the session content of the service call session, i.e., the request body in the service call request and the response body in the service call response, into an internal cache, generating a decorator such as Wrapper instance with caching functionality.

[0079] Step 302: Based on the service call session, determine the hook functions in the executable function used to override the hook functions provided by the proxy instrumentation module, and override the hook functions.

[0080] In this embodiment of the application, the hook function is used to obtain the session content of the service call session. The hook function includes a hook function before the executable function is processed and a hook function after the executable function is processed. The hook function before processing is the breforeMethod function, and the hook function after processing is the afterMethod function.

[0081] In this embodiment, the purpose of overriding hook functions is to insert a piece of logic code into a dynamically running independent application, dynamically changing the execution flow or function of the internal code. Here, overriding the hook functions before and after processing means that the inserted logic code is executed before or after processing the data to retrieve the request or response data according to the inserted logic code. For example, if user A sends a message, before overriding the hook function, user A sends the message to user B. After overriding the hook function provided by the proxy instrumentation module, user A not only sends the message to user B but also to user C.

[0082] Here, if the service call session is a service call request, the microservice node in the distributed system determines the breforeMethod hook function in the executable function used to override the proxy instrumentation module, and overrides the breforeMethod hook function in order to obtain the data content of the request body in the service call request.

[0083] If the service call session is a service call response, the afterMethod hook function provided by the proxy instrumentation module is determined in the executable function through the microservice node in the distributed system, and the afterMethod hook function is overridden in order to obtain the data content of the response body in the service call response returned by the service.

[0084] Step 303: Replace the parameters carried by the overridden hook function with the decorator instance, execute the replaced hook function, obtain the session content, and store it in the cache area.

[0085] In this embodiment, after the hook function is overridden, that is, after the logic execution program included in the hook function is changed, the parameters carried by the overridden hook function are replaced with decorator instances and loaded back into the runtime container, and the replaced hook function is executed to obtain the session content. The session content is then stored in the cache area to achieve data backup. The backed-up session content includes request content and response content.

[0086] In this embodiment, if the service call session is a service call request, the parameters carried by the overridden `breforeMethod` hook function are replaced with a `Wrapper` decorator instance by the microservice node in the distributed system. This ensures that the request data object obtained by the system is a processed data object with cache functionality. The replaced `breforeMethod` hook function is executed to obtain the request content, which is then stored in a cache area, such as a local file, database, middleware, or message queue. The request body can then be retrieved from the cache area by calling the decorator instance, thus obtaining the data content of the request body. If the service call session is a service call response, the parameters carried by the overridden `afterMethod` hook function are replaced with a `Wrapper` decorator instance by the microservice node in the distributed system. This ensures that the request data object obtained by the system is a processed data object with cache functionality. The replaced `afterMethod` hook function is executed to obtain the response content, which is then stored in a cache area, such as a local file, database, middleware, or message queue. The response body can then be retrieved from the cache area by calling the decorator instance, thus obtaining the data content of the response body.

[0087] For example, the microservice node normally processes user business requests, including querying bank account transaction information for October. The request information includes the user's ID and the transaction date, such as October. After processing by the microservice, the transaction record for October corresponding to the ID is found in the database. After executing the replaced hook function, the query result (referring to the transaction record) is obtained. The query result is intercepted, copied, and stored in a cache area, such as a local file, a database, middleware, or a message queue. The query result is then retrieved repeatedly from the cache area.

[0088] As mentioned above, if a microservice node uses the Apache-HttpClient component, and this component supports decorator replacement, a new decorator class can be generated that implements this interface. The pre-processing and post-processing hook functions can be overridden, meaning the hook function is rewritten. Before or after processing the data, instrumentation logic is executed to retrieve the request or response body data. This involves executing the hook function to store the response body data in a cache area, such as a local file, database, middleware, or message queue. The session content in the cache area is then read by calling the decorator instance. In other words, for HTTP behavioral data generated by components like Apache-HttpClient, implementing the interface class generates a decorator object with data caching functionality. The cache block can be read repeatedly. By storing the session content in the cache area, the microservice can repeatedly read the session content stored in the cache area, solving the problem in related technologies where data content can only be read once.

[0089] Here, in a scenario where microservice nodes do not support decorator objects replacing specific components, combined with... Figure 6 This section further explains step 202, which involves dynamically instrumenting the executable function using a proxy instrumentation module to obtain the session content from the service call session.

[0090] Step 401: If the microservice node does not support decorator objects replacing specific components, obtain the binary code block of the executable function.

[0091] The binary code block of an executable function can be understood as the machine code of the executable function that the computer can recognize.

[0092] In this embodiment of the application, when the microservice node does not support the replacement of specific components by decorator objects, the binary code block of the executable function is obtained by using the javassist tool on the microservice node in the distributed system.

[0093] Step 402: Use the HTTP data processing object in the object pool to convert and decompile the binary code block to obtain the source code block of the executable function.

[0094] In this embodiment of the application, the source code block of the executable function can be understood as a source code file that the user can understand; here, by converting the binary code block, the bytecode code block ClassFile of the executable function is obtained, and the source code block of the executable function is obtained by decompiling the bytecode code block ClassFile.

[0095] In this embodiment of the application, when the microservice node does not support the decoration object replacing a specific component, after obtaining the binary code block of the executable function, the microservice node in the distributed system obtains the HTTP data processing object from the ClassPool object pool according to the Http component ClassName (such as javax.servlet.http.HttpServletRequest), and uses getClassFile to convert the binary code block to obtain the bytecode code block ClassFile of the executable function; furthermore, the bytecode code block ClassFile of the executable function is decompiled to obtain the source code block of the executable function.

[0096] Step 403: Determine the keyword corresponding to the service call session.

[0097] The keywords include getInputStream (which returns an input stream) and getOutputStream (which returns an output stream).

[0098] In this embodiment of the application, if the service call session is a service call request, the keyword corresponding to the service call request is getInputStream; if the service call session is a service call response, the keyword corresponding to the service call response is getOutputStream.

[0099] Step 404: From the list of functions contained in the source code block, query the target function information object corresponding to the keyword, and obtain the target source code block corresponding to the target function information object.

[0100] In this embodiment of the application, after determining the keyword corresponding to the service call session through the microservice nodes in the distributed system, a function list, such as List, is determined from the source code block. <methodinfo>methods; and from the list of functions List <methodinfo>The `methods` method searches for the `MethodInfo` target function information object that matches the keyword; furthermore, the target source code block corresponding to the `MethodInfo` target function information object is obtained by calling the `read` method on the microservice node in the distributed system.

[0101] Here, taking the retrieval of response data as an example, where the service call session is the service call response, the application reads the response function with the keyword `getOutputStream` as `HttpServletResponse.getOutputStream`, and searches for a list of functions by matching the keyword `getOutputStream`. <methodinfo>The MethodInfo object corresponding to the methods object is accessed by calling the read method to obtain the target source code block corresponding to the MethodInfo object.

[0102] Step 405: Read the target source code block line by line to determine the line number of the source code that meets the matching conditions.

[0103] The matching conditions include the existence of an assignment identifier and the presence of a feature function corresponding to the keyword in the expression on the right side of the assignment identifier.

[0104] For example, taking the acquisition of response data as an example, i.e., the service call session is the service call response, the reading starts from the first line of the target source code block, looking for whether there is an assignment operation in that line of source code, such as whether there is an assignment operator `=` and whether there is a characteristic function corresponding to the keyword such as getOutputStream in the expression on the right. If the source code of that line of source code meets the matching condition, the line number of the source code is determined; if the source code of that line of source code does not meet the matching condition, the reading continues to the next line until the last line is read and still does not meet the matching condition, it is determined that there is no reading operation in the target source code block.

[0105] Step 406: Insert the capture function included in the proxy instrumentation module into the next line after the row number. The capture function copies and parses the session data returned by the feature function to obtain the session content.

[0106] In this embodiment, the capturing function is used to record the line number of the source code that meets the matching condition and to copy the result returned by the source code. For example, the capturing function can be CtMethod.insertAt(lineNumber, code); where lineNumber is the line number of the source code that meets the matching condition, and code is the result returned by the source code.

[0107] In this embodiment, the target source code block is read line by line. After determining the line number N where the source code that meets the matching conditions is located, the capture function CtMethod.insertAt(lineNumber,code) of the proxy instrumentation module is inserted in the next line, i.e., line N+1. The microservice node copies and parses the session data returned by the feature function through the capture function, that is, the feature function is converted into a String, and the String object generated in line N is obtained, thereby obtaining the session content.

[0108] As described above, if the microservice node does not support replacing the Apache-httpclient component with a decorator object, the process is as follows: First, using the original functionality provided by dynamic instrumentation, the binary code block of the executable function `doexecute` in the application is obtained and converted into source code that the user can understand, i.e., the source code block. Second, from the list of functions contained in the source code block, the target function information object corresponding to the keyword is queried, and the target source code block corresponding to the target function information object is obtained. Then, the target source code block is analyzed, read line by line, and the line number of the source code that meets the matching conditions is determined. Further, a capture function is inserted on the next line after the line number of the source code that meets the matching conditions. The capture function copies and parses the session result returned by the feature function corresponding to the keyword, thereby obtaining the session content. In other words, when the microservice node does not support replacing the Apache-httpclient component with a decorator object, data capture is performed at the function input and return points, and the function that handles HTTP requests is instrumented. During the request phase, instrumentation is used to set up a capture function to obtain the parameter list, track the target point where the application has already read the InputStream data structure, and inject code blocks to read the request body data again. During the response phase, instrumentation is again used to set up a capture function to track the function's return object, which has not yet been converted into an OutputStream data structure. This object can be parsed into the response body data for collection. In this way, by copying and parsing the session results returned by the feature function through the capture function, the microservice can repeatedly read the copied session content, solving the problem that data content can only be read once in related technologies.

[0109] Here, even if the capture function does not parse the session data returned by the feature function, the session content can still be obtained through... Figure 7 The steps shown are to be completed.

[0110] Step 501: Obtain the object hash value of the session data, and obtain the object set consisting of all objects during the runtime of the distributed system through the instrumentation functions included in the proxy instrumentation module.

[0111] In this embodiment of the application, when the microservices in the distributed system do not support decorator Wrapper replacement, and the capture function directly reads session data, namely the data of ServletRequest.getInputStream() in the request phase and the data of ServletResponse.getOutputStream() in the response phase, and the session data is not parsed, the object hash value of the session data is obtained through the microservice nodes of the distributed system; and the object collection consisting of all objects at runtime of the distributed system is obtained through the instrumentation function getAllLoadedClasses function of Instrumentation included in the proxy instrumentation module.

[0112] Step 502: In the object collection, use the object hash value as the key to find the data stream object corresponding to the object hash value.

[0113] In this embodiment of the application, in the object collection, the data stream object corresponding to the object hash value is found by using the object's full path name@object hash value as the key. The data stream object includes InputStream objects and OutputStream objects.

[0114] Step 503: Use bytecode analysis tools to parse the properties of the data stream object.

[0115] Among these attributes is a cursor pointer used to indicate the read status of data stored at a memory address.

[0116] In this embodiment of the application, the properties of the data stream object are parsed using the bytecode tool getDeclaredField, namely the pos cursor pointer used to indicate the read status of the data stored in the memory address.

[0117] Step 504: If the cursor pointer is pointing to the end address of the memory address where the session data is located, adjust the cursor pointer to the start address of the memory address.

[0118] Step 505: After serializing the data stream object, re-inject it into the object pool to read the session content.

[0119] In this embodiment, when the cursor pointer points to the end address of the memory address where the session data is located, the pointer of the pos cursor is adjusted to the start address of the memory address, thereby resetting the pos cursor pointer so that the session content can be read repeatedly; finally, the data stream object is serialized and re-injected into the object pool, and then the session content is read.

[0120] As described above, by instrumenting the data stream object's properties—that is, finding the memory address of the `pos` cursor pointer and resetting it to zero—then serializing and re-injecting it into the program allows for the re-reading of session content. In other words, bytecode modifies the `pos` pointer of the data storage object. Dynamic instrumentation provides the ability to retrieve all loaded objects during application runtime. Based on this, it first locates the target `InputStream` and `OutputStream` objects. After the instrumented incremental code block reads the complete data, the bytecode tool modifies the object's `pos` pointer, resets it, and re-injects it into the application, allowing it to be read again by the application itself. Thus, by resetting the `pos` cursor pointer to zero, the microservice can repeatedly read session content, solving the problem in related technologies where data can only be read once using the `pos` cursor pointer.

[0121] See Figure 8 , Figure 8 This is an optional flowchart illustrating the link tracing method provided in this application embodiment, which will be combined with... Figure 8 The steps shown are explained below.

[0122] Step 601: When the distributed system is started and the proxy instrumentation module is preloaded, if a user initiates a business request, the microservice node receives the service call request and determines the executable function of the service call request.

[0123] The service call request carries a tracking identifier.

[0124] Step 602: Through the proxy instrumentation module, the executable function is dynamically instrumented to obtain the request content in the service call request.

[0125] Step 603: Based on the request content and tracing identifier, generate the first tracing segment tag for the microservice node.

[0126] The session content includes the request content.

[0127] In this embodiment of the application, during the link tracing process, when a business request passes through each microservice node, the first tracing segment marker of the microservice node reflects the request call status of the microservice node, and the first tracing segment marker of the microservice node reflects the relative position of the microservice node in the business request call path.

[0128] In this embodiment, the first trace segment marker is used to reflect whether a service call request included in a certain business request passes through the microservice node, and to reflect the relative position of the microservice node in the business request call path. The first trace segment marker TraceSegment of the microservice node includes a set of span markers TraceSpan composed of at least one span marker TraceSpan when accessing the microservice node behavior, a trace identifier traceID corresponding to the business request, a set of all metadata data_binary, a trace segment identifier segmentID, i.e., the service identifier of the microservice node, and the number of span markers spanSpan in the set of span markers spanSpan, spansize.

[0129] The aforementioned span marker represents the user behavior data involved in the business request. The span marker TraceSpan is stored in a new data structure in key-value pair format. The span marker includes the span ID of the service unit in this microservice node, the span ID of the previous service unit of this service unit pspanID, the trace ID corresponding to the business request traceID, and the generation time of the span marker. Of course, the span marker can also include the plugin component name of this microservice node, the custom tag, the peer service type (also known as the service call type) peer, and the operation resource operationName. Here, the plugin component name can be a SpringHttp component, and the custom tag can be the content of the request header and request body stored as a custom tag, including the request URL (http.url), request method (http.method), request header (http.reqHeader), and request body (http.reqBody).

[0130] The aforementioned trace segment identifiers are codes for different nodes in a single task call, used to characterize and distinguish different nodes in a single task call.

[0131] In a feasible application scenario, refer to Figure 9 As shown, microservice node A is the entry service for processing business requests. After the tracing tag includes all the content of the service call request to microservice node A, for the behavior of accessing microservice node A, based on the span identifier spanID=1 of service unit 1 in microservice node A, the tracing identifier traceID=1 corresponding to the business request, the span identifier pspanID=0 of the previous service unit of service unit 1, and the timestamp T1, the span tag Tracespan-1 of service unit 1 in microservice node A is generated. Furthermore, microservice node A uses the username and password from the login request as parameters to query the database authentication of service unit 2. This request will generate a trace marker Tracespan-2 for service unit 2 based on the span identifier spanID=2, the trace identifier traceID=1 corresponding to the business request, the span identifier pspanID=1 of the previous service unit of service unit 2, and the timestamp T2. Thus, trace markers Tracespan for accessing two service units in microservice node A are formed. The data structures of the two trace markers Tracespan are merged into the first trace segment marker TraceSegment-1, so that one first trace segment marker TraceSegment includes multiple tracespan data.

[0132] Step 604: Based on the first trace segment marker of the microservice node, generate a downstream call request for the microservice node to make service calls to the downstream microservice node.

[0133] For example, the trace identifier (traceID) is extracted from the first trace segment marker of the microservice node, and the trace identifier (traceID) is encapsulated into the header custom field X-Weapm-TraceID in the HTTP call request of the microservice node to make a service call to the downstream microservice node, so as to obtain the downstream call request.

[0134] Step 605: Send a downstream call request to the downstream microservice node so that the downstream microservice node generates a first downstream trace segment marker based on the downstream call request, until all call requests between the microservice nodes included in the behavioral transaction are completed.

[0135] In this embodiment, after the downstream microservice node receives the downstream call request sent by the upstream microservice node, the downstream microservice node determines the executable function of the downstream call request, and performs dynamic instrumentation on the executable function of the downstream call request through the proxy instrumentation module to obtain the request content in the downstream call request. Based on the request content and trace identifier in the downstream call request, the first downstream trace segment marker of the downstream microservice node is generated until all call requests between the microservice nodes included in the behavioral transaction are completed.

[0136] Step 606: Report the first tracing segment marker through the microservice node, and report the first downstream tracing segment marker through the downstream microservice node.

[0137] Here, the first trace segment marker TraceSegment of the microservice node and the first downstream trace segment marker TraceSegment of the downstream microservice node are uploaded to the data receiving module, and the data receiving module parses the trace identifier traceID as the primary key and writes it to the storage.

[0138] Step 607: When the service call session is a service call response, the executable function is dynamically instrumented through the proxy instrumentation module to obtain the response content in the service call response, thereby obtaining the response content corresponding to all microservice nodes.

[0139] In this embodiment of the application, the session content includes response content.

[0140] Step 608: Based on the response content and tracing identifier, generate the second tracing segment tag for each microservice node, and report the corresponding second tracing segment tag through each microservice node.

[0141] In this embodiment of the application, during the link tracing process, when a business request passes through each microservice node, the second tracing segment marker of the microservice node reflects the situation of the microservice node being called by the response service, and the second tracing segment marker of the microservice node reflects the relative position of the microservice node in the path of the response service call.

[0142] In this embodiment, the second trace segment marker is used to reflect whether the response service call included in a certain business request passes through the microservice node, and to reflect the relative position of the microservice node in the business request call path. The second trace segment marker TraceSegment of the microservice node includes a set of span markers TraceSpan composed of at least one span marker TraceSpan when accessing the microservice node behavior, a trace identifier traceID corresponding to the business request, a set of all metadata data_binary, a trace segment identifier segmentID, i.e., the service identifier of the microservice node, and the number of span markers spanSpan in the set of span markers spanSpan, spansize.

[0143] The aforementioned span marker TraceSpan is stored in a new data structure in key-value pair format. The span marker includes the span ID of the service unit in this microservice node, the span ID of the previous service unit, the trace ID corresponding to the business request, and the generation time of the span marker. Of course, the span marker can also include the plugin component name of this microservice node, the custom tag, the peer service type (also known as the service call type), and the operation resource operationName. Here, the plugin component name can be a SpringHttp component, and the custom tag can be the content of the response header and response body stored as a custom tag, including but not limited to the response method, response header, and response content.

[0144] The aforementioned trace segment identifiers are codes for different nodes in a single task call, used to characterize and distinguish different nodes in a single task call.

[0145] In this embodiment of the application, a second trace segment marker is generated for each microservice node based on the response content and the trace identifier, and the corresponding second trace segment marker is reported by each microservice node.

[0146] Step 609: Based on the first tracking segment marker, the first downstream tracking segment marker, and the second tracking segment marker, determine the tracking of the user's behavioral transactions.

[0147] In this embodiment, the distributed system connects the first tracing segment marker, the first downstream tracing segment marker, and the second tracing segment marker according to the calling order of the microservice nodes corresponding to each tracing segment marker, thereby generating a service call chain of the microservice nodes and thus obtaining the tracking of user behavior transactions.

[0148] As described above, the above method can solve the problem of broken user behavior data due to the inability to obtain inter-service call data in traffic mirroring and gateway backup data based on hardware (switches / routers). This application embodiment achieves universal end-to-end behavior tracing from user-triggered transactions to distributed service entry points, inter-service calls, and storage layer services. By applying dynamic instrumentation technology combined with distributed link tracing technology, TraceID is embedded in various types of service calls (HTTP / RPC service calls, access to message queue middleware, access to in-memory and relational databases, etc.), ensuring that detailed behavioral data is captured for each microservice reached by the user transaction, and then linked together to form a complete traceable behavior. Simultaneously, this application embodiment, based on dynamic instrumentation technology, seamlessly embeds the collected data logic into the application. This deployment method can cover all service modules of the distributed system without requiring custom coding of the application itself, while achieving full service coverage and solving the problem of broken behavior data in the original technical solutions, i.e., the inability to obtain interaction data of some microservices in the distributed system. This enables the acquisition of user behavior data within each microservice, fully covering all microservices.

[0149] Here, the process of updating the service configuration information of microservice nodes is combined with Figure 10 The steps shown are explained below.

[0150] Step 701: Obtain the service configuration information of the microservice node.

[0151] The service configuration information includes the collection speed and collection range of the span marker.

[0152] In this embodiment of the application, the tracking segment marker includes a span marker.

[0153] The collection speed refers to the total number of span markers collected by the microservice node per unit of time, such as span_limit_per_sec, span_limit_per_min, and span_limit_per_hour.

[0154] The collection scope includes, but is not limited to, subsystem_list, microservice node scope, interface scope, wildcard matching of request path pattern_request_path, complete matching of request path pattern_request_path, and matching of custom parameter value request_parm_key.

[0155] It should be noted that in related technologies, the collection of span markers, i.e., behavioral data, is based on a full-volume mirroring traffic method, and data filtering can only be performed afterward, resulting in unnecessary waste of computational and storage resources during the collection process. In this embodiment, by setting the service configuration information for the collection range of span markers, it is possible to filter HTTP data generated by specific subsystem services, specific API interfaces, and HTTP data sets containing specific parameters, etc., as follows:

[0156] First, the configuration service client obtains the above configuration options, parses the Tag content of the TraceSpan span marker, and extracts the request URL and request header;

[0157] Second, for wildcard matching of the request path pattern_request_path, regular expression matching is performed on http.url; for complete matching of the request path pattern_request_path, full matching is performed on http.url; for subsystem scope subsystem_list, matching is performed based on the global variable SystemID of the meta package; for matching custom parameter values ​​request_parm_key, regular expression matching and full matching are performed on the param parameter set of the request header http.reqHeader.

[0158] In this way, by setting the data collection range, it is not necessary to collect all user behavior data; only the behavior data that meets the data collection range needs to be collected, stored, and recorded. This reduces resource computation and lowers storage costs.

[0159] Here, since the proxy instrumentation module integrates a general configuration center client implemented based on a reliable coordination system (ZooKeeper) of a distributed system, the proxy instrumentation module can perform batch changes on the service configuration information of microservice nodes that integrate the proxy instrumentation module in different scopes such as single-point systems or subsystems.

[0160] In this embodiment, before obtaining the service configuration information of the proxy instrumentation module, since the proxy instrumentation module is coupled to the software development kit (SDK) of the configuration center client of the distributed system, the microservice node is set to one of the following access modes: local configuration mode, remote mode, and remote non-update mode. The remote mode refers to loading the service configuration information configured by the server for the microservice node when starting the microservice node, and backing up the service configuration information to the disk. The local mode refers to reading the service configuration information of the microservice node from the disk when starting the microservice node.

[0161] Step 702: Obtain hot update information of the changed service configuration information issued by the configuration service center of the distributed system.

[0162] Step 703: Trigger the hot update mechanism of the configuration service of the microservice node. The configuration service pulls new configuration information from the configuration center server according to the hot update information and updates the service configuration information according to the new configuration information.

[0163] In this embodiment of the application, the microservice node of the distributed system obtains the hot update information of the changed service configuration information issued by the configuration service center of the distributed system, refreshes the status and content of the ZooKeeper node, triggers the configuration service SDK hot update mechanism in the microservice node, and pulls the new configuration information from the configuration center server according to the hot update information through the configuration service SDK, and updates the service configuration information according to the new configuration information.

[0164] Here, the process of recording the number of span markers collected per unit time (i.e., the collection speed) in the service configuration information of microservice nodes is combined with... Figure 11 Steps 801 to 803 are shown; or steps 801 to 802 and 804 are explained.

[0165] Step 801: Obtain the preset number of span markers included in the generated tracking segment markers within the preset time period.

[0166] In this embodiment, the preset data volume can be understood as the number of span markers included in the tracking segment markers per unit time. For example, it could be the number of span markers included in the tracking segment markers collected by the microservice node per second: span_limit_per_sec.

[0167] Step 802: When a new span marker is to be added to the tracking segment marker, obtain the time interval between the current time and the generation time of the span marker at the target position in the tracking segment marker.

[0168] In this embodiment of the application, the target location can be the location corresponding to each span mark when the span marks generated in chronological order are stored.

[0169] Step 803: If the time interval is greater than the preset duration, delete the span marker of the target position from the tracking segment marker.

[0170] In this embodiment of the application, when a new span marker needs to be added to the tracking segment marker, the time interval is calculated by subtracting the generation time of the span marker at each position in the tracking segment marker from the current time according to the generation time of the span marker. If the time interval corresponding to the span marker is greater than the preset duration, the span marker is deleted from the tracking segment marker.

[0171] Step 804: If the time interval is less than the preset duration and the number of span markers in the tracking segment markers is less than the preset number, add new span markers to the tracking segment markers.

[0172] In this embodiment, when a new span marker needs to be added to the tracking segment marker, the time interval is calculated by subtracting the generation time of the span marker at each position in the tracking segment marker from the current time, according to the order of the span marker generation times. If the time interval corresponding to the span marker is less than a preset duration, and the number of currently recorded span markers in the tracking segment marker is less than a preset data amount, then the new span marker is added to the tracking segment marker. If the time interval corresponding to the span marker is less than the preset duration, and the number of currently recorded span markers in the tracking segment marker is greater than or equal to the preset data amount, then the new span marker is discarded, that is, the new span marker is not added to the tracking segment marker.

[0173] In a feasible application scenario, if the TraceSegment is a singly linked list storing the collected data of span markers per unit time, in the form of "span ID + generation time timestamp", then, for example, if the preset data amount of span markers included in the TraceSegment generated per minute is set to 2 (span_limit_per_min), the singly linked list of TraceSegment markers is SpanNode1(span-1, T1) -> SpanNode2(span-2, T2). When a new SpanNode3(span-3, T3) is initialized and added to the singly linked list of TraceSegment markers, the following update action is performed.

[0174] First, calculate the time interval T_interval (T_interval = T_current – ​​T1) between the generation time T1 of the first node SpanNode1 in the tracking segment marker singly linked list and the current time T_current. Second, if the time interval T_interval is greater than one minute, delete the first SpanNode1 node from the tracking segment marker singly linked list. Then, take the next node and calculate the time interval between the generation time of the next node and the current time. If the time interval between the generation time of the next node and the current time is greater than one minute, delete the next node from the tracking segment marker singly linked list, until the time interval T_interval between the generation time of the next node and the current time in the tracking segment marker singly linked list is less than one minute or there is no next node. Finally, determine the remaining number of nodes in the tracking segment marker singly linked list. If the remaining number is greater than or equal to a preset number of 2, refuse to add SpanNode3 to the tracking segment marker singly linked list. If the remaining number is less than the preset number of 2, add SpanNode3 to the tracking segment marker singly linked list.

[0175] As can be seen from the above, by designing the spansize capacity field in the TraceSegment data structure and globally modifying this value after configuration updates, the initial capacity of subsequent TraceSegments is changed. This controls the upper limit of the number of span markers Tracespan contained in each TraceSegment per unit time, preventing a large number of span markers from being generated by too many collection components in a single microservice node, which would lead to an increase in the total time consumption of behavioral data sampling of the microservice node.

[0176] Referring to the above embodiments, Figure 12 and Figure 13 , Figure 12 The diagram shown is a timing diagram of link tracing for user service requests in an embodiment of this application. Figure 13 The diagram shown is a system block diagram of a distributed system according to an embodiment of this application. The distributed system includes a microservice node integrating a data acquisition agent module, a data receiving module, and a management module.

[0177] against Figure 12 First, the data acquisition agent module is pluggable, and the acquisition code is injected into the microservice node during application startup. Second, the microservice node registers clients and initializes the reporting subsystem, services, and instances to the management module. Then, the management module distributes service configuration information such as acquisition speed and acquisition scope to the microservice node. Further, the microservice node processes business requests, enables tracing, and reports the collected behavioral data to the data receiving module. The data receiving module collects and stores the behavioral data. Then, the microservice node can also periodically report client host load metrics to the management module. Load metrics include, but are not limited to, central processing unit (CPU) load, memory usage, disk input / output (I / O) usage, real-time traffic, application garbage collection frequency and time, etc. Based on the reported host load metrics, the management module predicts the load of the client host, obtains the predicted acquisition speed for the next stage, and dynamically distributes the calculated acquisition speed for the next stage so that the client can adjust in real time.

[0178] against Figure 13 The microservice nodes that integrate the data collection agent module include: agent instrumentation module, application indicator monitoring module, configuration service client, data reporting module, and distributed microservice nodes.

[0179] Here, the instrumentation module described above dynamically instrumentes the service during startup, preloading the pluggable module before the client starts and injecting the code into the microservice node via inter-process communication. The application metrics monitoring module calls the Java Virtual Machine (JVM) monitoring interface to obtain the microservice application's CPU load, memory usage, disk I / O utilization, real-time traffic, and application garbage collection frequency and time. The configuration service client receives real-time changes to service configuration information from the management module, including collection frequency and target scope. The data reporting module, an RPC-based data reporting client, reports load data and behavioral data separately. The distributed microservice node, instrumented with tracing logic code before startup, records user input and response during business request processing, synchronizes this to the data reporting module, and adds or transmits the TraceID during downstream service calls.

[0180] The aforementioned data receiving module includes data receiving service, data parsing service, data storage service, and storage cluster.

[0181] Here, the aforementioned data receiving service receives behavioral data traffic (corresponding to the aforementioned tracing tags) reported by microservice nodes. This behavioral data traffic includes a tracing identifier (TraceID), subsystem, service, client Internet Protocol (IP), request content set, response content set, timestamp, and custom tag information. The aforementioned data parsing service, based on the semantic concepts of the OpenTracing protocol, parses the reported behavioral data into storage objects with the data structures shown in Table 1. The aforementioned data storage service is used to dump the data to a message queue middleware and a relational database.

[0182] component Components Plugin component names, such as the HttpClient component that initiates HTTP calls. tag Label Key-value pair structure, which can be used to customize tag information. peer peer resource type This refers to accessing the downstream component type; for example, a database record would be recorded as DB. operationName Operational resource location For RPC remote calls, record the method name.

[0183] Table 1

[0184] The aforementioned management modules include client registration service, load data collection service, visualization service, and configuration center server.

[0185] Here, the aforementioned client registration service is used to report to the service registry center after the microservice node starts, recording the online / offline status of the integrated proxy instrumentation module. The load data collection service is used to collect host load metrics reported by the data reporting module of the microservice node. The visualization service is used to display information. The configuration center server is used to distribute service configuration information on demand to the clients where the microservice node resides.

[0186] In a feasible application scenario, combined with Figure 13 and Figure 14 , Figure 14 This illustration demonstrates a microservice node integration proxy instrumentation module for a distributed system. Taking a distributed system, such as an external business system, comprising three modules: microservice node A, microservice node B, and microservice node C, as an example, microservice node A provides account services, microservice node B provides product services, and microservice node C provides transaction services. A user's request to purchase a product triggers a transaction, and the implementation process is as follows.

[0187] The first step is to load the plugin into the agent instrumentation module.

[0188] Here, the proxy instrumentation module loads plugin modules included in Application Performance Management (WeAPM) such as Spring, HttpClient, Netty, and OkHttp, preloading the logic code of the behavior data collection plugin. Here, APM is an application behavior and problem analysis tool used in distributed architecture systems to capture application data and quickly locate and resolve problems.

[0189] The second step is to embed the agent instrumentation module into the microservice node.

[0190] Here, microservice nodes, such as Java applications, add the JavaAgent parameter to the startup parameters to specify the WeAPM agent program. After the microservice node starts, the agent instrumentation module communicates with the microservice node through inter-process communication to inject the bytecode converted by the plugin code into the microservice node's JVM, thereby completing the implantation.

[0191] The third step is to register the client service.

[0192] Here, the microservice node implantation logic includes reporting metadata and online status to the client registration service of the management module. The metadata includes subsystem number, service name, instance ID, initialization configuration key-value pairs, and client version number.

[0193] The fourth step is to trigger a transaction upon receiving a user's business request.

[0194] Here, a user's subscription operation triggers a transaction, enters the microservice gateway, the gateway routes to account service A for authentication, then to product service B to obtain the product list, and finally to transaction service C to complete the subscription.

[0195] The fifth step is to obtain the request content and response content of the business request based on the business request.

[0196] Here, based on the business request, the request content and response content can be obtained through the following process:

[0197] A1, Request Data Capture - Function Determined.

[0198] Here, Account Service A uses a common Spring framework service to handle user requests. The proxy instrumentation module, based on the JavaAgent mechanism and combined with the bytecode enhancement tool (bytebuddy), determines the code areas in Account Service A's JVM that use executable functions annotated with annotations such as "org.springframework.web.bind.annotation.GetMapping" as the code areas for injecting logic.

[0199] A2, requesting data capture - TraceSpan tag acquisition.

[0200] Here, the parameters Object[]Allarguments of the executable function are extracted to obtain the message content of the business request. The message content includes, but is not limited to, account information, destination service address, client characteristics, etc., and is stored in the data structure of the trace marker TraceSpan in key-value pair format. The data structure of the trace marker TraceSpan is shown in Table 2.

[0201]

[0202] Table 2

[0203] It should be noted that after a user initiates a business request, the authentication process is first recorded using [User ID, Password] as request parameters to Account Service A. At this time, the User ID and password are also stored as key-value pairs in the TraceSpan data structure used for account tracking. Next, a product query is initiated using [User ID, List of Wealth Management Companies, Deposit Products / Fund Products] as request parameters to Product Service B. Again, the User ID, List of Wealth Management Companies, and Deposit Products / Fund Products are stored as key-value pairs in the TraceSpan data structure used for product span tracking. Finally, a subscription is initiated using [Product ID, Subscription Amount, Redemption / Rollover at Maturity] as request parameters to Subscription Service C. Again, the User ID, Subscription Amount, and Redemption / Rollover at Maturity are stored as key-value pairs in the TraceSpan data structure used for subscription tracking. All three actions are marked as trace IDs for this business request.

[0204] A3, Request data capture - combination of span marker TraceSpan and trace segment marker TraceSegment.

[0205] Here, the data structure of the TraceSegment marker is shown in Table 3.

[0206] [TraceSpan] TraceSpan collection traceID Link tracing: The transaction ID to which this TraceSegment belongs [data_binary] All metadata contained in this TraceSegment segmentID The unique identifier of the TraceSegment spansize The number of TraceSpans contained in this TraceSegment

[0207] Table 3

[0208] Here, refer to Figure 9 As shown, Account Service A is the entry service for processing business requests. In addition to storing the message content of the business request, the TraceSpan data structure generates the span marker Tracespan-1 of service unit 1 in Account Service A based on the span identifier spanID=1 of service unit 1 in Account Service A, the trace identifier traceID=1 corresponding to the business request, the span identifier pspanID=0 of the previous service unit of service unit 1, and the timestamp T1. Furthermore, Account Service A uses the username and password from the login request as parameters to query the database authentication of Service Unit 2. This request will generate a trace marker Tracespan-2 for Service Unit 2 based on the span identifier spanID=2, the trace identifier traceID=1 corresponding to the business request, the span identifier pspanID=1 of the previous service unit of Service Unit 2, and the timestamp T2. Thus, two trace markers Tracespan for accessing Account Service A are formed. The data structures of the two trace markers Tracespan are merged into the first trace segment marker TraceSegment-1, so that one first trace segment marker TraceSegment includes multiple tracespan data.

[0209] A4, Request Data Capture - Transmission of trace ID (traceID) between microservice nodes.

[0210] Here, account service A initiates an HTTP request to call product service B. Based on the Agent probe function, the trace identifier traceID of TraceSegment-1 is generated, extracted and written into the custom field X-Weapm-TraceID in the HTTP request header, and then followed by the service call to product service B.

[0211] A5, request behavior data reporting.

[0212] Here, Account Service A, based on the Agent probe function, uploads the TraceSegment to the data receiving module, which then parses the trace ID as the primary key and writes it to the storage.

[0213] A6, Request Data Capture - Function Determined.

[0214] Here, product service B uses the Agent probe function to identify executable functions as the code region for implanted logic.

[0215] A7, requesting data capture - TraceSpan tag acquisition.

[0216] Here, the parameter list of the executable function is extracted, parsed, and stored in a new TraceSpan data structure in key-value pair format.

[0217] A8, Link Tracing - Trace ID Resolution.

[0218] Based on the new traceSpan data structure, the existence of the X-Weapm-TraceID field in the HTTP Header is determined and parsed as the trace identifier traceID. Subsequent actions of the thread in processing this request will all use this trace ID as the traceid.

[0219] A9. Each microservice makes a downstream call, repeating steps A1-A8, recording the HTTP traffic (behavior) data set generated by the TraceSegment in each microservice and reporting it to the data receiving module, and passing the trace identifier traceID as the trace ID of the user's business request.

[0220] A10, Response Data Capture - Content Acquisition.

[0221] Here, for the executable function identified above, the proxy instrumentation module injects logic before the executable function returns, parsing the response content returned by the executable function. Since the HTTP response content is stored using a Java IO stream base class object, and given its readable-only characteristic, to ensure the application can still read it after the instrumentation code is read, the following approach is adopted: a memory buffer is created, the IO stream base class data is copied, and its pointer to the number of bytes read is reset to the initial position. This allows for repeated retrieval of the HTTP response content without causing the microservice to be unable to read the response content.

[0222] It should be noted that after the business request is processed by the business system, the instrumented code enters the response data capture phase. The following actions are recorded sequentially: An authentication request is sent to Account Service A, which returns response data containing [User ID, authentication successful, and permission scope] after authentication; a product query request is sent to Product Service B, which returns response data containing [product number list and product description resources]; and a purchase request is sent to Purchase Service C, which returns response data containing [User ID, product ID, purchase amount, and purchase result]. The response data for these three actions is all marked with the current tracking identifier `traiceID`. At this point, the request and response data for user transactions processed separately by the distributed microservices are completely captured.

[0223] It is important to emphasize that current link tracing solutions fail to address the issue of data being read only once, making it impossible to obtain service response data. Specifically, they only reveal whether a user initiated an authentication query for an account service and whether it was successful; they query for a product service but the specific product list returned is unknown; and they submit a purchase application but the purchased product and amount are unknown. This results in user behavior data being presented with only the action known, not the result, creating a data gap that cannot support subsequent behavior auditing or application data asset auditing (whether the service involves sensitive data processing).

[0224] A11, HTTP Response Capture - Data Reporting.

[0225] Here, the microservice node writes the response data to the TraceSegment marker and uploads it to the data receiving module.

[0226] The sixth step is to update the service configuration information of the microservice nodes.

[0227] Here, the proxy instrumentation module integrates a general configuration center client based on ZooKeeper, enabling batch changes to the service configuration information of microservices with integrated proxy instrumentation modules at different scales, such as single points or subsystems. The specific process of updating the service configuration information of microservice nodes is explained below.

[0228] B1. Configuration Center Server - Access Enabled

[0229] Here, the proxy instrumentation module is coupled with the configuration center client SDK and the access mode is set (selectable local configuration mode / remote mode / remote non-update mode). In remote mode, the microservice starts, loads the server configuration, and backs it up to the disk. In local mode, the microservice starts and reads the configuration from the disk.

[0230] B2. Configuration Center Server - Change Configuration

[0231] Here, the configuration center server changes the configuration, refreshes the state and content of the ZooKeeper node, and triggers the hot update mechanism of the configuration service SDK (Software Development Kit) in the microservice. The configuration service SDK pulls the configuration according to the hot update notification and updates the microservice memory configuration, thereby updating the service configuration information of the microservice node.

[0232] It should be noted that, in order to control the upper limit of the number of TranceSpans contained in each TranceSegment and prevent a large number of TranceSpans from being generated by too many collection components in a single service, which would increase the total time spent on microservice behavior data sampling, a spansize capacity field can be designed in the TranceSegment data structure. After the configuration is updated, this value can be modified globally so that the initial capacity of subsequent segments changes.

[0233] As can be seen from the above, by using the proxy instrumentation module to implant the process of collecting microservice access behavior data, discrete behavior records can be linked into complete user transactions. This allows us to obtain the behavior data of a single point service in the distributed system, trace the user transaction operation path, reconstruct the user behavior profile, analyze the abnormal points and fault points in the behavior chain, and thus obtain the timing sequence and complete data of each sub-operation triggered by the user transaction in the distributed system, effectively solving the problems of data disorder and broken links.

[0234] The following continues to describe the exemplary structure of the link tracing device 154 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 3 As shown, the software module stored in the link tracing device 154 in the memory 150 can be the link tracing device in the distributed system 100, including:

[0235] The receiving module 1541, when the distributed system is started and the proxy instrumentation module is preloaded, receives the service call session through the microservice node if it detects that a user has initiated a business request.

[0236] The determination module 1542 is used to determine the executable functions of a service call session; wherein, the service call session includes a service call request and a service call response, and the service call session carries a trace identifier;

[0237] Processing module 1543 is used to dynamically instrument executable functions through the proxy instrumentation module to obtain the session content in the service call session;

[0238] The processing module 1543 is also used to track the behavioral transactions corresponding to the user's business requests based on the tracking identifier and session content.

[0239] It should be noted that the description of the apparatus in this application embodiment is similar to the description of the method embodiment described above, and has similar beneficial effects as the method embodiment; therefore, it will not be repeated. For technical details not disclosed in this apparatus embodiment, please refer to the description of the method embodiment of this application for understanding.

[0240] This application provides a storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to perform the method provided in this application, for example... Figure 4-8 The method shown in 10-11.

[0241] In some embodiments, the storage medium may be a computer-readable storage medium, such as a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic surface memory, optical disc, or a compact disk-read-only memory (CD-ROM); or it may be a device that includes one or any combination of the above-mentioned memories.

[0242] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0243] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file containing other programs or data, for example, in one or more scripts within a Hyper Text Markup Language (HTL) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files storing one or more modules, subroutines, or code sections). As an example, executable instructions may be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0244] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.< / methodinfo> < / methodinfo> < / methodinfo>

Claims

1. A link tracing method, characterized in that, include: When a distributed system is started and a proxy instrumentation module is preloaded, if a user initiates a business request, the microservice node receives the service call session and determines the executable function of the service call session; wherein, the service call session includes a service call request and a service call response, and the service call session carries a tracking identifier; When the microservice node supports decorator objects replacing specific components, a decorator instance with data caching function is generated using byte data streams; based on the service call session, the hook function used to override the proxy instrumentation module in the executable function is determined and overridden; the parameters carried by the overridden hook function are replaced with the decorator instance, the replaced hook function is executed, the session content is obtained, and stored in the cache area; If the microservice node does not support decorator objects replacing specific components, the binary code block of the executable function is obtained; the binary code block is converted and decompiled using HTTP data processing objects in the object pool to obtain the source code block of the executable function; a keyword corresponding to the service call session is determined, the keyword including returning an input stream and returning an output stream; from the function list contained in the source code block, a target function information object corresponding to the keyword is queried, and the target source code block corresponding to the target function information object is obtained; the target source code block is read line by line to determine the line number of the source code that meets the matching conditions, wherein the matching conditions include the existence of an assignment identifier and the presence of a feature function corresponding to the keyword in the expression on the right side of the assignment identifier; a capture function included in the proxy instrumentation module is inserted into the next line after the line number, and the session data returned by the feature function is copied and parsed through the capture function to obtain the session content; Based on the tracking identifier and the session content, the behavioral transactions corresponding to the user's business request are tracked.

2. The method according to claim 1, characterized in that, If the capture function does not parse the session data returned by the feature function, the method further includes: Obtain the object hash value of the session data, and obtain the object set composed of all objects at runtime of the distributed system through the instrumentation function included in the proxy instrumentation module; In the object collection, the data stream object corresponding to the object hash value is found using the object hash value as the key; The attributes of the data stream object are parsed using bytecode tools, including a cursor pointer that indicates the read status of data stored in a memory address. If the cursor pointer points to the end address of the memory address where the session data is located, adjust the cursor pointer to the start address of the memory address. The data stream object is serialized and then re-injected into the object pool to read the session content.

3. The method according to claim 1 or 2, characterized in that, The tracking of the user's behavioral transactions based on the tracking identifier and the session content includes: When the service call session is a service call request, a first trace segment marker for the microservice node is generated based on the request content and the trace identifier, wherein the session content includes the request content; Based on the first trace segment marker of the microservice node, a downstream call request is generated for the microservice node to make service calls to the downstream microservice node; Send the downstream call request to the downstream microservice node so that the downstream microservice node generates a first downstream trace segment marker based on the downstream call request, until all call requests between the microservice nodes included in the behavioral transaction are completed; The first trace segment marker is reported through the microservice node, and the first downstream trace segment marker is reported through the downstream microservice node; When the service call session is a service call response, a second trace segment tag is generated for each microservice node based on the response content and trace identifier, and the corresponding second trace segment tag is reported by each microservice node, wherein the session content includes the response content; The tracking of the user's behavioral transactions is determined based on the first tracking segment marker, the first downstream tracking segment marker, and the second tracking segment marker.

4. The method according to claim 3, characterized in that, The tracking segment markers include span markers, and the method further includes: Obtain the service configuration information of the microservice node, wherein the service configuration information includes the collection speed and collection range of the span marker; Obtain hot update information for changes to the service configuration information issued by the configuration service center of the distributed system; The hot update mechanism of the configuration service of the microservice node is triggered. The configuration service pulls new configuration information from the configuration service center according to the hot update information and updates the service configuration information according to the new configuration information.

5. The method according to claim 4, characterized in that, If the hot update information is information that updates the acquisition speed, the method includes: Obtain the preset number of span markers generated within a preset time period; When a new span marker is to be added to the tracking segment marker, obtain the time interval between the current time and the generation time of the span marker at the target position in the tracking segment marker; If the time interval is greater than the preset duration, the span marker of the target location is deleted from the tracking segment marker; If the time interval is less than the preset duration and the number of span markers in the tracking segment markers is less than the preset number, the new span marker is added to the tracking segment markers.

6. A link tracing device, characterized in that, include: The receiving module, when the distributed system is started and the proxy instrumentation module is preloaded, receives the service call session through the microservice node if it detects that a user has initiated a business request. A determination module is used to determine the executable functions of the service call session; wherein the service call session includes a service call request and a service call response, and the service call session carries a tracking identifier; The processing module is used to generate a decorator instance with data caching function using byte data stream when the microservice node supports decorator object replacement of specific components; based on the service call session, determine the hook function in the executable function used to override the hook function provided by the proxy instrumentation module, and override the hook function; replace the parameters carried by the overridden hook function with the decorator instance, execute the replaced hook function, obtain the session content, and store it in the cache area; The processing module is further configured to: obtain the binary code block of the executable function when the microservice node does not support decorator objects replacing specific components; convert and decompile the binary code block using HTTP data processing objects in the object pool to obtain the source code block of the executable function; determine the keyword corresponding to the service call session, the keyword including returning an input stream and returning an output stream; query the target function information object corresponding to the keyword from the function list contained in the source code block, and obtain the target source code block corresponding to the target function information object; read the target source code block line by line to determine the line number of the source code that meets the matching conditions, wherein the matching conditions include the existence of an assignment identifier and the presence of a feature function corresponding to the keyword in the expression on the right side of the assignment identifier; insert the capture function included in the proxy instrumentation module into the next line after the line number, and copy and parse the session data returned by the feature function through the capture function to obtain the session content; The processing module is also used to track the behavioral transactions corresponding to the user's business request based on the tracking identifier and the session content.

7. A distributed system, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the method according to any one of claims 1 to 5.

8. A storage medium, characterized in that, It stores executable instructions for causing a processor to execute, thereby implementing the method of any one of claims 1 to 5.

Images