Distributed tracking information enhancement method and system

Abstract

The invention discloses a distributed tracking information enhancement method and system, and belongs to the technical field of distributed tracking and Internet, and the method comprises the steps: 1) supplementing distributed tracking information data: complementing all calling behavior information related to a kernel system interface on the basis of the current distributed tracking data, therefore, the data is in butt joint with the data acquired by the kernel; 2) acquiring information data and docking information of a key function needing to be monitored from a kernel; and 3) enhancing the distributed tracking data based on the data acquired by the kernel: based on the docking information of the tracking information and the docking information monitored by the kernel, adding the information monitored by the kernel to the current distributed tracking information to complete information enhancement. According to the method, the content of underlying processing is added for distributed tracking information, so that richer information sources are added for rapid fault positioning and problem backtracking.

Description

Technical Field

[0001] The present invention relates to the field of distributed tracing and Internet technology, and in particular to a method and system for enhancing distributed tracing information. Background Art

[0002] Distributed tracing technology obtains the call information of each sub-application (microservice) through various means (active output of applications, passive library / system call extraction, etc.) to generate span content, and integrates spans with the same traceid to form a distributed tracing result of a call in the distributed system. This technology focuses on the tracing information of the 7-layer protocol, so that it can provide application developers / operators with statistics on the time consumption of each component in the distributed environment. However, the current technology does not include detailed information on the 4-layer protocol, storage, and IO processing. Therefore, operators cannot determine the root cause of the anomaly based on the current technical solution, and can only understand that the delay of a certain operation is long. Summary of the invention

[0003] The technical task of the present invention is to address the above deficiencies and provide a method and system for enhancing distributed tracing information, thereby adding underlying processing content to the distributed tracing information, thereby adding a richer information source for rapid fault location and problem backtracing.

[0004] The technical solution adopted by the present invention to solve its technical problem is:

[0005] A method for enhancing distributed tracing information, comprising:

[0006] 1) Complete the distributed tracing information data:

[0007] Based on the current distributed tracing data, complete the information of all call behaviors related to the kernel system interface, so as to connect with the data obtained by the kernel;

[0008] 2) Obtain information data and docking information of key functions to be monitored from the kernel;

[0009] 3) Enhance the distributed tracing data based on the data obtained by the kernel:

[0010] Based on the docking information of the tracing information and the docking information of the kernel monitoring, the kernel monitoring information is appended to the current distributed tracing information to complete the information enhancement.

[0011] This method provides a method for completing the docking information of distributed tracing information. On the basis of the current distributed tracing information, it adds a docking method for docking kernel data, increases the collection of kernel information, and adds the definition of docking methods for kernel data collection.

[0012] This method provides the definition of kernel information and related docking information, and then based on the compatible docking information, it can enhance the data hierarchy of distributed tracing technology based on the collected kernel information, thereby providing more support for abnormal problem location and definition.

[0013] Furthermore, the method of completing the distributed tracing information data and connecting it with the data obtained by the kernel includes the following types:

[0014] Complete the required data, including the complete five-tuple data and related process ID when establishing / listening to the connection, the timestamp and process ID during IO operation, the state change record of process control, and the timestamp data;

[0015] Add new data, including associating the global traceid in the call request with each key system call operation, thereby associating each request with the socket five-tuple, with the timestamp of each IO operation, and with similar key operations.

[0016] The interface information added to the distributed tracing information should be able to be used to connect to the kernel information, so it needs to contain the necessary information to define a system operation, including the process ID, user ID, execution timestamp, and parameters of the related calls (such as socket five-tuples, io addresses and offsets, etc.).

[0017] Furthermore, kernel information includes the time when the data packet is constructed and sent, the time when the IO operation is initiated and the file system completes the read, and the time when the process scheduling state changes.

[0018] Furthermore, the method for completing the information data of distributed tracking is to modify and enhance the current tracking system.

[0019] Furthermore, the docking information structure of distributed tracing data and kernel data contains the same fields;

[0020] The information data and docking information of the key functions to be monitored are obtained from the kernel, the key functions in the kernel are monitored based on the ebpf tool, and the information on the required monitoring content is obtained, and the docking information is prepared so that it can be embedded in the distributed tracking data.

[0021] Furthermore, the kernel information docking information provides all information of the interface call associated with the kernel operation, including the process number, user number, call timestamp, and related call parameter data of the call;

[0022] When it is necessary to monitor the socket sending time, record the time when the socket is created and the time when it is passed to the network card driver, so as to complete the time observation of data sending; at the same time, record the process number, timestamp, and user number information of the data packet sent, so as to enhance the tracking data.

[0023] This step is implemented by using the ebpf-based monitoring tool to complete kernel information observation.

[0024] Furthermore, the method for enhancing the distributed tracking data based on the data obtained by the kernel is to modify and enhance the associated input components of the current existing tracking system.

[0025] The present invention also claims to protect a distributed tracking information enhancement system, comprising:

[0026] The distributed tracing information data module is used to complete all the call behavior information related to the kernel system interface based on the current distributed tracing data, so as to connect with the data obtained by the kernel;

[0027] Obtain information data of key functions to be monitored and docking information modules from the kernel;

[0028] A module for enhancing distributed tracing data based on the data obtained by the kernel. Based on the docking information of the tracing information and the docking information of the kernel monitoring, the kernel monitoring information is attached to the current distributed tracing information to complete the information enhancement.

[0029] The system specifically implements the enhancement of distributed tracing information through the above-mentioned distributed tracing information enhancement method.

[0030] The present invention also claims protection for a distributed tracking information enhancement device, comprising: at least one memory and at least one processor;

[0031] The at least one memory is used to store a machine-readable program;

[0032] The at least one processor is used to call the machine-readable program to implement the above method.

[0033] The present invention also claims protection for a computer-readable medium having computer instructions stored thereon, which implement the above method when executed by a processor.

[0034] Compared with the prior art, the distributed tracking information enhancement method and system of the present invention has the following beneficial effects:

[0035] The present invention appends kernel monitoring data to distributed tracing data, enhances the monitoring capability of distributed tracing information from layer 7 to layer 4, and can conveniently expand the locating range to the kernel when locating anomalies, thereby being able to infer whether the abnormal problem occurs in the hardware device. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flow chart of an implementation method of the distributed tracing information enhancement method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0038] A method for enhancing distributed tracing information, comprising:

[0039] 1. Complete the distributed tracing information data:

[0040] Based on the current distributed tracing data, complete all the information about the calling behaviors related to the kernel system interface, so as to connect with the data obtained by the kernel. The related connection methods include the following types:

[0041] (1) Complete the required data, including the complete five-tuple data and related process ID when establishing a connection / listening to a connection, the timestamp and process ID during IO operations, the state change record of process control, and the timestamp data.

[0042] (2) Add new data, including associating the global traceid in the call request with each key system call operation, thereby associating each request with the socket five-tuple, with the timestamp of each IO operation, and with similar key operations.

[0043] This step is implemented by modifying and enhancing the existing tracking system.

[0044] 2. Obtain the information data and docking information of the key functions to be monitored from the kernel:

[0045] Based on the ebpf tool, key functions in the kernel are monitored, and information about the required monitoring content is obtained. At the same time, docking information is prepared so that it can be embedded in distributed tracing data.

[0046] For example, when you need to monitor the socket sending time, you need to record the time when the socket is created and the time when it is passed to the network card driver, so as to complete the time observation of data sending. However, you also need to record the process number, timestamp, user number and other information that completes the sending of the data packet, so as to enhance the tracking data.

[0047] This step is implemented by using the ebpf-based monitoring tool to complete kernel information observation.

[0048] 3. Enhance the distributed tracing data based on the data obtained by the kernel:

[0049] Based on the docking information of the tracing information and the docking information of the kernel monitoring, the kernel monitoring information is appended to the current distributed tracing information to complete the information enhancement.

[0050] This step is implemented by modifying and enhancing the associated input components of the existing tracking system.

[0051] This method adds a docking method for docking kernel data based on the current distributed tracing information, increases the collection of kernel information, and adds a definition of a docking method for kernel data collection.

[0052] The purpose of adding the docking method for distributed tracing information is to match the data from the kernel. Therefore, the docking information structure of distributed tracing data and kernel data must contain the same fields.

[0053] The interface information added to the distributed tracing information should be able to be used to connect to the kernel information, so it needs to contain the necessary information to define a system operation, including the process ID, user ID, execution timestamp, and parameters of the related calls (such as socket five-tuples, io addresses and offsets, etc.).

[0054] The main purpose of adding kernel information is to complete the status information of the system kernel execution after calling the system interface in distributed tracing. Therefore, the kernel information that can be provided includes but is not limited to the time of data packet construction and sending, the time of initiating IO operations and completing file system reading, and the time of process scheduling status change.

[0055] The purpose of the kernel information docking information is to match the distributed tracing information data. Therefore, it is necessary to provide all the information of the interface call associated with the kernel operation, including but not limited to the process ID, user ID, call timestamp, related call parameters and other data of the call.

[0056] This method provides the definition of kernel information and related docking information, and then based on the compatible docking information, it can enhance the data hierarchy of distributed tracing technology based on the collected kernel information, thereby providing more support for abnormal problem location and definition.

[0057] An embodiment of the present invention further provides a distributed tracing information enhancement system, which implements the enhancement of distributed tracing information through the distributed tracing information enhancement method described in the above embodiment.

[0058] The system includes:

[0059] 1. The distributed tracing information data module is used to complete all the call behavior information related to the kernel system interface based on the current distributed tracing data, so as to connect with the data obtained by the kernel; the related connection methods include the following types:

[0060] Complete the required data, including the complete five-tuple data and related process number when establishing / listening to the connection, the timestamp and process number during IO operation, the state change record and timestamp of process control, and other data.

[0061] Add new data, including associating the global traceid in the call request with each key system call operation, thereby associating each request with the socket five-tuple, with the timestamp of each IO operation, and with similar key operations.

[0062] The implementation of this module is to modify and enhance the existing tracking system.

[0063] 2. Obtain the information data of the key functions to be monitored and the docking information module from the kernel:

[0064] Based on the ebpf tool, key functions in the kernel are monitored, and information about the required monitoring content is obtained. At the same time, docking information is prepared so that it can be embedded in distributed tracing data.

[0065] For example, when you need to monitor the socket sending time, you need to record the time when the socket is created and the time when it is passed to the network card driver, so as to complete the time observation of data sending. However, you also need to record the process number, timestamp, user number and other information that completes the sending of the data packet, so as to enhance the tracking data.

[0066] The implementation of this module is based on the ebpf monitoring tool to complete kernel information observation.

[0067] 3. Enhancement module for distributed tracing data based on data obtained by the kernel:

[0068] Based on the docking information of the tracing information and the docking information of the kernel monitoring, the kernel monitoring information is added to the current distributed tracing information to complete the information enhancement;

[0069] The implementation of this module is to modify and enhance the associated input components of the existing tracking system.

[0070] An embodiment of the present invention further provides a distributed tracking information enhancement device, including: at least one memory and at least one processor;

[0071] The at least one memory is used to store a machine-readable program;

[0072] The at least one processor is used to call the machine-readable program to implement the distributed tracking information enhancement method described in the above embodiment.

[0073] The embodiment of the present invention further provides a computer-readable medium, on which computer instructions are stored, and when the computer instructions are executed by a processor, the distributed tracing information enhancement method described in the above embodiment is implemented. Specifically, a system or device equipped with a storage medium can be provided, on which software program codes implementing the functions of any of the above embodiments are stored, and a computer (or CPU or MPU) of the system or device is enabled to read and execute the program code stored in the storage medium.

[0074] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0075] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0076] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0077] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0078] The present invention is shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the code review methods in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the protection scope of the present invention.

Claims

1. A method for enhancing distributed tracing information, characterized in that: include: 1) Complete the distributed tracing information data: Based on the current distributed tracing data, complete the information of all call behaviors related to the kernel system interface, so as to connect with the data obtained by the kernel; 2) Obtain information data and docking information of key functions to be monitored from the kernel; 3) Enhance the distributed tracing data based on the data obtained by the kernel: Based on the docking information of the tracing information and the docking information of the kernel monitoring, the kernel monitoring information is appended to the current distributed tracing information to complete the information enhancement.

2. A method for enhancing distributed tracking information according to claim 1, characterized in that: The method of completing the distributed tracing information data and connecting it with the data obtained by the kernel includes the following types: Complete the required data, including the complete five-tuple data and related process ID when establishing / listening to the connection, the timestamp and process ID during IO operation, the state change record of process control, and the timestamp data; Add new data, including associating the global traceid in the call request with each key system call operation, thereby associating each request with the socket five-tuple, with the timestamp of each IO operation, and with similar key operations.

3. A method for enhancing distributed tracking information according to claim 1 or 2, characterized in that: Kernel information includes the time it takes to build and send a data packet, the time it takes for an IO operation to be initiated and for the file system to complete reading, and the time it takes for the process scheduling state to change.

4. A method for enhancing distributed tracking information according to claim 1 or 2, characterized in that: The method for completing the information data of distributed tracking is to modify and enhance the current existing tracking system.

5. The method for enhancing distributed tracking information according to claim 1, characterized in that: The interconnection information structure of distributed tracing data and kernel data contains the same fields; The information data and docking information of the key functions to be monitored are obtained from the kernel, the key functions in the kernel are monitored based on the ebpf tool, and the information on the required monitoring content is obtained, and the docking information is prepared so that it can be embedded in the distributed tracking data.

6. A method for enhancing distributed tracking information according to claim 5, characterized in that: The kernel information docking information provides all the information of the interface call associated with the kernel operation, including the process number, user number, call timestamp, and related call parameter data of the call; When it is necessary to monitor the socket sending time, record the time when the socket is created and the time when it is passed to the network card driver, so as to complete the time observation of data sending; at the same time, record the process number, timestamp, and user number information of the data packet sent, so as to enhance the tracking data.

7. A method for enhancing distributed tracking information according to claim 1, characterized in that: The method for enhancing the distributed tracking data based on the data obtained by the kernel is to modify and enhance the associated input components of the current existing tracking system.

8. A distributed tracking information enhancement system, characterized in that: include: The distributed tracing information data module is used to complete all the call behavior information related to the kernel system interface based on the current distributed tracing data, so as to connect with the data obtained by the kernel; Obtain information data of key functions to be monitored and docking information modules from the kernel; A module for enhancing distributed tracing data based on the data obtained by the kernel. Based on the docking information of the tracing information and the docking information of the kernel monitoring, the kernel monitoring information is attached to the current distributed tracing information to complete the information enhancement. The system specifically realizes the enhancement of distributed tracing information through the distributed tracing information enhancement method described in any one of claims 1 to 7.

9. A distributed tracking information enhancement device, characterized in that: include: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is used to call the machine-readable program to implement the method described in any one of claims 1 to 7.

10. A computer-readable medium, characterized in that The computer readable medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images