Link monitoring method and device for multi-type probe scene

Abstract

The invention relates to a link monitoring method and device for a multi-type probe scene, and the method comprises the following steps: S1, adding a preset field for data association and link series connection in an interaction process of all links of a calling link which is constructed in advance and comprises multi-type probes, application program performance management is realized by using a unified link propagation protocol; and S2, after the link propagation protocol is unified, realizing data reporting through kafka, performing data conversion through Datakit to obtain a preset form, and outputting the data to a preset observation platform. Compared with the prior art, communication across the front end and the rear end and between different types of probes at the rear end is achieved, the whole calling chain of the request can be seen in the observable platform no matter which technology is applied to the APM, the problem can be conveniently positioned in the problem checking process, and the link calling relation can be conveniently sorted under the micro-service architecture.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a link monitoring method and device for multi-type probe scenarios. Background Art

[0002] Under the microservice architecture, APM (Application Performance Management) and RUM (Real User Monitoring) are used to sort out the calling relationships between services and between services and front-end pages, and to troubleshoot problems in the process of mutual calling of services.

[0003] With the development of APM and RUM technologies, a variety of APM probes and RUM probes have been derived. Each probe has its own unique link propagation protocol. If different project teams use different probes, but there is interaction between projects, call chain disconnection may easily occur.

[0004] In response to the challenges of high concurrency and rapid demand changes, Chinese patent application publication number CN116232963A provides a link tracking method and system, which takes into account the response speed of problem discovery and the process of production and operation, and fully considers the design of production abnormal event processing, and can quickly perceive node failure. However, the application does not take into account the situation where the call chain is easily disconnected under the premise of multiple probes.

[0005] In summary, due to the different propagation protocols and data structures used by probes, the call chain data is easily fragmented and cannot be connected in series. Summary of the invention

[0006] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a link monitoring method and device for multi-type probe scenarios, so as to solve or partially solve the problem that link data is fragmented and cannot be connected in series in multi-type probe scenarios.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] One aspect of the present invention provides a link monitoring method for a multi-type probe scenario, comprising the following steps:

[0009] Step S1, for a pre-built call link including multiple types of probes, in the interaction process of each link of the call link, preset fields for data association and link concatenation are added, and a unified link propagation protocol is used to implement application performance management;

[0010] Step S2: After the link propagation protocol is unified, data reporting is realized through Kafka, and after the data is converted into a preset format through Datakit, it is output to the preset observation platform.

[0011] As a preferred technical solution, the call link includes a front-end link and multiple back-end links, wherein the multiple back-end links are deployed with multiple types of probes.

[0012] As a preferred technical solution, the multiple types of probes include at least two of DDtrace, OpenTelemetry, and Beifuyou.

[0013] As a preferred technical solution, in step S1, the interaction process of each link of the calling link includes WEB request and / or pinpoint protocol.

[0014] As a preferred technical solution, in step S1, the preset fields include a version parameter, a trace-id parameter, a parent-id parameter and a trace-flags parameter.

[0015] As a preferred technical solution, in step S1, the data association is an association between the front end and the back end, and the link series connection is a series connection between the links corresponding to the back end.

[0016] As a preferred technical solution, the link propagation protocol is the W3C link propagation protocol.

[0017] As a preferred technical solution, the step S2 specifically includes the following steps:

[0018] Use JSON and / or thrift protocol to report data through Kafka. Use Datakit to convert Kafka data into line protocol and then output it to the preset observation platform to display link data association.

[0019] Another aspect of the present invention provides an electronic device, comprising: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the aforementioned link monitoring method for multi-type probe scenarios.

[0020] Another aspect of the present invention provides a computer-readable storage medium, including one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the aforementioned link monitoring method for multi-type probe scenarios.

[0021] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0022] (1) Solving the problem of data communication failure between different probes: The present invention adds preset fields for data association and link concatenation in the interaction process of each link of the calling link, and uses a unified link propagation protocol to implement application performance management, thereby realizing the concatenation of back-end and front-end data, and the link concatenation between each link of the back-end.

[0023] (2) Solve the problem that data cannot be associated when displaying data after data collection: After the link propagation protocol is unified, the present invention realizes data reporting through Kafka, converts the data into a preset format through Datakit, and then outputs it to the preset observation platform to realize link data association display. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flowchart of a link monitoring method for a multi-type probe scenario in an embodiment;

[0025] Figure 2 Schematic diagram of the original call link including multiple types of probes and the observation platform in the embodiment;

[0026] Figure 3 The call link optimized by the solution of this embodiment;

[0027] Figure 4 It is a schematic diagram showing the association of link data in an embodiment. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] The definitions of the relevant protocols and data involved in the embodiment are as follows:

[0030] Beifuyou Probe uses the Protobuf protocol. Protobuf (Protocol Buffers) is a flexible and efficient serialization framework, mainly used for serialization and deserialization of structured data. Protobuf is used in RPC (remote procedure call) systems and data storage, especially in scenarios that require cross-language services and storage.

[0031] DDtrace uses the MessagePack protocol, which is an efficient binary serialization format that allows data to be exchanged between multiple languages, providing higher efficiency and smaller size.

[0032] Tingyun uses the JSON protocol. JSON (JavaScript Object Notation) is a lightweight data exchange format that is easy for people to read and write, and easy for machines to parse and generate. It is based on a subset of JavaScript. JSON has become an independent data format and is supported by multiple programming languages.

[0033] Observable uses line protocol, line protocol introduction:

[0034] Line Protocol is a text format used in the time series database InfluxDB to write data points into the database and to record and transmit time series data.

[0035] Observable consumes Kafka data, converts JSON and Protobuf protocol data in Tingyun and Beifuyou into line protocol data, receives MessagePack protocol of DDtrace through API interface, and then converts it into line protocol. Finally, the data is reported to the observable platform for unified data display.

[0036] Example 1

[0037] In view of the problems existing in the above-mentioned prior art, this embodiment provides a link monitoring method for multi-type probe scenarios, see Figure 1 , including the following steps:

[0038] Step S1, for the pre-built call link including multiple types of probes, in the interaction process of each link of the call link, preset fields for data association and link concatenation are added, and a unified link propagation protocol is used to implement application performance management.

[0039] The call link includes a front-end link and multiple back-end links, wherein the multiple back-end links are deployed with multiple types of probes.

[0040] Figure 2 The following is an example, where the front-end link includes Tingyun, and the back-end link includes DDtrace, Opentelemetry, and Beifuyou, which are deployed in three application software respectively. All three are APM probes. Probes have their own unique link propagation protocols. When different project teams use different probes, but there is interaction between projects, the call chain will be disconnected. Figure 2 In the project, Tingyun was used as the front-end user behavior probe, and Beifuyou, DDtrace and OpenTelemetry probes were used in the back-end. The propagation protocols and data structures used by each probe were different, resulting in problems such as fragmentation and inability to connect the call chain data.

[0041] To address this problem, this step uniformly uses the w3c protocol as the link propagation protocol to first implement link transparent transmission between DDtrace and OpenTelemetry.

[0042] To solve the link break problem between Tingyun and DDtrace, Tingyun's WEB supports adding traceId in the request header. The key is traceparent, the value is 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01, and the four parameters connected by "-" are version, trace-id, parent-id and trace-flags. After adding, when the front end sends a request, the back end can receive the link information of the front end, thereby associating the front end data with the back end data.

[0043] In order to solve the link-breaking problem between OpenTelemetry and Beifuyou, based on Beifuyou's native pinpoint protocol, Beifuyou needs to support the W3C link propagation protocol at the same time. On the basis of the original protocol, the traceparent field is added, and the value is 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01. The four parameters connected by "-" are version, trace-id, parent-id and trace-flags. After adding, it can realize the connection with the front-end data of Tingyun, and can also realize the connection with DDtrace and Opentelemetry.

[0044] Step S2: After the link propagation protocol is unified, data reporting is realized through Kafka, and after the data is converted into a preset format through Datakit, it is output to the preset observation platform.

[0045] See also Figure 4 ,In this step, after the links are uniformly transmitted using the W3C protocol, ,in order to display it in the observable platform, Tingyun data uses the JSON ,protocol and is transmitted through kafka, and Beifuyou uses the thrift protocol to transmit it through kafka. By setting the data converter, ,Tingyun and kafka data are converted into data that ,can be recognized by the observable platform, where the link conversion task is ,implemented in the Datakit plug-in.

[0046] Specifically, this step includes four stages: data collection, data reporting, data conversion and unified display.

[0047] Data collection stage: Data collection is achieved based on the W3C protocol between Tingyun deployed on the front end and three types of probes on the back end.

[0048] Data reporting stage: DDtrace and OpenTelemetry probes report data through http and grpc respectively, Tingyun and Beifuyou report data through json and trifle respectively, and Kafka.

[0049] In the data conversion stage, the listening cloud and kafka data are converted into data that can be recognized by the observability platform, and in this embodiment, it is converted into a line protocol.

[0050] Unified display: The converted data is sent to the observable platform to realize the associated display of link data.

[0051] In summary, this method uses the w3c link propagation protocol to achieve data communication between the front-end and back-end, and between different types of APM probes on the back-end to solve the problem of data communication failure between different APMs. On this basis, a data format converter is developed on Datakit, and the data format is converted into a line protocol through Datakit, and uniformly collected on the observable platform to achieve link data association display and solve the problem of data fragmentation and inability to associate. Regardless of the APM technology used in the application, the entire call chain of the request can be seen on the observable platform, which is convenient for locating problems during troubleshooting and is also conducive to sorting out link call relationships under the microservice architecture.

[0052] Example 2

[0053] This embodiment provides an electronic device, including: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the link monitoring method for multi-type probe scenarios as described in Example 1.

[0054] Example 3

[0055] This embodiment provides a computer-readable storage medium, including one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the link monitoring method for multi-type probe scenarios as described in Example 1.

[0056] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A link monitoring method for multi-type probe scenarios, characterized in that: The steps include: Step S1, for a pre-built call link including multiple types of probes, in the interaction process of each link of the call link, preset fields for data association and link concatenation are added, and a unified link propagation protocol is used to implement application performance management; Step S2: After the link propagation protocol is unified, data reporting is realized through Kafka, and after the data is converted into a preset format through Datakit, it is output to the preset observation platform.

2. A link monitoring method for multi-type probe scenarios according to claim 1, characterized in that: The call link includes a front-end link and multiple back-end links, wherein the multiple back-end links are deployed with multiple types of probes.

3. A link monitoring method for multi-type probe scenarios according to claim 2, characterized in that: The multiple types of probes include at least two of DDtrace, OpenTelemetry, and Beifuyou.

4. A link monitoring method for multi-type probe scenarios according to claim 1, characterized in that: In the step S1, the interaction process of each link of the calling link includes WEB request and / or pinpoint protocol.

5. A link monitoring method for multi-type probe scenarios according to claim 1, characterized in that: In the step S1, the preset fields include a version parameter, a trace-id parameter, a parent-id parameter, and a trace-flags parameter.

6. A link monitoring method for multi-type probe scenarios according to claim 1, characterized in that: In the step S1, the data association is an association between the front end and the back end, and the link series connection is a series connection between the links corresponding to the back end.

7. A link monitoring method for multi-type probe scenarios according to claim 1, characterized in that: The link propagation protocol is the W3C link propagation protocol.

8. A link monitoring method for multi-type probe scenarios according to claim 1, characterized in that: The step S2 specifically includes the following steps: Use JSON and / or thrift protocol to report data through Kafka. Use Datakit to convert Kafka data into line protocol and then output it to the preset observation platform to display link data association.

9. An electronic device, characterized in that: include: One or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the link monitoring method for multi-type probe scenarios as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: It comprises one or more programs executed by one or more processors of an electronic device, wherein the one or more programs comprise instructions for executing the link monitoring method for multi-type probe scenarios as described in any one of claims 1-8.

Images