Link monitoring method and device for distributed system, electronic equipment and medium

Abstract

The invention provides a link monitoring method and device for a distributed system, electronic equipment and a medium, and the method comprises the steps: capturing the operation state of an application program corresponding to a monitoring agent program based on the monitoring agent program installed at each computing node of the distributed system according to a preset link tracking protocol, corresponding operation information is obtained; wherein the application programs installed in the computing nodes are developed based on multiple programming languages; converting the operation information of the running state of each application program into tracking data in a uniform format; sending the tracking data to a monitoring platform; wherein the monitoring platform is used for analyzing tracking data of different application programs to identify an application layer event, and the application layer event is used for being associated with network layer transmission data to realize link monitoring. According to the method, full-link monitoring can be realized in a multi-language and multi-protocol scene, and the method is suitable for a complex application scene.

Description

Technical Field

[0001] The present invention relates to the field of operation and maintenance technology, and in particular to a link monitoring method, device, electronic equipment and medium for a distributed system. Background Art

[0002] With the widespread application of distributed systems, enterprises have built multi-level applications in complex network environments with multiple languages ​​and multiple protocols. These applications use different programming languages ​​and support different network communication protocols. How to achieve full-link monitoring of distributed systems is a key issue that needs to be solved urgently. The existing technology uses traditional monitoring tools to automatically monitor and track distributed links for applications in a single programming language, which has great limitations and is difficult to achieve full-link monitoring in complex scenarios with multiple languages ​​and multiple protocols. Summary of the invention

[0003] The present invention provides a link monitoring method, device, electronic device and medium for a distributed system, so as to solve the problem that the conventional monitoring tools used in the prior art to automatically monitor and track distributed links of applications in a single programming language have great limitations and are difficult to adapt to complex application scenarios of multiple programming languages.

[0004] The present invention provides a link monitoring method for a distributed system, comprising: According to a preset link tracking protocol, based on a monitoring agent installed in each computing node of the distributed system, the running status of the application corresponding to the monitoring agent is captured, and the corresponding operation information is obtained; wherein the application installed in each computing node is developed based on multiple programming languages; Convert the operational information of each application's running status into tracking data in a unified format; The tracking data is sent to a monitoring platform; wherein the monitoring platform is used to parse the tracking data of different applications to identify application layer events, and the application layer events are used to associate with network layer transmission data to achieve link monitoring.

[0005] In some embodiments, capturing the running status of the application corresponding to the monitoring agent program and obtaining the corresponding operation information includes: In an application program based on communication in a first programming language, performing bytecode enhancement based on the link tracking protocol, injecting a first monitoring logic, capturing key operations of the application program based on communication in the first programming language, and determining operation information of the application program based on communication in the first programming language; In the application program based on communication in the second programming language, the data transmission and tracking mechanism of the link tracking protocol is imitated, and the second monitoring logic is injected to capture the key operations of the application program based on communication in the second programming language and determine the operation information of the application program based on communication in the second programming language.

[0006] In some embodiments, the method further includes converting data transmitted by applications communicating based on different communication protocols into a format defined by the target protocol.

[0007] In some embodiments, the transmission protocols used by the network layer in the distributed system include Hypertext Transfer Protocol HTTP, Transmission Control Protocol TCP, and Session Initiation Protocol SIP.

[0008] The present invention also provides a link monitoring method for a distributed system, comprising: Obtain tracking data in a unified format converted from the operational information of each application's running status; Parse the tracking data of different applications, identify application layer events, and correlate the application layer events with network layer transmission data; The operation information of the running status of each application is captured based on a monitoring agent installed on each computing node of the distributed system according to a preset link tracking protocol; Furthermore, the applications installed in each computing node are developed based on a variety of programming languages.

[0009] In some embodiments, parsing the tracking data of different applications, identifying application layer events, and correlating and analyzing the application layer events with network layer transmission data includes: Parsing the tracking data of different applications to obtain parsed tracking data, wherein the parsed tracking data includes a plurality of link IDs and a plurality of session IDs; Determining a corresponding target application based on the multiple link IDs and the multiple session IDs; Obtaining the operation log of the target application, parsing the operation log, and obtaining the parsed operation log; According to the parsed operation log, application layer events are identified, and the application layer events are correlated and analyzed with network layer transmission data.

[0010] In some embodiments, after correlating and analyzing the application layer event with the network layer transmission data, the method further includes: Generate a full-link view of the distributed system, the full-link view including multiple links, each link including two application nodes; The full-link view is visualized.

[0011] The present invention also provides a link monitoring device for a distributed system, comprising: A first acquisition unit is used to capture the running status of the application corresponding to the monitoring agent program based on the monitoring agent program installed in each computing node of the distributed system according to a preset link tracking protocol, and obtain corresponding operation information; wherein the application program installed in each computing node is developed based on multiple programming languages; A processing unit, used to convert the operation information of the running status of each application into tracking data in a unified format; A sending unit is used to send the tracking data to a monitoring platform; wherein the monitoring platform is used to parse the tracking data of different applications to identify application layer events, and the application layer events are used to associate with network layer transmission data to achieve link monitoring.

[0012] The present invention also provides a link monitoring platform for a distributed system, comprising: A second acquisition unit is used to acquire tracking data in a unified format converted from the operation information of the running status of each application program; A correlation analysis unit, used to parse the tracking data of different applications, identify application layer events, and correlate and analyze the application layer events with network layer transmission data; The operation information of the running status of each application is captured based on a monitoring agent installed on each computing node of the distributed system according to a preset link tracking protocol; Furthermore, the applications installed in each computing node are developed based on a variety of programming languages.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a link monitoring method for a distributed system as described in any one of the above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the link monitoring method for a distributed system as described in any one of the above is implemented.

[0015] The link monitoring method, device, electronic device and medium for a distributed system provided by the present invention capture the running status of the corresponding application and obtain the corresponding operation information based on a monitoring agent installed in each computing node of the distributed system according to a preset link tracking protocol, wherein the application installed in each computing node is developed based on multiple programming languages, and the operation information of the running status of each application is converted into tracking data in a unified format, and the tracking data is sent to the monitoring platform, thereby enabling full-link monitoring of the distributed system in complex application scenarios of multiple programming languages. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a schematic diagram of the structure of a distributed system provided by an embodiment of the present invention.

[0018] Figure 2 This is one of the flow charts of the link monitoring method for a distributed system provided by an embodiment of the present invention.

[0019] Figure 3 It is a flow chart of a monitoring agent program provided in an embodiment of the present invention sending tracking data to a monitoring platform through an asynchronous task.

[0020] Figure 4 This is the second flow chart of the link monitoring method for a distributed system provided by an embodiment of the present invention.

[0021] Figure 5 This is the third flow chart of the link monitoring method for a distributed system provided by an embodiment of the present invention.

[0022] Figure 6 It is a full-link view of the distributed system provided by an embodiment of the present invention.

[0023] Figure 7 It is a schematic diagram of the structure of a link monitoring device for a distributed system provided by an embodiment of the present invention.

[0024] Figure 8 It is a structural diagram of a link monitoring platform for a distributed system provided by an embodiment of the present invention.

[0025] Fig. 9 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] The terms "first", "second", etc. in the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0028] In order to ensure the reliability and performance of distributed systems, it is necessary to monitor the entire link to capture the status and performance indicators of each link. Due to the diversity of programming languages ​​and communication protocols, the implementation of full-link monitoring solutions faces huge challenges.

[0029] The existing full-link monitoring solution is only applicable to application scenarios with a single programming language or a single communication protocol; traditional application performance management (APM) tools, such as SkyWalking, use Java Agent to achieve automated monitoring and distributed link tracking of Java applications, but their support capabilities are relatively limited when facing other programming languages ​​such as C++ and Go, as well as multi-language mixed programming environments. The existing technology has the following defects: 1) Limitations of cross-language support: SkyWalking’s core monitoring capabilities rely on Java Agent, which has good support for Java applications, but weak native support for C++ and Go applications, making it difficult to achieve unified link tracking in a multi-programming language environment; 2) Limitations of multi-protocol support: SkyWalking only supports HyperText Transfer Protocol (HTTP), Google Remote Procedure Call (gRPC) and other protocols. Support for other protocols such as Session Initiation Protocol (SIP) and Transmission Control Protocol (TCP) requires additional development work, making it difficult to achieve full-link monitoring in a multi-communication protocol environment; 3) It is difficult to integrate cross-language and cross-protocol data: In an environment with multiple programming languages ​​and multiple communication protocols, the formats and transmission methods of monitoring data vary, and existing solutions lack a unified data integration and analysis mechanism.

[0030] In response to the above problems, an embodiment of the present invention provides a link monitoring method for a distributed system. By following a preset link tracking protocol and based on a monitoring agent installed on each computing node of the distributed system, the running status of the corresponding application is captured, and the corresponding operation information is obtained. The operation information of the running status of each application is converted into tracking data in a unified format, and the tracking data is sent to a monitoring platform, thereby enabling full-link monitoring of the distributed system in complex application scenarios of multiple programming languages.

[0031] Figure 1 The structure diagram of the distributed system provided by the embodiment of the present invention is as follows. Figure 1 As shown, a distributed system is provided, which includes a client 110 and a server 120. The client 110 includes multiple computing nodes, and the server 120 includes a monitoring platform. A monitoring agent and an application are installed in each computing node. Each computing node is communicatively connected to the monitoring platform, and different computing nodes are communicatively connected to each other.

[0032] It should be noted that the computer nodes in a distributed system can be located in different physical locations. These nodes are connected together through a network and work together to complete a task or a group of tasks. The distributed system has strong computing power, reliability and scalability, and has high requirements for data consistency.

[0033] Optionally, the distributed system includes at least a hardware layer, an operating system layer, a network layer, a transport layer, an application layer, etc.

[0034] Optionally, the application programs in each computing node are developed based on multiple programming languages, such as Java, C++, Go and other programming languages.

[0035] Optionally, each computing node is installed with a monitoring agent program such as Java Agent, C++ Agent, Go Agent, etc.

[0036] Optionally, at least one monitoring agent and at least one application are installed in each computing node. For example, computing node 1 is installed with 1 monitoring agent (Java Agent) and 1 Java application. For another example, computing node 2 is installed with 3 monitoring agents (Java Agent, C++ Agent and Go Agent), 5 Java applications, 3 C++ applications and 1 Go application.

[0037] It should be noted that the Java Agent is used to monitor applications developed based on the Java programming language; the C++ Agent is used to monitor applications developed based on the C++ programming language; and the Go Agent is used to monitor applications developed based on the Go programming language.

[0038] Optionally, the applications in each computing node communicate based on different communication protocols, such as HTTP, TCP, SIP and other protocols.

[0039] Figure 2 This is a flow chart of a link monitoring method for a distributed system provided by an embodiment of the present invention. Figure 2 As shown, a link monitoring method for a distributed system is provided, and the link monitoring method for a distributed system is applied to Figure 1 The distributed system in is taken as an example to illustrate, which includes the following steps: step 210, step 220, and step 230. The steps of the method flow are only used as a possible implementation mode of the present invention.

[0040] Step 210, according to the preset link tracking protocol, based on the monitoring agent installed in each computing node of the distributed system, capture the running status of the application corresponding to the monitoring agent and obtain the corresponding operation information; wherein, the application installed in each computing node is developed based on multiple programming languages.

[0041] Among them, the link tracking protocol is a technical means used to track the request flow path in a distributed system.

[0042] It should be noted that the working principle of the link tracking protocol is to mark each request by inserting a unique trace ID in the request path. When the request flows through the various computing nodes in the distributed system, this ID will be passed on. Each computing node will record its own processing time, latency, and request status, and send this information to the monitoring platform.

[0043] Among them, Trace represents a complete call chain, which is composed of multiple Spans. Span represents an operation or service call. Span is the basic unit in link tracing; Trace ID is a unique identifier used to identify the complete path of a specific request or transaction.

[0044] Optionally, before each application is started, a corresponding monitoring agent is used to load the code of each application based on bytecode enhancement technology, and monitoring logic code is added before and after the call of key methods (such as methods marked with Trace annotations) so that each application can implement a unified link tracking protocol.

[0045] The core idea of ​​the Trace annotation method is to mark the link ID between any two communicating applications and mark each link with a unique Trace ID.

[0046] Optionally, the running state of the application includes at least: a start state, a normal running state, a stop state, a pause state, an error state, a maintenance state, an offline state and a high load state.

[0047] Optionally, the operation information includes at least: operation name, start time, end time, duration, status code, error information, metadata (such as session ID, Trace ID), resource usage, and dependencies between different operations.

[0048] Among them, the multiple programming languages ​​include at least Java, C++, Go and other programming languages; Java is a cross-platform object-oriented programming language, which can achieve cross-platform through the Java virtual machine. Java has strong stability and security and is the preferred programming language for many large enterprises; C++ supports object-oriented programming and generic programming, and is suitable for developing applications that require high performance and real-time response; Go is a statically typed, compiled programming language. The syntax of the Go language is concise and clear, and it has powerful concurrent processing capabilities, which has advantages in developing high-performance applications.

[0049] Optionally, in each computing node, according to the preset link tracking protocol, the running status of the Java application is captured based on the Java Agent, and the corresponding operation information is obtained; the running status of the C++ application is captured based on the C++ Agent, and the corresponding operation information is obtained; the running status of the Go application is captured based on the Go Agent, and the corresponding operation information is obtained.

[0050] It should be noted that the monitoring agent needs to interact with the corresponding application to obtain the running status of the application and the corresponding operation information; the monitoring agent can obtain the running status of the corresponding application and the corresponding operation information through various channels, such as reading the log files of the application, intercepting HTTP requests and responses, using the Software Development Kit (SDK) or Application Programming Interface (API), etc.

[0051] It can be understood that by adopting the preset link tracking protocol, cross-language support for distributed systems can be achieved. Based on the link tracking protocol, the running status and operation information of the corresponding application can be captured by using the monitoring agent installed on each computing node, thereby enabling unified monitoring of applications in different programming languages ​​in the distributed system, which helps to simplify the monitoring process and improve monitoring efficiency and user experience.

[0052] Step 220: Convert the operation information of each application program's running status into tracking data in a unified format.

[0053] The tracking data includes at least the Trace ID, session ID, operation name, start time, end time, duration and other data in a unified format.

[0054] Optionally, define a unified data structure to represent the tracing data; this data structure contains all necessary fields, such as Trace ID, session ID, operation name, start time, end time, duration, status code, error message, etc.

[0055] Optionally, data cleaning is performed on the operation information of the running status of each application.

[0056] Optionally, data mapping is performed on the operation information of the running status of each application program, and the operation information from different sources is mapped into a unified data structure. For example, the abnormal information of application programs in different programming languages ​​is converted into a unified error information field.

[0057] Optionally, data enhancement is performed on the operation information of each application's running status, and additional context information is added, such as session ID, Trace ID, etc.

[0058] Optionally, the tracking data in a unified format is serialized for storage and transmission.

[0059] It is understandable that by converting the operational information of the running status of each application into tracking data in a unified format, the compatibility of the distributed system with applications in different programming languages ​​can be improved and the monitoring process can be simplified. In addition, the tracking data in a unified format provides a standardized interface, allowing applications developed in different programming languages ​​to communicate and exchange data through this interface, which helps to simplify the integration process between different applications, reduce development and maintenance costs, and improve the observability, maintainability, scalability and collaboration efficiency of the distributed system.

[0060] Step 230: Send the tracking data to the monitoring platform; wherein the monitoring platform is used to parse the tracking data of different applications to identify application layer events, and the application layer events are used to associate with the network layer transmission data to achieve link monitoring.

[0061] The monitoring platform is also used to receive tracking data sent by the monitoring agent program of each computing node.

[0062] Optionally, the monitoring agent of each computing node sends the tracking data to the monitoring platform through an asynchronous task, thereby improving the transmission efficiency of the tracking data and reducing the impact on the main thread performance.

[0063] Figure 3 The following is a flow chart of the monitoring agent program provided in the embodiment of the present invention sending tracking data to the monitoring platform through an asynchronous task. Figure 3 As shown in the figure, librdkafka and Fork processes are newly created on the computing node. The monitoring platform includes Kafka and Observability Analysis Platform (OAP). The monitoring agent cpp2sky of the C++ application reports the captured tracking data to librdkafka, and then transmits it to Kafka of the monitoring platform through the Fork process, and then reports it to OAP from Kafka.

[0064] It should be noted that the original data reporting mechanism is that the monitoring agent directly reports the tracking data to the OAP of the monitoring platform.

[0065] Among them, librdkafka is a high-performance C / C++ client library that supports high-throughput and low-latency scenarios. librdkafka also has the characteristics of message reliability. Applications can distribute messages through librdkafka to enhance the security and efficiency of data transmission.

[0066] Among them, the Fork process is a mechanism used to create a new process in the operating system. The fork() function creates a new process by copying the address space of the current process. The new process is called a child process, and the original process is called a parent process. By using fork() to create multiple child processes, parallel processing can be achieved to prevent blocking of the main business process and ensure the stability of link monitoring under high concurrency.

[0067] Among them, Kafka is a distributed stream processing platform. Kafka stores data as a partitioned log file arranged in chronological order. Each topic can have multiple partitions, and each partition can have multiple copies. Kafka has the characteristics of high throughput and strong scalability.

[0068] It should be noted that asynchronous tasks allow the program to continue executing other tasks while waiting for certain operations to complete (such as I / O operations), thereby avoiding the waste of resources caused by waiting and improving the concurrency and efficiency of the program; by using asynchronous tasks, the sending of tracking data can be performed in the background without interfering with the normal operation of the main thread, thereby improving the user experience.

[0069] It should be noted that the application layer of a distributed system is the layer closest to the user, and the application program implements specific business logic at the application layer; the network layer of a distributed system is responsible for the transmission of data packets between different networks, including routing selection, logical address allocation, and cross-network data transmission.

[0070] Among them, application layer events refer to various operations or state changes that occur during the operation of an application. These events can be user interaction behaviors (such as clicks, slides, etc.), or state changes or errors within a distributed system. Application layer events are not only related to the responsiveness and accuracy of user interactions, but also directly affect the performance and stability of the application.

[0071] Optionally, the application layer event can be a page event, that is, an event involving the loading, unloading and visibility change of a page, for example, an event triggered when a user enters or exits a web page; the application layer event can also be a trigger event, that is, an event triggered by a specific user operation, such as a button click or a form submission.

[0072] The network layer transmission data at least includes IP data packets, routing control messages and network layer protocol data units.

[0073] In an embodiment of the present invention, based on a preset link tracking protocol and based on a monitoring agent installed in each computing node of a distributed system, the running status of a corresponding application is captured and corresponding operation information is obtained, wherein the application installed in each computing node is developed based on multiple programming languages, and the operation information of the running status of each application is converted into tracking data in a unified format, and the tracking data is sent to a monitoring platform, thereby enabling full-link monitoring of the distributed system in complex application scenarios of multiple programming languages.

[0074] In some embodiments, capturing and monitoring the running status of an application corresponding to the agent program and obtaining corresponding operation information includes: In an application program based on communication in a first programming language, bytecode enhancement is performed based on a link tracking protocol, a first monitoring logic is injected, key operations of the application program based on communication in the first programming language are captured, and operation information of the application program based on communication in the first programming language is determined; In the application based on the second programming language communication, the data transmission and tracking mechanism of the link tracking protocol is imitated, the second monitoring logic is injected, the key operations of the application based on the second programming language communication are captured, and the operation information of the application based on the second programming language communication is determined.

[0075] Among them, critical operations refer to operations in the application that have a significant impact on the overall performance, stability, and user experience.

[0076] Optionally, key operations include at least method calls and returns, exception handling, database operations, external service calls, user behaviors, and other operations.

[0077] Optionally, key operations that need to be monitored for different types of applications are determined, data collection points are set at the beginning and end of the key operations, and corresponding monitoring logic is determined.

[0078] It should be noted that applications based on the first programming language communication use the link tracking protocol for data transmission and tracking, such as Java applications based on SkyWalking's SW8 protocol for data transmission and tracking; applications based on the second programming language communication imitate the data transmission and tracking mechanism of the link tracking protocol for data transmission and tracking, such as C++ applications and Go applications, which perform data transmission and tracking in a manner compatible with the SW8 protocol.

[0079] It should be noted that the application based on the second programming language communication uses a specific library (such as the cross-platform C++ library Boost.Asio) for network communication and uses the customized SW8 protocol for serialization and deserialization of data packets.

[0080] In some embodiments, the above method further includes converting data transmitted by applications communicating based on different communication protocols into a format defined by the target protocol.

[0081] The application programs that communicate based on different communication protocols include at least application programs that communicate based on communication protocols such as HTTP, TCP, and SIP.

[0082] Optionally, the target protocol may be SkyWalking's SW8 protocol.

[0083] It should be noted that different applications interact with each other and transmit data based on different communication protocols. Each application can parse received messages of different protocols, such as SIP messages, TCP messages, etc.

[0084] In some embodiments, the transmission protocols based on the network layer in the distributed system include HTTP, TCP and SIP.

[0085] Among them, HTTP is an application layer protocol and the basis of data communication on the World Wide Web. It is highly flexible and easy to expand. In a distributed system, HTTP can be used to implement lightweight communication between computing nodes. TCP is a transport layer protocol that provides reliable, connection-oriented network data stream services. TCP is the basis of many upper-layer protocols (such as HTTP). In a distributed system, TCP can be used for communication between back-end services. SIP is an application layer protocol that is mainly used to create, modify and terminate real-time sessions, such as voice and video calls. In a distributed system, SIP can be used to implement real-time messaging.

[0086] It is understandable that HTTP, as one of the most widely used protocols on the Internet, has good compatibility and interoperability. By adopting the HTTP protocol, different computing nodes can communicate and exchange data efficiently; TCP, as the underlying transmission protocol, provides stable and reliable data transmission services for upper-layer applications, allowing different applications to communicate smoothly; SIP, as a standardized multimedia communication protocol, supports multiple media types and encoding formats, promoting interoperability between different multimedia applications.

[0087] Figure 4 The second flow chart of the link monitoring method for a distributed system provided by an embodiment of the present invention. Figure 4 As shown, an entry trace is created in the application, TCP, SIP and other requests are initiated and processed, contexts are reported, and full-link monitoring of multiple communication protocols such as TCP and SIP is achieved.

[0088] Specifically, the code of the Java application is modified and extended based on the Java Agent, and the code of the C++ application is modified and extended based on the C++Agent, so that multi-protocol communication can be achieved between the Java application and the C++ application.

[0089] Figure 5 The third flow chart of the link monitoring method for a distributed system provided by an embodiment of the present invention. Figure 5 As shown, the present invention also provides a link monitoring method for a distributed system, including the following steps 510 and 520.

[0090] Step 510: Acquire tracking data in a unified format converted from the operation information of the running status of each application program; Step 520: Parse the tracking data of different applications, identify application layer events, and correlate and analyze the application layer events with the network layer transmission data; Among them, the operational information of the running status of each application is captured based on the monitoring agent installed on each computing node of the distributed system according to the preset link tracking protocol; Furthermore, the applications installed in each computing node are developed based on a variety of programming languages.

[0091] Optionally, the application program installed in each computing node is developed based on a programming language such as Java, C++, or Go.

[0092] Optionally, the monitoring platform obtains the tracking data by interacting with the monitoring agent program of each computing node.

[0093] Optionally, the tracking data of a specific port is captured through a pre-set filter; and the application layer events are associated with the network layer transmission data through a preset reorganization logic.

[0094] Optionally, based on the timestamp and context information of the application layer event, the application layer event is correlated and analyzed with the network layer transmission data.

[0095] For example, in an e-commerce application, when a user clicks a “Buy” button, this event is recorded in the log, and the network layer transmission data is analyzed to identify the network requests related to the event.

[0096] It is understandable that by parsing the tracking data of different applications, identifying application layer events, and correlating and analyzing application layer events with network layer transmission data, it is possible to more accurately locate performance bottlenecks, quickly detect potential security threats, and help optimize the user experience.

[0097] Optionally, the monitoring agent of each computing node is used to: In an application program based on communication in a first programming language, bytecode enhancement is performed based on a link tracking protocol, a first monitoring logic is injected, key operations of the application program based on communication in the first programming language are captured, and operation information of the application program based on communication in the first programming language is determined; In the application based on the second programming language communication, the data transmission and tracking mechanism of the link tracking protocol is imitated, the second monitoring logic is injected, the key operations of the application based on the second programming language communication are captured, and the operation information of the application based on the second programming language communication is determined.

[0098] Among them, the first programming language refers to the most basic and widely used programming language, such as Java; the second programming language refers to a programming language suitable for special scenarios or emerging fields, such as C++ and Go.

[0099] Optionally, key operations of different applications are determined based on their characteristics and monitoring requirements, such as method calls, database operations, external service calls, etc.

[0100] Optionally, the monitoring agent of each computing node is also used to convert data transmitted by applications communicating based on different communication protocols into a format defined by a target protocol, such as a format defined by SkyWalking's SW8 protocol.

[0101] Optionally, the transmission protocols based on the network layer in the distributed system include HTTP, TCP and SIP.

[0102] In some embodiments, parsing the tracking data of different applications, identifying application layer events, and correlating the application layer events with network layer transmission data may include: Parse the tracking data of different applications to obtain parsed tracking data, where the parsed tracking data includes multiple link IDs and multiple session IDs; Determine a corresponding target application based on the multiple link IDs and the multiple session IDs; Obtain the running log of the target application, parse the running log, and obtain the parsed running log; Based on the parsed operation logs, application layer events are identified and application layer events are correlated and analyzed with network layer transmission data.

[0103] The link ID is used to identify a complete request or transaction, and each link ID includes only two application nodes; the session ID is used to identify a user's session.

[0104] Optionally, the tracking data includes at least multiple link IDs, multiple session IDs and operation information.

[0105] Optionally, a preset mapping table is searched according to the link ID and the session ID to determine the target application corresponding to the link ID and the session ID.

[0106] The parsed operation log includes information such as timestamp, event type, user ID, and operation result.

[0107] Optionally, the application layer event may be a user operation (such as registration, invitation, session termination), a system error, or a performance bottleneck.

[0108] Optionally, compare the timestamp of the application layer event and the timestamp of the network layer transmission data (such as network request / response). When the difference between the two timestamps is less than a preset threshold, determine that the application layer event is related to the network layer transmission data; use context information such as link ID and session ID to further verify the accuracy of the association.

[0109] For example, if the link ID of an HTTP request matches the user ID of an application layer event, and the request time coincides with the event occurrence time, then the two events are correlated.

[0110] It is understandable that by parsing the tracking data of different applications, determining the target application based on multiple link IDs and session IDs, parsing the operation log of the application, and correlating and analyzing the application layer events with the network layer transmission data, it is helpful to fully understand the operating status and user behavior of the distributed system, accurately locate performance bottlenecks and latency issues, and quickly and accurately detect potential security threats, which helps to reasonably allocate and plan resources, optimize the user interface and interaction design, and improve the user experience.

[0111] In some embodiments, after correlating and analyzing the application layer event with the network layer transmission data, the method further includes: Generate a full-link view of the distributed system, which includes multiple links, each of which includes two application nodes; Provides a visual display of the entire link view.

[0112] Each link is marked with a unique link ID, such as Trace ID.

[0113] Optionally, SkyWalking is used to process and analyze the tracking data in real time and generate a complete link view.

[0114] Optionally, the full-link view is visualized through a visualization interface.

[0115] Optionally, use the visualization tool Grafana to visualize the full-link view, create a topology diagram to display each node and their connection relationships, create a time series diagram to show how events on each link change over time, create a scatter plot to display performance indicators of different links, and create a heat map to display hot spots in the distributed system.

[0116] Figure 6 This is a full-link view of the distributed system provided by the embodiment of the present invention. Figure 6 As shown, in each link, one node sends an invitation (INVITE), a network protocol packet (PRACK) or an acknowledgment packet (ACK), and the other node receives the invitation, network protocol packet or acknowledgment packet. Each link is assigned a unique identifier, and the data of each link is summarized in the code debugger (Code Composer Studio, CCS).

[0117] It can be understood that by generating a full-link view of the distributed system and visualizing the full-link view, the interaction process between each node in the entire distributed system can be clearly displayed, which helps developers quickly identify abnormal links and thus speed up troubleshooting and repair.

[0118] The following describes a link monitoring device for a distributed system provided by an embodiment of the present invention. The link monitoring device for a distributed system described below and the link monitoring method for a distributed system described above can refer to each other.

[0119] Figure 7 A schematic diagram of the structure of a link monitoring device for a distributed system provided by an embodiment of the present invention, such as Figure 7 As shown, the link monitoring device 700 for a distributed system includes: The first acquisition unit 710 is used to capture the running state of the application corresponding to the monitoring agent program based on the monitoring agent program installed in each computing node of the distributed system according to the preset link tracking protocol, and obtain the corresponding operation information; wherein the application program installed in each computing node is developed based on multiple programming languages; Processing unit 720, used to convert the operation information of the running status of each application into tracking data in a unified format; The sending unit 730 is used to send the tracking data to the monitoring platform; wherein the monitoring platform is used to parse the tracking data of different applications to identify application layer events, and the application layer events are used to associate with the network layer transmission data to achieve link monitoring.

[0120] Optionally, capturing and monitoring the running status of the application corresponding to the agent program and obtaining corresponding operation information include: In an application program based on communication in a first programming language, bytecode enhancement is performed based on a link tracking protocol, a first monitoring logic is injected, key operations of the application program based on communication in the first programming language are captured, and operation information of the application program based on communication in the first programming language is determined; In the application based on the second programming language communication, the data transmission and tracking mechanism of the link tracking protocol is imitated, the second monitoring logic is injected, the key operations of the application based on the second programming language communication are captured, and the operation information of the application based on the second programming language communication is determined.

[0121] Optionally, the link monitoring device 700 for a distributed system further includes: The data conversion unit is used to convert data transmitted by applications communicating based on different communication protocols into a format defined by a target protocol.

[0122] Optionally, the transmission protocols used by the network layer in the distributed system include HTTP protocol, TCP protocol and SIP protocol.

[0123] The following describes a link monitoring platform for a distributed system provided by an embodiment of the present invention. The link monitoring platform for a distributed system described below and the link monitoring method for a distributed system described above can refer to each other.

[0124] Figure 8 A schematic diagram of the structure of a link monitoring platform for a distributed system provided in an embodiment of the present invention is shown in FIG800. The link monitoring platform 800 for a distributed system includes: The second acquisition unit 810 is used to acquire the tracking data in a unified format converted from the operation information of the running status of each application program; A correlation analysis unit 820 is used to parse the trace data of different applications, identify application layer events, and perform correlation analysis between application layer events and network layer transmission data; Among them, the operational information of the running status of each application is captured based on the monitoring agent installed on each computing node of the distributed system according to the preset link tracking protocol; Furthermore, the applications installed in each computing node are developed based on a variety of programming languages.

[0125] Optionally, parse the trace data of different applications, identify application layer events, and correlate application layer events with network layer transmission data, including: Parse the tracking data of different applications to obtain parsed tracking data, where the parsed tracking data includes multiple link IDs and multiple session IDs; Determine a corresponding target application based on the multiple link IDs and the multiple session IDs; Obtain the running log of the target application, parse the running log, and obtain the parsed running log; Based on the parsed operation logs, application layer events are identified and application layer events are correlated and analyzed with network layer transmission data.

[0126] Optionally, the link monitoring platform 800 for a distributed system further includes: A generation unit, used to generate a full-link view of the distributed system, where the full-link view includes multiple links, and each link includes two application nodes; Provides a visual display of the entire link view.

[0127] It should be noted here that the link monitoring device for a distributed system and the link monitoring platform for a distributed system provided in the embodiments of the present invention can implement all the method steps implemented in the above-mentioned link monitoring method embodiment for a distributed system, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0128] Fig. 9 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Fig. 9 As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930 and a communication bus 940, wherein the processor 910, the communication interface 920 and the memory 930 communicate with each other through the communication bus 940. The processor 910 may call the logic instructions in the memory 930 to execute a link monitoring method for a distributed system, the method comprising: based on a preset link tracking protocol, based on a monitoring agent installed in each computing node of the distributed system, capturing the running state of the application corresponding to the monitoring agent, and obtaining corresponding operation information; wherein the application installed in each computing node is developed based on multiple programming languages; converting the operation information of the running state of each application into tracking data in a unified format; sending the tracking data to the monitoring platform; wherein the monitoring platform is used to parse the tracking data of different applications to identify application layer events, and the application layer events are used to associate with network layer transmission data to realize link monitoring.

[0129] In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0130] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the link monitoring method for a distributed system provided by the above-mentioned methods, the method comprising: based on a preset link tracking protocol, based on a monitoring agent installed in each computing node of the distributed system, capturing the running status of an application corresponding to the monitoring agent, and obtaining corresponding operation information; wherein the application installed in each computing node is developed based on multiple programming languages; converting the operation information of the running status of each application into tracking data in a unified format; sending the tracking data to a monitoring platform; wherein the monitoring platform is used to parse the tracking data of different applications to identify application layer events, and the application layer events are used to associate with network layer transmission data to achieve link monitoring.

[0131] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0132] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A link monitoring method for a distributed system, characterized in that: include: According to a preset link tracking protocol, based on a monitoring agent installed in each computing node of the distributed system, the running status of the application corresponding to the monitoring agent is captured, and the corresponding operation information is obtained; wherein the application installed in each computing node is developed based on multiple programming languages; Convert the operational information of each application's running status into tracking data in a unified format; The tracking data is sent to a monitoring platform; wherein the monitoring platform is used to parse the tracking data of different applications to identify application layer events, and the application layer events are used to associate with network layer transmission data to achieve link monitoring.

2. The link monitoring method for a distributed system according to claim 1, characterized in that: The capturing of the running status of the application corresponding to the monitoring agent program and obtaining the corresponding operation information includes: In an application program based on communication in a first programming language, performing bytecode enhancement based on the link tracking protocol, injecting a first monitoring logic, capturing key operations of the application program based on communication in the first programming language, and determining operation information of the application program based on communication in the first programming language; In the application program based on communication in the second programming language, the data transmission and tracking mechanism of the link tracking protocol is imitated, and the second monitoring logic is injected to capture the key operations of the application program based on communication in the second programming language and determine the operation information of the application program based on communication in the second programming language.

3. The link monitoring method for a distributed system according to claim 1, characterized in that: The method further includes converting data transmitted by applications communicating based on different communication protocols into a format defined by a target protocol.

4. The link monitoring method for a distributed system according to claim 3, characterized in that: The transmission protocols used by the network layer in the distributed system include the Hypertext Transfer Protocol HTTP, the Transmission Control Protocol TCP and the Session Initiation Protocol SIP.

5. A link monitoring method for a distributed system, characterized in that: include: Obtain tracking data in a unified format converted from the operational information of each application's running status; Parse the tracking data of different applications, identify application layer events, and correlate the application layer events with network layer transmission data; The operation information of the running status of each application is captured based on a monitoring agent installed on each computing node of the distributed system according to a preset link tracking protocol; Furthermore, the applications installed in each computing node are developed based on a variety of programming languages.

6. The link monitoring method for a distributed system according to claim 5, characterized in that: The parsing of the tracking data of different applications, identifying application layer events, and correlating the application layer events with network layer transmission data includes: Parsing the tracking data of different applications to obtain parsed tracking data, wherein the parsed tracking data includes a plurality of link IDs and a plurality of session IDs; Determining a corresponding target application based on the multiple link IDs and the multiple session IDs; Obtaining the operation log of the target application, parsing the operation log, and obtaining the parsed operation log; According to the parsed operation log, application layer events are identified, and the application layer events are correlated and analyzed with network layer transmission data.

7. The link monitoring method for a distributed system according to claim 5, characterized in that: After correlating and analyzing the application layer event with the network layer transmission data, the method further includes: Generate a full-link view of the distributed system, the full-link view including multiple links, each link including two application nodes; The full-link view is visualized.

8. A link monitoring device for a distributed system, characterized in that: include: A first acquisition unit is used to capture the running status of the application corresponding to the monitoring agent program based on the monitoring agent program installed in each computing node of the distributed system according to a preset link tracking protocol, and obtain corresponding operation information; wherein the application program installed in each computing node is developed based on multiple programming languages; A processing unit, used to convert the operation information of the running status of each application into tracking data in a unified format; A sending unit is used to send the tracking data to a monitoring platform; wherein the monitoring platform is used to parse the tracking data of different applications to identify application layer events, and the application layer events are used to associate with network layer transmission data to achieve link monitoring.

9. A link monitoring platform for a distributed system, characterized in that: include: A second acquisition unit is used to acquire tracking data in a unified format converted from the operation information of the running status of each application program; A correlation analysis unit, used to parse the tracking data of different applications, identify application layer events, and correlate and analyze the application layer events with network layer transmission data; The operation information of the running status of each application is captured based on a monitoring agent installed on each computing node of the distributed system according to a preset link tracking protocol; Furthermore, the applications installed in each computing node are developed based on a variety of programming languages.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, it implements the link monitoring method for a distributed system as described in any one of claims 1 to 4, or implements the link monitoring method for a distributed system as described in any one of claims 5 to 7.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the link monitoring method for a distributed system as described in any one of claims 1 to 4, or implements the link monitoring method for a distributed system as described in any one of claims 5 to 7.

Images