Application monitoring method and system, electronic equipment and storage medium

Abstract

The invention relates to an application monitoring method and system, electronic equipment and a storage medium. The method is applied to the application monitoring system, the application monitoring system comprises a network probe and a central monitoring platform, the network probe is deployed in a host machine, the host machine is accessed to a plurality of virtual machines, each virtual machine is deployed with at least one service system, and the method comprises the following steps: respectively collecting the network communication flow of each virtual machine through the network probe; performing protocol analysis on the network communication flow of each virtual machine through the network probe to obtain application monitoring data of each service system; uniformly synchronizing all application monitoring data to a central monitoring platform through a network probe; and performing centralized storage and correlation analysis on all application monitoring data through the central monitoring platform. According to the scheme provided by the invention, the deployment efficiency and the resource utilization rate can be improved, the development and operation and maintenance cost is remarkably reduced, and the fault positioning efficiency and accuracy are improved.

Description

Technical Field

[0001] This application relates to the field of network security technology, and in particular to application monitoring methods, systems, electronic devices and storage media. Background Technology

[0002] Application monitoring can ensure that application services can provide users with the expected services stably, efficiently, and continuously, thereby supporting the achievement of business goals.

[0003] In related technologies, application monitoring is typically performed independently by each business system or microservice. Common methods include embedding or integrating log collection tools (such as Log4 and ELK), performance metric monitoring components (such as Prometheus), and distributed tracing systems (such as Zipkin and Jaeger). These monitoring components are usually deployed along with the business system or integrated by the development team during the application development phase. While this can achieve some observation of the operational status of the business system, the monitoring data is fragmented because each business system uses different monitoring solutions. Furthermore, monitoring data is often stored within each business system, making centralized analysis and display on a unified platform impossible. This decentralized monitoring approach severely impacts troubleshooting efficiency and fault location accuracy, especially in complex system environments involving multi-business system collaboration and cross-team development. Summary of the Invention

[0004] To address or partially address the problems existing in related technologies, this application provides an application monitoring method, system, electronic device, and storage medium, which can improve deployment efficiency and resource utilization, significantly reduce development and maintenance costs, and improve fault location efficiency and accuracy.

[0005] The first aspect of this application provides an application monitoring method applied to an application monitoring system, the application monitoring system including a network probe and a central monitoring platform, the network probe being deployed on a host machine, the host machine being connected to multiple virtual machines, each of the virtual machines deploying at least one business system, the method comprising: The network communication traffic of each virtual machine is collected using the network probes. The network probes are used to perform protocol parsing on the network communication traffic of each virtual machine to obtain application monitoring data for each business system. All application monitoring data are uniformly synchronized to the central monitoring platform through the network probe. The central monitoring platform centrally stores and correlates all application monitoring data.

[0006] In one embodiment, the host machine connects to multiple virtual machines via a bridge device; the step of collecting network communication traffic for each virtual machine using the network probe includes: The network probe monitors and collects the network communication traffic within the same virtual machine, and / or the network communication traffic between different virtual machines, and / or the network communication traffic between any virtual machine and an external network, which are forwarded by the bridge device respectively.

[0007] In one embodiment, each of the business systems consists of multiple application services, and the network communication traffic includes interactive data packets of each application service; the step of parsing the network communication traffic of each virtual machine using the network probe to obtain application monitoring data for each business system includes: The network probe performs protocol parsing on each of the interactive data packets to extract application monitoring data for each application service from each interactive data packet.

[0008] In one embodiment, the step of performing protocol parsing on each of the interactive data packets using the network probe to extract application monitoring data for each application service from each interactive data packet includes: The network probe decodes each layer of the protocol layer of each interactive data packet layer by layer, so as to extract the application monitoring data of each application service from each layer of the protocol layer of each interactive data packet. The multi-layered protocol system includes a link layer, a network layer, a transport layer, and an application layer. The application monitoring data includes key field information and business indicator information. The key field information includes at least one of the following: source IP, source port, destination IP, destination port, request identifier, request time, request duration, request method, request path, request body, and response body. The business indicator information includes at least one of the following: TCP handshake duration, connection error, retransmission rate, connection latency, return status code, response status, response time, response duration, database request processing time, and connection relationship.

[0009] In one embodiment, the step of uniformly synchronizing all application monitoring data to the central monitoring platform via the network probe includes: The network probes identify the protocol type of each interactive data packet at the application layer; the protocol type includes at least one of HTTP, DNS, MySQL, SSL, and SSH. The network probe writes all the application monitoring data into the corresponding log files according to the protocol type. The network probe is used to generate a corresponding target index for each of the log files; The network probe is used to synchronize all the log files and the corresponding target indexes to the central monitoring platform.

[0010] In one embodiment, the central monitoring platform includes Elasticsearch; the centralized storage and correlation analysis of all application monitoring data through the central monitoring platform includes: The central monitoring platform centrally stores all log files in Elasticsearch according to the corresponding target index. The central monitoring platform extracts the target log file from multiple log files already stored in Elasticsearch based on the target index and a preset time period. The central monitoring platform extracts abnormal monitoring data from the target log file based on the business indicator information. The central monitoring platform determines the target application service that matches the abnormal monitoring data from all the application services based on the key field information. The central monitoring platform associates the abnormal monitoring data with the target application service to push alarms.

[0011] In one embodiment, the central monitoring platform stores deployment information for each application service, including deployment IP and deployment port; the step of determining the target application service matching the abnormal monitoring data from all application services based on the key field information through the central monitoring platform includes: The central monitoring platform extracts the target IP and target port of the abnormal monitoring data from the key field information; the target IP is either the source IP or the destination IP, and the target port is either the source port or the destination port. The central monitoring platform matches the target IP with each of the deployed IPs and matches the target port with each of the deployed ports. The central monitoring platform identifies the application services corresponding to the deployment IP that successfully matches the target IP and the deployment port that successfully matches the target port as the target application services.

[0012] A second aspect of this application provides an application monitoring system, including a network probe and a central monitoring platform. The network probe is deployed on a host machine, and the host machine is connected to multiple virtual machines, each of which deploys at least one business system. The network probe is used to collect network communication traffic for each virtual machine; to perform protocol parsing on the network communication traffic of each virtual machine to obtain application monitoring data for each business system; and to uniformly synchronize all application monitoring data to the central monitoring platform. The central monitoring platform is used for centralized storage and correlation analysis of all application monitoring data.

[0013] A third aspect of this application provides an electronic device, comprising: Processor; and A memory that stores executable code, which, when executed by the processor, causes the processor to perform the method described above.

[0014] A fourth aspect of this application provides a computer-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method described above.

[0015] The fifth aspect of this application provides a computer program product comprising computer instructions that, when executed by a processor, implement the method described above.

[0016] The technical solution provided in this application may include the following beneficial results: The solution provided in this application is applied to an application monitoring system, which includes a network probe and a central monitoring platform. The network probe is deployed on a host machine, which hosts multiple virtual machines. Each virtual machine hosts at least one business system. The network probe collects network communication traffic from each virtual machine. The network probe then performs protocol parsing on the network communication traffic of each virtual machine to obtain application monitoring data for each business system. All application monitoring data is then synchronized to the central monitoring platform via the network probe. The central monitoring platform centrally stores and performs correlation analysis on all application monitoring data. This application, by deploying network probes on the host machine, covers the network communication traffic of all virtual machines, enabling the parsing and acquisition of application monitoring data from all business systems, thereby improving deployment efficiency and resource utilization. Furthermore, the central monitoring platform centrally stores and performs correlation analysis on the application monitoring data from all business systems, effectively avoiding redundant construction of monitoring systems, significantly reducing development and maintenance costs, and establishing monitoring links across business systems, thereby improving the efficiency and accuracy of fault location.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0018] The above and other objects, features and advantages of this application will become more apparent from the more detailed description of exemplary embodiments thereof in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments thereof.

[0019] Figure 1 This is a flowchart illustrating the application monitoring method in an embodiment of this application; Figure 2 This is another flowchart illustrating the application monitoring method in an embodiment of this application; Figure 3 This is a schematic diagram illustrating the network interaction of application services across host machines, as shown in the embodiments of this application. Figure 4 This is a schematic diagram illustrating the network interaction between the application service and the host machine, as shown in an embodiment of this application. Figure 5 This is a schematic diagram of the structure of the application monitoring system shown in the embodiments of this application; Figure 6 This is a schematic diagram of the structure of an electronic device shown in an embodiment of this application. Detailed Implementation

[0020] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to make this application more thorough and complete, and to fully convey the scope of this application to those skilled in the art.

[0021] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0022] It should be understood that although the terms "first," "second," "third," etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0023] In related technologies, application monitoring is usually completed independently by each business system or microservice. Common methods include embedding or integrating log collection tools (such as Log4, ELK), performance metric monitoring components (such as Prometheus), and distributed tracing systems (such as Zipkin, Jaeger).

[0024] It should be noted that Log4 is an open-source logging tool based on Java, ELK is Elasticsearch (a distributed search and analysis engine) + Logstash (a data processing pipeline) + Kibana (a data visualization platform), Prometheus is an open-source monitoring system and time series database, Zipkin is an open-source distributed tracing system, and Jaeger is a high-performance, highly scalable distributed tracing system.

[0025] The aforementioned monitoring components are typically deployed along with the business systems or integrated by the development team during the application development phase. While this allows for some observation of the operational status of the business systems, the fragmented distribution of monitoring data stems from the fact that each business system uses different monitoring solutions. Furthermore, monitoring data is often stored within each individual business system, making centralized analysis and visualization on a unified platform impossible. This decentralized monitoring approach severely impacts troubleshooting efficiency and fault location accuracy, especially in complex system environments involving multi-business system collaboration and cross-team development.

[0026] To address the aforementioned issues, this application provides an application monitoring method. By deploying network probes on the host machine, network communication traffic of all virtual machines can be covered, enabling the parsing and acquisition of application monitoring data for all business systems. This improves deployment efficiency and resource utilization. Furthermore, a central monitoring platform centrally stores and correlates the application monitoring data of all business systems, effectively avoiding redundant construction of monitoring systems, significantly reducing development and maintenance costs, and establishing monitoring links across business systems, thereby improving the efficiency and accuracy of fault location.

[0027] The technical solutions of the embodiments of this application are described in detail below with reference to the accompanying drawings.

[0028] Figure 1 This is a flowchart illustrating the application monitoring method in an embodiment of this application.

[0029] See Figure 1 The application monitoring method of this application may include: S110 collects network communication traffic for each virtual machine using network probes.

[0030] In this embodiment, it can be applied to an application monitoring system (or edge cloud application monitoring system), which is a unified and efficient monitoring system for the operational status of cross-business systems. Specifically, the application monitoring system may include network probes and a central monitoring platform. The central monitoring platform establishes an SSH (Secure Shell) connection with each host machine. The network probe may include at least one, and each network probe is deployed on a corresponding host machine. Each host machine connects to multiple virtual machines, and each virtual machine deploys at least one business system. In other words, the relationship between the network probe and the host machine is one-to-one, the relationship between the host machine and the virtual machine is one-to-many, and the relationship between the virtual machine and the business system is one-to-one or one-to-many. Therefore, the relationship between the network probe and the business system is one-to-many.

[0031] Each network probe can collect the network communication traffic of each virtual machine corresponding to its host in real time. The network communication traffic can include at least one of the following: business communication traffic within the same business system, business communication traffic between different business systems, and business communication traffic between any business system and an external network. This application embodiment uses a single network probe as an example: Assume that the host where the network probe is located has three virtual machines (virtual machine 1 to virtual machine 3). Virtual machine 1 is deployed with business system X, and virtual machines 2 and 3 are each deployed with business system Y and business system Z. Assume that there is business communication between business system X and business system Y of virtual machine 2, business communication between business system Y of virtual machine 2 and an external network (such as the Internet), and business communication within business system Z of virtual machine 3. Then the network probe collects the network communication traffic of virtual machines 1 to virtual machines 3 respectively. The network communication traffic of virtual machine 1 can include the business communication traffic between business system X and business system Y, the network communication traffic of virtual machine 2 can include the business communication traffic between business system X and business system Y and the business communication traffic between business system Y and an external network, and the network communication traffic of virtual machine 3 can include the business communication traffic within business system Z.

[0032] S120 uses network probes to analyze the network communication traffic of each virtual machine to obtain application monitoring data for each business system.

[0033] A business system is a complete software system designed to solve specific business problems. It is usually composed of multiple application services working together. Therefore, the operating status of a business system can be reflected by aggregating its application monitoring data. Application monitoring data refers to the data involved in business communication between different application services and / or between any application service and the external network. Therefore, by parsing the network communication traffic of each virtual machine, a network probe can obtain the application monitoring data of each application service within each virtual machine. The application monitoring data of multiple application services belonging to the same business system can be used as the application monitoring data of the business system.

[0034] Continuing with the example above: Assume business system X includes application services X1 and X2, business system Y includes application services Y1 to Y3, and business system Z includes application services Z1 to Z3. Application services Y1, Y2, and Z1 are all deployed on virtual machine 2, while application services Y3, Z2, and Z3 are all deployed on virtual machine 3. Business system X has business interactions with business system Y on virtual machine 2 (e.g., between application services X1 and Y1), business system Y on virtual machine 2 has business interactions with the external network (e.g., between application services Y2 and the external network), and business system Z on virtual machine 3 has internal business interactions (e.g., between application services Z2 and application services Z3). Since there are business interactions between service Z1 and service Z2, the network probe can obtain application monitoring data x1 for service X1 by parsing the network communication traffic of virtual machine 1, and application monitoring data y1 for service Y1 and application monitoring data y2 for service Y2 by parsing the network communication traffic of virtual machine 2, and application monitoring data z2 for service Z2 and application monitoring data z3 for service Z3 by parsing the network communication traffic of virtual machine 3. Among them, application monitoring data x1 can be used as application monitoring data for service X, application monitoring data y1 and application monitoring data y2 can be used as application monitoring data for service Y, and application monitoring data z2 and application monitoring data z3 can be used as application monitoring data for service Z.

[0035] S130 uses network probes to uniformly synchronize all application monitoring data to the central monitoring platform.

[0036] Each network probe can uniformly synchronize the application monitoring data of all business systems it parses to the central monitoring platform. For example, assuming network probe 1 corresponds to business systems R to T, network probe 2 corresponds to business systems U to W, and network probe 3 corresponds to business systems X to Z, then network probe 1 will synchronize the application monitoring data of business systems R to T to the central monitoring platform, network probe 2 will synchronize the application monitoring data of business systems U to W to the central monitoring platform, and network probe 3 will synchronize the application monitoring data of business systems X to Z (such as the application monitoring data x1, y1, y2, z2, and z3 mentioned above) to the central monitoring platform.

[0037] S140 centrally stores and analyzes all application monitoring data through a central monitoring platform.

[0038] After receiving the application monitoring data of all business systems synchronized by all network probes, the central monitoring platform can centrally store and perform correlation analysis on the application monitoring data of all business systems.

[0039] As can be seen, compared to related technologies where monitoring components (such as Log4j, ELK, and Prometheus) need to be embedded into specific business systems or application services, resulting in high coupling with applications, complex deployment, and difficulty in unified management, and each business system needs to independently implement monitoring logic, the monitoring system (or monitoring component) requires a large amount of repetitive development, deployment, and maintenance, which undoubtedly greatly increases development, deployment, and maintenance costs. In response, this application's embodiment decouples the network probe from the business system by deploying it independently on the host machine. Without relying on application code modifications, deploying a single network probe enables centralized data collection from the entire complex system environment of the host machine, greatly improving deployment efficiency and resource utilization, and thus significantly reducing development, deployment, and maintenance costs.

[0040] As can be seen from this example, the solution provided in this application can cover the network communication traffic of all virtual machines by deploying network probes on the host machine, so as to obtain application monitoring data of all business systems, thereby improving deployment efficiency and resource utilization. In addition, the application monitoring data of all business systems are centrally stored and correlated through the central monitoring platform, which effectively avoids the duplication of monitoring system construction, significantly reduces development and operation and maintenance costs, and can open up monitoring links across business systems, thereby improving the efficiency and accuracy of fault location.

[0041] Figure 2 This is another flowchart illustrating the application monitoring method shown in this application.

[0042] See Figure 2 The application monitoring method of this application may include: S210, monitors and collects network communication traffic within the same virtual machine, and / or, network communication traffic between different virtual machines, and / or, network communication traffic between any virtual machine and the external network, respectively, forwarded by the bridge device via network probes; wherein, network communication traffic includes interactive data packets of each application service.

[0043] The application monitoring system may include network probes and a central monitoring platform. The network probe may include at least one, and each network probe is deployed on a corresponding host machine. Each host machine is connected to multiple virtual machines, and each virtual machine is deployed with at least one business system. In other words, the network probe and the host machine have a one-to-one relationship, the host machine and the virtual machine have a one-to-many relationship, and the virtual machine and the business system have a one-to-one or one-to-many relationship. Therefore, the network probe and the business system have a one-to-many relationship.

[0044] Each host machine includes a bridge device, which is the host machine's network interface card (NIC), such as... Figures 3-4 As shown ( Figures 3-4 The virtual machine in the text is a virtual machine. For ease of display, only one application service is shown inside each virtual machine, but in reality, multiple application services can be deployed inside each virtual machine. Figure 3 This is a diagram illustrating the network interaction between application services and their hosts. Figure 4 This is a diagram illustrating the network interaction between the application service and the host machine. Figure 3 and Figure 4 As can be seen, regardless of whether it is a cross-host scenario or a same-host scenario, the network communication traffic of each virtual machine is forwarded by the host machine's bridge device (such as br0). That is to say, whether it is business communication between the same or different business systems within a virtual machine, business communication between the same or different business systems between different virtual machines, or business communication between any business system within any virtual machine and the external network, the resulting business communication traffic will eventually converge to the bridge device. Based on this bridge architecture characteristic, each network probe can be bound to the bridge device of the corresponding host machine when it is deployed. In practical applications, each network probe only needs to listen to the bridge device of its host machine to achieve unified, complete, and comprehensive traffic collection of the network communication behavior of all virtual machines corresponding to its host machine. In this way, regardless of the creation or destruction of the virtual machine instance, the network probe can continuously and stably capture the network communication traffic of the virtual machine without the need for additional configuration adjustments.

[0045] A business system is a complete software system designed to solve specific business problems. It is usually composed of multiple application services working together. For example, an e-commerce system (one type of business system) may include front-end web services (responsible for displaying web pages to users), back-end application services (handling business logic, such as placing orders and making payments), database services (storing data such as products, users, and orders), and caching services (improving access speed). These application services can be deployed in the same virtual machine or in different virtual machines. Therefore, the network communication traffic of each virtual machine can include the business communication traffic of each application service within it. Business communication traffic can include interactive data packets. In other words, the network communication traffic of each virtual machine can include the interactive data packets of each application service within it.

[0046] Continuing with the example above: Since there is business communication between application service X1 and application service Y1, business communication between application service Y2 and the external network, and business communication between application service Z2 and application service Z3, the network communication traffic of virtual machine 1 includes the interactive data packet X1 of application service X1, the network communication traffic of virtual machine 2 includes the interactive data packet Y1 of application service Y1 and the interactive data packet Y2 of application service Y2, and the network communication traffic of virtual machine 3 includes the interactive data packet Z2 of application service Z2 and the interactive data packet Z3 of application service Z3.

[0047] It should be noted that each network probe can use libpcap (library for packetcapture) as the data capture engine by default, or it can use a higher performance capture framework, such as AF_Packet (Address Family-Packet) or DPDK (Data Plane Development Kit), to meet the high throughput requirements of high traffic scenarios.

[0048] It is evident that, compared to related technologies where monitoring systems (or monitoring components) can only focus on the internal behavior of a single business system and cannot observe the interaction process between different application services at the network level, resulting in a lack of global monitoring capabilities for operational status, this application addresses this issue by independently deploying a network probe on the host machine. The network probe, by monitoring the host machine's bridge device, achieves non-intrusive traffic collection and covers the business communication traffic of all application services across all business systems. This allows for the capture of interaction data packets from the business communication traffic of each application service. Therefore, this application not only monitors the internal behavior of the same business system but also observes the interaction process between different application services at the network level.

[0049] S220 uses network probes to perform protocol parsing on each interactive data packet in order to extract application monitoring data for each application service from each interactive data packet.

[0050] Each network probe includes a protocol analyzer. Each network probe can use its built-in protocol analyzer to parse the protocol of each captured interactive data packet to extract the application monitoring data of each application service within each interactive data packet. The application monitoring data of multiple application services belonging to the same business system can be used as the application monitoring data of that business system.

[0051] Continuing with the above example: The network probe uses its built-in protocol analyzer to parse the interaction data packet X1 of application service X1, obtaining application monitoring data x1 of application service X1; it also parses the interaction data packet Y1 of application service Y1, obtaining application monitoring data y1 of application service Y1; it parses the interaction data packet Y2 of application service Y2, obtaining application monitoring data y2 of application service Y2; it parses the interaction data packet Z2 of application service Z2, obtaining application monitoring data z2 of application service Z2; and it parses the interaction data packet Z3 of application service Z3, obtaining application monitoring data z3 of application service Z3. Application monitoring data x1 can be used as application monitoring data for business system X; application monitoring data y1 and y2 can be used as application monitoring data for business system Y; and application monitoring data z2 and z3 can be used as application monitoring data for business system Z.

[0052] In one embodiment, each interactive data packet is parsed using a network probe to extract application monitoring data for each application service from each interactive data packet. This may include: By using network probes to decode each interactive data packet layer by layer across multiple protocol layers, application monitoring data for each application service can be extracted from each protocol layer of each interactive data packet. The multiple protocol layers include the link layer, network layer, transport layer, and application layer. The application monitoring data includes key field information and business indicator information. The key field information includes at least one of the following: source IP, source port, destination IP, destination port, request identifier, request time, request duration, request method, request path, request body, and response body. The business indicator information includes at least one of the following: TCP handshake duration, connection error, retransmission rate, connection latency, return status code, response status, response time, response duration, database request processing time, and connection relationship.

[0053] Because related technologies mainly focus on data at the business system resource level, such as CPU, memory, disk, and network throughput, they lack the ability to deeply analyze network communication traffic. To address this, this application's embodiment uses a network probe to perform layer-by-layer decoding based on the network protocol stack, enabling it to delve into application layer protocol types (such as HTTP, DNS, MySQL, SSL, SSH, etc.), supporting TCP stream reassembly and field extraction. This results in finer-grained data collection and richer semantic information, allowing for subsequent aggregated monitoring of the business system's operational status based on traffic collection and analysis combined with application service metadata information.

[0054] Taking the interactive data packet X1 of application service X1 as an example, the network probe uses its built-in protocol analyzer to decode the multiple protocol layers of interactive data packet X1 layer by layer. These multiple protocol layers can include the link layer, network layer, transport layer, and application layer. The link layer protocol type can include Ethernet; the network layer protocol type can include IP (Internet Protocol); the transport layer protocol type can include at least one of TCP (Transmission Control Protocol) and UDP (User Datagram Protocol); and the application layer protocol type can include at least one of HTTP (Hypertext Transfer Protocol), DNS (Domain Name System), MySQL (relational database), SSL (Secure Sockets Layer), and SSH (Secure Shell). The protocol analyzer can decode upwards from the link layer, network layer, transport layer, and application layer, i.e., the decoding order is: link layer → network layer → transport layer → application layer, in order to extract the application monitoring data x1 of application service X1 from each protocol layer of interactive data packet X1. Using this method, application monitoring data for each application service can be extracted.

[0055] The application monitoring data includes key field information, which includes at least one of the following: source IP, source port, destination IP, destination port, request identifier, request time, request duration, request method, request path, request body, and response body. The request method, request path, request body, and response body are specific to the HTTP protocol, while other key fields such as source IP, source port, destination IP, destination port, request identifier, request time, and request duration are universal. In other words, if the protocol type of an interaction data packet at the application layer is HTTP, the extracted key field information includes IP, source port, destination IP, destination port, request identifier, request time, request duration, request method, request path, request body, and response body. If the protocol type of the interaction data packet at the application layer is any of DNS, MySQL, SSL, or SSH, the extracted key field information includes IP, source port, destination IP, destination port, request identifier, request time, and request duration, but does not include the request method, request path, request body, and response body.

[0056] The application monitoring data also includes business metrics information, which includes at least one of the following: TCP handshake time, connection error, retransmission rate, connection latency, return status code, response status, response time, response time, database request processing time, and connection relationship.

[0057] Specifically, the source and destination IP addresses can be extracted from the network layer (IP); the source and destination ports can be extracted from the transport layer (TCP / UDP); when the request identifier is a TCP sequence number, it can be extracted from the transport layer (TCP); when the request identifier is an HTTP request ID, it can be extracted from the application layer (HTTP); the request time and request duration are calculated across multiple protocol layers; the request method, request path, request body, and response body can be extracted from the application layer (HTTP); the TCP handshake duration can be extracted from the transport layer (TCP); and when the connection error is ICMP (Internet Control Message Protocol)... When a connection error occurs (e.g., destination unreachable) in the Internet Control Message Protocol, the connection exception can be extracted from the network layer (IP); when the connection exception is a connection reset or timeout, it can be extracted from the transport layer (TCP); when the connection exception is an exception status code (e.g., 5XX), it can be extracted from the application layer (HTTP); the retransmission rate can be extracted from the transport layer (TCP); the connection latency can be extracted from either the transport layer (TCP) or the network layer (IP); the return status code can be extracted from the application layer (HTTP); when the response status is a status code, the response status can be extracted from the application layer (HTTP); when the response status is a connected status, the response status can be extracted from the transport layer (TCP); the response time can be extracted from the application layer (HTTP / MySQL); the response time can be extracted from the application layer (HTTP / MySQL); the database request processing time can be extracted from the application layer (MySQL); and the connection relationship can be extracted from either the transport layer (TCP / UDP) or the network layer (IP).

[0058] As can be seen, the protocol analyzer of the network probe can not only identify the protocol type of the interactive data packets in each protocol layer, but also perform complex operations such as TCP stream reassembly, application monitoring data extraction, and field parsing. For example, when processing HTTP requests, the protocol analyzer can accurately extract key field information such as request method, source IP, source port, destination IP, destination port, request path, and request time from the application layer (HTTP) to provide rich semantic information for the upper-layer event mechanism.

[0059] The S230 uses network probes to uniformly synchronize all application monitoring data to the central monitoring platform.

[0060] This step can be found in the description in S130, and will not be repeated here.

[0061] In one implementation, synchronizing all application monitoring data to a central monitoring platform via network probes may include: The network probe identifies the protocol type of each interactive data packet at the application layer; the protocol type includes at least one of HTTP, DNS, MySQL, SSL, and SSH; the network probe writes all application monitoring data to the corresponding log files according to the protocol type; the network probe generates a corresponding target index for each log file; and the network probe synchronizes all log files and their corresponding target indexes to the central monitoring platform.

[0062] Because each business system in related technologies independently collects and stores monitoring data, the format and content of monitoring data from different business systems are inconsistent, making unified processing and correlation analysis difficult. To address this, the embodiments of this application can structure protocol data to enable unified processing and correlation analysis of application monitoring data from all business systems, thereby improving observability.

[0063] In its implementation, after protocol parsing, the network probe automatically generates structured "events" based on different protocol behaviors. These protocol behaviors consist of application-layer protocol types (such as HTTP / DNS / MySQL / SSL / SSH) and event types (such as request / response). "Events" are the core concept in the network probe's analysis logic, abstracting complex underlying network data into high-level semantic objects that can be processed. For example, when a complete HTTP request is parsed, the network probe can trigger the `http_request` event; when a DNS query request is parsed, it can trigger the `dns_request` event. These events contain all the parsed key field information and are passed to the network probe's script engine for processing. This event generation mechanism enables the network probe to achieve a modular, decoupled protocol analysis architecture, facilitating expansion and maintenance.

[0064] This application uses the request and response message bodies obtained by parsing an HTTP request (one type of interactive data packet) as an example to illustrate the following: Network probes define log output format fields, including request time (ts), request identifier (uid), request method (method), request path (uri), request body (request_body), and response body (response_body).

[0065] Network probes cache and store HTTP message content by defining a global structure (SessionBody). This global structure manages the complete HTTP session request and response data within the network probe. The global structure contains two fields: request_body and response_body, which store the content of the HTTP protocol request and response message bodies, respectively.

[0066] Network probes use an event-driven approach to process network communication traffic. Custom event handling scripts can register callback events for specific protocols to achieve accurate parsing and processing of HTTP messages. The specific data collection process is as follows: 1. Event Registration and Initialization: When the network probe loads, it can register to listen for key events such as http_request (HTTP request event), http_entity_data (HTTP entity data event), and http_message_done (HTTP message completion event) via script. Whenever a new HTTP request arrives, the http_request event is triggered. During this stage, the corresponding global structure SessionBody is initialized, and the fields of request_body and response_body are set to empty.

[0067] 2. Data Packet Listening and Concatenation: When the HTTP packet body arrives, the `http_entity_data` event is triggered. Then, the `is_orig` is extracted from the current packet. The logic of `is_orig` is based on the initiator of the connection (such as the initiator of the SYN (Synchronize) in the three-way handshake). Therefore, the network probe distinguishes whether the current packet is a request packet body or a response packet body based on `is_orig`. is_orig = true → The initiator (i.e., the party that sends the first TCP SYN packet, usually the client), for example: A initiates the connection → A will always have is_orig=true.

[0068] is_orig = false → Receiver (usually the server), for example: B responds to the connection → B will always have is_orig = false.

[0069] Therefore, if is_orig = true, the current message is determined to be flowing in the inbound direction (i.e., a request sent by the client to the server), and the current message is written to request_body; if is_orig = false, the current message is determined to be flowing out the outbound direction (i.e., a response returned by the server to the client), and the current message is written to response_body.

[0070] A complete HTTP request may include multiple HTTP message body data, which may trigger the http_entity_data event multiple times in sequence. The network probe can concatenate multiple HTTP message body data belonging to the same HTTP request until the data is complete.

[0071] 3. Message completion and persistence: When a complete HTTP request processing ends, the http_message_done event is triggered. At this time, the network probe determines that the collection of interactive data packets (HTTP requests) has been completed, and then persists the global structure (SessionBody) in the current session to the host machine's disk, usually in structured text or JSON file format.

[0072] Following the above method, each interactive data packet is processed by a script. The analysis results after script processing are output to the host machine's disk in the form of a structured log file, which facilitates subsequent analysis and integration.

[0073] Network probes can write all application monitoring data (including key field information and business metrics information) to the corresponding log files according to the protocol type (such as HTTP / DNS / MySQL / SSL / SSH). For example, conn.log is used to store all connection information, http.log is used to store all HTTP requests and responses, and dns.log is used to store all DNS query logs.

[0074] The network probe also includes Filebeat (a lightweight log synchronization tool). Filebeat is responsible for monitoring changes in the content of various protocol log files in the log generation directory of the network probe and synchronizing the newly added log files to Elasticsearch (a distributed search and analysis engine) of the central monitoring platform in real time for storage and indexing.

[0075] The synchronization strategy follows the indexing principle of "by protocol type + by date": each type of protocol log file (such as conn.log, http.log, dns.log, etc.) corresponds to an independent target index every day (as shown in Table 1 below), which facilitates subsequent log retrieval, alarm analysis and index lifecycle management.

[0076]

[0077] Table 1 Considering the limited storage resources of the host machine's disk, in order to avoid the long-term accumulation of log files causing disk pressure, this application embodiment can archive and clean up log files through scheduled tasks. For example, log files can be archived daily. Therefore, after the log files are successfully synchronized to Elasticsearch on the central monitoring platform, the synchronized log files can be automatically deleted from the disk periodically, thereby ensuring that local space usage is controllable and guaranteeing the continuous and stable operation of the network probe service.

[0078] The S240 centrally stores and analyzes all application monitoring data through a central monitoring platform.

[0079] This step can be found in the description in S140, and will not be repeated here.

[0080] In one embodiment, the central monitoring platform includes Elasticsearch; the centralized storage and correlation analysis of all application monitoring data through the central monitoring platform may include: The central monitoring platform centrally stores all log files in Elasticsearch according to the corresponding target index. Based on the target index and a preset time period, the platform extracts the target log file from multiple existing log files in Elasticsearch. It then extracts abnormal monitoring data from the target log file based on business metrics. Finally, the platform identifies the target application service matching the abnormal monitoring data from all application services based on key field information. Finally, the platform associates the abnormal monitoring data with the target application service for alert push notifications.

[0081] After receiving all log files synchronized by the network probe via Filebeat and their corresponding target indexes, the central monitoring platform can centrally store all log files in Elasticsearch according to the corresponding target indexes.

[0082] Because related technologies lack monitoring mechanisms across business systems and network layers, it's difficult to quickly determine whether a performance bottleneck or service anomaly is caused by the application service itself or by factors such as communication or network links between application services. This severely impacts troubleshooting efficiency and fault location accuracy. To address this, the central monitoring platform provides data analysis and early warning mechanisms, including application latency warnings, HTTP service request status warnings, and application service network operation monitoring and fault location assistance mechanisms.

[0083] For the application latency alert mechanism, at the macro view level, the central monitoring platform periodically extracts target log files from multiple log files stored in Elasticsearch through scheduled tasks. For example, based on the target index and a preset time period (such as 1 day / 7 days / 1 month / custom time range), it extracts recent log files from multiple log files stored in Elasticsearch as target log files. The target log files store monitoring data for multiple applications of the same protocol type (such as HTTP protocol). The central monitoring platform determines abnormal monitoring data with abnormal access latency (such as response time exceeding a preset duration threshold) based on business indicators such as response time and response latency in each application monitoring data. Based on the key field information in the abnormal monitoring data, the central monitoring platform determines the target application service that matches the abnormal monitoring data from all application services, thereby accurately locating the specific application service to which it belongs. This allows the abnormal monitoring data to be associated with the target application service for alert push. For example, the central monitoring platform triggers an alert mechanism according to preset rules, and promptly notifies the application manager of the target application service via SMS, email, etc., to assist them in quickly investigating and optimizing the performance of the target application service.

[0084] For the HTTP service request status alert mechanism, the central monitoring platform periodically extracts target log files from multiple http.log files stored in Elasticsearch through scheduled tasks. For example, based on the target index http.log-XXX (XXX being the date) and a preset time period (such as 1 day / 7 days / 1 month / custom time range), the platform extracts the most recent http.log files as target log files. These target log files store recent HTTP request and response data (i.e., application monitoring data of the HTTP protocol). The central monitoring platform identifies abnormal monitoring data with abnormal status codes (such as 4XX or 5XX) based on the return status code (or response status code) in each application monitoring data. Using the key field information in this abnormal monitoring data, the platform determines the target application service that matches the abnormal monitoring data from all application services, thus accurately locating the specific application service to which it belongs. This allows the platform to associate the abnormal monitoring data with the target application service for alert push. For example, the central monitoring platform triggers an alert mechanism according to preset rules, promptly notifying the application manager of the target application service via SMS, email, etc., to assist them in quickly investigating service logic anomalies and ensuring business stability and availability.

[0085] For application service network operation monitoring and fault location assistance mechanisms, the central monitoring platform supports periodically querying network monitoring data of a specified application service from multiple log files stored in Elasticsearch within preset time periods (such as 1 day / 7 days / 1 month / custom time range), displaying its operational status within the selected time period. The central monitoring platform provides visual charts, including latency fluctuation line graphs and HTTP status code distribution graphs for application services, to help achieve observability and traceability of application service network monitoring data.

[0086] The data analysis and early warning mechanism provided by the central monitoring platform offers strong data support for application service fault diagnosis. For example, when an application service suddenly becomes unavailable or experiences increased response latency, related technologies often attribute the problem to network jitter due to the inability to reproduce the issue or the lack of logs, failing to effectively pinpoint the problem. In this embodiment, the central monitoring platform provides a time window to query the overall latency of the application service and the network latency at the transport layer. This effectively determines whether the problem occurs at the network transport layer or the application processing layer, thus clarifying the fault boundary and facilitating accurate and efficient problem location and resolution.

[0087] In one embodiment, the central monitoring platform stores deployment information for each application service, including deployment IP and deployment port. The central monitoring platform determines the target application service matching the abnormal monitoring data from all application services based on key field information, which may include: The central monitoring platform extracts the target IP and target port of abnormal monitoring data from key field information; the target IP is the source IP or destination IP, and the target port is the source port or destination port; the central monitoring platform matches the target IP with each deployment IP and the target port with each deployment port; the central monitoring platform identifies the application services corresponding to the deployment IP that successfully matches the target IP and the deployment port that successfully matches the target port as the target application services.

[0088] Since the application monitoring data collected and reported by network probes is essentially stateless and difficult to directly reflect the specific application services associated with it, after querying abnormal monitoring data, in order to achieve effective association between the stateless abnormal monitoring data and the actual application services (i.e., the target application services), the central monitoring platform can pre-manage the metadata information of global application services in a unified manner, including mechanisms such as business system registration management, application service registration management, and application service deployment management.

[0089] Regarding the business system registration management mechanism, the central monitoring platform supports unified registration and basic information management for each business system. For example, users can enter metadata information such as the system name, business description, unique code, system owner, and contact information (such as telephone and email) of each business system into the central monitoring platform. This information serves as the basis for subsequent application service registration and the attribution of abnormal monitoring data, ensuring that abnormal monitoring data can be accurately classified into the business system to which it belongs.

[0090] Regarding the application service registration management mechanism, a business system typically consists of multiple application services. This application embodiment allows users to register all application services under each business system to the central monitoring platform one by one, including the service name, service purpose, service code, business system to which it belongs, service path list, service type, etc., so as to achieve fine-grained monitoring and management of service dimensions.

[0091] For the application service deployment management mechanism, the central monitoring platform maintains the deployment information of each registered application service, including deployment IP, deployment port, service to which it belongs, deployment environment, etc., so that the deployment information can be bound with the data collected by the network probe to realize the dynamic association between the network layer and the application layer, and thus accurately map traffic behavior to specific service instances.

[0092] As can be seen, the central monitoring platform maintains basic information at the system level, such as system name, system contact person and contact information (telephone, email), while managing key attributes at the service level, including service name, service purpose, service code, business system to which it belongs, service path list, and deployment information (such as deployment IP and deployment port), as shown in Table 2 below.

[0093]

[0094] Table 2 The central monitoring platform, through unified management of metadata information of business systems and application services, can intelligently match abnormal monitoring data based on fields such as target IP and target port, thereby identifying the business affiliation of abnormal monitoring data and significantly improving the accuracy, relevance, and operability of monitoring analysis.

[0095] In one example, if the interaction data packet corresponding to the abnormal monitoring data is a request data packet (e.g., application service X1 cannot send the request data packet to application service Y1), the central monitoring platform can extract the source IP as the target IP of the abnormal monitoring data from the key field information of the abnormal monitoring data, and extract the source port as the target port of the abnormal monitoring data, so as to match the target IP (source IP) with each deployment IP and match the target port (source port) with each deployment port. Through the target IP (source IP) and target port (source port), the target application service (application service X1) that matches the abnormal monitoring data can be accurately located. The central monitoring platform can determine the application service (application service X1) corresponding to the deployment IP that successfully matches the target IP (source IP) and the deployment port that successfully matches the target port (source port) as the target application service.

[0096] In another example, if the interaction data packet corresponding to the abnormal monitoring data is a response data packet (such as application service Y1 responding to a timeout), the central monitoring platform can extract the source IP as the target IP of the abnormal monitoring data from the key field information of the abnormal monitoring data, and extract the source port as the target port of the abnormal monitoring data, so as to match the target IP (source IP) with each deployment IP and match the target port (source port) with each deployment port. Through the target IP (source IP) and target port (source port), the target application service (application service Y1) that matches the abnormal monitoring data can be accurately located. The central monitoring platform can determine the application service (application service Y1) corresponding to the deployment IP that successfully matches the target IP (source IP) and the deployment port that successfully matches the target port (source port) as the target application service.

[0097] In another example, if the response data packet of application service Y1 is not captured for a long time (e.g., application service Y1 suddenly becomes unavailable), the central monitoring platform can only capture the request data packet of application service X1. Therefore, the request data packet of application service X1 is regarded as abnormal monitoring data. The central monitoring platform can extract the destination IP as the target IP of the abnormal monitoring data from the key field information of the abnormal monitoring data, and extract the destination port as the target port of the abnormal monitoring data, so as to match the target IP (destination IP) with each deployment IP and match the target port (destination port) with each deployment port. Through the target IP (destination IP) and target port (destination port), the target application service (application service Y1) that matches the abnormal monitoring data can be accurately located. The central monitoring platform can determine the application service (application service Y1) corresponding to the deployment IP that successfully matches the target IP (destination IP) and the deployment port that successfully matches the target port (destination port) as the target application service.

[0098] As can be seen, this application embodiment maintains the metadata information (including deployment IP and deployment port) of each application service through a central monitoring platform, and dynamically matches it with the target IP and target port in the abnormal monitoring data. This enables the abnormal monitoring data to be automatically attributed to the specific application service, thus building a unified application perspective. This automatic mapping mechanism effectively solves the problem that it is difficult to translate data collected from the network layer into business dimensions. It supports accurate monitoring and visualization analysis across business systems and can clearly display the connection relationships, response status, and connection latency of each application service, thereby assisting in the rapid diagnosis of performance bottlenecks or network anomalies and improving the efficiency and accuracy of fault location.

[0099] Compared to related technologies that focus on system resource-level performance metrics such as CPU utilization, memory usage, disk I / O, or network interface bandwidth, which only provide coarse-grained system-level health status monitoring and are difficult to pinpoint performance problems or root causes of specific application services, the network probe in this application embodiment can deeply analyze service-level request and response processes based on network communication behavior. It supports the collection of key indicators such as interface-level response time, connection latency, and return status codes to obtain service performance profiles. This enables the identification of anomalies, bottlenecks, or sources of failure in individual application services, thereby significantly improving the observability and operational efficiency of business systems.

[0100] In summary, the embodiments of this application can establish a monitoring link across business systems, realize the visualization of service communication paths, rapid identification of fault boundaries, and global diagnosis of performance bottlenecks, effectively avoid the duplication of monitoring system construction, and significantly reduce development and maintenance costs.

[0101] As can be seen from this example, the solution provided in this application has the following technical advantages: 1. Centralize and migrate monitoring functions: By uniformly deploying network probes at the host layer, the method of deploying monitoring components in various business systems or application services in related technologies is replaced, avoiding redundant development and deployment, reducing system resource consumption, and improving operational consistency and maintainability.

[0102] 2. Enhanced network layer observability: Based on the original network communication traffic, protocol layer parsing can be performed to obtain business indicator information such as TCP handshake time, connection anomalies, retransmission rate, return status code, response time, and database request processing time. Compared with the log-based tracking method in related technologies, the embodiments of this application have more comprehensive and low-interference observation capabilities.

[0103] 3. Improve fault location efficiency: Anomaly monitoring data can support rapid fault location based on the communication five-tuple (such as source IP, destination IP, source port, destination port, and protocol type). Combined with link path restoration and historical backtracking analysis, it can shorten the time for problem location and processing.

[0104] 4. Supports unified traffic business identification: By matching the target IP and target port in abnormal monitoring data with the deployment IP and deployment port in deployment information, the system can determine the attribution of business traffic in cross-business system scenarios, ensuring the integrity and consistency of monitoring data.

[0105] 5. Excellent horizontal scalability: Deploying network probes on the host machine has no intrusive impact on business systems, can be adapted to multiple virtualization platforms, and is suitable for unified traffic awareness and application behavior monitoring in large-scale edge cloud environments.

[0106] Therefore, the solution provided in this application has the following beneficial effects: 1. Non-intrusive virtual machine application monitoring capabilities based on host-machine bridge traffic analysis By deploying network probes on the virtualized host and uniformly monitoring the bridge device (such as br0) between the host and virtual machines, the network communication traffic of all virtual machines entering and leaving the virtual machines and between them can be captured and analyzed without any obstruction. No monitoring components need to be deployed in the business system or application service, thus ensuring that the operation of the business system is not interfered with. At the same time, no configuration changes are required when creating or destroying virtual machines, which has higher compatibility and maintainability. Compared with the deployment method in related technologies (that is, deploying monitoring components in each business system or application service), the deployment method of this application embodiment is more lightweight and non-intrusive.

[0107] 2. Protocol-level in-depth analysis and formatting processing Leveraging the powerful protocol parsing capabilities of network probes, the TCP / IP stack and its upper-layer protocols such as HTTP, DNS, MySQL, SSL, and SSH can be decoded layer by layer. The session-level information is then output in a structured manner into various log formats (such as conn.log, http.log, dns.log, ssl.log, mysql.log, ssl.log, etc.). Compared to related technologies that only provide raw data packets or brief summaries, the embodiments of this application have stronger context restoration capabilities and data mining potential, providing semantically rich raw data for subsequent analysis.

[0108] 3. Visualized network operation analysis based on latency trends and status code distribution The central monitoring platform supports querying various protocol log files in Elasticsearch within preset time periods (such as 1 day / 7 days / 1 month / custom time range), dynamically generating latency fluctuation line charts and HTTP status code distribution charts for application services, and realizing visual analysis of the network operation status of application services. Compared with the manual troubleshooting of log queries in related technologies, the embodiment of this application can intuitively present the problem trend and support backtracking analysis through the central monitoring platform, providing a clear basis for quickly determining whether it is a network bottleneck or application anomaly.

[0109] 4. Intelligent early warning mechanism for application time consumption and HTTP service request status The central monitoring platform periodically analyzes target log files (such as http.log) in Elasticsearch, monitoring for anomalies where response times exceed preset thresholds or abnormal status codes (such as 4XX or 5XX). By extracting the target IP and port from this anomaly data and combining it with pre-stored deployment information for each application service (such as deployment IP and port), it automatically associates stateless anomaly monitoring data with specific application services within the central monitoring platform. This early warning mechanism does not rely on context passing or intrusive data entry, accurately reconstructing request paths and identifying target application services. Furthermore, the central monitoring platform supports centralized and unified management of application monitoring. Once all application services are connected, unified log collection, anomaly detection, and alarm push can be achieved without the need for dedicated maintenance personnel. Alarm information is precisely sent to the corresponding application managers via SMS, email, etc. Compared to log comparison or single-state monitoring methods in related technologies, the application monitoring method in this embodiment offers higher accuracy, automation, and real-time performance, significantly improving the overall fault detection and response efficiency of the business system.

[0110] 5. Supports historical analysis capabilities with multi-dimensional time windows. The central monitoring platform allows users to select preset time periods (such as 1 day / 7 days / 1 month / custom time range) to extract application monitoring data from Elasticsearch and perform trend analysis based on the set time aggregation granularity, helping users quickly identify abnormal fluctuations or performance changes within specific time periods. The central monitoring platform also supports viewing indicator data by application and cluster dimensions. Users can conduct in-depth analysis from the perspective of a single application service, or observe resource utilization and abnormal distribution at the overall cluster level, meeting the multi-level and multi-role operation and maintenance analysis needs. Compared to static reports or fixed-period analysis methods in related technologies, the analysis method of this application embodiment has higher flexibility and interactivity, significantly improving problem localization and root cause tracing capabilities, and is suitable for comprehensive operation and maintenance scenarios involving multiple scenarios and multiple business lines.

[0111] Corresponding to the aforementioned application function implementation method embodiments, this application also provides an application monitoring system, electronic device, and corresponding embodiments.

[0112] Figure 5 This is a schematic diagram of the structure of the application monitoring system shown in the embodiments of this application.

[0113] See Figure 5 This application provides an application monitoring system 500, which may include: a network probe 510 and a central monitoring platform 520. The network probe is deployed on a host machine, and the host machine is connected to multiple virtual machines, each of which has at least one business system deployed. The Network Probe 510 is used to collect network communication traffic for each virtual machine; to perform protocol parsing on the network communication traffic of each virtual machine to obtain application monitoring data for each business system; and to uniformly synchronize all application monitoring data to the central monitoring platform. The central monitoring platform 520 is used for centralized storage and correlation analysis of all application monitoring data.

[0114] In one implementation, the host machine connects to multiple virtual machines via a bridge device; the network probe 510 is used to collect network communication traffic for each virtual machine separately for: Monitor and collect network communication traffic within the same virtual machine forwarded by the bridge device, and / or, network communication traffic between different virtual machines, and / or, network communication traffic between any virtual machine and the external network.

[0115] In one embodiment, each business system consists of multiple application services, and the network communication traffic includes interactive data packets from each application service; the network probe 510, when performing protocol parsing on the network communication traffic of each virtual machine to obtain application monitoring data for each business system, is used for: Each interactive data packet is parsed according to the protocol in order to extract application monitoring data for each application service from each interactive data packet.

[0116] In one embodiment, the network probe 510 is used to: perform protocol parsing on each interactive data packet to extract application monitoring data for each application service from each interactive data packet. Each interactive data packet is decoded layer by layer at each protocol layer in order to extract application monitoring data for each application service from each protocol layer of each interactive data packet. The multi-layer protocol includes the link layer, network layer, transport layer, and application layer. The application monitoring data includes key field information and business indicator information. The key field information includes at least one of the following: source IP, source port, destination IP, destination port, request identifier, request time, request duration, request method, request path, request body, and response body. The business indicator information includes at least one of the following: TCP handshake duration, connection error, retransmission rate, connection latency, return status code, response status, response time, response duration, database request processing time, and connection relationship.

[0117] In one implementation, the network probe 510 is used to: synchronize all application monitoring data to the central monitoring platform. Identify the protocol type of each interactive data packet at the application layer; the protocol type includes at least one of HTTP, DNS, MySQL, SSL, and SSH; According to the protocol type, write all application monitoring data to the corresponding log file; Generate a corresponding target index for each log file; All log files and their corresponding target indexes are synchronized to the central monitoring platform.

[0118] In one implementation, the central monitoring platform includes Elasticsearch; the central monitoring platform 520 is used for: centralized storage and correlation analysis of all application monitoring data. All log files are centrally stored in Elasticsearch according to the corresponding target index; Based on the target index and a preset time period, extract the target log file from multiple log files already stored in Elasticsearch; Based on business metrics information, extract abnormal monitoring data from the target log file; Based on key field information, identify the target application service that matches the anomaly monitoring data from all application services; Link abnormal monitoring data to the target application service for alarm push.

[0119] In one embodiment, the central monitoring platform stores deployment information for each application service, including deployment IP and deployment port; the central monitoring platform 520, when determining the target application service matching the abnormal monitoring data from all application services based on key field information, uses the following methods: Extract the target IP and target port of the abnormal monitoring data from the key field information; the target IP is the source IP or the destination IP, and the target port is the source port or the destination port; Match the target IP with each deployment IP, and match the target port with each deployment port. The application services corresponding to the deployment IP that successfully matches the target IP and the deployment port that successfully matches the target port are identified as the target application services.

[0120] As can be seen from this example, the solution provided in this application can cover the network communication traffic of all virtual machines by deploying network probes on the host machine, so as to obtain application monitoring data of all business systems, thereby improving deployment efficiency and resource utilization. In addition, the application monitoring data of all business systems are centrally stored and correlated through the central monitoring platform, which effectively avoids the duplication of monitoring system construction, significantly reduces development and operation and maintenance costs, and can open up monitoring links across business systems, thereby improving the efficiency and accuracy of fault location.

[0121] Regarding the system in the above embodiments, the specific ways in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated further here.

[0122] Figure 6 This is a schematic diagram of the structure of an electronic device shown in an embodiment of this application.

[0123] See Figure 6 The electronic device 600 includes a memory 610 and a processor 620.

[0124] The processor 620 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. Memory 610 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. ROM may store static data or instructions required by the processor 620 or other modules of the computer. Permanent storage devices may be read-write storage devices. Permanent storage devices may be non-volatile storage devices that retain stored instructions and data even when the computer is powered off. In some embodiments, permanent storage devices use mass storage devices (e.g., magnetic or optical disks, flash memory) as permanent storage devices. In other embodiments, permanent storage devices may be removable storage devices (e.g., floppy disks, optical drives). System memory may be a read-write storage device or a volatile read-write storage device, such as dynamic random access memory. System memory may store some or all of the instructions and data required by the processor during operation. Furthermore, memory 610 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, memory 610 may include a removable storage device that is readable and / or writable, such as a laser disc (CD), a read-only digital multifunction optical disc (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disc, an ultra-high density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not contain carrier waves or transient electronic signals transmitted wirelessly or via wired connections.

[0125] The memory 610 stores executable code, which, when processed by the processor 620, can cause the processor 620 to execute part or all of the methods described above.

[0126] Furthermore, the method according to this application can also be implemented as a computer program or computer program product, which includes computer program code instructions for performing some or all of the steps in the method described above.

[0127] Alternatively, this application may be implemented as a computer-readable storage medium (or a non-transitory machine-readable storage medium or a machine-readable storage medium) storing executable code (or computer program or computer instruction code) that, when executed by a processor of an electronic device (or server, etc.), causes the processor to perform part or all of the steps of the methods described above according to this application.

[0128] This application also provides a computer program product, which includes computer instructions that, when executed by a processor, implement the method described above.

[0129] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An application monitoring method, characterized in that, The method is applied to an application monitoring system, which includes a network probe and a central monitoring platform. The network probe is deployed on a host machine, and the host machine is connected to multiple virtual machines. Each virtual machine deploys at least one business system. The network communication traffic of each virtual machine is collected using the network probes. The network probes are used to perform protocol parsing on the network communication traffic of each virtual machine to obtain application monitoring data for each business system. All application monitoring data are uniformly synchronized to the central monitoring platform through the network probe. The central monitoring platform centrally stores and correlates all application monitoring data.

2. The method according to claim 1, characterized in that, The host machine connects to multiple virtual machines via a bridge device; the collection of network communication traffic for each virtual machine via the network probe includes: The network probe monitors and collects the network communication traffic within the same virtual machine, and / or the network communication traffic between different virtual machines, and / or the network communication traffic between any virtual machine and an external network, which are forwarded by the bridge device respectively.

3. The method according to claim 1, characterized in that, Each of the aforementioned business systems consists of multiple application services, and the network communication traffic includes interactive data packets from each of the application services; the application monitoring data for each business system is obtained by parsing the network communication traffic of each virtual machine using the network probe, including: The network probe performs protocol parsing on each of the interactive data packets to extract application monitoring data for each application service from each interactive data packet.

4. The method according to claim 3, characterized in that, The step of parsing the protocol of each interaction data packet using the network probe to extract application monitoring data for each application service from each interaction data packet includes: The network probe decodes each layer of the protocol layer of each interactive data packet layer by layer, so as to extract the application monitoring data of each application service from each layer of the protocol layer of each interactive data packet. The multi-layered protocol system includes a link layer, a network layer, a transport layer, and an application layer. The application monitoring data includes key field information and business indicator information. The key field information includes at least one of the following: source IP, source port, destination IP, destination port, request identifier, request time, request duration, request method, request path, request body, and response body. The business indicator information includes at least one of the following: TCP handshake duration, connection error, retransmission rate, connection latency, return status code, response status, response time, response duration, database request processing time, and connection relationship.

5. The method according to claim 4, characterized in that, The process of uniformly synchronizing all application monitoring data to the central monitoring platform via the network probe includes: The network probes identify the protocol type of each interactive data packet at the application layer; the protocol type includes at least one of HTTP, DNS, MySQL, SSL, and SSH. The network probe writes all the application monitoring data into the corresponding log files according to the protocol type. The network probe is used to generate a corresponding target index for each of the log files; The network probe is used to synchronize all the log files and the corresponding target indexes to the central monitoring platform.

6. The method according to claim 5, characterized in that, The central monitoring platform includes Elasticsearch; the centralized storage and correlation analysis of all application monitoring data through the central monitoring platform includes: The central monitoring platform centrally stores all log files in Elasticsearch according to the corresponding target index. The central monitoring platform extracts the target log file from multiple log files already stored in Elasticsearch based on the target index and a preset time period. The central monitoring platform extracts abnormal monitoring data from the target log file based on the business indicator information. The central monitoring platform determines the target application service that matches the abnormal monitoring data from all the application services based on the key field information. The central monitoring platform associates the abnormal monitoring data with the target application service to push alarms.

7. The method according to claim 6, characterized in that, The central monitoring platform stores deployment information for each application service, including deployment IP and deployment port. The process of determining the target application service matching the abnormal monitoring data from all application services based on the key field information via the central monitoring platform includes: The central monitoring platform extracts the target IP and target port of the abnormal monitoring data from the key field information; the target IP is either the source IP or the destination IP, and the target port is either the source port or the destination port. The central monitoring platform matches the target IP with each of the deployed IPs and matches the target port with each of the deployed ports. The central monitoring platform identifies the application services corresponding to the deployment IP that successfully matches the target IP and the deployment port that successfully matches the target port as the target application services.

8. An application monitoring system, characterized in that, It includes a network probe and a central monitoring platform. The network probe is deployed on a host machine, which is connected to multiple virtual machines. Each virtual machine has at least one business system deployed on it. The network probe is used to collect network communication traffic for each virtual machine; and to perform protocol parsing on the network communication traffic for each virtual machine to obtain application monitoring data for each business system. All application monitoring data will be uniformly synchronized to the central monitoring platform. The central monitoring platform is used for centralized storage and correlation analysis of all application monitoring data.

9. An electronic device, characterized in that, include: processor; as well as A memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method as described in any one of claims 1-10.

Images