End-to-end monitoring method and apparatus, and storage medium

Abstract

The present disclosure provides an end-to-end monitoring method and apparatus, and a storage medium. The method comprises: acquiring configuration information of a service corresponding to a service request; on the basis of the configuration information of the service, determining a monitoring configuration of a user plane device on a communication path of the service, wherein the monitoring configuration is used for instructing the user plane device to collect monitoring information of the service; and sending the monitoring configuration to the user plane device.

Description

End-to-end monitoring method, device and storage medium

[0001] This disclosure claims priority to Chinese patent application No. 202410202023.4, filed on February 22, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present disclosure relates to the field of information processing technology, and in particular to an end-to-end monitoring method, device, and storage medium. Background Art

[0003] Currently, the monitoring methods for application services mainly adopt the following technical routes: using Application Performance Monitoring (APM) technology to monitor business functions, framework functions or library functions at the L5-L7 layers; or using Extended Berkeley Packet Filter (eBPF) technology to monitor links, communication paths, system calls, etc. below the L3 layer.

[0004] APM technology uses instrumentation code, which modifies application source code and cannot be embedded in infrastructure, creating observation blind spots. eBPF technology struggles to obtain application performance data through common methods like correlation and aggregation in complex scenarios like cross-threading and asynchronous calls, making it unsuitable for in-depth analysis of application performance.

[0005] Summary of the Invention

[0006] In one aspect, an end-to-end monitoring method is provided, which is applied to a control plane device. The method includes:

[0007] Get the configuration information of the service corresponding to the service request;

[0008] determining, based on the configuration information of the service, a monitoring configuration of a user plane device on a communication path of the service, wherein the monitoring configuration is used to instruct the user plane device to collect monitoring information of the service;

[0009] Send the monitoring configuration to the user plane device.

[0010] On the other hand, an end-to-end monitoring method is provided, which is applied to a user plane device. The method includes:

[0011] receiving a monitoring configuration of a service sent by a control plane device, wherein the user plane device is located on a communication path of the service;

[0012] Based on the monitoring configuration, monitoring information of the service is obtained.

[0013] In another aspect, an end-to-end monitoring device is provided, which is applied to a control plane device. The monitoring device includes:

[0014] The acquisition module is used to obtain the configuration information of the service corresponding to the service request;

[0015] a determining module, configured to determine, based on the configuration information of the service, a monitoring configuration of a user plane device on a communication path of the service, wherein the monitoring configuration is used to instruct the user plane device to collect monitoring information of the service;

[0016] The communication module is further configured to send the monitoring configuration to the user plane device.

[0017] On the other hand, an end-to-end monitoring device is provided, which is applied to a user plane device. The monitoring device includes:

[0018] a communication module, configured to receive a monitoring configuration of a service sent by a control plane device, the user plane device being located on a communication path of the service;

[0019] An acquisition module is used to acquire monitoring information of the service based on the monitoring configuration.

[0020] On the other hand, a communication device is provided, comprising: a memory and a processor; the memory and the processor are coupled; the memory is used to store a computer program; and the processor implements the end-to-end monitoring method described in any one of the above aspects or embodiments when executing the computer program.

[0021] On the other hand, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the end-to-end monitoring method described in any one of the above aspects or embodiments is implemented.

[0022] On the other hand, a computer program product is provided, which includes computer program instructions, and when the computer program instructions are executed by a processor, the end-to-end monitoring method according to any one of the above aspects or embodiments is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the present disclosure, the following briefly introduces the drawings required for use in some embodiments of the present disclosure. Obviously, the drawings described below are only drawings of some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0024] FIG1 is a schematic diagram of a packet sending and receiving path provided by some embodiments of the present disclosure.

[0025] FIG2 is a schematic diagram of another packet sending and receiving path provided by some embodiments of the present disclosure.

[0026] FIG3 is a schematic diagram of another packet sending and receiving path provided by some embodiments of the present disclosure.

[0027] FIG4 is a schematic diagram of a communication scenario provided by some embodiments of the present disclosure.

[0028] FIG5 is a schematic diagram of another communication scenario provided by some embodiments of the present disclosure.

[0029] FIG6 is a schematic diagram of another communication scenario provided by some embodiments of the present disclosure.

[0030] FIG7 is a schematic diagram of another communication scenario provided by some embodiments of the present disclosure.

[0031] FIG8 is a schematic diagram of another communication scenario provided by some embodiments of the present disclosure.

[0032] FIG9 is a schematic diagram of another communication scenario provided by some embodiments of the present disclosure.

[0033] FIG10 is a flowchart of an end-to-end monitoring method provided by some embodiments of the present disclosure.

[0034] FIG11 is a schematic diagram of another communication scenario provided by some embodiments of the present disclosure.

[0035] FIG12 is a flowchart of another end-to-end monitoring method provided by some embodiments of the present disclosure.

[0036] FIG13 is a flowchart of another end-to-end monitoring method provided by some embodiments of the present disclosure.

[0037] FIG14 is a schematic diagram of another communication scenario provided by some embodiments of the present disclosure.

[0038] FIG15 is a flowchart of another end-to-end monitoring method provided by some embodiments of the present disclosure.

[0039] FIG16 is a schematic diagram of another communication scenario provided by some embodiments of the present disclosure.

[0040] FIG17 is a schematic diagram of another communication scenario provided by some embodiments of the present disclosure.

[0041] FIG18 is a schematic diagram of another communication scenario provided by some embodiments of the present disclosure.

[0042] FIG19 is a schematic diagram of another communication scenario provided by some embodiments of the present disclosure.

[0043] FIG20 is a schematic structural diagram of an end-to-end monitoring device provided in some embodiments of the present disclosure.

[0044] FIG21 is a schematic structural diagram of another end-to-end monitoring device provided in some embodiments of the present disclosure.

[0045] FIG22 is a schematic structural diagram of a communication device provided in some embodiments of the present disclosure. DETAILED DESCRIPTION

[0046] The following will clearly and completely describe the technical solutions of this disclosure in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this disclosure, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of this disclosure without making any creative efforts shall fall within the scope of protection of this disclosure.

[0047] It should be noted that in this disclosure, words such as "exemplarily" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this disclosure as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts by way of example.

[0048] In the following, the terms "first," "second," etc. are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the quantity of the technical features indicated. Therefore, a feature specified as "first," "second," etc. may explicitly or implicitly include one or more of the features.

[0049] In the description of this disclosure, unless otherwise specified, " / " means "or." For example, A / B can mean A or B. "And / or" herein is simply a description of an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: only A, only B, and A and B. Furthermore, "at least one" means one or more, and "a plurality" means two or more.

[0050] To facilitate understanding of the technical solutions of the embodiments of the present disclosure, the following briefly introduces the technologies involved in the embodiments of the present disclosure.

[0051] 1. APM technology.

[0052] APM technology primarily uses instrumentation to obtain a rich set of observation signals from microservices, including metrics, traces, logs, and function performance analysis. Therefore, APM technology can provide systematic, in-depth, and comprehensive measurements of microservices. However, instrumentation also introduces numerous challenges for APM, primarily in the following areas:

[0053] (1) Inserting code will modify the application source code and require re-release. For example, using bytecode enhancement technologies such as Java Agent also requires modifying the application startup parameters and re-releasing it. In addition, when an application injects multiple Java Agents, a series of compatibility issues may arise, such as conflicts between different Agents at runtime, making code maintenance much more difficult.

[0054] (2) Different instrumentation codes may cause maintenance difficulties due to different versions and logical conflicts.

[0055] (3) Some applications (such as infrastructure such as gateways on the communication path) become observation blind spots because they cannot be instrumented, resulting in the inability to detect and locate faults in a timely manner. For example, with the development of cloud native and service mesh technologies, the communication link between two services may span multiple sidecars, virtual bridges, gateways, etc. APM usually only focuses on the business code or framework / library function level, and cannot instrument nodes such as gateways on the communication path to obtain observation data. Therefore, when a fault occurs on the communication path, it is difficult to detect the fault in a timely manner relying solely on APM technology.

[0056] 2. eBPF technology.

[0057] eBPF technology is a kernel technology that allows developers to run specific functions without modifying the kernel code. For example, eBPF technology allows regular user-space applications to package the logic to be executed in the Linux kernel into bytecode. When certain events (called hooks, which can be system calls or network events, etc.) occur, the Linux kernel dynamically loads and executes the compiled code.

[0058] For example, as shown in FIG1 , a conventional packet receiving and sending path includes: a network card receives a data packet through an Ethernet interface (ethθ); the data packet passes through a point (or hook) in the eXpress Data Path (XDP) where custom code can be inserted, and is then transmitted to the Linux kernel. The Linux kernel allocates memory for the data packet based on the alloc skb function and then sends it to the kernel protocol stack; the kernel protocol stack performs GRO (Generic Receive Offloading) processing on the packet and reassembles the fragmented packets; the data packet enters the ingress hook of the traffic control (TC) and then enters the processing point of the Linux subsystem (Netfilter): a hook (PREROUTING hook) at the stage before the data packet enters the network protocol stack processes the iptables rules in the permanent table (raw table); the data packet passes through the kernel's connection tracking (conntrack) module; the iptables rules in the mangle table used to modify the data packet are processed at the PREROUTING hook; the table used for network address translation operations (NAT) is processed at the PREROUTING hook. table) of iptables rules.

[0059] Furthermore, the forwarding information base (FIB) used to quickly forward packets is searched. Next, there are four Netfilter processing points: the FORWARD hook, which processes iptables rules in the mangle table before forwarding the packet to the destination host after the routing decision is made; the FORWARD hook, which processes iptables rules in the filter table used to filter data; the POSTROUTING hook, which processes iptables rules in the mangle table after the packet leaves the local host; and the POSTROUTING hook, which processes iptables rules in the NAT table.

[0060] When a packet reaches the TC's egress hook, it performs an egress check, determining whether the packet is destined for the local device or an external host. Furthermore, before sending the packet, it undergoes Generic Segmentation Offload (GSO) fragmentation.

[0061] As shown in Figure 2, compared to the traditional packet sending and receiving path, the eBPF data transmission path is short-circuited, creating a shortcut from TC ingress to TC egress, eliminating intermediate steps. As can be seen, using eBPF technology can implement packet redirection and flow control within the kernel, significantly shortening the packet processing path and latency.

[0062] In addition, as shown in FIG3 , based on the eBPF technology, it is also possible to realize the transmission of data packets when the destination node is a service node on another host.

[0063] From the above, we can see that, in contrast to APM technology, eBPF technology does not require the implantation of instrumentation code. Instead, it extends kernel functionality by running programs in a sandbox, obtaining external data from applications (for example, using a variety of probe functions (such as kprobe, tracepoint, uprobe, etc.) to collect metrics, tracing, logs, and other information from kernel system calls and user function calls), and using thread IDs to associate this information. Due to the influence of the mapping relationship between kernel-mode threads and user-mode threads, association using thread IDs can be simple (for example, different user calls correspond to different user-mode threads and have different threadIDs), or complex (for example, a user call needs to be implemented by different threads), and may even require the assistance of other information (such as the introduction of service semantics). After the association is completed, it is necessary to identify the application protocol, further aggregate the information of different flows to obtain the application call log, and further calculate the performance indicators of the full stack.

[0064] Because eBPF technology does not require instrumentation, it does not modify applications, thus avoiding the need for re-release and code conflicts. This feature allows eBPF technology to run securely on infrastructure (such as gateways on communication paths), effectively avoiding observation blind spots. However, because eBPF technology collects raw byte streams, it is not possible to use general methods to extract service semantics. Furthermore, in complex scenarios such as cross-threading and asynchronous calls, it is difficult to obtain application performance data through common methods such as correlation and aggregation. Therefore, eBPF technology is not suitable for in-depth analysis of application performance.

[0065] 3. Operations, Administration and Maintenance (OAM) technology.

[0066] To monitor and manage the status and performance of Ethernet devices, various OAM technologies have been designed. These technologies can operate in different scenarios. For example, Bidirectional Forwarding Detection (BFD) detects data link errors between two given points (which can be two network nodes connected by a single link or two network nodes connected by a multi-hop link); Inband Network Telemetry (INT) collects, carries, organizes, and reports network status information through data plane services; and Ping tests connectivity and latency between hosts by sending Internet Control Message Protocol (ICMP) echo requests and receiving echo replies.

[0067] However, current network OAM technologies were originally designed to facilitate network operations and maintenance for operators. Therefore, these technologies struggle to perceive services, collect service-related data, and feed it back to the controller for orchestration, scheduling, routing, and other policy adjustments. Furthermore, each cloud provider typically deploys network OAM technology within its central cloud for performance monitoring. However, in multi-cloud scenarios spanning different physical locations (e.g., edge cloud, central cloud), or even across different cloud providers, there is a lack of effective performance monitoring solutions to perceive services and observe network infrastructure performance.

[0068] The above is an introduction to some of the technologies involved in the embodiments of the present disclosure, which will not be repeated below.

[0069] Traditional developers typically develop applications as a single, indivisible monolith. However, as application functionality grows in complexity, this development model exponentially increases the difficulty of operating and maintaining the application. To address this, developers are beginning to break applications down into logically independent functional modules and develop each module separately. These modules are called "microservices." In a microservices architecture, the functionality of a monolithic application can be implemented by users accessing the first microservice and then enabling communication between microservices.

[0070] Traditional monolithic applications are typically deployed and run on a single service node provided by a cloud service provider. When a user runs an application, data flows north-south from the user to the service node. With the increasing complexity of application functionality and the ubiquitous deployment of cloud services, the deployment model for microservice modules in a microservices architecture has evolved from deploying all services on a single service node of a single cloud service provider to deploying them across multiple service nodes of a single cloud service provider, or even across multiple service nodes of different cloud service providers. Therefore, when a user runs an application, data flows north-south from the user to the service node, and east-west between service nodes.

[0071] Furthermore, a single end-user request often involves multiple services. Different services may be implemented by different teams and programming languages, and deployed across thousands of service nodes across various data centers and data centers. Therefore, understanding system health and service performance, and promptly identifying and troubleshooting issues, is crucial to maintaining normal application operation.

[0072] For example, as shown in Figure 4 (a), when microservices are deployed in a service instance of a Pod, microservices can communicate directly with each other. As shown in Figure 4 (b), when microservices are deployed in multiple service nodes of a Pod, the communication path between microservices needs to pass through the service node's proxy device, such as sidecar. As shown in Figure 4 (c), when microservices are deployed in different service nodes of different cloud service providers (CSPs), the communication path between microservices may pass through multiple checkpoints such as proxy devices and cloud gateways, making the communication path very complex. Considering that the input parameters of a microservice may be the operating results of other microservices, monitoring the status of each microservice and the communication path between them is crucial to maintaining the normal operation of the application.

[0073] Currently, there are two main technical routes for monitoring microservices: using APM technology to monitor business functions, framework functions, or library functions at the L5-L7 layers, or using eBPF technology to monitor links, communication paths, system calls, etc. below the L3 layer.

[0074] APM technology uses instrumentation code, which modifies application source code and cannot be embedded in infrastructure, creating observation blind spots. eBPF technology struggles to obtain application performance data through common methods like correlation and aggregation in complex scenarios like cross-threading and asynchronous calls, making it unsuitable for in-depth analysis of application performance.

[0075] At the same time, with the emergence of new applications such as extended reality (XR) and the metaverse, users are increasingly demanding high-quality viewing experiences within limited computing and network resources. To ensure a high-quality experience despite fluctuations in computing and network resources, achieving zero-intrusion, full-stack observability in future multi-cloud communications and edge computing scenarios is a crucial challenge, and the industry currently lacks a clear solution.

[0076] To address these technical issues, we designed a simple and feasible observability solution based on a comprehensive consideration of the advantages and disadvantages of related technologies (such as APM and eBPF). This solution can understand the observability needs of different applications, flexibly and dynamically set observability solution parameters, and focus on collecting performance metrics in key areas (such as performance bottlenecks or suspected fault areas), thereby achieving efficient and rapid fault diagnosis and performance optimization.

[0077] The disclosed embodiments provide an end-to-end monitoring method, the idea of ​​which is that: a control plane device obtains configuration information of a service corresponding to a service request, and then, based on the service configuration information, determines and sends corresponding monitoring configurations to user plane devices on the service's communication path. The monitoring configurations are used to instruct the user plane devices to collect monitoring information of the service. In this way, for different service requests, the monitoring configurations of the user plane devices on the service's communication path can be flexibly and dynamically configured to adapt to the service monitoring requirements of different scenarios, and can achieve zero-intrusion full-stack performance monitoring across the user side, network side, and cloud side.

[0078] The end-to-end monitoring method provided by the embodiments of the present disclosure is described in detail below with reference to the accompanying drawings.

[0079] It should be noted that the services involved in the embodiments of the present disclosure include microservices, or other application services, etc., which are not limited in the embodiments of the present disclosure.

[0080] First, an embodiment of the present disclosure provides an end-to-end monitoring system, which includes: a user plane device and a control plane device.

[0081] In some embodiments, the user plane device includes at least one of the following: a router device deployed on the network side, a proxy device deployed on the cloud side service node, a gateway device deployed on the cloud side, and an access gateway device deployed on the user side.

[0082] For example, the cloud side may be a data center (DC), an edge cloud, etc. For ease of explanation, the following description is made taking the cloud side as a DC as an example.

[0083] In some embodiments, the control plane device includes, but is not limited to, at least one of the following: a controller and an orchestrator.

[0084] In some embodiments, the embodiments of the present disclosure deploy a performance monitoring and orchestration logic functional module in the control plane device, which is used to deploy differentiated monitoring configurations (a mechanism for monitoring service performance) on user plane devices (e.g., user-side devices, network-side devices, and cloud-side devices, etc.) on the communication path based on the configuration information of the service (e.g., service semantics) to collect measurement information reported by the user plane devices, such as traces, logs, metrics, etc.; by integrating and associating scattered measurement information, end-to-end performance indicators can be obtained.

[0085] The deployment model of the aforementioned performance monitoring and orchestration logic function module can be distributed or centralized. For example, as shown in Figure 5 , the performance monitoring and orchestration logic function module can be centrally deployed in the orchestrator, which then sends the respective monitoring configurations to the user plane devices on the communication path. For example, as shown in Figure 6 , the performance monitoring and orchestration logic function module can be distributedly deployed in various controllers of the network.

[0086] In some embodiments, an interface exists between the performance monitoring and orchestration logic function module and other logic function modules of the control plane device.

[0087] Exemplarily, there is an interface a between the performance monitoring and orchestration logic function module and the service registration management logic function module, which is used to obtain the service identification and the address information of the service node running the service (for example, the Internet Protocol (IP) address). In this way, the performance monitoring and orchestration logic function module can select a differentiated performance monitoring mechanism based on the information obtained.

[0088] Exemplarily, there is an interface b between the performance monitoring and orchestration logic function module and the routing decision logic function module, which is used to obtain routing information of the service, for example, the north-south path between the user and the service node running the service and the communication path between the service nodes running the service. In this way, the performance monitoring and orchestration logic function module can configure the communication infrastructure (for example, gateways, routers, etc.) on the communication path based on the obtained routing information to start the performance monitoring mechanism of the perception service.

[0089] In some embodiments, the end-to-end monitoring system can be applied in scenarios including the user side, network side, and cloud side, involving north-south data flow from the user side to the cloud side and east-west data flow between different clouds. The application running on the user-side device consists of at least two services, which may have a variety of deployment scenarios, for example, including the following scenarios:

[0090] Scenario 1: Deployed in a DC.

[0091] As shown in FIG7 , the service nodes running the services are deployed in a DC, and the communication path between the services passes through a proxy device (Proxy). Exemplarily, the proxy device can be a sidecar.

[0092] Exemplarily, as shown in FIG7 , the data center includes a service node A and a service node B. The data flow flows in from the traffic inlet (ingress) of the service node A, passes through the proxy device A, and reaches the service node A. When flowing out, the data flow passes through the proxy device A and flows out from the traffic outlet (egress) of the service node A. Then, the data flow flows in through the traffic inlet of the service node B, passes through the proxy device B, and reaches the service node B. When flowing out, the data flow passes through the proxy device B and flows out from the traffic outlet of the service node B.

[0093] Scenario 2: Cross-DC deployment

[0094] As shown in FIG8 , service nodes running services are deployed in multiple DCs, and the communication paths between services pass through the DC's gateway device and proxy device.

[0095] Exemplarily, as shown in Figure 8, data center A includes service node A, and data center B includes service node B; the data flow flows into data center A through gateway device A of data center A, flows in from the traffic inlet of service node A, and reaches service node A through proxy device A; when flowing out, the data flow passes through proxy device A and flows out from the traffic outlet of service node A; then, the data flow passes through gateway device A of data center A to reach gateway device B of data center B, flows into data center B through gateway device B, and then flows in from the traffic inlet of service node B, passes through proxy device B to service node B; when flowing out, the data flow passes through proxy device B and flows out from the traffic outlet of service node B.

[0096] Scenario 3: Edge deployment

[0097] As shown in Figure 9, the service nodes running the services are deployed in the edge DC, and the communication path between services passes through the gateway device, proxy device and wide area network of the edge DC.

[0098] For example, as shown in Figure 9, service node A is located in edge data center A, and service node B is located in edge data center B; the data flow flows into edge data center A through the gateway device of edge data center A, flows in from the traffic inlet of service node A, and reaches service node A through proxy device A; when flowing out, the data flow passes through proxy device A, flows out from the traffic outlet of service node A, reaches the gateway device of edge data center A, and then passes through the router device in the wide area network to reach the gateway device of edge data center B, flows into edge data center B through the gateway device of edge data center B, and then flows in from the traffic inlet of service node B, passes through proxy device B to reach service node B; when flowing out, the data flow passes through proxy device B and flows out from the traffic outlet of service node B.

[0099] It is understandable that the application scenarios of the embodiments of the present disclosure are not limited. The system architecture and business scenarios described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Those skilled in the art will appreciate that with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present disclosure are equally applicable to similar technical problems.

[0100] The present disclosure provides an end-to-end monitoring method, as shown in FIG10 , which includes the following steps:

[0101] S201. A control plane device obtains configuration information of a service corresponding to a service request.

[0102] In some embodiments, the configuration information of the service includes at least one of the following: an identifier of the service, address information of a service node running the service, and routing information of the service.

[0103] In some embodiments, the service request is issued by a user equipment (UE). Therefore, after the control plane device obtains configuration information of the service corresponding to the service request based on the service request, the method further includes: the control plane device sends a response message to the service request to the UE, where the response message to the service request includes an identifier of the service.

[0104] Exemplarily, as shown in FIG11 , when a user issues a new service request, the service request is sent to the control plane device for processing; in response to the service request, the control plane device calls interface a to obtain the service identifier and the address information of the service node running the service from the service registration management logic function module; the control plane device calls interface b to obtain the routing information of the service from the routing decision logic function module.

[0105] S202: The control plane device determines the monitoring configuration of the user plane device on the communication path of the service based on the configuration information of the service.

[0106] The monitoring configuration is used to instruct the user plane device to collect monitoring information of the service.

[0107] Exemplarily, the monitoring configuration includes at least one of the following: APM monitoring configuration, eBPF mechanism, OAM mechanism, and service identification.

[0108] Exemplarily, the monitoring information includes at least one of the following: trace information, metrics information, and logs information.

[0109] In some embodiments, when the user-side device includes: a router device deployed on the network side, a proxy device deployed on the cloud-side service node, a gateway device deployed on the cloud side, and an access gateway device deployed on the user side, the above step S202 can be implemented as: based on the configuration information of the service, determining the first monitoring configuration of the router device, the second monitoring configuration of the proxy device, and the third monitoring configuration of the gateway device.

[0110] The first monitoring configuration, the second monitoring configuration and the third monitoring configuration may be the same or different.

[0111] In some embodiments, as shown in FIG12 , the above step S202 can be implemented as the following steps S2021 - S2023 .

[0112] S2021. The control plane device determines a monitoring plan for the service based on the service identifier.

[0113] It is understandable that the control plane device can select a suitable monitoring solution for the service based on the service identification. In this way, the control plane device can select an effective monitoring solution for performance monitoring based on the differences between different services, thereby enhancing the observability of the service's communication link.

[0114] S2022: The control plane device determines a user plane device on the communication path based on the address information of the service node running the service and the routing information of the service.

[0115] Exemplarily, the routing information of the service includes a communication path. When the service nodes running the service are located on the same cloud side, the user plane devices on the communication path include: a proxy device deployed on the cloud side service node.

[0116] Exemplarily, the routing information of the service includes a communication path. When the service nodes running the service are located on different cloud sides, the user plane devices on the communication path include: a gateway device deployed on the cloud side and a proxy device deployed on the cloud side service node.

[0117] Exemplarily, the routing information of the service includes a communication path. When the service nodes running the service are respectively located on the edge cloud side, the user plane devices on the communication path include: a gateway device deployed on the cloud side, a router device deployed on the network side, and a proxy device deployed on the cloud side service node.

[0118] S2023. The control plane device configures the monitoring configuration of the user plane device based on the monitoring solution of the service.

[0119] Exemplarily, when the user-plane devices include a gateway device deployed on the cloud side, a router device deployed on the network side, and a proxy device deployed on the cloud-side service node, step S2023 may be implemented by configuring, based on the service's monitoring solution, a monitoring configuration for the gateway device, a monitoring configuration for the router device, and a monitoring configuration for the proxy device. The monitoring configuration for the gateway device, the monitoring configuration for the router device, and the monitoring configuration for the proxy device may be the same or different.

[0120] Exemplarily, in the case where the user-side device includes: a gateway device deployed on the cloud side, a router device deployed on the network side, and a proxy device deployed on the cloud-side service node, based on the service monitoring solution, the gateway device deployed on the cloud side is configured with context information (context) delivery rules for the service identifier (for example, the delivery rules can specify the sampling frequency, and the content of the context delivery, etc.), which are used to trigger the APM mechanism for the service identifier; based on the service monitoring solution, the proxy device deployed on the cloud-side service node is configured with kernel information collection rules for the service identifier (for example, the kernel information collection rules include collection parameter categories, such as input or output (IO) throughput, latency, error rate, etc.), which are used to load the eBPF mechanism for the service identifier; based on the service monitoring solution, the router device deployed on the network side is configured with packet header field addition rules for the service identifier, which are used to trigger the OAM mechanism (for example, INT) for the service identifier.

[0121] S203: The control plane device sends a monitoring configuration to the user plane device; accordingly, the user plane device receives the monitoring configuration sent by the control plane device.

[0122] It should be noted that the embodiments of the present disclosure do not limit the transmission method of the monitoring configuration.

[0123] S204: The user plane device obtains monitoring information of the service based on the monitoring configuration.

[0124] In some embodiments, as shown in FIG13 , the above step S204 can be implemented as the following steps S2041 - S2043 .

[0125] S2041. The user plane device receives a first message.

[0126] The first message is generated by the UE. For example, as shown in Figure 14, after receiving the response message to the service request, the UE obtains the service identifier from the response message and then generates a first message carrying the service identifier. During this process, the service identifier must be passed down from the L7 layer to the L4 and L3 layers and carried in the message header; for example, this can be done through the uprobe function in eBPF technology.

[0127] S2042: The user plane device detects whether the first message carries a service identifier.

[0128] Exemplarily, the service identifier may be carried in a message header, and the user plane device may detect whether the message header of the first message carries the service identifier.

[0129] S2043: When the first message carries the service identifier, the user plane device triggers the monitoring configuration of the service based on the service identifier and obtains the monitoring information of the service.

[0130] For example, as shown in FIG14 , it is assumed that the user plane device includes: a router device, a gateway device, and a proxy device; it is assumed that the OAM mechanism is configured in the router device, the APM mechanism is configured in the gateway device, and the eBPF mechanism is configured in the proxy device. When the first message passes through the router device in the wide area network, the router device can determine whether to trigger the OAM mechanism (such as BFD, INT, etc.) and set relevant parameters based on the service identifier in the message header of the first message. When the first message passes through the gateway device, the gateway device reads the service identifier in the message header, determines whether to start the APM mechanism (such as the distributed tracing mechanism) based on the service identifier, and sets relevant parameters (such as the sampling frequency, the content of the context transmission, etc.). When the first message passes through the proxy device in the service node, the proxy device triggers the eBPF program based on the service identifier in the message header, determines whether to collect information and the category of collected information (such as IO throughput, latency, error rate, etc.), and uses the service identifier to associate and obtain flow-related trace, metrics, logs, and other information.

[0131] In some embodiments, when the first service completes its operation, the next-hop service is called. At this point, the service node running the first service obtains the identifier of the second service and sends a message carrying that identifier. For example, the identifier of the second service can be obtained from a domain name resolution service node. Similarly, during this process, the identifier of the second service is passed down from the L7 layer to the L4 and L3 layers and carried in the message header.

[0132] In some embodiments, as shown in FIG15 , the method further includes the following steps:

[0133] S205: The user plane device sends monitoring information to the control plane device; correspondingly, the control plane device receives the monitoring information sent by the user plane device.

[0134] S206: The control plane device obtains an end-to-end performance indicator of the service based on the monitoring information of the service sent by the user plane device.

[0135] In some embodiments, the control plane device integrates the monitoring information of the service collected by each user plane device on the communication path to obtain an end-to-end performance indicator of the service.

[0136] Exemplarily, the end-to-end performance indicator may include at least one of the following: integrated metrics, traces, and logs.

[0137] It is understandable that since the information collected by a single device cannot reflect the end-to-end performance, all user-plane devices that trigger the monitoring configuration (such as proxy devices, router devices, gateway devices, etc.) need to report the collected service monitoring information to the control-plane device, which will be integrated by the control-plane device to obtain end-to-end performance indicators.

[0138] Exemplarily, as shown in Figure 16, the user plane device (such as a proxy device, a router device, a gateway device, etc.) reports the collected service monitoring information (such as tracking information, log information, indicator information) to the control plane device for analysis and processing; wherein, the reported information will carry the identification of the user plane device, the identification of the service, and the obtained data flow and user information (such as a five-tuple or a network traffic identifier (flow ID)), so that the control plane device can integrate and obtain end-to-end (across end, network, and cloud) data flow information related to the application corresponding to the service request.

[0139] S207. The control plane device performs operation and maintenance management on the service based on the end-to-end performance indicators.

[0140] It is understandable that the data flow information can be directly used by the control plane equipment, for example, to obtain a panoramic view of the entire application call; or to quickly locate the fault or performance bottleneck position for operation and maintenance management.

[0141] In some embodiments, the method further includes: the control plane device sending the end-to-end performance indicator to other control plane devices, so that the other control plane devices optimize the performance of the service based on the end-to-end performance indicator.

[0142] It is understandable that the end-to-end performance indicators can also be fed back to other control plane devices or other logical functional modules (for example, routing decision logical functional modules) through interfaces to optimize service performance.

[0143] For ease of understanding, the end-to-end monitoring method provided by the embodiment of the present disclosure is described below in the form of examples.

[0144] For example, as shown in Figure 17, in scenario 1, the service node running the service is deployed in a DC. Therefore, the service communication path only passes through the proxy devices in the two service nodes. Assuming that the eBPF mechanism is configured in the proxy device, when a message carrying the service identifier passes through the proxy device, the eBPF mechanism is triggered based on the service identifier in the message header, and determines whether information needs to be collected and the type of information to be collected (for example, I / O throughput, latency, or error rate). Then, based on the service identifier, service monitoring information is collected and associated to obtain flow-related traces, metrics, and logs.

[0145] For example, as shown in FIG18 , in scenario 2, the service nodes running the service are deployed in two DCs, and the communication path between the services passes through the gateway devices of the two DCs and the proxy devices of the two service nodes. Assume that the eBPF mechanism is configured in the proxy device and the APM mechanism is configured in the gateway device. When a message carrying the service identifier passes through the gateway device, the service identifier in the message header is read to determine whether to start the corresponding APM mechanism (for example, a distributed tracing mechanism) and set relevant parameters (for example, sampling frequency, context content, etc.). When a message carrying the service identifier passes through the proxy device, the eBPF mechanism is triggered based on the service identifier in the message header, and it is determined whether information needs to be collected and the category of information to be collected (for example, IO throughput, latency or error rate, etc.). Then, the monitoring information of the service is collected based on the service identifier, and the flow-related traces, metrics and logs are obtained by association.

[0146] For example, as shown in FIG19 , in scenario three, the service node running the service is deployed in the edge DC, and the communication path between services needs to cross the wide area network, passing through the gateway device, proxy device of the edge DC and the router device of the wide area network. Assume that the eBPF mechanism is configured in the proxy device, the APM mechanism is configured in the gateway device, and the OAM mechanism is configured in the router device. When a message carrying the service identifier passes through the router device in the wide area network, it can be determined whether the OAM mechanism needs to be triggered based on the service identifier in the message header, and the relevant parameters can be set. When a message carrying the service identifier passes through the gateway device, the gateway device reads the service identifier in the message header, determines whether the APM mechanism needs to be triggered (for example, it can be a distributed tracing mechanism) based on the service identifier, and sets the relevant parameters (for example, sampling frequency, content passed by context, etc.). When a message carrying a service identifier passes through the proxy device, the eBPF mechanism can be triggered based on the service identifier in the message header to determine whether information needs to be collected and the category of information to be collected (for example, IO throughput, latency, or error rate, etc.). The service identifier is used to collect service monitoring information and associate it to obtain flow-related traces, metrics, and logs.

[0147] The above mainly introduces the solution of the embodiment of the present disclosure from the perspective of method. It can be understood that in order to realize the above functions, the end-to-end monitoring device includes at least one of the hardware structure and software modules corresponding to the execution of each function. It should be easy for those skilled in the art to realize that, in combination with the units and algorithm steps of each example described in the embodiment disclosed herein, the embodiment of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiment of the present disclosure.

[0148] The embodiment of the present disclosure can divide the end-to-end monitoring device into functional modules according to the above-mentioned method embodiment. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one functional module. The above-mentioned integrated module can be implemented in the form of hardware or software. It should be noted that the division of modules in the embodiment of the present disclosure is schematic and is only a logical function division. There may be other division methods in actual implementation. The following is an example of dividing each functional module corresponding to each function.

[0149] Figure 20 is a schematic diagram of the structure of an end-to-end monitoring device provided in an embodiment of the present disclosure. This end-to-end monitoring device is applied to a control plane device and can execute the end-to-end monitoring method provided in the above method embodiment. As shown in Figure 20, the end-to-end monitoring device 600 includes: an acquisition module 601, a determination module 602, and a communication module 603. In other embodiments, the end-to-end monitoring device 600 also includes: a management module 604.

[0150] The acquisition module 601 is used to acquire configuration information of the service corresponding to the service request.

[0151] The determination module 602 is configured to determine a monitoring configuration of a user plane device on a communication path of the service based on configuration information of the service, where the monitoring configuration is used to instruct the user plane device to collect monitoring information of the service.

[0152] The communication module 603 is configured to send monitoring configuration to the user plane device.

[0153] In some embodiments, the configuration information of the service includes at least one of the following: an identifier of the service, address information of a service node running the service, and routing information of the service.

[0154] In some embodiments, the determination module 602 can be used to determine the monitoring plan of the service based on the identification of the service; determine the user plane device on the communication path based on the address information of the service node running the service and the routing information of the service; and configure the monitoring configuration of the user plane device based on the monitoring plan of the service.

[0155] In some embodiments, the user plane device includes at least one of the following: a router device deployed on the network side, a proxy device deployed on the cloud side service node, a gateway device deployed on the cloud side, and an access gateway device deployed on the user side.

[0156] In some embodiments, the determination module 602 may be configured to determine a first monitoring configuration of the router device, a second monitoring configuration of the proxy device, and a third monitoring configuration of the gateway device based on configuration information of the service.

[0157] In some embodiments, the monitoring configuration includes at least one of the following: APM monitoring configuration, eBPF mechanism, OAM mechanism, and service identification.

[0158] In some embodiments, the monitoring information of the service includes at least one of the following: tracking information, log information, and indicator information.

[0159] In some embodiments, the communication module 603 is further configured to receive service monitoring information sent by a user plane device.

[0160] In some embodiments, the management module 604 is configured to obtain end-to-end performance indicators of the service based on monitoring information of the service sent by the user plane device; and perform operation and maintenance management of the service based on the end-to-end performance indicators.

[0161] In some embodiments, the communication module 603 is further configured to send the end-to-end performance indicator to other control plane devices, so that the other control plane devices optimize the performance of the service based on the end-to-end performance indicator.

[0162] In some embodiments, the communication module 603 is further configured to send a response message of the service request to the user equipment, where the response message of the service request includes an identifier of the service.

[0163] FIG21 is a schematic diagram of the structure of another end-to-end monitoring device provided by an embodiment of the present disclosure. The end-to-end monitoring device is applied to a user plane device and can execute the end-to-end monitoring method provided by the above method embodiment. As shown in FIG21 , the end-to-end monitoring device 700 includes: a communication module 701 and an acquisition module 702.

[0164] The communication module 701 is configured to receive a monitoring configuration of a service sent by a control plane device, where the user plane device is located on a communication path of the service.

[0165] The acquisition module 702 is used to acquire monitoring information of the service based on the monitoring configuration.

[0166] In some embodiments, the communication module 701 is further configured to send monitoring information to the control plane device.

[0167] In some embodiments, the acquisition module 702 can be used to receive a first message; detect whether the first message carries a service identifier; if the first message carries a service identifier, trigger the service monitoring configuration based on the service identifier to obtain service monitoring information.

[0168] In the case of implementing the functions of the above-mentioned integrated modules in hardware, the embodiments of the present disclosure provide a possible structure of the communication device involved in the above-mentioned embodiments. As shown in Figure 22, the communication device 800 includes: a processor 802 and a bus 804. In some embodiments, the communication device may also include a memory 801; in some embodiments, the communication device 800 may also include a communication interface 803.

[0169] The processor 802 can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the embodiments of the present disclosure. The processor 802 can be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof, and can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the embodiments of the present disclosure. The processor 802 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0170] The communication interface 803 is used to connect to other devices via a communication network, such as Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0171] The memory 801 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0172] As a possible implementation, the memory 801 can exist independently of the processor 802. The memory 801 can be connected to the processor 802 via a bus 804 to store instructions or program codes. When the processor 802 calls and executes the instructions or program codes stored in the memory 801, the end-to-end monitoring method provided by the embodiment of the present disclosure can be implemented. In another possible implementation, the memory 801 can also be integrated with the processor 802.

[0173] Bus 804 can be an Extended Industry Standard Architecture (EISA) bus, etc. Bus 804 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, FIG22 shows only one thick line, but this does not mean that there is only one bus or only one type of bus.

[0174] Some embodiments of the present disclosure provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium), which stores computer program instructions. When the computer program instructions are executed on a computer, the computer executes the end-to-end monitoring method as described in any of the above embodiments.

[0175] Exemplarily, the above-mentioned computer-readable storage media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes, etc.), optical disks (e.g., compact disks (CDs), digital versatile disks (DVDs), etc.), smart cards, and flash memory devices (e.g., erasable programmable read-only memories (EPROMs), cards, sticks, or key drives, etc.). The various computer-readable storage media described in the present disclosure may represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0176] An embodiment of the present disclosure provides a computer program product comprising instructions. When the computer program product is run on a computer, the computer is enabled to execute the end-to-end monitoring method described in any one of the above embodiments.

[0177] The above is only a specific embodiment of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or replacements within the technical scope disclosed in the present disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. An end-to-end monitoring method, applied to a control plane device, comprising: Get the configuration information of the service corresponding to the service request; determining, based on the configuration information of the service, a monitoring configuration of a user plane device on a communication path of the service, wherein the monitoring configuration is used to instruct the user plane device to collect monitoring information of the service; Send the monitoring configuration to the user plane device.

2. The method according to claim 1, wherein The configuration information of the service includes at least one of the following: an identifier of the service, address information of a service node running the service, and routing information of the service.

3. The method according to claim 2, wherein: The determining, based on the configuration information of the service, a monitoring configuration of a user plane device on a communication path of the service includes: Determining a monitoring plan for the service based on the identifier of the service; determining the user plane device on the communication path based on the address information of the service node running the service and the routing information of the service; Based on the monitoring solution of the service, a monitoring configuration of the user plane device is configured.

4. The method according to claim 1, wherein The user plane device includes at least one of the following: a router device deployed on the network side, a proxy device deployed on the cloud side service node, a gateway device deployed on the cloud side, and an access gateway device deployed on the user side.

5. The method according to claim 4, wherein The determining, based on the configuration information of the service, a monitoring configuration of a user plane device on a communication path of the service includes: Based on the configuration information of the service, a first monitoring configuration of the router device, a second monitoring configuration of the proxy device, and a third monitoring configuration of the gateway device are determined.

6. The method according to claim 1, wherein The monitoring configuration includes at least one of the following: application performance monitoring APM monitoring configuration, extended Berkeley packet filter eBPF mechanism, operation maintenance management OAM mechanism, and identification of the service.

7. The method according to claim 1, wherein The monitoring information of the service includes at least one of the following: tracking information, log information, and indicator information.

8. The method according to claim 1, further comprising: Receive monitoring information of the service sent by the user plane device.

9. The method according to claim 8, further comprising: Obtaining an end-to-end performance indicator of the service based on the monitoring information of the service sent by the user plane device; The service is operated and maintained based on the end-to-end performance indicators.

10. The method according to claim 9, further comprising: The end-to-end performance indicator is sent to other control plane devices, so that the other control plane devices optimize the performance of the service based on the end-to-end performance indicator.

11. The method according to claim 2, further comprising: A response message to the service request is sent to the user equipment, where the response message to the service request includes an identifier of the service.

12. An end-to-end monitoring method, applied to a user plane device, comprising: receiving a monitoring configuration of a service sent by a control plane device, wherein the user plane device is located on a communication path of the service; Based on the monitoring configuration, monitoring information of the service is obtained.

13. The method according to claim 12, further comprising: The monitoring information is sent to the control plane device.

14. The method according to claim 12, wherein: The acquiring monitoring information of the service based on the monitoring configuration includes: receiving a first message; Detecting whether the first message carries the identifier of the service; In a case where the first message carries the identifier of the service, monitoring configuration of the service is triggered based on the identifier of the service, and monitoring information of the service is obtained.

15. A communication device comprising: memory and processor; The memory is coupled to the processor; The memory is used to store instructions executable by the processor; When the processor executes the instructions, the end-to-end monitoring method according to any one of claims 1 to 14 is performed.

16. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the end-to-end monitoring method according to any one of claims 1 to 14.

Images