Inter-container network monitoring and visualization system and method
By combining the eBPF module and the K8s module, non-intrusive inter-container network monitoring and visualization are achieved, solving the problem of low monitoring efficiency in existing technologies and improving monitoring efficiency and accuracy.
Patent Information
- Application Number
- CN202511167536.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-20
- Publication Date
- 2025-11-28
AI Technical Summary
Existing container network monitoring technologies rely on business code instrumentation or sidecar injection, resulting in high maintenance costs and an inability to capture second-level container lifecycles and millisecond-level transient connections, thus affecting monitoring efficiency.
By combining the eBPF module and the K8s module, network activity data is captured in the Linux kernel through the eBPF module, and the cluster status is monitored in real time using the K8s module. Combined with the OpenTelemetry reporting module, non-intrusive inter-container network traffic monitoring and visualization are achieved.
It achieves zero-intrusion inter-container network monitoring, improves monitoring efficiency and accuracy, reduces deployment costs, and supports second-level lifecycle network behavior tracking and visualization analysis.
Smart Images
Figure CN121037271A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of network monitoring, and in particular to a system and method for monitoring and visualizing inter-container networks. Background Technology
[0002] With the deep application of cloud-native technologies, the dynamic scheduling characteristics of container orchestration systems and the elastic scaling mechanism of microservice architectures have made the network performance characteristics of cloud applications highly dynamic and subject to instantaneous fluctuations.
[0003] However, existing container network monitoring technologies often rely on business code instrumentation or sidecar injection. This approach is highly intrusive, resulting in high maintenance costs. Furthermore, the fixed-period sampling mechanism cannot capture second-level container lifecycles and millisecond-level transient connections, leading to missed traffic peaks and affecting the monitoring efficiency of container network monitoring.
[0004] Therefore, how to efficiently monitor inter-container networks has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0005] This invention provides a system and method for monitoring and visualizing inter-container network traffic, addressing how to achieve comprehensive monitoring and visualization of inter-container network traffic in Kubernetes (K8s) non-intrusively through eBPF technology, thereby improving the efficiency and accuracy of monitoring.
[0006] To address the aforementioned technical problems, this invention provides a container network monitoring and visualization system, which includes: an eBPF module, a K8s module, an OpenTelemetry reporting module, and an observability backend.
[0007] The eBPF module is deployed on the Linux kernel and integrates pre-compiled eBPF programs for capturing network activity data.
[0008] The K8s module integrates a K8s API Watch mechanism, which is used to monitor and synchronize the cluster status in real time in order to establish a mapping database;
[0009] The K8s module pulls the network activity data from the eBPF module, performs contextual association matching between the network activity data and the information in the mapping database, outputs target monitoring data, and sends it to the OpenTelemetry reporting module.
[0010] The OpenTelemetry reporting module integrates the OpenTelemetry protocol, which is used to convert the received target monitoring data into OpenTelemetry attribute indicators and send them to the observability backend.
[0011] The observability backend is used to receive and visualize the OpenTelemetry attribute metrics.
[0012] Furthermore, the K8s module has a metrics-agent unit deployed in a bypass configuration.
[0013] Furthermore, the metrics-agent unit integrates the BCC tool, which is used to load the pre-compiled eBPF program into the Linux kernel and trigger the kprobe mechanism to capture the network activity data.
[0014] Furthermore, the K8s module queries the mapping database based on the IP address to inject context-related tags into the network activity data.
[0015] Furthermore, the network activity data includes: six-tuple data corresponding to sockets, network packets, and retransmission packets, as well as the mapping relationship between each six-tuple data and the process PID.
[0016] Furthermore, the network activity data is temporarily stored in the eBPF Hash MAP of the eBPF module.
[0017] Furthermore, the mapping database includes Pod mapping, Node host association mapping, and Service load relationship matching mapping.
[0018] Furthermore, the OpenTelemetry attribute metrics include counter types.
[0019] Furthermore, the OpenTelemetry attribute metrics include: number of bytes transmitted over the network, number of packets transmitted over the network, number of retransmitted packets over the network, cumulative network round-trip time, and number of network round-trip time samples.
[0020] Another embodiment of the present invention provides a method for monitoring and visualizing inter-container networks, including:
[0021] Run the metrics-agent program, which is deployed in a bypass manner on the K8s cluster nodes;
[0022] After running, the pre-compiled eBPF program is loaded into the Linux kernel using the BCC tool, and the network activity data is obtained from the Linux kernel.
[0023] The cluster status is monitored and synchronized in real time according to the K8s API Watch mechanism to establish the mapping database;
[0024] The network activity data is matched with the mapping database in a contextual relationship to obtain the target monitoring data;
[0025] The target monitoring data is converted into the OpenTelemetry attribute metrics;
[0026] The OpenTelemetry attribute metrics are reported to the observability backend using the OpenTelemetry protocol.
[0027] Compared with the prior art, the beneficial effects of the embodiments of the present invention are at least one of the following:
[0028] This invention deploys metrics-agent in a bypass manner, while simultaneously capturing real-time network activity data between containers through an eBPF module deployed in the kernel in conjunction with the Kprobe interception mechanism. This achieves zero-intrusion monitoring, avoiding the performance loss caused by frequent context switching in traditional user-space tools and reducing deployment costs. Real-time information synchronization via a K8s module, linking network activity data with K8s metadata, enables precise tracking of network behavior of containers with a lifecycle of seconds, improving monitoring efficiency and reliability. Standardized output via OpenTelemetry allows for visualized analysis from raw network activity data to business semantics, enhancing the scalability of container network monitoring. Attached Figure Description
[0029] Figure 1 This is a schematic diagram of the inter-container network monitoring and visualization system structure in one embodiment of the present invention;
[0030] Figure 2 This is an overall execution flowchart of network monitoring in one embodiment of the present invention;
[0031] Figure 3 This is a schematic diagram of a method for monitoring and visualizing inter-container networks in one embodiment of the present invention. Detailed Implementation
[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.
[0033] In the description of this application, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," "third," etc., may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.
[0034] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal communication between two components. The terms "vertical," "horizontal," "left," "right," "upper," "lower," and similar expressions used herein are for illustrative purposes only and do not indicate or imply that the device or component referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as limiting the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.
[0035] In the description of this application, it should be noted that, unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is for the purpose of describing specific embodiments only and is not intended to limit the invention. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.
[0036] One embodiment of the present invention provides a container network monitoring and visualization system. For details, please refer to [link to documentation]. Figure 1 , Figure 1 The diagram illustrates the architecture of an inter-container network monitoring and visualization system according to one embodiment of the present invention. The system includes: an eBPF module 1, a K8s module 2, an OpenTelemetry reporting module 3, and an observability backend 4. eBPF (extended Berkeley Packet Filter) is an extended Berkeley packet filter.
[0037] In this embodiment, a user-space metrics-agent unit is deployed in bypass mode within Kubernetes module 2. The metrics-agent unit integrates the BCC tool, which can load pre-compiled eBPF programs into the Linux kernel (kernel / kernel space) and trigger the kprobe mechanism to capture the network activity data. In this embodiment, the BCC tool and Python 3.x are installed on the node where the metrics-agent resides, enabling interaction with the kernel and loading C code. Specifically, the eBPF collection program compiled by BCC is converted from C language to bytecode in a Python script and loaded into the kernel as bytecode.
[0038] It should be understood that Kubernetes (K8s) is responsible for scheduling containerized workloads (K8s is a tool for managing containerized applications across multiple hosts on a cloud platform). This embodiment does not make any configuration changes to the K8s cluster itself; it only deploys the data acquisition component in a bypass manner, completely independent of the application and Kubernetes cluster configuration, and does not rely on TraceProvider, Sidecar, service mesh, or code-level instrumentation. This non-intrusive design significantly reduces access costs, avoids version compatibility issues, and greatly improves the system's portability and ease of use.
[0039] eBPF module 1 is deployed in the Linux kernel and integrates a pre-compiled eBPF program for capturing network activity data. This network activity data includes: six-tuple data corresponding to sockets, network packets, and retransmission packets, as well as the mapping relationship between each six-tuple data and the process PID. In this embodiment, eBPF module 1 utilizes the kprobe mechanism to intercept network activity-related system calls (such as connect, sendmsg, recvmsg) and protocol stack hooks (such as socket tracepoints). It should be understood that kprobe is the Linux kernel's debugging mechanism; mounting means binding the eBPF program to a target kernel function via BCC. Specifically:
[0040] After loading the eBPF program into the Linux kernel, BCC intercepts the following system calls via the kprobe mechanism: udp_sendmsg(), udp_recvmsg(), udp_v6_sendmsg(), udp_v6_recvmsg(), tcp_v4_connect(), tcp_v6_connect(), inet_csk_accept(), tcp_sendmsg(), and tcp_recvmsg(). It also intercepts and collects the six-tuple information corresponding to the socket, including: IP version, TCP or UDP, source IP, source port, destination IP, destination port, and the mapping to the process PID, which is then stored in lruhash.
[0041] BCC will also intercept the following functions via the kprobe mechanism: udp_send_skb(), udp_queue_rcv_skb(), __tcp_transmit_skb(), and tcp_queue_rcv(). Meanwhile, tcp_queue_rcv() can retrieve the round-trip time (RTT); obtain the six-tuple information corresponding to the network packet; match this six-tuple information with the six-tuple information obtained from the socket; obtain and save the PID corresponding to the network packet; the network packet size; and increment the network packet counter.
[0042] BCC also intercepts tcp_retransmit_skb() through the kprobe mechanism, obtains the six-tuple information corresponding to the retransmitted packet, matches the six-tuple information obtained from the socket, obtains and saves the PID corresponding to the network packet, the network packet size, and increments the network packet counter.
[0043] This embodiment uses the above design as the foundation for cloud-native network observability, dynamically mounting to achieve zero-intrusion monitoring. eBPF intercepts critical system calls through Kprobe, aggregating network transmission metrics such as byte count, packet count, retransmission count, and round-trip latency in real time within the kernel, effectively reducing CPU utilization and mitigating interference with business processes. This network activity data is temporarily stored in the eBPF Hash MAP (a storage structure) of the eBPF module.
[0044] Furthermore, Kubernetes module 2 integrates a Kubernetes API Watch mechanism, which can be used to monitor and synchronize cluster status in real time, thereby establishing a mapping database. Specifically, using the Kubernetes API watch mechanism, the following objects are monitored in real time: all Node information, which is used to associate with the host machine; all Pod information, which is indexed by Pod IP; and all Service information, which can match load relationships. Based on the monitored metadata, the following mappings are established: Pod-IP → Pod metadata (including namespace, tag, UID), Pod UID → Node metadata, and Service IP → Service metadata, i.e., the mapping database, which maintains the dynamic association relationships of Pod mapping, Node host association mapping, and Service load relationship matching mapping.
[0045] To ensure that the collected network activity data has accurate contextual information, Kubernetes module 2 periodically pulls network activity data from the eBPF Hash Map of eBPF module 1 and performs contextual association matching between the network activity data and information in the mapping database to output target monitoring data. This operation essentially associates network activity data with Kubernetes metadata, thus attaching multi-dimensional Kubernetes tags to each data entry to support subsequent visualization analysis and alerting policy configuration based on services, namespaces, workloads, and other dimensions. Specifically, Kubernetes module 2 queries the mapping database based on IP addresses, matching the corresponding Pod / Service / Node metadata to the network activity data based on the IP address, thereby injecting multi-dimensional contextual association tags into the network activity data.
[0046] For example, the system searches for corresponding pod / service information in the mapping database using the IP address. If the pod information exists, it then searches for node information based on the pod information. It should be understood that an IP address may be associated with a Kubernetes pod or service, therefore a maximum of two searches are possible: first, checking if it corresponds to a pod (using the "Pod-IP->pod metadata" table); if not, then searching for the corresponding service (using the "Service IP->Service metadata" table). If the IP address corresponds to a pod, the Kubernetes pod is bound to a node, and the corresponding node metadata (using the "Pod UID->Node metadata" table) can be found using the UID (Unique ID) in the Pod metadata.
[0047] This embodiment uses eBPF+K8s API Watch to effectively track short-lifecycle communications between containers (such as function cold starts, Job tasks, etc.), improving the monitoring accuracy and timeliness in high-concurrency and elastic scaling scenarios.
[0048] Next, the target monitoring data with injected context-related tags is sent to OpenTelemetry reporting module 3. OpenTelemetry reporting module 3 integrates the OpenTelemetry protocol / OTLP (an open-source observability protocol). For details, please refer to [link / reference needed]. Figure 2 As shown, the OpenTelemetry reporting module 3 is used to convert the received target monitoring data into OpenTelemetry attribute indicators and send them to the compatible observability backend 4.
[0049] The OpenTelemetry attribute metrics include: number of bytes transmitted over the network, number of packets transmitted over the network, number of retransmitted packets over the network, cumulative network round-trip time, and number of network round-trip time samples. In other words, this embodiment converts the associated target monitoring data into a metering data type defined by the OpenTelemetry protocol. These OpenTelemetry attribute metrics include those converted into counter types.
[0050] Finally, OpenTelemetry property metrics are received and visualized via observability backend 4. For example, observability backend 4 could be Prometheus, Grafana, Jaeger, etc.
[0051] In summary, this embodiment deploys eBPF in kernel space and directly mounts it to system calls related to network transmission, collecting network activity data of inter-container communication in a non-embedded manner. By synchronizing container metadata in real time through Kubernetes, it ensures that the collected metrics have accurate context information and uniformly converts them into the OpenTelemetry protocol format for reporting to the observability backend. No modifications are required to the Kubernetes cluster configuration, business application code, or service deployment method, resulting in higher deployment flexibility and system compatibility. It can be directly deployed in various Kubernetes distributions (such as Kubeadm, ACK, GKE, etc.).
[0052] One embodiment of the present invention provides a method for monitoring and visualizing inter-container networks, applied to the aforementioned system. For details, please refer to [link to relevant documentation]. Figure 3 , Figure 3 The diagram shown illustrates a method for monitoring and visualizing inter-container networks according to one embodiment of the present invention, including the following steps:
[0053] S1. Run the metrics-agent program, which is deployed on the K8s cluster nodes in a bypass manner.
[0054] In this embodiment, K8s is first deployed, and the Python environment and BCC tools are installed. Then, the metrics-agent program is started.
[0055] Since this embodiment does not rely on changes to K8s network plugins (such as CNI, Cilium) or control plane configurations, and does not require the introduction of Sidecar proxies or CRD custom resources, it has good portability and compatibility.
[0056] S2. After running, use the BCC tool to load the pre-compiled eBPF program into the Linux kernel and obtain network activity data from the Linux kernel.
[0057] In this embodiment, when metrics-agent starts, it automatically performs the eBPF program loading and attaching kprobe to critical system calls, collects network activity data in real time and stores it in the eBPF Hash Map.
[0058] S3. Monitor and synchronize cluster status in real time according to the K8s API Watch mechanism to establish a mapping database.
[0059] Next, metrics-agent monitors the cluster status through the K8s API Watch and creates a mapping relationship.
[0060] S4. Perform contextual association matching between the network activity data and the information in the mapping database to obtain the target monitoring data.
[0061] S5. Convert target monitoring data into OpenTelemetry attribute metrics.
[0062] S6. Use the OpenTelemetry protocol to report OpenTelemetry attribute metrics to the observability backend.
[0063] The technical features and effects of the inter-container network monitoring and visualization method proposed in this embodiment are the same as those of the inter-container network monitoring and visualization system proposed in this embodiment, and will not be repeated here.
[0064] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.
Claims
1. A container network monitoring and visualization system, characterized in that, The system includes: an eBPF module, a K8s module, an OpenTelemetry reporting module, and an observability backend; The eBPF module is deployed on the Linux kernel and integrates pre-compiled eBPF programs for capturing network activity data. The K8s module integrates a K8s API Watch mechanism, which is used to monitor and synchronize the cluster status in real time in order to establish a mapping database; The K8s module pulls the network activity data from the eBPF module, performs contextual association matching between the network activity data and the information in the mapping database, outputs target monitoring data, and sends it to the OpenTelemetry reporting module. The OpenTelemetry reporting module integrates the OpenTelemetry protocol, which is used to convert the received target monitoring data into OpenTelemetry attribute indicators and send them to the observability backend. The observability backend is used to receive and visualize the OpenTelemetry attribute metrics.
2. The inter-container network monitoring and visualization system as described in claim 1, characterized in that, The K8s module has a metrics-agent unit deployed in a bypass configuration.
3. The inter-container network monitoring and visualization system as described in claim 2, characterized in that, The metrics-agent unit integrates the BCC tool, which is used to load the pre-compiled eBPF program into the Linux kernel and trigger the kprobe mechanism to capture the network activity data.
4. The inter-container network monitoring and visualization system as described in claim 1, characterized in that, The K8s module queries the mapping database based on the IP address to inject context-related tags into the network activity data.
5. The inter-container network monitoring and visualization system as described in claim 1, characterized in that, The network activity data includes: six-tuple data corresponding to sockets, network packets, and retransmission packets, as well as the mapping relationship between each six-tuple data and the process PID.
6. The inter-container network monitoring and visualization system as described in claim 1, characterized in that, The network activity data is temporarily stored in the eBPF Hash MAP of the eBPF module.
7. The inter-container network monitoring and visualization system as described in claim 1, characterized in that, The mapping database includes Pod mapping, Node host association mapping, and Service load relationship matching mapping.
8. The inter-container network monitoring and visualization system as described in claim 1, characterized in that, The OpenTelemetry attribute metrics include counter types.
9. The inter-container network monitoring and visualization system as described in claim 1, characterized in that, The OpenTelemetry attribute metrics include: number of bytes transmitted over the network, number of packets transmitted over the network, number of retransmitted packets over the network, cumulative network round-trip time, and number of network round-trip time samples.
10. A method for monitoring and visualizing inter-container networks, applied to the system described in any one of claims 1 to 9, characterized in that, include: Run the metrics-agent program, which is deployed in a bypass manner on the K8s cluster nodes; After running, the pre-compiled eBPF program is loaded into the Linux kernel using the BCC tool, and the network activity data is obtained from the Linux kernel. The cluster status is monitored and synchronized in real time according to the K8s API Watch mechanism to establish the mapping database; The network activity data is matched with the mapping database in a contextual relationship to obtain the target monitoring data; The target monitoring data is converted into the OpenTelemetry attribute metrics; The OpenTelemetry attribute metrics are reported to the observability backend using the OpenTelemetry protocol.