Monitoring method and device based on Kubernetes container
By defining eBPF data in the Kubernetes container environment and using Operator to monitor CRD status, the problems of inconvenient monitoring deployment and high resource consumption are solved, realizing fast and resource-efficient container monitoring, improving development efficiency and user experience.
Patent Information
- Application Number
- CN202510955670.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-11
- Publication Date
- 2025-11-21
AI Technical Summary
In existing technologies, monitoring and deploying Kubernetes containers is inconvenient and consumes a lot of resources. The traditional sidecar mode increases resource consumption, and eBPF programs require careful design and debugging, and resource consumption remains high even when deployed on a large scale.
Define eBPF data in the Kubernetes container environment, monitor CRD status through Operator, load pre-compiled eBPF programs, obtain container process data and store it in eBPF Map, and export it to data storage terminals such as OpenTelemetry, ELK or Prometheus to optimize eBPF program development to avoid resource consumption.
It enables fast and resource-efficient Kubernetes container monitoring, improving development efficiency and user experience, and automatically collects data to facilitate rapid integration into existing monitoring systems.
Smart Images
Figure CN120994487A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of computer, and in particular, to a monitoring method and device based on Kubernetes container. BACKGROUND
[0002] At present, the deployment of containers is becoming more and more intensive, and the observability of communication between containers is becoming more and more difficult to sort out. The elastic scaling period of containers is becoming shorter and shorter, and the transience and stateless characteristics make the real-time requirement of monitoring higher and higher. In addition, the resource consumption caused by the traditional use of the sidecar mode may increase the consumption of resources.
[0003] In terms of data collection, the underlying tool may only collect part of the indicators or collect all the indicators, which may cause missing of key indicators or excessive collection of unnecessary indicators.
[0004] eBPF is a technology that runs programs in the Linux kernel, which provides a revolutionary solution for container monitoring, allowing monitoring of each container without consuming a lot of resources, and allowing monitoring of application performance without modifying application code, which is particularly useful for microservice architecture in Kubernetes environment, as it may contain multiple programming languages and frameworks. eBPF can monitor network traffic at the kernel level, which includes detailed indicators from the data link layer to the transport layer, which helps to diagnose network-related problems.
[0005] However, the eBPF technology has the following defects:
[0006] Firstly, eBPF programs need to be carefully designed and written to ensure that they do not have a negative impact on system performance. This requires constant debugging and optimization in the container environment, which brings inconvenience to developers in distribution and deployment. In addition, although eBPF consumes less resources than traditional monitoring methods, it still needs to consume certain computing and memory resources when deployed on a large scale, which may pose a challenge to resource-constrained environments. SUMMARY
[0007] The present application provides a monitoring method and device based on Kubernetes container to solve the problems of inconvenient monitoring deployment and large resource impact in the prior art.
[0008] In a first aspect, the present application provides a monitoring method based on Kubernetes container, specifically comprising the following steps:
[0009] Defining eBPF data in CRD (Custom Resource Definitions) in the Kubernetes container environment;
[0010] Monitoring the state of the CRD through an Operator in a Kubernetes container environment, loading a pre-compiled eBPF program when the CRD changes, and updating the data in the CRD;
[0011] When a new process is executed each time, it is determined whether the new process is a container process, and if the new process is a container process, the data information of the container process is obtained and stored in an eBPF Map (wherein the eBPF Map is used to transmit data to a user end);
[0012] The data in the eBPF Map is exported in one or at least two formats and stored in a corresponding data storage end and displayed.
[0013] Preferably, the eBPF data includes an eBPF program, a data mapping relationship, and a filtering rule; wherein the data mapping includes a mapping of kernel data and monitoring data, and the filtering rule includes a loading filtering rule and an execution filtering rule.
[0014] Preferably, the pre-compiled eBPF program specifically includes the following steps:
[0015] The latest eBPF program in the CRD is saved as a.bpf.c format file and stored in a persistent volume of Kubernetes;
[0016] An eBPF compiler is started for compilation, the program is compiled through a libbpf mode, and a pre-compiled eBPF program is formed.
[0017] Preferably, the loading process specifically includes the following steps:
[0018] The CRD loads a filtering rule, determines whether there is a node filtering rule, if there is, filters out unnecessary nodes and loads or unloads the eBPF program, and if there is not, directly loads or unloads the eBPF program;
[0019] The CRD self-filtering rule determines whether there is another filtering rule, and if there is, stores it in the eBPF Map.
[0020] Preferably, a target file is generated through a Kubernetes persistent volume, and the eBPF program is loaded or unloaded by executing the target file.
[0021] Preferably, the data information of the container includes a cgroupid and a pid, and is stored in the eBPFMap with the pid as a key.
[0022] Preferably, the data storage end includes one or at least two of OpenTelemetry, ELK, and Prometheus.
[0023] Preferably, the eBPF program is developed through libbpf, avoiding incompatibility between kernel versions.
[0024] Preferably, the development process of the eBPF program specifically checks whether the pid and cgroupid information in the eBPF Map matches the current process or thread, and if so, acquires data from the Linux system kernel and stores it in the eBPF Map.
[0025] In a second aspect, the application also provides a Kubernetes container-based monitoring device, specifically including the following modules:
[0026] A data definition module is configured to define eBPF data in a CRD in a Kubernetes container environment.
[0027] A data compilation and update module is configured to monitor the state of the CRD through an Operator in the Kubernetes container environment, load a pre-compiled eBPF program when the CRD changes, and update the data in the CRD.
[0028] A new process data acquisition module is configured to determine whether a new process is a container process each time the new process is executed, acquire data information of the container process if the new process is a container process, and store the data information in an eBPF Map.
[0029] A data storage module is configured to export the data in the eBPF Map to one or at least two formats, store the data in a corresponding data storage end, and display the data.
[0030] The eBPF Map is configured to transmit data to a user space.
[0031] Preferably, in the data definition module, the eBPF data includes an eBPF program, a data mapping relationship, and a filtering rule; the data mapping includes a mapping of kernel data and monitoring data, and the filtering rule includes a loading filtering rule and an execution filtering rule.
[0032] Preferably, the data compilation and update module includes a pre-compilation submodule and a data update submodule; the pre-compilation submodule is configured to pre-compile an eBPF program, and the data update submodule is configured to update data loading.
[0033] Preferably, the pre-compilation submodule specifically includes the following units:
[0034] A pre-compilation first unit is configured to save the latest eBPF program in the CRD as a file in the.bpf.c format and store the file in a persistent volume of Kubernetes.
[0035] A pre-compilation second unit is configured to start the eBPF compiler for compilation, compile the program through libbpf, and form a pre-compiled eBPF program.
[0036] Preferably, the data updating sub-module specifically comprises the following units:
[0037] A data loading first unit is configured to load the filtering rules through the CRD, judge whether there are node filtering rules, filter out unnecessary nodes and load or unload the eBPF program if there are the node filtering rules, or directly load or unload the eBPF program if there are no the node filtering rules.
[0038] A data loading second unit is configured to judge whether there are other filtering rules through the CRD self-filtering rules, and store the other filtering rules in the eBPF Map if there are the other filtering rules.
[0039] Preferably, in the data loading second unit, a target file is generated through the Kubernetes persistent volume, and the eBPF program is loaded or unloaded by executing the target file.
[0040] Preferably, in the new process data obtaining module, the data information of the container includes a cgroupid and a pid, and the pid is stored in the eBPF Map as a key.
[0041] Preferably, in the data storage module, the data storage end comprises one or at least two of OpenTelemetry, ELK and Prometheus.
[0042] Preferably, the eBPF program is developed through libbpf, which avoids incompatibility between kernel versions.
[0043] Preferably, the development process of the eBPF program specifically comprises the following steps: verifying whether the pid and cgroupid information in the eBPF Map match the current process or thread, obtaining data from the Linux system kernel if the pid and cgroupid information match the current process or thread, and storing the data in the eBPF Map.
[0044] In a third aspect, the present application further provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the monitoring method based on the Kubernetes container according to any one of the first aspect of the present application.
[0045] In a fourth aspect, the present application also provides an electronic device, comprising: a memory, storing a computer program; a processor, connected with the memory in communication, invoking the computer program to execute the method for monitoring based on the Kubernetes container according to any one of the first aspect.
[0046] Compared with the prior art, the present application has the following obvious substantial features and advantages:
[0047] The present application provides a method and device for monitoring based on the Kubernetes container, which solves the problems of inconvenient monitoring deployment, large resource influence on monitoring deployment and the like in the prior art. The development and deployment of the eBPF program in the manner of the present application can quickly monitor the network based on the Kubernetes container without compiling and loading the code, thereby improving the development efficiency. Meanwhile, the user state program does not need to be developed to obtain the eBPF data, and the existing monitoring tools can automatically collect data through the definition of indexes, tracking and mapping relationships, thereby realizing the quick integration of the existing monitoring display system and improving the use experience. BRIEF DESCRIPTION OF DRAWINGS
[0048] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and the illustrative embodiments of the present application and their description serve the purpose of explaining the present application. The accompanying drawings should not be construed as an improper limitation of the present application. In the drawings:
[0049] Figure 1 is a flow chart of a method for monitoring based on the Kubernetes container according to a preferred embodiment of the present application.
[0050] Figure 2 is a structural schematic diagram of a device for monitoring based on the Kubernetes container according to a preferred embodiment of the present application. DETAILED DESCRIPTION
[0051] The present application provides a method and device for monitoring based on the Kubernetes container, in order to make the purpose, technical scheme and effects of the present application more clear and explicit, the present application is further described in detail below with reference to the drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.
[0052] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and in the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence, and it should be understood that the data thus used can be interchanged under appropriate circumstances. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0053] Example 1:
[0054] As Figure 1 shown, the method for monitoring based on the Kubernetes container according to the present embodiment specifically includes steps S1-S4.
[0055] Step S1, define eBPF data in CRD (Custom Resource Definitions) in the Kubernetes container environment. Specifically, the eBPF data includes eBPF programs, data mapping relationships and filtering rules; wherein the data mapping includes the mapping of kernel data and monitoring data, and the filtering rules include loading filtering rules and executing filtering rules.
[0056] The eBPF program mainly contains kernel function mounting points, and eBPF programs can be attached to these mounting points to trigger execution when a specific event occurs. The kernel data definition in the data mapping relationship, that is, the definition of the data format and data type of the Linux system kernel code written into the eBPF Map, is used for subsequent data display at the data storage end. The purpose of the filtering rule is to avoid unnecessary system overhead, and it is stipulated that only the loading filtering rule will load the eBPF program to the Linux system kernel when it matches the target node, and then for the execution filtering rule, only after the execution filtering rule matches successfully can the eBPF program continue to be executed, reducing the Linux system kernel execution overhead.
[0057] Step S2, monitoring the state of the CRD through the Operator in the Kubernetes container environment (the design concept of the Kubernetes Operator, which is a tool for extending the functions of Kubernetes, is to encode the professional knowledge and operation process of the operation and maintenance personnel into software to efficiently manage the application running on Kubernetes by using Custom Resource (CR) and Custom Controller (CC)) when the CRD changes, loading the pre-compiled eBPF program, and updating the data in the CRD.
[0058] Specifically, in step S2, the pre-compiled eBPF program specifically includes steps S201a-S201b.
[0059] Step S201a, save the latest eBPF program in the CRD as a.bpf.c format file and store it in the persistent volume of Kubernetes.
[0060] Step S201b, start the eBPF compiler for compilation, compile the program through libbpf, and form a pre-compiled eBPF program. Specifically, the compiler internally pre-installs dependencies such as Clang and LLVM required for compilation.
[0061] Specifically, in step S2, the loading process specifically includes steps S202a-S202b.
[0062] Step S202a, the CRD loads the filtering rules, judges whether there are node filtering rules, if there are, filters out the unnecessary nodes and loads or unloads the eBPF program, if there are not, directly loads or unloads the eBPF program.
[0063] Step S202b, the CRD self-filtering rules, judge whether there are other filtering rules, if there are, store them in the eBPF Map (wherein the eBPF Map is used to transmit data to the user mode). Specifically, generate a target file through the Kubernetes persistent volume, and load or unload the eBPF program by executing the target file.
[0064] Step S3, when a new process is executed each time, it is judged whether the new process is a container process. If the new process is a container process, the data information of the container process is obtained and stored in the eBPF Map. The data information of the container process includes cgroupid and pid, and is stored in the eBPF Map with pid as the key. During the running of the Kubernetes container, some information related to the running state of the container will be generated, including process identifier (PID), cgroup to which the container belongs, and other information. Cgroupid refers to the ID of the cgroup (Control Group). The cgroup is a mechanism provided by the Linux kernel, which is used to limit, manage and monitor the resources (such as CPU, memory, I / O, etc.) of a process group. Each container is usually assigned to a separate cgroup to isolate and manage its resource usage. Pid refers to the process identifier (Process ID), which is a unique identifier assigned to each process by the operating system. The pid of the process is used as the key (key), and the data (such as cgroupid) related to the pid is used as the value (value) and stored in the eBPF Map.
[0065] Step S4, the data in the eBPF Map is exported to one or at least two formats and stored in the corresponding data storage end and displayed.
[0066] Optionally, the data storage end includes one or at least two of OpenTelemetry, ELK and Prometheus. OpenTelemetry, ELK and Prometheus are three different tools. OpenTelemetry is an observability framework and toolkit designed to create and manage telemetry data such as links, metrics and logs. It focuses on the generation, collection, management and export of telemetry data, and is vendor and tool independent. ELK is a set of open source log analysis solutions composed of Elasticsearch, Logstash and Kibana. It is mainly used for log collection, storage, analysis and visualization. Prometheus is an open source system monitoring and alerting toolset. It is mainly used for monitoring and collecting, storing, querying and monitoring alarm of metric data.
[0067] Specifically, the eBPF program is developed by libbpf, which avoids the incompatibility between kernel versions.
[0068] The development process of the eBPF program specifically involves verifying whether the pid and cgroupid information in the eBPF Map match the current process or thread. If they match, the data is retrieved from the Linux system kernel and stored in the eBPF Map.
[0069] Example 2:
[0070] like Figure 2 As shown in the figure, the monitoring device based on Kubernetes containers described in this embodiment specifically includes a data definition module, a data compilation and update module, a new process data acquisition module, and a data storage module.
[0071] The data definition module is used to define eBPF data in the CRD of the Kubernetes container environment. The eBPF data includes eBPF programs, data mapping relationships, and filtering rules; the data mapping includes mappings between kernel data and monitoring data, and the filtering rules include loading filtering rules and executing filtering rules.
[0072] The eBPF program primarily contains kernel function mount points. The eBPF program can be attached to these mount points to trigger execution when specific events occur. The kernel data definition in the data mapping relationship defines the data format and data type of the Linux system kernel-mode code written into the eBPF Map, used for subsequent data display at the data storage end. The filtering rules aim to avoid unnecessary system overhead. They stipulate that the eBPF program is only loaded into the Linux system kernel when the loading of the filtering rule matches the target node. Furthermore, for executing the filtering rule, the eBPF program can only continue execution after a successful match, reducing the execution overhead of the Linux system kernel.
[0073] The data compilation and update module is used to monitor the status of the CRD through the Operator in the Kubernetes container environment. When the CRD changes, the pre-compiled eBPF program is loaded to update the data in the CRD.
[0074] Specifically, the data compilation and update module includes a pre-compilation submodule and a data update submodule; wherein, the pre-compilation submodule is used to pre-compile the eBPF program, and the data update submodule is used for data loading and updating.
[0075] Specifically, the pre-compiled submodule includes a first pre-compiled unit and a second pre-compiled unit.
[0076] The pre-compilation first unit is configured to save the latest eBPF program in the CRD as a.bpf.c format file and store the file in a persistent volume of Kubernetes.
[0077] The pre-compilation second unit is configured to start the eBPF compiler for compilation, compile the program in a libbpf mode, form a pre-compiled eBPF program, and support BPF-RE (Compile Once-Run Everywhere) to allow developers to create binary files that can run on different kernel versions, enhance the portability of the program, and save the compiled program file in a persistent volume in a Kubernetes container environment for subsequent loading of the program.
[0078] Specifically, the data updating submodule specifically includes a data loading first unit and a data loading second unit.
[0079] The data loading first unit is configured to load filtering rules through the CRD, determine whether there are node filtering rules, filter out unnecessary nodes and load or unload the eBPF program if there are node filtering rules, and directly load or unload the eBPF program if there are no node filtering rules.
[0080] The data loading second unit is configured to filter rules through the CRD, determine whether there are other filtering rules, and store the other filtering rules in the eBPF Map if there are other filtering rules. Specifically, a target file is generated through a Kubernetes persistent volume, and the eBPF program is loaded or unloaded by executing the target file.
[0081] The new process data acquisition module is configured to determine whether a new process is a container process each time the new process is executed, acquire data information of the container process if the new process is the container process, and store the data information in the eBPF Map. The data information of the container process includes a cgroupid and a pid, and the pid is stored in the eBPF Map as a key. The data storage module is configured to export data in the eBPF Map in one or at least two formats and store the data in a corresponding data storage end and display the data. Specifically, the data storage end includes one or at least two of OpenTelemetry, ELK, and Prometheus. The eBPF program is developed in a libbpf mode to avoid incompatibility between kernel versions.
[0082] The development process of the eBPF program specifically includes verifying whether pid and cgroupid information in the eBPF Map matches a current process or thread, acquiring data from a Linux system kernel if the pid and cgroupid information match the current process or thread, and storing the data in the eBPF Map.
[0083] The foregoing detailed description of the application has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the application to the precise form disclosed, and obviously many modifications and variations are possible in light of the above teaching. The described embodiments were chosen in order to best explain the principles of the application and its practical application to thereby enable others skilled in the art to best utilize the application. It is intended that the described embodiments be considered merely exemplary and that all modifications and variations are possible given the scope of the application.
Claims
1. A Kubernetes container-based monitoring method, characterized in that, Specifically comprising the following steps: Defining eBPF data in CRD in a Kubernetes container environment; Monitoring the state of the CRD through an Operator in the Kubernetes container environment, loading a pre-compiled eBPF program when the CRD changes, and updating data in the CRD; When a new process is executed each time, determining whether the new process is a container process, if the new process is a container process, obtaining data information of the container process and storing it in an eBPF Map; Exporting data in the eBPF Map into one or at least two formats and storing it in a corresponding data storage end and displaying it.
2. The Kubernetes container-based monitoring method of claim 1, wherein, The eBPF data includes eBPF programs, data mapping relationships, and filtering rules; wherein the data mapping includes the mapping of kernel data and monitoring data, and the filtering rules include loading filtering rules and executing filtering rules.
3. The Kubernetes container-based monitoring method of claim 2, wherein, The pre-compiled eBPF program specifically comprises the following steps: Save the latest eBPF program in the CRD as a.bpf.c format file and store it in the persistent volume of Kubernetes; Start the eBPF compiler for compilation, compile the program through libbpf, and form a pre-compiled eBPF program.
4. The Kubernetes container-based monitoring method of claim 2, wherein, The loading process specifically comprises the following steps: The CRD loads the filtering rules, judges whether there are node filtering rules, if there are, filters out unnecessary nodes and loads or unloads the eBPF program, if there are not, directly loads or unloads the eBPF program; The CRD self-filtering rules judge whether there are other filtering rules, if there are, store them in the eBPFMap.
5. The Kubernetes container-based monitoring method of claim 2, wherein, Generate a target file through the Kubernetes persistent volume, and load or unload the eBPF program by executing the target file.
6. The Kubernetes container-based monitoring method of claim 1, wherein, The data storage end includes one or at least two of OpenTelemetry, ELK, and Prometheus.
7. The Kubernetes container-based monitoring method of claim 2, wherein, The development process of the eBPF program specifically verifies whether the pid and cgroupid information in the eBPF Map matches the current process or thread, if it matches, obtains data from the Linux system kernel and stores it in the eBPF Map. 8.A Kubernetes container-based monitoring device, characterized in that, Specifically comprising the following modules: A data definition module for defining eBPF data in CRD in a Kubernetes container environment; A data compilation and update module for monitoring the state of the CRD through an Operator in the Kubernetes container environment, loading a pre-compiled eBPF program when the CRD changes, and updating data in the CRD; A new process data acquisition module for determining whether a new process is a container process when the new process is executed each time, if the new process is a container process, obtaining data information of the container process and storing it in an eBPF Map; A data storage module is configured to export data in the eBPF Map into one or at least two formats, store the data into a corresponding data storage end and display the data.
9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the Kubernetes container-based monitoring method in any one of claims 1-7.
10. An electronic device, comprising: A computer program product includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the Kubernetes container-based monitoring method in any one of claims 1-7.