A resource monitoring method based on Kubernetes

Abstract

The present invention discloses a resource monitoring method based on Kubernetes, which belongs to the field of cloud platform management technology. Kubernetes collects various indicators of node and pod through heapster, stores the acquired data in InfluxDB, and displays the monitoring data in InfluxDB through Grafana. The present invention can well complete the production of cloud monitoring indicators, and provides the monitoring alarm platform with the ability to issue alarms according to different states. After receiving the alarm, the operation and maintenance personnel can quickly check according to the alarm content to avoid unnecessary losses.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud platform management, and in particular to a resource monitoring method based on Kubernetes. Background Art

[0002] Kubernetes (k8s) is an open source container cluster management system from Google. Based on Docker technology, it provides a series of complete functions such as deployment and operation, resource scheduling, service discovery, and dynamic scaling for containerized applications, improving the convenience of large-scale container cluster management.

[0003] Kubernetes is a new leading distributed architecture solution based on container technology. All applications exist in the form of resources in Kubernetes. These resources may become unavailable due to network, environmental and other factors. Therefore, it is necessary to be able to monitor these resources in real time and issue alarms when the status of these resources is abnormal, so that operation and maintenance personnel can conduct investigations after receiving the alarms to avoid unnecessary losses. Summary of the invention

[0004] The technical task of the present invention is to address the above shortcomings and provide a Kubernetes-based resource monitoring method that can well complete the production of cloud monitoring indicators and provide the monitoring alarm platform with the ability to issue alarms according to different states.

[0005] The technical solution adopted by the present invention to solve its technical problem is:

[0006] A resource monitoring method based on Kubernetes. Kubernetes collects various indicators of nodes and pods through heapster, stores the acquired data in InfluxDB, and displays the monitoring data in InfluxDB through Grafana.

[0007] This method can provide a resource monitoring solution for Kubernetes-based container services. The cloud monitoring data generated by this method can monitor the resource status of Kubernetes in real time and display it through Grafana.

[0008] Preferably, the cluster management program ICE regularly pushes cluster metadata to the cloud monitoring program agent; the cloud monitoring program agent regularly obtains monitoring data corresponding to the cluster from InfluxDB; and pushes the produced cloud monitoring indicators to Grafana through Kafka.

[0009] Preferably, multiple go routines are used to acquire and push data for a large amount of data. InfluxDB is an open source time series database developed in the go language, which is particularly suitable for processing and analyzing time series related data such as resource monitoring data.

[0010] Specifically, for cluster metadata, ICE provides an API interface for the agent program to call regularly, which can obtain all managed cluster metadata;

[0011] Furthermore, a cache of cluster status is set. Each time new cluster metadata is received, the original cache is cleared and the new metadata is added to the cache. A scheduled task is started, and the agent updates the cluster status in the cache at fixed intervals.

[0012] Push the successfully created cloud monitoring indicators to Grafana.

[0013] Specifically, for the resource metadata in the cluster, a scheduled task is started, and the agent traverses and obtains the cluster list in the cache at a fixed interval;

[0014] ICE collects monitoring data of each cluster Kubernetes-heapster and pushes it to InfluxDB;

[0015] The agent obtains cluster information from ICE and cluster-related monitoring data from InfluxDB, and pushes the assembled data to Grafana for display.

[0016] Preferably, the pushing to Grafana,

[0017] Initialize the Kafka Producer instance and establish a connection with the Kafka consumer;

[0018] Put the cloud monitoring data indicators into the producer value and push them to Grafana.

[0019] Preferably, the cloud monitoring agent is made into a docker image, deployed in the form of statefulset, and the corresponding service is configured as headless;

[0020] When the ICE manager distributes data to the agent at a scheduled time, it can distribute a certain part of the data to a specified pod based on the headless configuration of different services. This can avoid the problem of too much data to be handled by a single pod.

[0021] The present invention also claims protection for a Kubernetes-based resource monitoring device, comprising: at least one memory and at least one processor;

[0022] The at least one memory is used to store a machine-readable program;

[0023] The at least one processor is configured to call the machine-readable program to execute the method according to any one of claims 1 to 8.

[0024] The present invention also claims protection for a computer-readable medium having computer instructions stored thereon, wherein when the computer instructions are executed by a processor, the processor is caused to execute any one of the methods of claims 1 to 8.

[0025] Compared with the prior art, the resource monitoring method based on Kubernetes of the present invention has the following beneficial effects:

[0026] By using this method, the cloud monitoring indicators can be well produced, and the monitoring alarm platform is provided with the ability to issue alarms according to different states. After receiving the alarm, the operation and maintenance personnel can quickly check according to the alarm content to avoid unnecessary losses. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a design architecture diagram of a Kubernetes-based resource monitoring method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0029] The embodiment of the present invention provides a resource monitoring method based on Kubernetes. Kubernetes collects various indicators of nodes and pods through heapster, stores the acquired data in InfluxDB, and displays the monitoring data in InfluxDB through Grafana.

[0030] This method implements CPU and memory resource monitoring based on Kubernetes, and can provide a resource monitoring solution for Kubernetes-based container services. The cloud monitoring data generated by this method can monitor the resource status of Kubernetes in real time and display it through Grafana.

[0031] Heapster is a k8s monitoring and performance analysis tool that naturally supports k8s. After we build the cluster, we use heapster to collect the CPU and memory resource usage of nodes and pods, and use third-party tools influxdb and grafana to monitor cluster resources.

[0032] InfluxDB is an open source time series database developed in GO language, which is particularly suitable for processing and analyzing time series data such as resource monitoring data. InfluxDB comes with various special functions such as standard deviation, random sampling data, statistical data change ratio, etc., which makes data statistics and real-time analysis very convenient.

[0033] Grafana is an open source program for visualizing large amounts of measurement data. It provides a powerful and elegant way to create, share, and browse data.

[0034] refer to Figure 1 The design architecture diagram of this method is shown in FIG.

[0035] The cluster management program ICE regularly pushes cluster metadata to the cloud monitoring program agent;

[0036] The cloud monitoring agent periodically obtains the monitoring data corresponding to the cluster from InfluxDB;

[0037] Push the generated cloud monitoring indicators to Grafana through Kafka.

[0038] Among them, for cluster metadata,

[0039] (1) ICE provides an API interface for the agent program to call regularly, which can obtain the metadata of all managed clusters;

[0040] (2) Set up a cluster status cache. Every time new cluster metadata is received, the original cache is cleared and the new metadata is added to the cache.

[0041] (3) Start a scheduled task, and the agent updates the cluster status in the cache every 3 minutes;

[0042] (4) Push the successfully created cloud monitoring indicators to Grafana.

[0043] For resource metadata within the cluster,

[0044] (1) Start a scheduled task, and the agent will traverse and obtain the cluster list in the cache every 3 minutes;

[0045] (2) ICE collects monitoring data of each cluster Kubernetes-heapster and pushes it to InfluxDB;

[0046] (3) The agent obtains cluster information from ICE and cluster-related monitoring data from InfluxDB, assembles the data and pushes it to Grafana for display.

[0047] The push to Grafana,

[0048] (1) Initialize the Kafka Producer instance and establish a connection with the Kafka consumer;

[0049] (2) Put the cloud monitoring data indicators into the producer value and push them to Grafana.

[0050] For the high availability of cloud monitoring indicator production program,

[0051] (1) Make the cloud monitoring program agent into a Docker image, deploy it in the form of a statefulset, and configure the corresponding service as headless;

[0052] (2) When the ICE manager distributes data to the agent at a scheduled time, it can distribute a certain part of the data to a specified pod based on the headless configuration of different services. This can avoid the problem of too much data to be handled by a single pod.

[0053] This method uses scheduled tasks to push and pull cloud monitoring index data, uses cache to save cluster metadata, and exposes an API interface to receive cluster metadata; and a large amount of data is acquired and pushed using multiple go coroutines. This method uses kafka message queues to push a large number of cloud monitoring indicators and achieve high-availability deployment of cloud monitoring programs.

[0054] Through this method, the production of cloud monitoring indicators can be well completed, and the monitoring alarm platform is provided with the ability to issue alarms according to different states.

[0055] The embodiment of the present invention further provides a Kubernetes-based resource monitoring device, comprising: at least one memory and at least one processor;

[0056] The at least one memory is used to store a machine-readable program;

[0057] The at least one processor is used to call the machine-readable program to execute a resource monitoring method based on Kubernetes described in the above embodiment of the present invention.

[0058] An embodiment of the present invention further provides a computer-readable medium having computer instructions stored thereon, and when the computer instructions are executed by a processor, the processor executes a resource monitoring method based on Kubernetes described in the above embodiment of the present invention. Specifically, a system or device equipped with a storage medium may be provided, on which a software program code implementing the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or device is enabled to read and execute the program code stored in the storage medium.

[0059] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0060] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0061] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0062] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0063] The present invention is shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the code review methods in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the protection scope of the present invention.

Claims

1. A resource monitoring method based on Kubernetes, characterized in that: Kubernetes collects various indicators of nodes and pods through heapster, stores the acquired data in InfluxDB, and displays the monitoring data in InfluxDB through Grafana; The cluster management program ICE regularly pushes cluster metadata to the cloud monitoring program agent; the cloud monitoring program agent regularly obtains the monitoring data corresponding to the cluster from InfluxDB; and pushes the generated cloud monitoring indicators to Grafana through Kafka; The push to Grafana initializes the Producer instance of Kafka and establishes a connection with the consumer of Kafka; the cloud monitoring data indicators are put into the value of the producer and pushed to Grafana; Make the cloud monitoring agent into a Docker image, deploy it in the form of a statefulset, and configure the corresponding service as headless; When the ICE manager distributes data to the agent at a fixed time, it distributes a certain part of the data to the specified pod according to the headless configuration of different services.

2. According to a Kubernetes-based resource monitoring method according to claim 1, it is characterized in that: For large amounts of data, multiple go coroutines are used to acquire and push data.

3. According to a Kubernetes-based resource monitoring method according to claim 1, it is characterized in that: For cluster metadata, ICE provides an API interface for the agent program to call regularly to obtain all managed cluster metadata.

4. A Kubernetes-based resource monitoring method according to claim 3, characterized in that: Set the cache of cluster status. Every time new cluster metadata is received, the original cache is cleared and the new metadata is added to the cache. Start the scheduled task, and the agent updates the cluster status in the cache at a fixed interval; Push the successfully created cloud monitoring indicators to Grafana.

5. A Kubernetes-based resource monitoring method according to claim 1 or 4, characterized in that: For the resource metadata in the cluster, a scheduled task is started, and the agent traverses and obtains the cluster list in the cache at a fixed interval; ICE collects monitoring data of each cluster Kubernetes-heapster and pushes it to InfluxDB; The agent obtains cluster information from ICE and cluster-related monitoring data from InfluxDB, and pushes the assembled data to Grafana for display.

6. A resource monitoring device based on Kubernetes, characterized in that: include: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is configured to call the machine-readable program to execute the method according to any one of claims 1 to 5.

7. A computer-readable medium, characterized in that The computer readable medium stores computer instructions, and when the computer instructions are executed by a processor, the processor executes the method according to any one of claims 1 to 5.

Images