Extensible large-scale cloud-native database monitoring and alarming system

Abstract

The application discloses an expandable large-scale cloud native database monitoring and alarming system, which comprises the following components: users, a management and control platform and a K8s cluster; the users interact with the management and control platform to set alarm configurations; the management and control platform controls the K8s cluster; and the K8s cluster is used for monitoring cloud native database instances. The application supports the horizontal expansion of the cloud native environment of multiple K8s through the creation of a K8s connection by using the authentication file kubeconfig of K8s, and the elastic deployment of prometheus+alertmanager is adaptively detected, so that the effect of cross-K8s cluster monitoring and alarming in large-scale cloud native databases is achieved. The application solves the problem that it is difficult to monitor and alarm due to the cross-cluster container network isolation and complex architecture in large-scale cloud native databases.

Description

Technical Field

[0001] This invention relates to the field of cloud-native database monitoring and alerting, and in particular to a scalable, large-scale cloud-native database monitoring and alerting system. Background Technology

[0002] Cloud-native databases are a type of cloud-native data infrastructure, a database service that fully leverages the advantages of public clouds. They possess extreme elasticity and scalability, serverless characteristics, a globally distributed architecture with high availability and low cost, and can integrate and interact with other cloud services. Alarms refer to the monitoring unit issuing alarm signals when a system failure occurs, depending on the fault type. All faults are accompanied by audible and visual alarms and text prompts. When an alarm occurs, the red alarm light on the monitoring unit illuminates, the buzzer sounds an alarm, and an alarm message is sent to the remote monitoring center.

[0003] Under the national strategies of Digital China, Technological Innovation, and Self-Reliance, and with the in-depth advancement of the Group's "Cloud Transformation and Digital Transformation" strategy and the implementation of the removal of IOE from underlying technologies, replacing foreign commercial databases with technologies such as open-source, independently developed databases has become a trend. The requirements for the efficient and stable operation of cloud-native databases are also increasing. In particular, timely alerts when cloud-native databases are abnormal are especially important. This helps operations and maintenance personnel quickly locate problems and resolve anomalies, effectively avoiding major failures. To this end, we propose a scalable, large-scale cloud-native database monitoring and alerting system. Summary of the Invention

[0004] The purpose of this invention is to provide a scalable, large-scale cloud-native database monitoring and alerting system. It uses the Kubernetes authentication file kubeconfig to create Kubernetes connections, supporting horizontal scaling across multiple Kubernetes cloud-native environments. It also adaptively detects and performs elastic deployment of Prometheus + AlertManager, achieving cross-Kubernetes cluster monitoring and alerting in large-scale cloud-native databases. This solves the problem of difficult monitoring and alerting caused by cross-cluster container network isolation and complex architecture in large-scale cloud-native databases.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a scalable, large-scale cloud-native database monitoring and alerting system, comprising the following components:

[0006] Users, management platform, and Kubernetes cluster;

[0007] Users interact with the management platform to set alarm configurations;

[0008] The management platform manages and controls the Kubernetes cluster;

[0009] A Kubernetes cluster is used to monitor cloud-native database instances.

[0010] Preferably, the specific operating method is as follows:

[0011] S1. Based on the native fabric8 framework and corresponding authentication files of Kubernetes, create Kubernetes connections to operate on k8s resources to support cross-cluster management of cloud-native databases and monitoring and alarms;

[0012] S2. When deploying an instance on the management platform, determine whether the instance's Kubernetes cluster is a new Kubernetes cluster. If it is a new cluster, save the corresponding authentication file of the new cluster to the backend database, and deploy monitoring and alarm tools on the new Kubernetes cluster and initialize them so that the new cluster can support monitoring cloud-native databases.

[0013] S3. When initializing monitoring and alarm tools, the management platform copies the default configuration and existing user monitoring and alarm configurations to the new cluster, so users do not need to reconfigure them in the new cluster. At the same time, the management platform will configure the webhook interface address opened by the management platform in the new alarm tool to receive alarm information from the new cluster instance.

[0014] S4. Cloud-native databases deployed through the management platform use the open-source mysqld_exporter by default to monitor the metrics of the cloud-native database; and use the open-source node_exporter to monitor the database containers and pods, obtaining various monitoring information of the instance containers, with complete monitoring data.

[0015] S5. Through the management and control platform, the monitoring tools can be flexibly configured for monitoring, and the collected monitoring information can be managed in a unified manner using the monitoring tools. The content that meets the alarm conditions can be sent to the alarm tools according to the alarm rules.

[0016] S6. The management platform filters all alarm information sent by the alarm tool according to the tag content in the alarm information, generates user-defined alarm information according to the alarm template configured by the user, and pushes the alarm message.

[0017] S7. Users configure monitoring through the front end of the management platform. The back end management platform will parse the monitoring configuration selected by the user, generate PromQL language, and synchronize it to the monitoring tool.

[0018] Preferably, the corresponding authentication file is the kubeconfig authentication file.

[0019] Preferably, the acquisition of various monitoring information of the instance container specifically refers to the CPU, I / O, memory, and disk monitoring information of the instance container.

[0020] Preferably, the management and control platform provides flexible monitoring configuration for the monitoring tools, and the monitoring configuration specifically includes alarm rules, alarm thresholds, and alarm frequency configuration.

[0021] Preferably, the user configures the monitoring through the front end of the management platform. The monitoring configuration specifically includes the monitoring indicators, thresholds, instances, and alarm contacts that need to be alerted.

[0022] Preferably, the K8s cluster is one or more.

[0023] Preferably, the monitoring tool is the open-source Prometheus monitoring tool.

[0024] Preferably, the alarm tool is the open-source AlertManager alarm tool.

[0025] Preferably, the management platform supports cross-Kubernetes cluster monitoring and alarming of cloud-native databases. At the same time, the management platform front-end configures the alarm rules, alarm contacts, and alarm templates of cloud-native database instances. The management platform stores the corresponding relevant data in the back-end and synchronizes the alarm rules to Prometheus for immediate effect.

[0026] The technical effects and advantages of this invention are as follows:

[0027] (1) By comparing with existing cloud-native database monitoring and alarm technologies, this method uses the k8s authentication file kubeconfig to connect and operate k8s resources to support cloud-native environments with multiple k8s clusters. At the same time, it supports the adaptive deployment of monitoring and alarm systems prometheus and alertmanager in new clusters without manual deployment. This ensures that in cloud-native environments, the underlying components do not need to be manually deployed to monitor and alarm systems after the platform horizontally scales the k8s cluster, supporting the extreme scaling of large-scale cloud-native environments.

[0028] (2) By comparing with existing cloud-native database monitoring and alarm technologies, the alarm rule configuration in this method allows users to write their own PromQL alarm rules or set thresholds, etc. The management and control platform parses and generates PromQL language and synchronizes it to Prometheus, which is convenient and efficient.

[0029] (3) The management platform in this invention allows users to view and modify alarm configurations on the interface, avoiding direct operation of Prometheus by users; at the same time, the management platform isolates users to prevent alarm information of different users from affecting each other, and users cannot see or operate alarms and alarm data of other users. Attached Figure Description

[0030] Figure 1 This is a diagram of the alarm structure of the present invention.

[0031] Figure 2 This is a basic schematic diagram of the present invention.

[0032] Figure 3 This is a logic diagram for order processing in this invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] This invention provides, for example Figure 1-3 The scalable, large-scale cloud-native database monitoring and alerting system shown includes the following components:

[0035] Users, management platform, and Kubernetes cluster;

[0036] Users interact with the management platform to set alarm configurations;

[0037] The management platform manages and controls the Kubernetes cluster;

[0038] A Kubernetes cluster is used to monitor cloud-native database instances.

[0039] The specific operating method is as follows:

[0040] S1. Based on the native fabric8 framework and corresponding authentication files of Kubernetes, create Kubernetes connections to operate on k8s resources to support cross-cluster management of cloud-native databases and monitoring and alarms;

[0041] S2. When deploying an instance on the management platform, determine whether the instance's Kubernetes cluster is a new Kubernetes cluster. If it is a new cluster, save the corresponding authentication file of the new cluster to the backend database, and deploy monitoring and alarm tools on the new Kubernetes cluster and initialize them so that the new cluster can support monitoring cloud-native databases.

[0042] S3. When initializing monitoring and alarm tools, the management platform copies the default configuration and existing user monitoring and alarm configurations to the new cluster, so users do not need to reconfigure them in the new cluster. At the same time, the management platform will configure the webhook interface address opened by the management platform in the new alarm tool to receive alarm information from the new cluster instance.

[0043] S4. Cloud-native databases deployed through the management platform use the open-source mysqld_exporter by default to monitor the metrics of the cloud-native database; and use the open-source node_exporter to monitor the database containers and pods, obtaining various monitoring information of the instance containers, with complete monitoring data.

[0044] S5. Through the management and control platform, the monitoring tools can be flexibly configured for monitoring, and the collected monitoring information can be managed in a unified manner using the monitoring tools. The content that meets the alarm conditions can be sent to the alarm tools according to the alarm rules.

[0045] S6. The management platform filters all alarm information sent by the alarm tool according to the tag content in the alarm information, generates user-defined alarm information according to the alarm template configured by the user, and pushes the alarm message.

[0046] S7. Users configure monitoring through the front end of the management platform. The back end management platform will parse the monitoring configuration selected by the user, generate PromQL language, and synchronize it to the monitoring tool.

[0047] The corresponding authentication file is the kubeconfig authentication file.

[0048] The specific monitoring information obtained for the instance container includes CPU, I / O, memory, and disk monitoring information.

[0049] The management platform allows for flexible monitoring configuration of monitoring tools, including alarm rules, alarm thresholds, and alarm frequency settings.

[0050] Users configure monitoring through the front end of the management platform. The specific monitoring configuration includes the monitoring indicators, thresholds, instances, and alarm contact persons that need to be alerted.

[0051] A Kubernetes cluster can consist of one or more clusters.

[0052] The monitoring tool used is the open-source Prometheus.

[0053] The alerting tool is the open-source AlertManager.

[0054] The management platform supports cross-Kubernetes cluster monitoring and alarming of cloud-native databases. At the same time, alarm rules, alarm contacts, and alarm templates for cloud-native database instances can be configured through the management platform front end. The management platform will store the corresponding relevant data in the back end and synchronize the alarm rules to Prometheus for immediate effect.

[0055] Specific descriptions of the different attached figures:

[0056] like Figure 1The basic flowchart illustrates how this monitoring and alarm system utilizes a management platform, AlertManager, and Prometheus. Users only need to interact with the management platform; the internal workings of the monitoring and alarm system remain seamless. Kubernetes authentication files, monitoring and alarm configurations, and alarm information are all centrally managed by the management platform.

[0057] 2. For example Figure 2 Basic schematic diagram and Figure 3 The order processing logic diagram provides a detailed explanation of the implementation of this method:

[0058] 1) Users place orders to deploy instances through the cloud platform. The management platform determines whether the Kubernetes cluster to which the instance belongs is a new cluster. If so, it deploys and initializes Prometheus and AlertManager in the new Kubernetes cluster and synchronizes the alarm rules and configurations. Then, it deploys the new instance, saves the kubeconfig file of the new cluster, and manages the new Kubernetes cluster.

[0059] 2) The management platform is responsible for receiving user configurations, storing the configured alarm rules, alarm contacts, alarm templates, etc., in the backend, and updating the alarm rules to Prometheus in real time. The management platform is also responsible for filtering and analyzing alarm information, and pushing active alarms to the corresponding alarm contacts according to the user-configured alarm contacts.

[0060] 3) Prometheus collects and aggregates metrics as time-series data. All metrics have a name, and each name corresponds to multiple labels. Prometheus aggregates the cloud-native database metric data collected by mysqld_exporter and node_exporter and generates time-series monitoring data. Then, it pushes this labeled time-series monitoring data to AlertManager.

[0061] 4) After receiving the time-series monitoring data from Prometheus, AlertManager will forward the alarm information of the corresponding tag to the management platform according to the configuration information.

[0062] 5) When the management platform receives the alarm data sent by AlertManager, it parses the alarm data, extracts the user information and instance information from the data, retrieves the alarm configuration corresponding to the user from the backend for comparison, and if the alarm is active, it fills the alarm information into the alarm template configured by the user and pushes the final alarm information to the alarm contact person configured by the user.

[0063] One embodiment of the present invention provides a cloud-native database monitoring and alerting system based on AlertManager and Prometheus. The system utilizes the open-source Prometheus monitoring tool, the open-source AlertManager alerting tool, and a self-developed management platform to form a cloud-native database monitoring and alerting system.

[0064] Another embodiment of the present invention is that in the “2+4+31+X” cloud-network converged resource layout formed in the promotion of cloud IT in cloud companies, the TeleDB cloud-native database used all adopt the cloud-native database monitoring and alarm system of the present invention, which can be flexibly configured, accurately monitored, and accurately alarmed to achieve precise and efficient operation and maintenance and effectively avoid major production failures.

[0065] Finally, it should be noted that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An extensible large-scale cloud-native database monitoring and alerting system, characterized in that, It includes the following components: Users, management platform, and Kubernetes cluster; Users interact with the management platform to set alarm configurations; The management platform manages and controls the Kubernetes cluster; A Kubernetes cluster is used to monitor cloud-native database instances. S1. Based on the native fabric8 framework and corresponding authentication files of Kubernetes, create Kubernetes connections to operate on k8s resources to support cross-cluster management of cloud-native databases and monitoring and alarms; S2. When deploying an instance on the management platform, determine whether the instance's Kubernetes cluster is a new Kubernetes cluster. If it is a new cluster, save the corresponding authentication file of the new cluster to the backend database, and deploy monitoring and alarm tools on the new Kubernetes cluster and initialize them so that the new cluster can support monitoring cloud-native databases. S3. When initializing monitoring and alarm tools, the management platform copies the default configuration and existing user monitoring and alarm configurations to the new cluster, so users do not need to reconfigure them in the new cluster. At the same time, the management platform will configure the webhook interface address opened by the management platform in the new alarm tool to receive alarm information from the new cluster instance. S4. Cloud-native databases deployed through the management platform use the open-source mysqld_exporter by default to monitor the metrics of the cloud-native database; and use the open-source node_exporter to monitor the database containers and pods, obtaining various monitoring information of the instance containers, with complete monitoring data. S5. Through the management and control platform, the monitoring tools can be flexibly configured for monitoring, and the collected monitoring information can be managed in a unified manner using the monitoring tools. The content that meets the alarm conditions can be sent to the alarm tools according to the alarm rules. S6. The management platform filters all alarm information sent by the alarm tool according to the tag content in the alarm information, generates user-defined alarm information according to the alarm template configured by the user, and pushes the alarm message. S7. Users configure monitoring through the front end of the management platform. The back end management platform will parse the monitoring configuration selected by the user, generate PromQL language, and synchronize it to the monitoring tool. The corresponding authentication file is specifically the kubeconfig authentication file; The specific monitoring information obtained for the instance container includes the CPU, I / O, memory, and disk monitoring information of the instance container. The management and control platform allows for flexible monitoring configuration of the monitoring tools, including alarm rules, alarm thresholds, and alarm frequency configuration. The user configures the monitoring through the front end of the management platform. The specific monitoring configuration includes the monitoring indicators, thresholds, instances, and alarm contacts that need to be alerted. The K8s cluster can be one or more; The monitoring tool mentioned is the open-source Prometheus monitoring tool; The alarm tool mentioned is the open-source AlertManager alarm tool.

2. The scalable large-scale cloud-native database monitoring and alarm system according to claim 1, characterized in that, The management platform supports cross-Kubernetes cluster monitoring and alarming of cloud-native databases. At the same time, alarm rules, alarm contacts, and alarm templates for cloud-native database instances can be configured through the management platform front end. The management platform will store the corresponding relevant data in the back end and synchronize the alarm rules to Prometheus for immediate effect.

Images