A method for managing monitoring and alarm notification based on multi-tenant mode under cloud container

Abstract

The application aims to provide a method for managing monitoring alarm notification under a multi-tenant mode based on a cloud container, steps comprising: S1: extending information by using a label rewriting mechanism; S2: realizing alarm information classification according to tenants by using a grouping mechanism; S3: optimizing a notification channel structure; S4: adding a front service. Through the method for managing monitoring alarm notification under a multi-tenant mode based on a cloud container, Prometheus is combined with a multi-tenant scene of a cloud platform, a tenant dimension identification label is introduced in a link of index information collection and alarm rule management, the correlation between monitoring data is greatly improved, technical support is provided for subsequent realization of multi-tenant notification, a notification service is introduced and a notification configuration custom object is adapted, and alarm information notification process tenant information dynamic management is realized.

Description

Technical Field

[0001] This invention relates to the field of monitoring and alarm operation and maintenance, and in particular to a method for managing monitoring and alarm notifications based on a multi-tenant mode under cloud containers. Background Technology

[0002] With the rapid development of technologies such as big data, IoT, and cloud computing, emerging container technology, with its many advantages such as loose coupling, distributed architecture, elastic scaling, and resource sharing, is gradually being accepted by major enterprises. More and more enterprises are beginning to deploy applications in container clouds in their production environments. During this information transformation process, the platform generates a large amount of monitoring data. The relationships between these monitoring data are intricate and the structures are not uniform. How to persistently store and efficiently analyze this data has become a new challenge for major enterprises. To ensure that the platform provides stable and reliable services, the platform's monitoring and early warning functions have received unprecedented attention.

[0003] Prometheus is a complete open-source monitoring solution for the cloud era. It completely revolutionizes the traditional monitoring and alerting model. Addressing the characteristics of monitoring services—high write speeds, low query speeds, strong time-series data requirements, and the need for easy integration—Prometheus has rebuilt a centralized data model for rule calculation and verification, and unified analysis of alerts. With its simplicity, efficiency, ease of management, and ease of integration, Prometheus has quickly attracted widespread attention. In 2016, Prometheus joined the CNCF (Cloud Native Computing Foundation), becoming the second most popular cloud-native open-source project after Kubernetes. Joining the CNCF also signifies that Prometheus's monitoring solution has gained widespread industry recognition and has been deeply adapted and integrated with other cloud-native services.

[0004] Rometheus provides a monitoring solution that can be divided into the following modules: monitoring metric information collection, metric data system management, alarm rule system management, and user interaction management. User interaction management can be further divided into web terminal management and alarm notification management. Since the organizational structure of operation and maintenance teams varies from project to project, the alarm notification process is often a part that needs to be customized for each project. Summary of the Invention

[0005] The purpose of this invention is to address the problem that the two parts of the alarm notification process often have different structures, making it difficult to identify tenant information and manage alarm notification channel information. This invention proposes a method for managing monitoring alarm notifications based on a multi-tenant mode under cloud containers, which realizes a method for monitoring and early warning notifications. The method optimizes the three aspects of monitoring indicator information collection, alarm rule system management, and alarm notification management, and integrates monitoring services with business scenarios.

[0006] A method for managing monitoring and alarm notifications based on multi-tenant mode in cloud containers, comprising the following steps: S1: Expand information using tag rewriting mechanism; S2: Use a grouping mechanism to classify alarm information by tenant; S3: Notification channel structure optimization; S4: Added front-end services.

[0007] Furthermore, a method for managing monitoring and alarm notifications based on a multi-tenant mode under cloud containers, wherein step S1 includes the following sub-steps: S11: The Prometheus service collects data from the tenant's xporter service; S12: Expand the required metrics by using the tag rewriting mechanism during information collection.

[0008] Furthermore, a method for managing monitoring and alarm notifications based on a multi-tenant mode under cloud containers, wherein step S2 includes the following sub-steps: S21: When the Prometheus service generates an alarm message, the alarm message carries an extended tag; S22: The alertmanager service receives alert information and groups data based on the tenant to which the tag belongs; S23: Configure different notification channels for different tenant groups based on tags.

[0009] Furthermore, a method for managing monitoring and alarm notifications based on multi-tenant mode under cloud containers, wherein step S3 includes the following sub-steps: S31: Redefine the information administrator hierarchy; S32: Administrators are divided into three levels: platform administrators, tenant administrators, and project administrators. The platform administrator is responsible for managing the entire system's monitoring and alarm system, and the system administrator receives and manages all alarm information; The tenant administrator is defined in the system. There are two sets of tenant logic, and the data between tenants is isolated from each other. The tenant administrator can only receive and manage alarm information of projects under its own tenant. A tenant can manage multiple projects. The project administrator can only receive and manage alarm messages for their own project; S33: Alarm messages are generated using a bubbling strategy. They are first sent to the project administrator, then to the tenant administrator of the project, and finally to the platform administrator.

[0010] Furthermore, a method for managing monitoring and alarm notifications based on multi-tenant mode under cloud containers, wherein step S4 includes the following sub-steps: S41: Alert manager service alert information is first sent to the front-end service; S42: The front-end service searches for notification channels that meet the conditions according to the three-tier notification structure; S43: Implement message push by calling the actual message sending interface according to different channel configurations.

[0011] The beneficial effects of this invention are as follows: This solution provides a method for managing monitoring and alarm notifications based on a multi-tenant mode under cloud containers. It combines Prometheus with the multi-tenant scenario of the cloud platform, introduces tenant dimension identification tags in the indicator information collection and alarm rule management stages, which greatly improves the correlation between monitoring data and provides technical support for the subsequent implementation of multi-tenant notifications. It introduces a notification service and adapts to the notification configuration custom object to realize dynamic management of tenant information in the alarm information notification process. Attached Figure Description

[0012] Figure 1 This is a diagram of the Prometheus monitoring and alerting service architecture.

[0013] Figure 2 This is a flowchart of the present invention.

[0014] Figure 3 This is a flowchart of the plaintiff's police information.

[0015] Figure 4 This is a flowchart for implementing alarm information using a grouping mechanism.

[0016] Figure 5 This is a structural diagram of the original notification channels.

[0017] Figure 6 This is a structural diagram of the information administrator hierarchy.

[0018] Figure 7 This is a structural diagram of the notification service. Detailed Implementation

[0019] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described with reference to the accompanying drawings.

[0020] As attached Figure 1 The following is a diagram of the Prometheus monitoring and alerting service architecture: Exporter services: This refers to various metric generation services in the monitoring system. Common exporter services include node-exporter for server monitoring scenarios, redis-exporter for monitoring cache services, and kafka-exporter for monitoring message queue services. Exporter services are responsible for interacting with the monitored objects and generating metric information according to a unified specification. The Prometheus service is responsible for periodically accessing various exporter services to obtain monitoring information.

[0021] Prometheus Service: To obtain monitoring metrics, the Prometheus monitoring service typically initiates a pull operation via an HTTP request. The metric content is provided by various exporter monitoring services. The return values ​​of the exporter services must meet the Prometheus specification requirements. The format of the returned metric information is as follows: <metric name> { <label name>= <label value>, ...}<metric value> Format information can be divided into indicator name (<metric name> ), tag information ( <label name>= <labelvalue>), indicator value (<metric value> The system consists of three parts: the indicator name is used to identify the monitoring information, the tag information is used as a key / value attribute to describe the monitoring object, and the indicator value is used in float64 floating-point format to describe the indicator's trend.

[0022] The Prometheus service stores the collected data in a time-series database, and the Prometheus server can maintain PrometheusRule alert rules. These alert rules are uniformly scheduled, calculated, and managed by the Prometheus service. In addition to necessary descriptive information, alert content can also carry some tag information. <label name>= <label value>Text information is used for problem description, and tag information ( <label name>= <label value>This is mainly used for classifying alarm information. When the Prometheus server triggers an alarm rule, the alarm information will be uniformly sent to the Alertmanager server.

[0023] Alertmanager Service: As an alarm information management service, Alertmanager receives and manages alarm information from various clients, including Prometheus. Common management operations include deduplication, grouping, caching, silencing, suppression, and forwarding. Grouping, silencing, and suppression require administrators to customize configurations based on the actual system requirements. Due to the diversity of user scenarios, Alertmanager does not directly send alarm information to users; it typically sends it first to a third-party notification server, which then forwards the alarm information to the user's terminal.

[0024] Notification services: Various notification services are responsible for sending messages to users or other notification services when alarm information is generated. Alertmanager supports a limited number of notification channels by default, and they are not easy to manage. When the scenario changes, the configuration file needs to be updated manually and the service needs to be restarted.

[0025] The Prometheus service architecture focuses on the monitoring business itself and is not strongly coupled with the tenant's business data. When it is necessary to send alarm information, the Alertmanager service does not support many notification channels by default, and it does not support the function of dynamically querying the notification object. In addition, the original intention of the exporter pattern is to let the exporter service focus on the monitoring object's metrics itself. Monitoring data often becomes isolated and low-value data because it is not related to business data.

[0026] As attached Figure 2 As shown, to address the above pain points, a method for managing monitoring and alarm notifications based on multi-tenant mode in cloud containers is proposed. The steps include: S1: Expand information using tag rewriting mechanism; S2: Use a grouping mechanism to classify alarm information by tenant; S3: Notification channel structure optimization; S4: Added front-end services.

[0027] S1 includes the following sub-steps: S11: The Prometheus service collects data from the tenant's xporter service; S12: Expand the required metrics by using the tag rewriting mechanism during information collection.

[0028] In the implementation example shown in the diagram, the Prometheus service needs to collect data from three exporter services from tenants 1 and 2 respectively, using an indicator named "up". The "up" indicator is used to identify whether the monitored object can be monitored normally. The data is 1 when monitoring is normal and 0 when monitoring is abnormal. Usually, an alarm is triggered when the data is 0. Before optimization, the collected indicator is: up{job="Tenant1-MonitoringService1"} 1 up{job="Tenant1-MonitoringService2"} 1 up{job="Tenant 2 - Monitoring Service 1"} 1 The tag information contains no other business description information besides the job used to identify the exporter's source. Based solely on the metric information, it's impossible to determine where the service is deployed, what type of exporter it is, or which tenant it belongs to. After optimizing the collected metrics using a tag rewriting mechanism during information collection, the following results were obtained: up{job="Tenant1-Monitoring Service1",namespace="default",type="node",system_name="Tenant1"} 1 up{job="Tenant1-MonitoringService2",namespace="default",type="redis",system_name="Tenant1"} 1 up{job="Tenant 2 - Monitoring Service 1",namespace="default",type="node",system_name="Tenant 2"} 1 The namespace, type, and system_name tags have been expanded respectively. The namespace is used to identify the exporter's data isolation logic, the type is used to identify the exporter type, and the system_name is used to identify the tenant to which it belongs.

[0029] As attached Figure 3 As shown, step S2 includes the following sub-steps: S21: When the Prometheus service generates an alarm message, the alarm message carries an extended tag; S22: The alertmanager service receives alert information and groups data based on the tenant to which the tag belongs; S23: Configure different notification channels for different tenant groups based on tags.

[0030] The implementation example is the up indicator shown in the figure. When the value of the up indicator is 1, it means that the monitoring status is normal. When the value of the up indicator is 0, it means that the monitoring status is abnormal. When the status is abnormal, an alarm is triggered.

[0031] Before optimization, when Prometheus generated alarm messages, all alarm information was sent to the Alertmanager service. Because there was not enough basic tag data, the Alertmanager service could only put all alarm information into one group when it received alarm information.

[0032] As attached Figure 4 As shown, after optimization, because the namespace, type, and system_name tags are extended during information collection, alarm messages can also carry these three extended tags. After receiving the alarm message, AlertManager can use the system_name tag (the tenant to which it belongs) as the data basis for grouping, and finally configure different notification channels for different tenant groups.

[0033] As attached Figure 5 As shown, when the Alertmanager server does not have multi-tenant configuration, all alarm messages are temporarily stored in the same group. In the traditional mode, alarm channels are bound to group information, which indirectly leads to the singleness of alarm channel types. In addition, the group channels in the traditional mode are on the same level, while in the multi-tenant scenario, not only are different tenants isolated from each other, but there are also data isolation or hierarchical permission requirements within a single tenant.

[0034] As attached Figure 6 As shown, step S3 includes the following sub-steps: S31: Redefine the information administrator hierarchy; S32: Administrators are divided into three levels: platform administrators, tenant administrators, and project administrators. The platform administrator is responsible for managing the entire system's monitoring and alarm system, and the system administrator receives and manages all alarm information; The tenant administrator is defined in the system. There are two sets of tenant logic, and the data between tenants is isolated from each other. The tenant administrator can only receive and manage alarm information of projects under its own tenant. A tenant can manage multiple projects. The project administrator can only receive and manage alarm messages for their own project; S33: Alarm messages are generated using a bubbling strategy. They are first sent to the project administrator, then to the tenant administrator of the project, and finally to the platform administrator.

[0035] As attached Figure 7 As shown, step S4 includes the following sub-steps: S41: Alert manager service alert information is first sent to the front-end service; S42: The front-end service searches for notification channels that meet the conditions according to the three-tier notification structure; S43: Implement message push by calling the actual message sending interface according to different channel configurations.

[0036] This solution employs a multi-tenant management approach for monitoring and alarm notifications within a cloud container environment. It integrates Prometheus with the multi-tenant scenario of a cloud platform, introducing tenant-level identifiers in the metric information collection and alarm rule management stages. This significantly improves the correlation between monitoring data, providing technical support for subsequent multi-tenant notification implementation. The solution introduces a notification service and adapts to custom notification configuration objects, enabling dynamic management of tenant information throughout the alarm notification process. Metric tag information, as crucial data throughout the entire monitoring and alarm process, is further enhanced by expanding monitoring tag dimensions and associating tag information, custom notification configuration objects, and message notification services. This ultimately achieves dynamic management of monitoring and notification functions in a multi-tenant scenario. The implementation of multi-tenant alarm notification improves information transmission efficiency and reduces project operation and maintenance costs.

[0037] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.< / label> < / label> < / label> < / label> < / labelvalue> < / label> < / label> < / label>

Claims

1. A method for managing monitoring and alarm notifications based on multi-tenant mode in cloud containers, characterized by the following steps: include: S1: Expand information using tag rewriting mechanism; S2: Use a grouping mechanism to classify alarm information by tenant; S3: Notification channel structure optimization; S4: Added front-end services; S2 includes the following sub-steps: S21: When the Prometheus service generates an alarm message, the alarm message carries an extended tag; S22: The alertmanager service receives alert information and groups data based on the tenant to which the tag belongs; S23: Configure different notification channels for different tenant groups based on tags; S3 includes the following sub-steps: S31: Redefine the information administrator hierarchy; S32: Administrators are divided into three levels: platform administrators, tenant administrators, and project administrators. The platform administrator is responsible for managing the entire system's monitoring and alarm system, receiving and managing all alarm information; The tenant administrator is defined in the system. There are two sets of tenant logic, and the data between tenants is isolated from each other. The tenant administrator can only receive and manage alarm information of projects under its own tenant. A tenant can manage multiple projects. The project administrator can only receive and manage alarm messages for their own project; S33: Alarm messages are generated using a bubbling strategy. They are first sent to the project administrator, then to the tenant administrator of the project, and finally to the platform administrator. S4 includes the following sub-steps: S41: Alert manager service alert information is first sent to the front-end service; S42: The front-end service searches for notification channels that meet the conditions according to the three-tier notification structure; S43: Implement message push by calling the actual message sending interface according to different channel configurations; S1 includes the following sub-steps: S11: The Prometheus service collects data from the tenant's exporter service; S12: Expand the required indicators by utilizing the tag rewriting mechanism during information collection; The extended metrics in S12 include the namespace, type, and system_name tags, where namespace is used to identify the exporter data isolation logic, type is used to identify the exporter type, and system_name is used to identify the tenant to which it belongs.

Images