Monitoring alarm method and monitoring alarm system of big data platform

By configuring alarm rules and processing alarm information in the monitoring and alarm system of the big data platform, the problem of inability to meet diversified monitoring needs and alarm processing in the existing technology is solved, and effective monitoring and maintenance of the big data platform is achieved.

CN118939516APending Publication Date: 2024-11-12ZHEJIANG SUPCON INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411085080.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-08
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing big data platform monitoring solutions cannot meet users' diverse monitoring needs, and cannot effectively process alarm information when monitoring abnormalities.

Method used

Provide a monitoring and alarm method for a big data platform. Through monitoring services in the monitoring alarm system, the target monitoring object is monitored, pre-configured alarm rules are obtained, and whether the monitoring information meets the alarm triggering conditions, and the alarm information is reported to the alarm platform. Perform processing.

Benefits of technology

Users can configure alarm rules on the front-end interface of the alarm platform to meet diverse monitoring needs, and effectively process alarm information when the alarm conditions are met, so as to realize the operation and maintenance of the big data platform.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118939516A_ABST
    Figure CN118939516A_ABST
Patent Text Reader

Abstract

The invention provides a monitoring alarm method and a monitoring alarm system for a big data platform, and relates to the technical field of operation and maintenance monitoring. The method comprises the following steps: monitoring a target monitoring object corresponding to a monitoring service in a big data platform by adopting the monitoring service in a monitoring platform in a monitoring alarm system to obtain monitoring information of the target monitoring object; acquiring an alarm rule of the target monitoring object by adopting the monitoring service, wherein the alarm rule is an alarm rule configured in advance through a front-end interface of an alarm platform in the monitoring alarm system; determining whether the monitoring information meets an alarm triggering condition of an alarm rule or not by adopting a monitoring service; and if the alarm triggering condition of the alarm rule is met, the monitoring service reports alarm information of the target monitoring object to an alarm platform through an alarm management service in the monitoring platform. By adopting the method provided by the invention, the monitoring requirement of the user is met through the configured alarm rule, and the alarm information is reported, so that the alarm platform can effectively process the alarm information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of operation and maintenance monitoring technology, and more specifically, to a monitoring and alarm method and system for a big data platform. Background Technology

[0002] A big data platform is an infrastructure primarily designed for scenarios involving the storage, computation, and real-time computation of massive amounts of data. It provides users with functions such as data viewing and data analysis.

[0003] The existing methods can only monitor fixed items on the big data platform. Users cannot configure monitoring rules for the big data platform, which cannot meet the diverse monitoring needs of users. Furthermore, when abnormal problems are detected, users can only view alarm information and cannot effectively handle the alarm information. Summary of the Invention

[0004] This application addresses the shortcomings of the prior art by providing a monitoring and alarm method and system for a big data platform, in order to solve the problems existing in the prior art.

[0005] The technical solution adopted in the embodiments of this application is as follows:

[0006] In a first aspect, embodiments of this application provide a monitoring and alarm method for a big data platform, applied to the monitoring and alarm system of the big data platform, the method comprising:

[0007] The monitoring service in the monitoring platform of the monitoring and alarm system is used to monitor the target monitoring object corresponding to the monitoring service in the big data platform, and the monitoring information of the target monitoring object is obtained.

[0008] The monitoring service is used to obtain the alarm rules of the target monitoring object. The alarm rules are alarm rules that are configured in advance through the front-end interface of the alarm platform in the monitoring alarm system.

[0009] Using the monitoring service, determine whether the monitoring information meets the alarm triggering conditions of the alarm rule;

[0010] If the alarm triggering conditions of the alarm rule are met, the monitoring service will report the alarm information of the target monitored object to the alarm platform through the alarm management service in the monitoring platform.

[0011] Secondly, this application also provides a monitoring and alarm system, including: a processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor. When the monitoring and alarm system is running, the processor communicates with the storage medium through the bus, and the processor executes the program instructions to implement the monitoring and alarm method of the big data platform described in any of the above embodiments.

[0012] Thirdly, this application also provides a monitoring and alarm device for a big data platform, applied to the monitoring and alarm system of the big data platform, the device comprising:

[0013] The monitoring module is used to monitor the target monitoring object corresponding to the monitoring service in the big data platform using the monitoring service in the monitoring and alarm system, and to obtain the monitoring information of the target monitoring object.

[0014] The acquisition module is used to acquire the alarm rules of the target monitoring object using the monitoring service. The alarm rules are alarm rules configured in advance through the front-end interface of the alarm platform in the monitoring alarm system.

[0015] The determination module is used to determine, using the monitoring service, whether the monitoring information meets the alarm triggering conditions of the alarm rule;

[0016] The alarm module is used to report the alarm information of the target monitored object to the alarm platform through the alarm management service in the monitoring platform if the alarm triggering conditions of the alarm rule are met.

[0017] Fourthly, this application also provides a storage medium storing program instructions, which are executed by a processor to implement the monitoring and alarm method of the big data platform described in any of the above embodiments.

[0018] The beneficial effects of this application are as follows: This application provides a monitoring and alarm method for a big data platform, comprising: using a monitoring service in a monitoring platform within a monitoring and alarm system to monitor a target monitoring object corresponding to the monitoring service in the big data platform, thereby obtaining monitoring information of the target monitoring object; using the monitoring service to obtain alarm rules for the target monitoring object, wherein the alarm rules are alarm rules pre-configured through the front-end interface of the alarm platform in the monitoring and alarm system; using the monitoring service to determine whether the monitoring information meets the alarm triggering conditions of the alarm rules; if the alarm triggering conditions of the alarm rules are met, the monitoring service reports the alarm information of the target monitoring object to the alarm platform through the alarm management service in the monitoring platform.

[0019] Using the method of this application, users can configure alarm rules on the front-end interface of the alarm platform. The alarm rules determine whether the big data platform meets the alarm triggering conditions, thus meeting the diverse monitoring needs of users. Furthermore, when the alarm triggering conditions of the alarm rules are met, the alarm information can be reported to the alarm platform, which can effectively process the alarm information and realize the operation and maintenance of the big data platform. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 One of the flowcharts of a monitoring and alarm method for a big data platform provided in an embodiment of this application;

[0022] Figure 2 A second schematic flowchart of a monitoring and alarm method for a big data platform provided in an embodiment of this application;

[0023] Figure 3 The third flowchart illustrates the monitoring and alarm method of a big data platform provided in an embodiment of this application.

[0024] Figure 4 The fourth flowchart illustrates the monitoring and alarm method of a big data platform provided in an embodiment of this application.

[0025] Figure 5 Fifth flowchart illustrating a monitoring and alarm method for a big data platform provided in an embodiment of this application;

[0026] Figure 6 A flowchart illustrating the monitoring and alarm method of a big data platform provided in an embodiment of this application (Sixth).

[0027] Figure 7 The seventh flowchart illustrates the monitoring and alarm method of a big data platform provided in an embodiment of this application.

[0028] Figure 8 This is the eighth flowchart illustrating a monitoring and alarm method for a big data platform provided in an embodiment of this application.

[0029] Figure 9 A schematic diagram of the structure of a monitoring and alarm device for a big data platform provided in an embodiment of this application;

[0030] Figure 10 This is a schematic diagram of the monitoring and alarm system provided in this application. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of this application, but not all embodiments.

[0032] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0033] Furthermore, the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Additionally, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] It should be noted that, where there is no conflict, the features in the embodiments of this application can be combined with each other.

[0035] A big data platform is a network platform that provides services through content sharing, resource sharing, channel co-construction, and data sharing. Big data platforms can fully utilize big data resources to support innovative development. In the digital economy era, big data platforms have not only become a new key production factor but also a powerful engine driving economic and social development.

[0036] Monitoring big data platforms can promptly identify program vulnerabilities or business logic design flaws, playing a crucial role in their operation. However, existing monitoring solutions can only monitor fixed items on the big data platform; users cannot configure monitoring rules, failing to meet diverse monitoring needs. Furthermore, when anomalies are detected, only alarm information can be viewed, without effective processing of the alarms.

[0037] Therefore, this application provides a monitoring and alarm method for a big data platform to solve the above problems. This method can be generated by any monitoring and alarm platform with computing and processing capabilities. The monitoring and alarm platform can be, for example, a computer device facing the terminal or a backend server.

[0038] The following examples, in conjunction with the accompanying drawings, provide specific illustrations of the monitoring and alarm methods of the big data platform provided in this application.

[0039] Figure 1 This is one of the flowcharts illustrating a monitoring and alarm method for a big data platform provided in an embodiment of this application, such as... Figure 1 As shown, the method includes:

[0040] S101. Use the monitoring service in the monitoring platform of the monitoring and alarm system to monitor the target monitoring object corresponding to the monitoring service in the big data platform and obtain the monitoring information of the target monitoring object.

[0041] First, it should be noted that the method in this application is implemented by a monitoring and alarm system, which is a platform developed for big data platforms and has functions such as monitoring, alarming, and alarm processing.

[0042] Specifically, the monitoring and alarm system includes a big data deployment platform, a monitoring platform, and an alarm platform. The big data deployment platform is used to deploy the big data platform to be tested; the monitoring platform is used to monitor the deployed big data platform and issue alarm information when anomalies occur; the alarm platform is used to receive alarm information and handle any anomalies in the big data platform to ensure its stable operation.

[0043] Big data deployment platforms can be, for example, the Ambari open-source system; monitoring platforms can be, for example, the Prometheus open-source system; and alarm platforms can be developed systems with functions such as rule configuration, alarm information reception, alarm information processing, alarm information display, and status tracking.

[0044] Ambari can be deployed on the big data platform to be tested. The Ambari open-source system itself can support the installation of various components in the big data platform. Simply download the required version package from the official website of the corresponding big data platform, upload it to the Ambari open-source system, and follow the on-screen prompts to select the installation package and install it step by step.

[0045] In this application, the components deployed by Ambari for the big data platform include Hadoop's HDFS service, Yarn service, Spark framework, Flink framework, and Zookeeper middleware. HDFS is a file system used to store files and locate them through a directory tree; Yarn is a resource scheduling platform; Spark and Flink are big data computing systems; Zookeeper provides consistency services for distributed applications, including configuration services, name services, distributed synchronization, group services, etc. Zookeeper's goal is to encapsulate complex and error-prone critical services, providing users with simple, easy-to-use interfaces and a high-performance, stable system.

[0046] In this application, the installation process requires first installing Zookeeper, then installing HDFS and Yarn services, and finally installing the Spark and Flink frameworks. The service can be deployed as a single-node service or a distributed service according to actual needs.

[0047] The Ambari open-source system itself has functions such as configuring alarms, restarting services, and viewing service status. Specific operation details can be viewed on the interface accessed via Ambari's default preset port (e.g., port 8080). Simultaneously, the Ambari open-source system also provides service interfaces for these functions. According to the design of this application, the monitoring and alarm system can call the service status check and service restart interface functions provided by the Ambari open-source system. For example, the monitoring service in the monitoring platform can call the service status check interface provided by the Ambari open-source system to monitor the target monitoring object corresponding to the monitoring service in the big data platform, obtaining the monitoring information of the target monitoring object. The monitoring information may include threshold monitoring information and / or status monitoring information.

[0048] Among them, threshold monitoring information is used to indicate the hardware utilization rate of the target monitored object, such as the storage usage of the target monitored object, the CPU utilization rate of the target monitored object, and the disk utilization rate of the node of the target monitored object; status monitoring information is used to indicate whether the target monitored object is running normally, such as whether the target monitored object has gone offline or whether it has been restarted after going offline.

[0049] S102. Use monitoring services to obtain alarm rules for the target monitored object.

[0050] Alarm rules include threshold alarm rules and disconnection alarm rules. Alarm rules are pre-configured through the front-end interface of the alarm platform in the monitoring alarm system. Users can create or modify alarm rules on the front-end interface of the alarm platform in the monitoring alarm system. For example, they can directly reference the alarm templates built into the alarm platform and modify the templates to obtain alarm rules.

[0051] Specifically, this can be done by using a monitoring service, obtaining the alarm file directory from the monitoring service's configuration file, and then obtaining alarm rules from the alarm rule files within that directory. In this application, one alarm rule corresponds to one alarm rule file. The file containing the uploaded alarm quality rules, configured in the parameter settings, is located in the `rules` directory of the root directory where the Prometheus monitoring platform is installed.

[0052] Taking threshold alarm rules as an example, threshold alarm rules mainly target threshold monitoring information obtained by the monitoring platform. The built-in threshold alarm rule templates include, but are not limited to, the rule templates shown in Table 1:

[0053] Table 1 Built-in Threshold Alarm Rule Template

[0054]

[0055]

[0056] The alarm platform's rule configuration function allows users to customize the threshold of alarm rules. For example, modifying the value of {key=val} in the alarm rule corresponding to any monitored object in Table 1 will complete the modification of the alarm rule.

[0057] For example, to set an alarm to be triggered when the node CPU utilization is greater than 80%, you can directly select the built-in "Node CPU Utilization" alarm rule, configure the threshold to 80, and the platform will generate the alarm rule expression:

[0058] (1-((sum(node_cpu_seconds_total{mode="idle",key=val})by(instance))

[0059] / (sum(node_cpu_seconds_total{key=val})by(instance))))*100>=80

[0060] In the rule configuration function of the alarm platform, users can customize the Promsql expression for threshold alarm rules. If the expression can be successfully tested, the monitoring alarm system enters the expression and the threshold alarm rules contained in the expression, generates a threshold alarm rule file, uploads it to the alarm file directory of the monitoring platform, and then executes the command of the monitoring platform to reload the configuration, so that the monitoring platform can read the specific threshold alarm rules and make the threshold alarm rules take effect.

[0061] In the rule configuration function of the alarm platform, users can also directly create disconnection alarm rules. Then, similarly, the monitoring platform can execute commands to reload the configuration, enabling the monitoring platform to read the specific disconnection alarm rules and make them effective. Based on the disconnection alarm rules, it supports restarting the disconnected service. Users can configure the restart time interval, restart count threshold information, and conditions for disabling restarts on the front-end interface of the alarm platform.

[0062] S103. Use monitoring services to determine whether the monitoring information meets the alarm triggering conditions of the alarm rules.

[0063] After obtaining the monitoring information of the target monitoring object and the alarm rules for the target monitoring object, it is possible to determine whether the target monitoring object meets the threshold alarm triggering conditions based on the threshold alarm rules and threshold monitoring information, and / or, based on the offline alarm rules and status monitoring information, determine whether the target monitoring object meets the offline alarm triggering conditions.

[0064] S104. If the alarm triggering conditions of the alarm rule are met, the monitoring service will report the alarm information of the target monitored object to the alarm platform through the alarm management service in the monitoring platform.

[0065] When the monitoring information meets the alarm triggering conditions of the alarm rules, the monitoring service reports the alarm information of the target monitored object to the alarm platform through the alarm management service in the monitoring platform. The alarm platform can then maintain the operation of the big data platform based on the alarm information.

[0066] The specific operation can be as follows: The monitoring platform is configured with an alarm management service, Alertmanager. Alertmanager reports the alarm information of the target monitored object to the alarm receiving function of the alarm platform. The alarm receiving function can then determine whether the alarm is a new alarm, a duplicate alarm, or an alarm that has already been repaired based on the alarm information and alarm time information, and update the record accordingly. The alarm processing function in the alarm platform maintains the operation of the big data platform based on the alarm information, marks the alarm processing status, and saves the alarm record.

[0067] Using the method of this application, users can configure alarm rules on the front-end interface of the alarm platform, and determine whether the big data platform meets the alarm triggering conditions through the alarm rules, thus meeting the diverse monitoring needs of users. Furthermore, when the alarm triggering conditions of the alarm rules are met, the alarm platform can effectively process the alarm information and realize the operation and maintenance of the big data platform.

[0068] Figure 2 This is a second flowchart illustrating a monitoring and alarm method for a big data platform provided in an embodiment of this application.

[0069] like Figure 2 As shown, in one embodiment, step S101, which uses the monitoring service in the monitoring platform of the monitoring and alarm system to monitor the target monitoring object corresponding to the monitoring service in the big data platform and obtain the monitoring information of the target monitoring object, may include:

[0070] S201. Use the monitoring service to obtain the configuration information of the target monitoring object from the monitoring service's configuration file.

[0071] After configuring the big data deployment platform and monitoring platform, the configuration information of the target monitored object (such as address and port number information) is pre-configured in the monitoring platform configuration file prometheus.yml as a job. Therefore, the configuration information of the target monitored object can be obtained from the configuration file of the monitoring service.

[0072] S202. Using monitoring services, based on the configuration information of the target monitoring object, the target monitoring object is monitored to obtain the monitoring information of the target monitoring object.

[0073] If the big data cluster is deployed in a distributed, multi-node manner, a first monitoring service component (hadoop_exporter) and / or a second monitoring service component (node_exporter) can be installed on multiple nodes. The monitoring service of the monitoring platform can be implemented, for example, through the first monitoring service component (hadoop_exporter). The hadoop_exporter can obtain the native monitoring metrics of the big data platform (e.g., Hadoop cluster) based on the configuration information of the target monitored object. This application supports multi-dimensional statistical monitoring information, and the hadoop_exporter can provide the most basic Hadoop cluster monitoring metric information. The monitoring service of the monitoring platform can also be implemented, for example, through the second monitoring service component (node_exporter). The node_exporter component can obtain the usage information of CPU, memory, disk, network, and other resources of the nodes in the target monitored object based on the configuration information (e.g., address and port number information) of the target monitored object.

[0074] Furthermore, since hadoop_exporter has a problem when acquiring monitoring information: if the target monitored object goes offline, there will be log output, but it will not trigger the reporting of information to the monitoring platform and ultimately trigger alarm information, this application can also set up a third monitoring service component (custom exporter, which can be customized and installed by the user to overcome the problems of the first or second monitoring service component).

[0075] The monitoring service of the monitoring platform can also be implemented through a third-party monitoring service component (custom exporter). A custom exporter requires creating a Maven project, obtaining the status of a specified service based on the interface provided by the Ambari open-source system, defining the obtained status information as monitoring metrics that the monitoring platform (Prometheus) can recognize, and reporting the metrics to Prometheus. According to the status query interface provided by Ambari, the status information of the specified service is: "STARTED" returns 1, and if it is not "STARTED", it returns 0. A return of 1 indicates that the target monitored object is running normally, and a return of 0 indicates that the target monitored object is offline.

[0076] The process information monitored by the custom Exporter includes, but is not limited to: the namenode and datanode processes of the HDFS service, the resourceManager and nodeManager processes of the Yarn service, and the master and worker processes of the Spark service. The names are custom names, and the process information is obtained through an interface provided by the Ambari open-source system. Based on the obtained status, it determines whether the status value is "STARTED," returning 1 if it is and 0 otherwise, and encapsulates the status value into a Prometheus data type.

[0077] Figure 3 This is the third flowchart illustrating a monitoring and alarm method for a big data platform provided in an embodiment of this application. Figure 3 As shown in S104, the monitoring service reports alarm information of the target monitored object to the alarm platform through the alarm management service in the monitoring platform, including:

[0078] S301. Use the monitoring service to obtain the IP address of the alarm management service from the monitoring service's configuration file.

[0079] Alarm management services can be implemented, for example, by using the Alertmanager component installed in the monitoring platform (Prometheus). Simply modify the Prometheus configuration file prometheus.yml, add the Alertmanager configuration, and replace the localhost configuration with the actual deployed IP address of the Alertmanagers component. This will add the alarm management service to the monitoring platform.

[0080] S302. Using the monitoring service, alarm information is transmitted to the alarm management service based on the IP address of the alarm management service.

[0081] After obtaining the IP address of the alarm management service, the alarm information obtained by the monitoring service component can be transmitted to the alarm management service.

[0082] S303. Use alarm management service to obtain the configuration information of the alarm platform from the configuration file of alarm management service, and transmit the alarm information to the alarm platform based on the configuration information of the alarm platform.

[0083] The alarm management service configuration file contains the alarm platform configuration information. Based on the alarm platform configuration information, alarm information can be transmitted to the alarm platform. That is, the alarm information reporting process is as follows: the node_exporter component, hadoop_exporter component, or custom exporter component obtains the monitoring information of the target monitored object. If the monitoring information meets the alarm triggering conditions of the alarm rule, the alarm information is sent to the Alertmanagers component, and then the Alertmanagers component reports the alarm information to the alarm platform.

[0084] Based on the above embodiments, the method of this application will be further illustrated below with reference to specific embodiments.

[0085] Figure 4 This is the fourth flowchart illustrating a monitoring and alarm method for a big data platform provided in an embodiment of this application. Figure 4 As shown in S101, the monitoring service in the monitoring platform of the monitoring and alarm system is used to monitor the target monitoring object corresponding to the monitoring service in the big data platform, and the monitoring information of the target monitoring object is obtained, including:

[0086] S401. Using the first monitoring service component in the monitoring service, monitor the first target monitoring object corresponding to the first monitoring service component in the big data platform, and obtain the monitoring information of the first target monitoring object.

[0087] The first monitoring service component could be, for example, the hadoop_exporter component. The first target monitoring objects include basic metrics monitoring items of the big data platform, such as HDFS storage usage in the past week, HDFS storage usage percentage, HDFS daily storage usage, NameNode document status, HDFS CPU usage, YARN ResourceManager service CPU utilization, YARN NodeManager available node percentage, etc., as shown in Table 1.

[0088] The hadoop_exporter component monitors the aforementioned basic metrics, thus obtaining monitoring information for those metrics.

[0089] S402. Use the second monitoring service component in the monitoring service to monitor the second target monitoring object corresponding to the second monitoring service component in the big data platform, and obtain the monitoring information of the second target monitoring object.

[0090] The second monitoring service component could be, for example, the node_exporter component. The second target monitoring object includes basic indicator monitoring items of the server node where the big data platform is located, such as node memory usage, node CPU usage, and node disk usage in Table 1.

[0091] The node_exporter component monitors the basic metrics of the aforementioned server nodes, thus obtaining monitoring information for those basic metrics.

[0092] S403. Use the third monitoring service component in the monitoring service to monitor the third target monitoring object corresponding to the third monitoring service component in the big data platform, and obtain the monitoring information of the third target monitoring object.

[0093] The third monitoring service component is a custom exporter component. The third target monitoring object includes the status monitoring items of software processes in the big data platform, such as whether the software is offline.

[0094] By using a custom exporter component to monitor the status of the software process, you can obtain information on whether the software has gone offline.

[0095] Based on this, such as Figure 5 As shown, S103, which uses a monitoring service to determine whether the monitoring information meets the alarm triggering conditions of the alarm rules, may further include:

[0096] S501. Using the first monitoring service component, based on the first threshold alarm rule of the first target monitoring object, determine whether the monitoring information of the first target monitoring object meets the alarm triggering condition of the first threshold alarm rule.

[0097] As described in the above embodiments, the first monitoring service component hadoop_exporter can obtain the native monitoring metrics of the big data platform (e.g., Hadoop cluster) based on the configuration information of the target monitoring object. Then, the hadoop_exporter component can determine whether the monitoring information of the first target monitoring object meets the alarm triggering conditions of the first threshold alarm rule based on the first threshold alarm rule of the first target monitoring object.

[0098] The first target monitoring object includes, but is not limited to, the HDFS storage usage in the past week, HDFS storage usage percentage, HDFS daily storage usage, NameNode document status, HDFS CPU usage, YARN ResourceManager service CPU utilization, YARN NodeManager available node percentage, etc. in Table 1. The first threshold alarm rule for the first target monitoring object is the alarm rule represented by a Promsql expression corresponding to the above monitoring object.

[0099] S502. Using the second monitoring service component, based on the second threshold alarm rule of the second target monitoring object, determine whether the monitoring information of the second target monitoring object meets the alarm triggering condition of the second threshold alarm rule.

[0100] As described in the above embodiments, the second monitoring service component node_exporter can obtain the usage information of CPU, memory, disk, network and other resources of the nodes in the target monitoring object based on the configuration information of the target monitoring object (such as address and port number information). Then, the node_exporter component can determine whether the monitoring information of the second target monitoring object meets the alarm triggering conditions of the second threshold alarm rule based on the second threshold alarm rule of the second target monitoring object.

[0101] The second target monitoring object includes, but is not limited to, node memory usage, node CPU usage, node disk usage, etc. in Table 1. The second threshold alarm rule for the second target monitoring object is the alarm rule represented by a Promsql expression corresponding to the above monitoring object.

[0102] S503. Using a third monitoring service component, based on the offline alarm rules of the third target monitoring object, determine whether the monitoring information of the third target monitoring object meets the alarm triggering conditions of the offline alarm rules.

[0103] As described in the above embodiments, the third monitoring service component (custom exporter) can monitor the status monitoring items of software processes in the big data platform. Then, the custom exporter can determine whether the monitoring information of the third target monitoring object meets the alarm triggering conditions of the disconnection alarm rule based on the disconnection alarm rule of the third target monitoring object.

[0104] The third target monitoring objects include, but are not limited to, the namenode and datanode processes of HDFS service, the resourceManager and nodeManager processes of Yarn service, and the master and worker processes of Spark service.

[0105] The third target monitoring object's disconnection alarm rules are the disconnection alarm rules configured by the user for the aforementioned monitoring objects.

[0106] Of course, alarm rules need to be obtained before executing S103. Therefore, the method of this application may also include:

[0107] S601. Use the first monitoring service component to obtain the first threshold alarm rule for the first target monitoring object.

[0108] The first threshold alarm rule can be obtained by modifying the value of {key=val} in the Promsql expressions corresponding to the following in Table 1: HDFS storage usage in the most recent week, HDFS storage usage percentage, HDFS daily storage usage, NameNode document status, HDFS CPU usage, YARN ResourceManager service CPU utilization, and YARN NodeManager available node percentage. Alternatively, it can be a threshold alarm rule directly provided by the user.

[0109] Before determining whether the monitoring information of the first target monitoring object meets the alarm triggering conditions of the first threshold alarm rule, the first threshold alarm rule of the first target monitoring object is obtained using the hadoop_exporter component.

[0110] S602. Use the second monitoring service component to obtain the second threshold alarm rule for the second target monitoring object.

[0111] The second threshold alarm rule can be obtained by modifying the value of {key=val} in the Promsql expressions corresponding to node memory utilization, node CPU utilization, and node disk utilization in Table 1, or it can be a threshold alarm rule directly provided by the user.

[0112] Before determining whether the monitoring information of the second target monitoring object meets the alarm triggering conditions of the second threshold alarm rule, the node_exporter component is used to obtain the second threshold alarm rule of the second target monitoring object.

[0113] S603. Use a third monitoring service component to obtain the offline alarm rules of the third target monitoring object.

[0114] The third threshold alarm rules may include, but are not limited to, the offline alarm rules corresponding to the namenode and datanode processes of HDFS service, the resourceManager and nodeManager processes of Yarn service, and the master and worker processes of Spark service.

[0115] Before determining whether the monitoring information of the third target monitoring object meets the alarm triggering conditions of the third threshold alarm rule, the third threshold alarm rule of the third target monitoring object is obtained using a custom exporter component.

[0116] This application also provides a possible implementation method for the alarm platform to maintain the operation of the big data platform based on the alarm information after reporting the alarm information of the target monitored object to the alarm platform. The following is combined with Figure 7 and Figure 8 Let's illustrate with examples.

[0117] like Figure 7 As shown, after S104 reports the alarm information of the target monitored object to the alarm platform, the method of this application may further include:

[0118] S701. If the alarm message indicates that the target monitored object is offline, the alarm processing service in the alarm platform will be used to restart the target monitored object.

[0119] If the alarm message indicates that the target monitored object is offline, and the alarm platform's alarm processing service is configured with a restart function, the alarm platform can restart the target monitored object through the alarm processing service. Users can configure the automatic restart time interval, the automatic restart failure threshold, and the threshold information for disabling automatic restart on the front-end interface.

[0120] The automatic restart time interval is used to set the minimum time interval between two service restarts, and the failure count threshold is used to define the maximum number of consecutive restart failures.

[0121] It should be noted that if the alarm processing service of the alarm platform is not configured with a restart function, the target monitored object cannot be restarted. In this case, the alarm information corresponding to the offline target monitored object will be marked as unprocessed.

[0122] S702. Detect whether the target monitoring object has been pulled up.

[0123] After restarting the target monitored object, check whether the target monitored object has been successfully started, that is, whether the target monitored object is running normally.

[0124] S703. If the target monitored object is not restarted, check whether the number of restarts of the target monitored object exceeds the failure count threshold.

[0125] If the target monitored object is successfully started, the process ends.

[0126] If the target monitored object is not restarted, check whether the number of restarts of the target monitored object exceeds the failure count threshold.

[0127] S704. If the number of restarts of the target monitored object does not exceed the failure count threshold, then restart the target monitored object again.

[0128] Taking a failure threshold of 10 as an example, if the number of restarts of the target monitored object does not exceed 10, the target monitored object can be restarted again. If the number of restarts of the target monitored object has exceeded 10 and it has not been successfully restarted, the target monitored object may have some problems that cannot be solved by restarting. Therefore, it will not be restarted and the alarm information will be transferred to manual handling.

[0129] This embodiment enables the reactivation of a target monitoring object after it goes offline, thus achieving the operation and maintenance of the big data platform.

[0130] In one implementation, after S702 detects whether the target monitored object has been pulled up, the processing method may further include:

[0131] S801. If the target monitored object is restarted, check whether the number of restarts of the target monitored object within a historical unit of time exceeds the preset number threshold.

[0132] If the alarm processing service of the alarm platform has the pause and restart function enabled, it will determine whether the number of successful restart records of the target monitored object in the historical unit time exceeds the preset number threshold. If it does not exceed the threshold, the alarm status will be marked as processed and the process will end.

[0133] S802. If so, then silence the target monitored object and check whether the target monitored object is running normally.

[0134] The preset number of times threshold can be used to indicate whether the target monitored object is frequently successfully restarted. Taking a preset number of times threshold of 20 times as an example, if the number of successful restarts of the target monitored object exceeds 20 times in a historical unit of time, it can be considered that although the target monitored object is successfully restarted, it frequently drops offline in a short period of time. There may be some problems that cannot be solved by restarting. The alarm information will be transferred to manual handling, and the target monitored object will be silenced and cannot be restarted again within the specified silence time.

[0135] After the silence period, the restart task is awakened, and it checks whether the target monitored object is running normally. If so, it marks the alarm information as processed and ends the process. If the target monitored object is in an abnormal state (including offline, running errors, etc.), it re-enters the main process and executes... Figure 7 The process is shown below.

[0136] It should also be noted that the alarm platform of this application also includes an alarm display function, which is used to display alarm information in a paginated list. The alarm information reported by the monitoring platform, the alarm processing status, and the associated rule information can be directly viewed on the front-end interface of the alarm platform.

[0137] The alarm platform also includes a status tracking function, which allows users to view historical status information of alarm processing, including the time point when the alarm was repaired and the time point when the alarm was transferred to manual processing, etc.

[0138] The following will continue to explain the apparatus, device and storage medium for implementing the data platform monitoring and alarm method provided in any of the above embodiments of this application. The specific implementation process and the resulting technical effects are the same as those in the corresponding method embodiments. For the sake of brevity, the parts not mentioned in the following embodiments can be referred to the corresponding content in the method embodiments.

[0139] like Figure 9 As shown in the illustration, this application also provides a monitoring and alarm device for a big data platform, applied to the monitoring and alarm system of the big data platform. The device includes:

[0140] Monitoring module 10 is used to monitor the target monitoring object corresponding to the monitoring service in the big data platform using the monitoring service in the monitoring and alarm system, and to obtain the monitoring information of the target monitoring object.

[0141] The acquisition module 20 is used to acquire the alarm rules of the target monitored object using the monitoring service. The alarm rules are alarm rules that are configured in advance through the front-end interface of the alarm platform in the monitoring alarm system.

[0142] The determination module 30 is used to use the monitoring service to determine whether the monitoring information meets the alarm triggering conditions of the alarm rules.

[0143] The alarm module 40 is used to report the alarm information of the target monitored object to the alarm platform through the alarm management service in the monitoring platform if the alarm triggering conditions of the alarm rules are met.

[0144] Optionally, the monitoring module 10 is also used to use a monitoring service to obtain the configuration information of the target monitoring object from the configuration file of the monitoring service; and to use the monitoring service to monitor the target monitoring object based on the configuration information of the target monitoring object to obtain the monitoring information of the target monitoring object.

[0145] Optionally, the acquisition module 20 is also used to acquire the alarm file directory from the configuration file of the monitoring service and acquire the alarm rules from the alarm file directory.

[0146] Optionally, the alarm module 40 is also used to: use the monitoring service to obtain the IP address of the alarm management service from the configuration file of the monitoring service; use the monitoring service to transmit alarm information to the alarm management service based on the IP address of the alarm management service; use the alarm management service to obtain the configuration information of the alarm platform from the configuration file of the alarm management service, and transmit the alarm information to the alarm platform based on the configuration information of the alarm platform.

[0147] Optionally, the monitoring module 10 is further configured to use a first monitoring service component in the monitoring service to monitor a first target monitoring object corresponding to the first monitoring service component in the big data platform, and obtain monitoring information of the first target monitoring object, which includes: basic indicator monitoring items of the big data platform; use a second monitoring service component in the monitoring service to monitor a second target monitoring object corresponding to the second monitoring service component in the big data platform, and obtain monitoring information of the second target monitoring object, which includes: basic indicator monitoring items of the server node where the big data platform is located; and use a third monitoring service component in the monitoring service to monitor a third target monitoring object corresponding to the third monitoring service component in the big data platform, and obtain monitoring information of the third target monitoring object, which includes: status monitoring items of software processes in the big data platform.

[0148] Optionally, the determining module 30 is further configured to: employ a first monitoring service component to determine whether the monitoring information of the first target monitoring object meets the alarm triggering condition of the first threshold alarm rule based on the first threshold alarm rule of the first target monitoring object; employ a second monitoring service component to determine whether the monitoring information of the second target monitoring object meets the alarm triggering condition of the second threshold alarm rule based on the second threshold alarm rule of the second target monitoring object; and employ a third monitoring service component to determine whether the monitoring information of the third target monitoring object meets the alarm triggering condition of the disconnection alarm rule based on the disconnection alarm rule of the third target monitoring object.

[0149] Optionally, the acquisition module 20 is further configured to acquire the first threshold alarm rule of the first target monitoring object using the first monitoring service component; acquire the second threshold alarm rule of the second target monitoring object using the second monitoring service component; and acquire the offline alarm rule of the third target monitoring object using the third monitoring service component.

[0150] Optionally, the monitoring and alarm method of the data platform further includes a processing module, used to restart the target monitoring object by using the alarm processing service in the alarm platform if the alarm information indicates that the target monitoring object is offline; detect whether the target monitoring object has been restarted; if the target monitoring object has not been restarted, detect whether the number of restarts of the target monitoring object exceeds the failure count threshold; if the number of restarts of the target monitoring object does not exceed the failure count threshold, restart the target monitoring object again.

[0151] Optionally, the processing module is also used to detect whether the number of times the target monitoring object has been restarted within a historical unit of time exceeds a preset threshold if the target monitoring object is restarted; if so, the target monitoring object is silenced and the running status of the target monitoring object is checked to see if it is normal.

[0152] The above-described device is used to execute the method provided in the foregoing embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0153] These modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more microprocessors, or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a system-on-a-chip (SOC).

[0154] Optionally, this application also provides a monitoring and alarm system. Figure 10 This is a schematic diagram of the monitoring and alarm system provided in this application. The monitoring and alarm system can be a local terminal device or a server. The monitoring and alarm system includes a processor 100, a storage medium 200 and a bus 300. The processor and the storage medium communicate with each other through the bus.

[0155] The storage medium stores program instructions that can be executed by the processor. When the monitoring and alarm system is running, the processor executes the program instructions to implement the monitoring and alarm method of the big data platform described in any of the above embodiments.

[0156] Optionally, this application also provides a storage medium storing program instructions, which, when executed by a processor, implement the monitoring and alarm method of the big data platform described in any of the above embodiments.

[0157] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0158] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0159] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0160] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0161] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A monitoring and alarm method for a big data platform, characterized in that: The monitoring and alarm system applied to the big data platform comprises: Using the monitoring service in the monitoring platform of the monitoring alarm system, monitoring the target monitoring object corresponding to the monitoring service in the big data platform is performed to obtain monitoring information of the target monitoring object; The monitoring service is used to obtain the alarm rule of the target monitoring object, wherein the alarm rule is: the alarm rule configured in advance through the front-end interface of the alarm platform in the monitoring alarm system; Using the monitoring service, determining whether the monitoring information satisfies the alarm triggering condition of the alarm rule; If the alarm triggering condition of the alarm rule is met, the monitoring service reports the alarm information of the target monitoring object to the alarm platform through the alarm management service in the monitoring platform.

2. The method according to claim 1, characterized in that The monitoring service in the monitoring platform of the monitoring alarm system is used to monitor the target monitoring object corresponding to the monitoring service in the big data platform to obtain the monitoring information of the target monitoring object, including: Using the monitoring service, obtaining configuration information of the target monitoring object from a configuration file of the monitoring service; The monitoring service is used to monitor the target monitoring object based on the configuration information of the target monitoring object to obtain monitoring information of the target monitoring object.

3. The method according to claim 1, characterized in that The adopting the monitoring service to obtain the alarm rule of the target monitoring object includes: The monitoring service is used to obtain an alarm file directory from a configuration file of the monitoring service, and the alarm rule is obtained from the alarm file directory.

4. The method according to claim 1, characterized in that: The monitoring service reports the alarm information of the target monitoring object to the alarm platform through the alarm management service in the monitoring platform, including: Using the monitoring service, obtain the IP address of the alarm management service from the configuration file of the monitoring service; Using the monitoring service, based on the IP address of the alarm management service, the alarm information is transmitted to the alarm management service; The alarm management service is used to obtain the configuration information of the alarm platform from the configuration file of the alarm management service, and based on the configuration information of the alarm platform, the alarm information is transmitted to the alarm platform.

5. The method according to claim 1, characterized in that: The monitoring service in the monitoring platform of the monitoring alarm system is used to monitor the target monitoring object corresponding to the monitoring service in the big data platform to obtain the monitoring information of the target monitoring object, including: Using the first monitoring service component in the monitoring service, monitoring the first target monitoring object corresponding to the first monitoring service component in the big data platform, and obtaining monitoring information of the first target monitoring object, wherein the first target monitoring object includes: basic indicator monitoring items of the big data platform; Using the second monitoring service component in the monitoring service, monitoring the second target monitoring object corresponding to the second monitoring service component in the big data platform, and obtaining monitoring information of the second target monitoring object, wherein the second target monitoring object includes: basic indicator monitoring items of the server node where the big data platform is located; The third monitoring service component in the monitoring service is used to monitor the third target monitoring object corresponding to the third monitoring service component in the big data platform to obtain monitoring information of the third target monitoring object, and the third target monitoring object includes: status monitoring items of the software process in the big data platform.

6. The method according to claim 5, characterized in that The adopting the monitoring service to determine whether the monitoring information satisfies the alarm triggering condition of the alarm rule includes: Using the first monitoring service component, based on the first threshold alarm rule of the first target monitoring object, determining whether the monitoring information of the first target monitoring object meets the alarm triggering condition of the first threshold alarm rule; Using the second monitoring service component, based on the second threshold alarm rule of the second target monitoring object, determining whether the monitoring information of the second target monitoring object meets the alarm triggering condition of the second threshold alarm rule; The third monitoring service component is used to determine whether the monitoring information of the third target monitoring object meets the alarm triggering condition of the offline alarm rule based on the offline alarm rule of the third target monitoring object.

7. The method according to claim 6, characterized in that Before using the monitoring service to determine whether the monitoring information satisfies the alarm triggering condition of the alarm rule, the method further includes: Using the first monitoring service component to obtain a first threshold alarm rule for the first target monitoring object; Using the second monitoring service component to obtain a second threshold alarm rule for the second target monitoring object; The third monitoring service component is used to obtain the offline alarm rule of the third target monitoring object.

8. The method according to claim 1, characterized in that After reporting the alarm information of the target monitoring object to the alarm platform, the method further includes: If the alarm information indicates that the target monitored object is in an offline state, the alarm processing service in the alarm platform is used to restart the target monitored object; Detecting whether the target monitoring object is pulled up; If the target monitoring object is not pulled up, detecting whether the restart times of the target monitoring object exceeds the failure times threshold; If the restart times of the target monitoring object does not exceed the failure times threshold, the target monitoring object is restarted again.

9. The method according to claim 8, characterized in that After detecting whether the target monitoring object is pulled up, the method further includes: If the target monitoring object is pulled up, detecting whether the restart times of the target monitoring object in the historical unit time exceeds the preset times threshold; If so, the target monitoring object is silenced and the operating status of the target monitoring object is detected to be normal.

10. A monitoring alarm system, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor. When the monitoring and alarm system is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to implement the monitoring and alarm method for the big data platform described in any one of claims 1 to 9.