An alarm method, system and device based on Ambari big data platform

Abstract

The present application discloses an alarm method, system and device based on the Ambari big data platform, the method comprising: first, in response to the user's input operation on the alarm template, generate an alarm definition and write it into the alarm file, then, regularly read the latest running time of each alarm task from the database, and compare it with the preset time threshold, so as to judge whether the running status of the alarm task is normal according to the comparison result, then, obtain the running result of at least one alarm task, and push it through the preset push channel; at the same time, regularly obtain the error log, and analyze the currently undefined alarm definition from it; and then calculate the priority of each alarm task according to the number of occurrences and preset level of each alarm task, so as to re-sort the execution order of each alarm task. Thus, the existing alarm function of the Ambari big data platform can be expanded and improved to provide a better cluster alarm effect.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an alarm method, system and device based on the Ambari big data platform. Background Art

[0002] Ambari is a big data platform management software based on the Hadoop ecosystem. It supports the installation and use of multiple big data components. Users log in to the Ambari platform and can create, install, start, stop, and maintain big data service components by clicking on the page without having to execute a large number of commands and scripts as in the traditional way, which greatly simplifies user use.

[0003] At present, in order to timely expose problems that occur during the operation of the big data cluster and avoid system operation risks, Ambari has preset Figure 1 The alarm system shown in the figure can provide basic alarm functions, but it also has the following five problems: first, it does not support custom alarms and cannot add new alarms; second, it lacks monitoring of the alarm task itself, resulting in the inability to provide accurate real-time alarm information; third, it lacks real-time push of alarm information; fourth, it lacks automatic planning of new alarm definitions based on cluster anomalies; and fifth, it lacks flexible scheduling of alarm execution. Therefore, how to improve the alarm effect of the Ambari big data platform to provide better cluster alarms is an urgent problem to be solved. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to provide an alarm method, system and device based on the Ambari big data platform, which can expand and improve the existing alarm functions to provide better cluster alarm effects.

[0005] In a first aspect, an embodiment of the present application provides an alarm method based on the Ambari big data platform, including:

[0006] In response to a user input operation on an alarm template, an alarm definition is generated, and the alarm definition is written into an alarm file;

[0007] Read the latest running time of each alarm task from the database regularly, and compare it with the preset time threshold, so as to judge whether the running status of the alarm task is normal according to the comparison result;

[0008] Obtaining the running result of at least one of the alarm tasks, and pushing the running result to the user's terminal device through a preset push channel;

[0009] Obtain the error log of the platform operation regularly and analyze the new alarm definitions that are currently undefined;

[0010] The priority of each alarm task is calculated according to the number of times each alarm task occurs and the preset level; and according to the ranking of the priorities, the execution order of each alarm task is re-ranked to achieve flexible scheduling of each alarm task.

[0011] Optionally, after the latest running time of each alarm task is read from the database at a fixed time and compared with a preset time threshold to determine whether the running state of the alarm task is normal according to the comparison result, the method further includes:

[0012] If it is determined that the running state of the alarm task is abnormal, the running state of the corresponding alarm task in the database is updated as abnormal.

[0013] Optionally, obtaining the running result of at least one of the alarm tasks and pushing the running result to the user's terminal device through a preset push channel includes:

[0014] Obtaining the running result of at least one of the alarm tasks, and selecting at least one push channel according to a preset correspondence between the running result of the alarm task and the push channel;

[0015] Assembling a push message corresponding to the operation result by using the at least one push channel;

[0016] The push message is pushed to the user's terminal device.

[0017] Optionally, the periodic acquisition of the error log of the platform operation and the analysis of the currently undefined new alarm definition therefrom include:

[0018] Obtain error logs of platform operation at regular intervals, and extract error keywords from the error logs;

[0019] Match the keywords of the error report with the existing alarm definition to obtain a matching result;

[0020] When the matching result indicates that the existing alarm definition does not contain the keyword of the error report, a new alarm definition that is currently undefined is determined.

[0021] Optionally, the periodically obtaining an error log of the platform operation and extracting error keywords from the error log include:

[0022] The error log of the platform operation is obtained regularly, and the error log is input into the pre-built keyword recognition algorithm model to calculate the keyword of the error through the model.

[0023] Optionally, calculating the priority of each alarm task according to the number of occurrences of each alarm task and a preset level includes:

[0024] Read the historical occurrence number of each alarm task from the database;

[0025] Determine the service level for each of the alarm tasks described;

[0026] The sum of the historical occurrence times and the service level of each alarm task is calculated, and the priority of each alarm task is determined according to the sum.

[0027] Optionally, after reordering the execution order of each alarm task according to the priority order to achieve flexible scheduling of each alarm task, the method further includes:

[0028] The reordered execution order of each alarm task is synchronously updated to the server management node of the Ambari big data platform so that each alarm task is executed according to the updated execution order.

[0029] In a second aspect, the embodiment of the present application further provides an alarm system based on the Ambari big data platform, including:

[0030] A custom alarm module, used to generate an alarm definition in response to a user's input operation on an alarm template, and write the alarm definition into an alarm file;

[0031] The alarm status monitoring module is used to periodically read the latest running time of each alarm task from the database and compare it with a preset time threshold to determine whether the running status of the alarm task is normal based on the comparison result;

[0032] An alarm information push module, used to obtain the running result of at least one of the alarm tasks, and push the running result to the user's terminal device through a preset push channel;

[0033] New alarm planning module, used to periodically obtain error logs of platform operation and analyze new alarm definitions that are currently undefined;

[0034] The alarm flexible scheduling module is used to calculate the priority of each alarm task according to the number of times each alarm task occurs and the preset level; and re-order the execution order of each alarm task according to the priority order to achieve flexible scheduling of each alarm task.

[0035] Optionally, the device further comprises:

[0036] The first update module is used to read the latest running time of each alarm task from the database at the timing, and compare it with a preset time threshold to determine whether the running status of the alarm task is normal based on the comparison result. If it is determined that the running status of the alarm task is abnormal, the running status of the corresponding alarm task in the database is updated as abnormal.

[0037] Optionally, the alarm information push module includes:

[0038] A selection submodule, used for obtaining the running result of at least one of the alarm tasks, and selecting at least one push channel according to a preset corresponding relationship between the running result of the alarm task and the push channel;

[0039] An assembling submodule, used to assemble a push message corresponding to the operation result by using the at least one push channel;

[0040] The push submodule is used to push the push message to the user's terminal device.

[0041] Optionally, the new alarm planning module includes:

[0042] The extraction submodule is used to periodically obtain the error log of the platform operation and extract the error keywords from the error log;

[0043] A matching submodule, used to match the keywords of the error report with the existing alarm definition to obtain a matching result;

[0044] The first determination submodule is used to determine a new alarm definition that is not currently defined when the matching result indicates that the keyword of the error is not included in the existing alarm definition.

[0045] Optionally, the extraction submodule is specifically used for:

[0046] The error log of the platform operation is obtained regularly, and the error log is input into the pre-built keyword recognition algorithm model to calculate the keyword of the error through the model.

[0047] Optionally, the alarm elastic scheduling module includes:

[0048] A reading submodule, used for reading the historical occurrence number of each alarm task from the database;

[0049] A second determination submodule is used to determine the service level of each alarm task;

[0050] The calculation submodule is used to calculate the sum of the historical occurrence times and the service level of each alarm task, and determine the priority of each alarm task according to the sum.

[0051] Optionally, the device further comprises:

[0052] The second update module is used to reorder the execution order of each alarm task according to the priority sorting to achieve flexible scheduling of each alarm task, and then synchronously update the reordered execution order of each alarm task to the server management node of the Ambari big data platform, so that each alarm task can be executed according to the updated execution order.

[0053] The embodiment of the present application also provides an alarm device based on the Ambari big data platform, including: a processor, a memory, and a system bus;

[0054] The processor and the memory are connected via the system bus;

[0055] The memory is used to store one or more programs, and the one or more programs include instructions, which, when executed by the processor, enable the processor to execute any one of the implementation methods of the above-mentioned alarm method based on the Ambari big data platform.

[0056] An embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a terminal device, the terminal device executes any one of the implementation methods of the above-mentioned alarm method based on the Ambari big data platform.

[0057] The embodiment of the present application provides an alarm method, system and device based on the Ambari big data platform. First, in response to the user's input operation on the alarm template, an alarm definition is generated and written into the alarm file. Then, the latest running time of each alarm task is read from the database regularly, and it is compared with the preset time threshold to determine whether the running status of the alarm task is normal according to the comparison result. Then, the running result of at least one alarm task is obtained, and the running result is pushed to the user's terminal device through a preset push channel; at the same time, the error log of the platform operation is obtained regularly, and the new alarm definition that is currently undefined is analyzed therefrom; and then the priority of each alarm task is calculated according to the number of occurrences and the preset level of each alarm task; and according to the priority ranking, the execution order of each alarm task is re-ordered to achieve flexible scheduling of each alarm task. In this way, the existing alarm function of the Ambari big data platform can be expanded and improved to provide a better cluster alarm effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0059] Figure 1 This is a schematic diagram of the structure of the existing alarm system on the Ambari big data platform;

[0060] Figure 2 A flowchart of an alarm method based on the Ambari big data platform provided in an embodiment of the present application;

[0061] Figure 3 A flowchart for monitoring the alarm status provided in an embodiment of the present application;

[0062] Figure 4 A flowchart for pushing alarm information provided in an embodiment of the present application;

[0063] Figure 5 A flowchart for determining a new alarm definition provided in an embodiment of the present application;

[0064] Figure 6 A flowchart for flexible scheduling of alarm tasks provided in an embodiment of the present application;

[0065] Figure 7 A schematic diagram of the composition of an alarm system based on the Ambari big data platform provided in an embodiment of the present application. DETAILED DESCRIPTION

[0066] At present, in order to timely expose problems that occur during the operation of the Ambari big data cluster and avoid system operation risks, Ambari has preset the following Figure 1 In the alarm system shown in the figure, an alarm definition file is pre-set in Ambari-Server, which pre-defines many alarm tasks (including Ambari-Server alarms, Ambari-Agent alarms, and component service alarms). After the server is started, it will read the definition file and actually run the alarm during the cluster operation. Ambari Server alarms run on the server, Ambari Agent alarms run on the agent, and service component alarms run on the corresponding machines, and finally write the operation results to the database. Although the alarm system can provide basic alarm functions, it also has the following five problems:

[0067] First, it does not support custom alarms and cannot add new alarms. If the preset alarms cannot meet the existing needs, they cannot be customized. Second, there is a lack of monitoring of the alarm task itself. If the alarm itself runs abnormally, the latest alarm status cannot be obtained, which may lead to the inability to provide accurate real-time alarm information. Third, there is a lack of real-time push of alarm information, resulting in users only being able to log in to the platform to view alarm information. Fourth, there is a lack of automatic planning of new alarm definitions based on cluster anomalies. When some problems often occur during cluster operation, if there is no alarm about this problem in the alarm definition at this time, the problem cannot be discovered in time through the alarm. It can be seen that new alarms need to be planned and run in time according to the anomaly. Fifth, there is a lack of flexible scheduling of alarm execution, resulting in all alarm tasks being executed at the same priority, and alarm tasks are not executed according to priority based on the importance of the service and the frequency of alarms. Therefore, how to improve the alarm effect of the Ambari big data platform to provide better cluster alarms is an urgent problem to be solved.

[0068] To solve the above defects, the embodiment of the present application provides an alarm method based on the Ambari big data platform. First, in response to the user's input operation on the alarm template, an alarm definition is generated and written into the alarm file. Then, the latest running time of each alarm task is read from the database regularly, and it is compared with the preset time threshold to determine whether the running status of the alarm task is normal according to the comparison result. Then, the running result of at least one alarm task is obtained, and the running result is pushed to the user's terminal device through a preset push channel; at the same time, the error log of the platform operation is obtained regularly, and the new alarm definition that is currently undefined is analyzed therefrom; and then the priority of each alarm task is calculated according to the number of occurrences and the preset level of each alarm task; and according to the priority ranking, the execution order of each alarm task is re-ordered to achieve flexible scheduling of each alarm task. Thus, the existing alarm function of the Ambari big data platform can be expanded and improved to provide a better cluster alarm effect.

[0069] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0070] First embodiment

[0071] See also Figure 2, is a flow chart of an alarm method based on the Ambari big data platform provided in this embodiment, the method comprising the following steps:

[0072] S201: In response to a user input operation on an alarm template, an alarm definition is generated, and the alarm definition is written into an alarm file.

[0073] In this embodiment, an alarm template is usually preset in Ambari (for details, see the following Figure 7 The custom alarm module contains the block diagram content) so that users can fill in the custom alarm according to the template. In this way, the Ambari big data platform can respond to the user's input operation on the alarm template (such as filling in or selecting operations, etc.), generate an alarm definition, and write the alarm definition to the alarm file so that Ambari-Server can re-read the alarm definition file and run the alarm task.

[0074] S202: Regularly read the latest running time of each alarm task from the database, and compare it with a preset time threshold, so as to determine whether the running status of the alarm task is normal according to the comparison result.

[0075] In this embodiment, in order to expand and improve the existing alarm function to provide a better cluster alarm effect, it is necessary to read the latest running time of each alarm task from the database regularly, such as Figure 3 As shown, it is compared with the preset time threshold. If the comparison result exceeds the time threshold, a test alarm task is sent. If the test task does not return the result normally, it is determined that the alarm task is running abnormally, that is, it is determined that the running status of the alarm task is abnormal. At this time, it is necessary to update the corresponding alarm task running status in the database as abnormal.

[0076] The preset time threshold refers to a critical value used to determine whether the current state of the alarm task is the latest state. The specific value of the preset time threshold can be determined according to actual conditions and experience, and the embodiment of the present application does not limit this. For example, it can be set to 1 minute or 2 minutes.

[0077] S203: Obtain the running result of at least one alarm task, and push the running result to the user's terminal device through a preset push channel.

[0078] In this embodiment, in order to expand and improve the existing alarm function and provide a better cluster alarm effect, it is necessary to obtain the running result of at least one alarm task and push the running result to the user's terminal device through a preset push channel. Among them, an optional implementation method is that the preset push channel includes but is not limited to SMS, email, mobile phone application, etc.

[0079] An optional implementation method is that the implementation process of this step S203 may include: first obtaining the running result of at least one of the alarm tasks, and selecting at least one push channel according to the preset correspondence between the running result of the alarm task and the push channel; then, using the selected at least one push channel, assembling a push message corresponding to the running result of the alarm task; and then the push message can be pushed to the user's terminal device (such as a mobile phone, tablet computer, etc.).

[0080] Specifically, if Figure 4 As shown, when Ambari-Server receives the running result of the alarm task, it first selects the corresponding push channel according to the running result of the alarm task (the correspondence between the running result of the alarm task and the push channel is pre-configured, and can be selected as email by default, and the user can flexibly configure it, or it can be selected as all users by default, and the number of users can be flexibly configured), and then assembles the push messages to be pushed according to different channels, and finally sends the push messages to the push channels (such as SMS, email, mobile APP, etc.), and supports timely push or delayed push.

[0081] S204: Regularly obtain the error log of the platform operation, and analyze the error log to find out the new alarm definition that is not currently defined.

[0082] In this embodiment, in order to expand and improve the existing alarm function to provide a better cluster alarm effect, it is also necessary to regularly obtain the error log of the platform operation and analyze the new alarm definition that is currently undefined.

[0083] Specifically, if Figure 5 As shown, an optional implementation method is that, first, it is necessary to read logs from Ambari-Server regularly and filter out error logs from them. Then, the error log is input into the pre-built keyword recognition algorithm model (the specific construction process is consistent with the existing method and will not be repeated here), so as to calculate the keyword of the error through the model (the format of the keyword is such as Ambari-Agent / heartbeat timeout, HDFS / disk capacity is full); then, the keyword of the error can be matched with the existing alarm definition to obtain the matching result. When the matching result shows that the existing alarm definition does not contain the keyword of the error, that is, these keywords cannot be found in the existing alarm definition, it can be determined that this is a new alarm definition that is not currently defined. At this time, a new alarm can be generated according to the format of the alarm template and synchronized to AmbariServer. At the same time, the alarm generated by the new plan is not enabled by default, and the user can decide whether to enable it after checking.

[0084] S205: Calculate the priority of each alarm task according to the number of occurrences of each alarm task and the preset level; and re-order the execution order of each alarm task according to the priority order to achieve flexible scheduling of each alarm task.

[0085] In this embodiment, in order to expand and improve the existing alarm function to provide better cluster alarm effect, it is also necessary to calculate the priority of each alarm task based on the number of times each alarm task occurs and the preset level; and according to the priority ranking, the execution order of each alarm task is re-sorted to achieve flexible scheduling of each alarm task.

[0086] Specifically, if Figure 6 As shown in the figure, in order to provide a better cluster alarm effect, it is necessary to regularly query the historical records of each alarm task from the database, count the number of historical occurrences of each alarm task, and determine the service level of each alarm task to calculate the priority of each alarm task. The specific calculation formula is: Alarm task priority = alarm number + service level. The service level defaults to 1. Users can assign different service levels to each alarm task according to the actual situation of the cluster, and then dynamically adjust the execution cycle and priority of the alarm task according to the preset priority matching rules. The matching rules are configurable (example execution rules are as follows: Figure 6 Finally, the execution order of each alarm task after reordering can be synchronously updated to the server management node Ambari-Server of the Ambari big data platform, so that each alarm task can be executed according to the updated execution order to achieve flexible scheduling of each alarm task.

[0087] In this way, by executing the above steps S201-S205, this embodiment adds custom alarms on the first hand, and users can add alarms that are not in the existing alarm definition; on the second hand, monitoring is added to the running status of the alarm, and for alarms that do not promptly feedback the alarm results, a test task is sent to check the running status of the alarm; on the third hand, real-time push of alarms is added, and the latest alarm information is pushed to the user terminal in a timely manner through multiple channels. On the fourth hand, intelligent planning of new alarms is added, and the alarms that are currently undefined and required can be analyzed and inferred by scanning the error log, which makes up for the problems caused by the lack of alarms. On the fifth hand, flexible scheduling of alarms is added, which can increase the execution frequency for important services and high-frequency alarms, and reduce the execution frequency for unimportant services and low-frequency alarms. In this way, the existing alarm function in Ambari is expanded and improved, making it more powerful, more able to meet customer customization needs, and optimizing the execution efficiency of alarms. Intelligent planning of alarms effectively makes up for the system problems that may be caused by the lack of alarms and reduces the risks of system operation.

[0088] In summary, the present embodiment provides an alarm method based on the Ambari big data platform. First, in response to the user's input operation on the alarm template, an alarm definition is generated and written into the alarm file. Then, the latest running time of each alarm task is read from the database regularly, and it is compared with the preset time threshold to determine whether the running status of the alarm task is normal according to the comparison result. Then, the running result of at least one alarm task is obtained, and the running result is pushed to the user's terminal device through the preset push channel. At the same time, the error log of the platform operation is obtained regularly, and the new alarm definition that is currently undefined is analyzed therefrom. Then, the priority of each alarm task is calculated according to the number of occurrences and the preset level of each alarm task. And according to the priority ranking, the execution order of each alarm task is re-sorted to achieve flexible scheduling of each alarm task. In this way, the existing alarm function of the Ambari big data platform can be expanded and improved to provide better cluster alarm effect.

[0089] Second embodiment

[0090] This embodiment will introduce an alarm system based on the Ambari big data platform. For related content, please refer to the above method embodiment.

[0091] See also Figure 7 , is a schematic diagram of the composition of an alarm system 700 based on the Ambari big data platform provided in this embodiment, and the system 700 includes:

[0092] The custom alarm module 701 is used to generate an alarm definition in response to a user's input operation on an alarm template, and write the alarm definition into an alarm file;

[0093] In this way, users can add new alarms that are not included in the definition file based on actual needs.

[0094] The alarm status monitoring module 702 is used to periodically read the latest running time of each alarm task from the database and compare it with a preset time threshold to determine whether the running status of the alarm task is normal according to the comparison result;

[0095] In this way, the running status of the alarm task is monitored. When the alarm runs abnormally, it is displayed through the interface to prompt the abnormality in time.

[0096] The alarm information push module 703 is used to obtain the running result of at least one of the alarm tasks, and push the running result to the user's terminal device through a preset push channel;

[0097] In this way, the latest alarm information can be pushed to the user's terminal display through multiple channels in a timely manner.

[0098] New alarm planning module 704, used to periodically obtain error logs of platform operation and analyze new alarm definitions that are currently undefined;

[0099] In this way, for alarms that may be missing in the existing system, new alarms can be planned through regular scanning and calculation to make up for the deficiencies in the existing system in a timely manner.

[0100] The alarm flexible scheduling module 705 is used to calculate the priority of each alarm task according to the number of times each alarm task occurs and the preset level; and re-sort the execution order of each alarm task according to the priority sorting, so as to realize flexible scheduling of each alarm task.

[0101] In this way, the execution of alarm tasks can be dynamically scheduled according to the importance of component services and the frequency of alarm occurrence. Important services and frequently occurring alarms can be given high priority and high frequency execution, while unimportant services and infrequently occurring alarms can be given low priority and infrequent execution or not executed.

[0102] In an implementation of this embodiment, the device further includes:

[0103] The first update module is used to read the latest running time of each alarm task from the database at the timing, and compare it with a preset time threshold to determine whether the running status of the alarm task is normal based on the comparison result. If it is determined that the running status of the alarm task is abnormal, the running status of the corresponding alarm task in the database is updated as abnormal.

[0104] In one implementation of this embodiment, the alarm information push module 703 includes:

[0105] A selection submodule, used for obtaining the running result of at least one of the alarm tasks, and selecting at least one push channel according to a preset corresponding relationship between the running result of the alarm task and the push channel;

[0106] An assembling submodule, used to assemble a push message corresponding to the operation result by using the at least one push channel;

[0107] The push submodule is used to push the push message to the user's terminal device.

[0108] In one implementation of this embodiment, the new alarm planning module 704 includes:

[0109] The extraction submodule is used to periodically obtain the error log of the platform operation and extract the error keywords from the error log;

[0110] A matching submodule, used to match the keywords of the error report with the existing alarm definition to obtain a matching result;

[0111] The first determination submodule is used to determine a new alarm definition that is not currently defined when the matching result indicates that the keyword of the error is not included in the existing alarm definition.

[0112] In an implementation of this embodiment, the extraction submodule is specifically used for:

[0113] The error log of the platform operation is obtained regularly, and the error log is input into the pre-built keyword recognition algorithm model to calculate the keyword of the error through the model.

[0114] In an implementation of this embodiment, the alarm flexible scheduling module 704 includes:

[0115] A reading submodule, used for reading the historical occurrence number of each alarm task from the database;

[0116] A second determination submodule is used to determine the service level of each alarm task;

[0117] The calculation submodule is used to calculate the sum of the historical occurrence times and the service level of each alarm task, and determine the priority of each alarm task according to the sum.

[0118] In an implementation of this embodiment, the device further includes:

[0119] The second update module is used to reorder the execution order of each alarm task according to the priority sorting to achieve flexible scheduling of each alarm task, and then synchronously update the reordered execution order of each alarm task to the server management node of the Ambari big data platform, so that each alarm task can be executed according to the updated execution order.

[0120] In summary, the alarm system based on the Ambari big data platform provided in this embodiment first responds to the user's input operation on the alarm template, generates an alarm definition, and writes it into the alarm file, then reads the latest running time of each alarm task from the database regularly, and compares it with the preset time threshold, so as to judge whether the running status of the alarm task is normal according to the comparison result, then obtains the running result of at least one alarm task, and pushes the running result to the user's terminal device through the preset push channel; at the same time, regularly obtains the error log of the platform operation, and analyzes the currently undefined new alarm definition from it; then calculates the priority of each alarm task according to the number of occurrences and preset level of each alarm task; and reorders the execution order of each alarm task according to the priority order, so as to realize the flexible scheduling of each alarm task. Thus, the existing alarm function of the Ambari big data platform can be expanded and improved to provide better cluster alarm effect.

[0121] Furthermore, the embodiment of the present application also provides an alarm device based on the Ambari big data platform, including: a processor, a memory, and a system bus;

[0122] The processor and the memory are connected via the system bus;

[0123] The memory is used to store one or more programs, and the one or more programs include instructions, which, when executed by the processor, enable the processor to execute any implementation method of the above-mentioned alarm method based on the Ambari big data platform.

[0124] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a terminal device, the terminal device executes any one of the implementation methods of the above-mentioned alarm method based on the Ambari big data platform.

[0125] It can be known from the description of the above implementation mode that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0126] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0127] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0128] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An alarm method based on the Ambari big data platform, characterized in that: include: In response to a user input operation on an alarm template, an alarm definition is generated, and the alarm definition is written into an alarm file; Read the latest running time of each alarm task from the database regularly, and compare it with the preset time threshold, so as to judge whether the running status of the alarm task is normal according to the comparison result; Obtaining the running result of at least one of the alarm tasks, and pushing the running result to the user's terminal device through a preset push channel; Obtain the error log of the platform operation regularly and analyze the new alarm definitions that are currently undefined; Calculate the priority of each alarm task according to the number of occurrences and the preset level of each alarm task; and re-order the execution order of each alarm task according to the order of the priority to achieve flexible scheduling of each alarm task; The step of comparing the time with a preset time threshold to determine whether the running state of the alarm task is normal according to the comparison result includes: If the latest running time is greater than the preset time threshold, the alarm task is sent; if the alarm task does not return normally, it is determined that the running state of the alarm task is abnormal.

2. The method according to claim 1, characterized in that: After the latest running time of each alarm task is read from the database at a fixed time and compared with the preset time threshold to determine whether the running state of the alarm task is normal according to the comparison result, the method further includes: If it is determined that the running state of the alarm task is abnormal, the running state of the corresponding alarm task in the database is updated as abnormal.

3. The method according to claim 1, characterized in that: The obtaining of the running result of at least one of the alarm tasks and pushing the running result to the user's terminal device through a preset push channel includes: Obtaining the running result of at least one of the alarm tasks, and selecting at least one push channel according to a preset correspondence between the running result of the alarm task and the push channel; Assembling a push message corresponding to the operation result by using the at least one push channel; The push message is pushed to the user's terminal device.

4. The method according to claim 1, characterized in that The method of periodically obtaining the error log of the platform operation and analyzing the error log to find out the new alarm definition which is not currently defined includes: Obtain error logs of platform operation at regular intervals, and extract error keywords from the error logs; Match the keywords of the error report with the existing alarm definition to obtain a matching result; When the matching result indicates that the existing alarm definition does not contain the keyword of the error report, a new alarm definition that is currently undefined is determined.

5. The method according to claim 4, characterized in that The step of regularly obtaining the error log of the platform operation and extracting the error keyword from the error log includes: The error log of the platform operation is obtained regularly, and the error log is input into the pre-built keyword recognition algorithm model to calculate the keyword of the error through the model.

6. The method according to claim 1, characterized in that The calculating the priority of each alarm task according to the number of occurrences of each alarm task and the preset level includes: Read the historical occurrence number of each alarm task from the database; Determine the service level for each of the alarm tasks described; The sum of the historical occurrence times and the service level of each alarm task is calculated, and the priority of each alarm task is determined according to the sum.

7. The method according to claim 1, characterized in that After reordering the execution order of each alarm task according to the priority order to achieve flexible scheduling of each alarm task, the method further includes: The reordered execution order of each alarm task is synchronously updated to the server management node of the Ambari big data platform so that each alarm task is executed according to the updated execution order.

8. An alarm system based on the Ambari big data platform, characterized in that: include: A custom alarm module, used to generate an alarm definition in response to a user's input operation on an alarm template, and write the alarm definition into an alarm file; The alarm status monitoring module is used to periodically read the latest running time of each alarm task from the database and compare it with a preset time threshold to determine whether the running status of the alarm task is normal based on the comparison result; An alarm information push module, used to obtain the running result of at least one of the alarm tasks, and push the running result to the user's terminal device through a preset push channel; New alarm planning module, used to periodically obtain error logs of platform operation and analyze new alarm definitions that are currently undefined; The alarm flexible scheduling module is used to calculate the priority of each alarm task according to the number of occurrences and the preset level of each alarm task; and re-order the execution order of each alarm task according to the order of the priority to achieve flexible scheduling of each alarm task; The alarm status monitoring module is specifically used to send the alarm task if the latest running time is greater than the preset time threshold, and to determine that the running status of the alarm task is abnormal if the alarm task does not return normally.

9. An alarm device based on the Ambari big data platform, characterized in that: include: Processor, memory, system bus; The processor and the memory are connected via the system bus; The memory is used to store one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by the processor, the processor executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the method according to any one of claims 1 to 7.

Images