Monitoring and warning method and system for batch processing task timeout

By adopting distributed locking and competition mechanism methods in batch tasks, we ensure that only one node performs monitoring and alarm tasks in each cycle. Combined with the redundant compensation mechanism, the problems of single-point failure and missing alarms in the existing technology are solved. , improves the reliability and scalability of the system.

CN119088634APending Publication Date: 2024-12-06CHINA CITIC BANK CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411023439.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing batch task monitoring and alarm methods pose a single point of failure risk, it is difficult to support customized scheduling and execution requirements, it is impossible to effectively monitor and alert abnormal situations such as crashes or dead cycles in the system, and it is easy to miss one-time monitoring and alarms.

Method used

Using distributed locking and competition mechanisms to deploy monitoring and alarm programs at cluster nodes, and implement distributed locking through databases to ensure that only one node performs monitoring and alarm tasks in each cycle, decouple the monitoring process and alarm process, and add a redundant compensation mechanism. To reduce the risk of missed monitoring and missed alarms.

Benefits of technology

It solves the problem of single-point deployment failure, prevents repeated alarms, and realizes the decoupling of the monitoring process and the alarm process, reduces the risk of missing monitoring and alarms, and improves the reliability and scalability of the system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088634A_ABST
    Figure CN119088634A_ABST
Patent Text Reader

Abstract

The invention relates to a batch processing task timeout monitoring and warning method and system. The method comprises the steps that a monitoring and warning program is deployed at a cluster node, a distributed lock is achieved through a database, a polling task is started for a distributed node obtained through successful competition, and a monitoring and warning process is entered; obtaining a first task from a timing batch monitoring alarm configuration table, and inserting the first task into a monitoring operation table or a delay monitoring table; acquiring a second task of which the current day state of the monitoring operation table is initialized, acquiring a second abnormal task, and inserting a record of the second abnormal task into an alarm operation table; and acquiring a third task with an initialized state from the alarm operation table, sending alarm information, and changing the state in the alarm operation table into an alarmed state. A distributed lock is utilized to ensure that only one node executes a monitoring alarm task in a period, repeated alarm is prevented, decoupling of a monitoring process and an alarm process is realized by designing different tables, and the risk of monitoring alarm omission and error is greatly reduced by adding redundancy compensation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of batch processing tasks, and in particular to a batch processing task timeout monitoring and alarm method and system. Background Art

[0002] Batch processing tasks are computationally intensive tasks that occupy a large amount of CPU and memory resources for a long time. If the batch processing exceeds the expected running time, it may be due to the following reasons: failure of the communication link between the scheduling node and the execution node, exhaustion of computing resources of the execution node, problems with the quality of the data source or batch processing program, etc. The above anomalies will affect the efficiency of data acquisition in the downstream business, and in serious cases may infect the entire system. Therefore, batch processing monitoring and alarming are crucial. When the batch processing task times out, the operation and maintenance personnel are notified in time for manual intervention.

[0003] The current batch processing monitoring and alarm methods mainly include the following: Trigger mode, monitoring and alarm methods based on batch processing task errors. If there is a task error, the error log is captured by the log center, and the operation and maintenance personnel can know the batch processing anomaly; Polling mode: On the node where the database is located, the SQL check is executed regularly. If there is any non-conformity, the alarm log is output and captured by the log center, and the operation and maintenance personnel can know the batch processing anomaly; Scheduling verification program: pre-embed the scheduling configuration information to be verified, and the program obtains the configuration and verifies it in time, and outputs reminders and alarms for verification success or failure; Create a listener for each batch processing task, and monitor its status by scanning the scheduled task execution table through the listener; Supplementary means: Manually observe the CPU and memory usage of the process to find faulty batch processing tasks.

[0004] Most existing monitoring methods are deployed on a single machine, which poses a risk of single point failure. Most existing monitoring methods are one-time monitoring and one-time alarming. If an unpredictable exception happens to occur when the monitoring task is executed, the one-time monitoring and alarming will be missed. Database script monitoring requires the configuration of the entire SQL, which places high demands on development and maintenance personnel. The configuration is cumbersome and it is difficult to meet customized scheduling and execution requirements. It is difficult to support customization of fixed alarm time. The monitoring method of batch task log error keywords cannot support situations where there is no way to capture error logs, such as system crashes and dead loops. Summary of the invention

[0005] To solve the problems of the prior art, the present invention uses distributed locks and competition mechanisms to ensure that only one node executes the monitoring and alarm tasks within a cycle to prevent repeated alarms; decouples the monitoring process from the alarm process by designing different tables to facilitate expansion; and adds a redundant compensation mechanism to reduce the risk of errors and omissions.

[0006] To achieve the above objectives, the technical solutions adopted by the present invention include:

[0007] A batch task timeout monitoring and alarm method, characterized by comprising:

[0008] S1. Deploy monitoring and alarm programs on cluster nodes, use the database to implement distributed locks, start polling tasks for successfully competed distributed nodes, and enter the monitoring and alarm process;

[0009] S2. Get the first task from the scheduled batch monitoring alarm configuration table, and insert the first task into the monitoring operation table or the delay monitoring table;

[0010] S3. Get the second task whose status is initialized on the day of the monitoring operation table, get the second abnormal task, and insert the record of the second abnormal task into the alarm operation table;

[0011] S4. Obtain the third task with the status of initialization from the alarm operation table, send alarm information, and change the status in the alarm operation table to alarmed.

[0012] Further, the obtaining of the first task from the timing batch monitoring alarm configuration table and inserting the first task into the monitoring operation table or the delay monitoring table includes:

[0013] Get the current system time;

[0014] Obtain the first task whose task start time or task end time is within the current polling coverage period from the scheduled batch monitoring alarm configuration table;

[0015] Traversing the first task;

[0016] Calculating a scheduling date for the first task;

[0017] Verify whether the first task should be called up on the day according to the scheduling configuration. If the verification fails, continue to traverse the first task; if the verification passes, calculate the expected execution date of the first task and insert it into the monitoring operation table or the delayed monitoring table according to whether the execution is postponed.

[0018] Furthermore, the expected execution date includes: daily execution and execution postponed to working days during holidays; the calculation of the expected execution date of the first task, and inserting the monitoring operation table or the delay monitoring table according to whether the execution is postponed, includes:

[0019] If the first task is to be executed daily, inserting the first task into the monitoring operation table and setting the monitoring state to initialization;

[0020] If the first task is postponed to a working day due to a holiday, if the current day is a holiday, the first task will be inserted into the delay monitoring table and the monitoring status will be set to delayed monitoring; if the current day is a working day, the first task will be inserted into the monitoring operation table and the monitoring status will be set to initialization.

[0021] Furthermore, the method also includes: monitoring the first task in a redundant compensation manner, traversing all tasks in the scheduled batch monitoring alarm configuration table before the current polling period of the day, traversing all tasks in the scheduled batch monitoring alarm configuration table of the previous day, querying the monitoring operation table and the delay monitoring table, and inserting unrecorded tasks into the monitoring operation table and the delay monitoring table according to the scheduling configuration.

[0022] Further, the step of obtaining a second task whose status on the day in the monitoring operation table is initialized, obtaining a second abnormal task, and inserting a record of the second abnormal task into the alarm operation table includes:

[0023] Get the current system time;

[0024] Obtain the second task whose status is initialized on the day in the monitoring operation table, query whether the second task and its batch information have records in the batch operation table, mark the second task with no records or with records and abnormal batch status as the second abnormal task, and mark the second tasks in other situations as the second normal tasks;

[0025] Calculate the expected alarm time of the second abnormal task according to the alarm configuration, insert the record of the second abnormal task into the alarm operation table, mark the state of the alarm operation table as initialized, and change the state of the monitoring operation table to task abnormality, registered;

[0026] The calculating the expected alarm time of the second abnormal task according to the alarm configuration includes: if the alarm configuration is urgent, obtaining the current system time as the expected alarm time; if the alarm configuration is non-urgent, if the current system time is working time, using the current system time as the expected alarm time, otherwise obtaining the most recent working time as the expected alarm time;

[0027] Change the status of the second normal task in the monitoring operation table to normal task, no alarm required.

[0028] Furthermore, the batch processing status exception can be determined as a batch processing status exception hit success when any of the following conditions occurs, including:

[0029] This record in the monitoring operation table monitors the task start monitoring, and the task status is not started;

[0030] This record in the monitoring operation table monitors task completion monitoring, and the task status is any status other than completed.

[0031] The present invention also relates to a batch processing task timeout monitoring and alarm system, which is characterized by comprising:

[0032] The startup module is used to deploy monitoring and alarm programs on cluster nodes, use the database to implement distributed locks, start polling tasks for successfully competed distributed nodes, and enter the monitoring and alarm process;

[0033] The pre-monitoring and delay monitoring module is used to obtain the first task from the timed batch monitoring alarm configuration table and insert the first task into the monitoring operation table or the delay monitoring table;

[0034] A monitoring and pre-alarm module, used for obtaining a second task with a status of initialization on the day in the monitoring operation table, obtaining a second abnormal task, and inserting a record of the second abnormal task into the alarm operation table;

[0035] The alarm module is used to obtain the third task with the status of initialization from the alarm operation table, send alarm information, and change the status in the alarm operation table to alarmed.

[0036] The present invention also relates to a computer-readable storage medium, characterized in that a computer program is stored on the storage medium, and the computer program implements the above method when executed by a processor.

[0037] The present invention also relates to an electronic device, characterized in that it comprises a processor and a memory;

[0038] The memory is used to store program instructions;

[0039] The processor is used to execute the above method by calling program instructions.

[0040] The present invention also relates to a computer program product, comprising a computer program and / or instructions, characterized in that the computer program and / or instructions implement the steps of the above method when executed by a processor.

[0041] The beneficial effects of the present invention are:

[0042] The batch task timeout monitoring and alarm method and system described in the present invention adopts cluster redundancy to solve the single-point deployment failure problem of deployment. The distributed lock and competition mechanism determines that only one node executes the monitoring and alarm task within a cycle to prevent multiple repeated alarms. By recording the monitoring status and the alarm status separately, the monitoring process and the alarm process are decoupled, the process is clear, and it is easy to expand. The addition of redundancy compensation greatly reduces the risks of missed monitoring, missed alarms, wrong monitoring, and wrong alarms. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flow chart of a batch task timeout monitoring and alarm method according to the present invention;

[0044] Figure 2 A schematic diagram of the structure of a batch processing task timeout monitoring and alarm system of the present invention;

[0045] Figure 3 A schematic diagram of the deployment of the monitoring alarm program of the present invention;

[0046] Figure 4 The present invention is a flowchart of an embodiment of a method for monitoring a database using a monitoring alarm program. DETAILED DESCRIPTION

[0047] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present invention.

[0048] Those skilled in the art can understand that, unless otherwise stated, the singular forms "one", "an", "said" and "the" used herein may also include plural forms, and the "first", "second", etc., are only used to distinguish and define objects for the purpose of introducing the scheme clearly, and do not limit the objects themselves. Of course, the objects defined by "first" and "second" may be the same terminal, device, user, etc., or the same type of terminal, device and user. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0049] In addition, it should be understood that in the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c, or a, b and c, where a, b, c can be single or multiple.

[0050] In order to make the objectives, technical solutions and advantages of the present application more clear, a detailed description will be given below with reference to the accompanying drawings and embodiments.

[0051] The following specific embodiments are used to describe in detail the technical solution of the present application and how the technical solution of the present application solves the above technical problems. The following embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0052] The first aspect of the present invention relates to a process of steps as follows Figure 1 A batch task timeout monitoring and alarm method is shown, comprising:

[0053] Step S1. Deploy a monitoring and alarm program on the cluster nodes, use the database to implement distributed locks, start a polling task for the distributed nodes that have successfully competed, and enter the monitoring and alarm process.

[0054] In an embodiment of the present invention, the monitoring alarm program is periodically called up by a polling task, which is repeated repeatedly. The frequency of the polling task is set according to specific needs. The monitoring alarm steps will be executed in the same polling cycle, including: pre-monitoring and delayed monitoring, monitoring and pre-alarm, and alarm.

[0055] It supports clustered deployment and uses database to implement distributed locks, which can prevent multiple nodes from starting the same polling task at the same time, ensuring that each task will only be executed by one node, avoiding repeated processing or node competition for the execution right of the monitoring alarm; within a cycle, multiple nodes compete for monitoring to avoid the problem of single-point suspension and inability to complete the monitoring task within the cycle. If the node that successfully competes does not complete the established monitoring alarm task, the polling time can be set, for example, setting a new node to check and complete the monitoring alarm task after 10 minutes.

[0056] The present application may also use a timed task framework (such as Quartz) to ensure that tasks are triggered according to predetermined time intervals or rules.

[0057] Step S2: Obtain a first task from the scheduled batch monitoring alarm configuration table, and insert the first task into the monitoring operation table or the delay monitoring table.

[0058] Specifically, step S2 includes: step S201, step S202, step S203, step S204, step S205:

[0059] Step S201. Get the current system time in the format of year, month, day, week, hour, minute, and second;

[0060] Step S202. Obtain the first task whose task start time or task end time is in the current polling coverage period from the scheduled batch monitoring alarm configuration table;

[0061] Step S203. Traverse the first task;

[0062] Step S204. Calculate the scheduling date of the first task, and calculate the batch date actually scheduled on that day according to the current system time plus or minus the daily difference;

[0063] Step S205. Verify whether the first task should be called up on the day according to the scheduling configuration. If the verification fails, there is no need to monitor the task and execute step S203. If the verification passes, calculate the expected execution date of the first task and insert it into the monitoring operation table or the delayed monitoring table according to whether the execution is postponed.

[0064] In this embodiment, the start monitoring and completion monitoring of the batch task are two independent monitorings, which can help the operation and maintenance personnel to accurately determine whether the fault is not started or not completed.

[0065] Specifically, step S205 also includes: the expected execution date includes: daily execution and holiday postponement to workday execution;

[0066] If the first task is to be executed daily, inserting the first task into the monitoring operation table and setting the monitoring state to initialization;

[0067] If the first task is postponed to a working day due to a holiday, if the current day is a holiday, the first task will be inserted into the delay monitoring table and the monitoring status will be set to delayed monitoring; if the current day is a working day, the first task will be inserted into the monitoring operation table and the monitoring status will be set to initialization.

[0068] The first task has other execution delays besides holiday delays. Customization is implemented by inserting the tasks accumulated in the delay monitoring table into the monitoring operation table, and the customized requirements are realized by linking with the holiday calendar table parameters.

[0069] In an embodiment of the present invention, tasks are executed according to the expected execution date, including: daily execution, working day execution, trading day execution, non-working day execution, monthly designated day execution, weekly designated day execution, designated date execution, etc.

[0070] In the embodiment of the present invention, for other execution extensions other than holiday extensions, due to certain force majeure factors, the execution can be configured to be temporarily postponed to a specific day.

[0071] Preferably, a redundancy compensation mechanism is added in step S2. Traverse all the tasks in the scheduled batch monitoring alarm configuration table before the current polling period on the day, and use the method of step S204 and step S205 to calculate whether they are called up on the day. If so, query whether there are monitoring records in the monitoring operation table and the delay monitoring table. If there are no records, insert the monitoring operation table or the delay monitoring table according to the specific method of S205, and set the state to initialization or delayed monitoring. This redundancy compensation is to compensate for the situation that other periods of the day cannot be normally inserted into the monitoring queue due to unpredictable reasons, resulting in the inability to perform monitoring. Multiple redundancies are used to ensure that all previous tasks can be inserted into the monitoring operation table or the delay monitoring table.

[0072] Additionally, a special redundancy compensation mechanism is added in the present invention. In order to ensure that the last few cycles of the day have sufficient redundancy compensation, the redundancy compensation of the previous day should be considered after the day cut, that is, the first five monitoring cycles of each day should check whether the scheduled batch tasks of the previous day are correctly inserted into the monitoring operation table or the delayed monitoring table. If there is a problem, the insertion process is performed according to the redundancy compensation mechanism added in the above step S2.

[0073] Step S2 of the present invention also includes: processing delayed monitoring due tasks. Specifically, for holiday delayed tasks, first determine whether the current day is the first working day after the holiday. If so, insert the first task with a monitoring status of delayed monitoring into the monitoring operation table, set the monitoring status to initialized, and change the status of the delayed monitoring table to have been taken out of the monitoring operation table.

[0074] In the specific implementation process of the present invention, a redundancy compensation mechanism and a special redundancy compensation mechanism are added to handle the expiration of delayed monitoring tasks during holidays.

[0075] If there are other execution delays besides holiday delays, they should be inserted into the monitoring operation table, and a redundant compensation mechanism should be added to ensure that the accumulated tasks are moved from the delay monitoring table to the monitoring operation table on the day they can be executed. When configuring a temporary execution delay to a specific day, mark the specific execution day in the calendar table in advance, and on the specific execution day, insert the accumulated delay monitoring uniformly into the monitoring operation table.

[0076] The above steps of the present invention insert the monitored scheduled batch tasks into the monitoring operation table and insert the scheduled batch tasks that are scheduled but not executed on the same day into the delayed monitoring table, thereby realizing the pre-monitoring and delayed monitoring of the scheduled batch tasks.

[0077] Additionally, the present invention provides a pre-monitoring and redundancy compensation method for online batch tasks: obtain the tasks whose start and end times are within the current polling coverage period in the online batch monitoring alarm configuration table, insert the tasks into the monitoring operation table, and if there is no record, the execution of this monitoring process ends; the redundancy compensation mechanism for online batch pre-monitoring is to traverse all the tasks in the online batch monitoring alarm configuration table before the current period on the same day. Since the online batch does not consider the situation where the trigger cycle and the actual execution cycle are different, there is no need to consider temporary storage and accumulation in the delay monitoring table.

[0078] Step S3. Obtain the second task whose status on the day is initialized in the monitoring operation table, obtain the second abnormal task, and insert the record of the second abnormal task into the alarm operation table.

[0079] Specifically, step S3 includes: step S301, step S302, step S303, step S304, step S305:

[0080] Step S301. Get the current system time in the format of year, month, day, week, hour, minute, and second;

[0081] Step S302. Obtain the second task whose status is initialized on the day in the monitoring operation table, query whether the second task and its batch information are recorded in the batch operation table, mark the second task with no record or with record and abnormal batch status as the second abnormal task, and mark the second task in other cases as the second normal task;

[0082] Step S303. Calculate the expected alarm time of the second abnormal task according to the alarm configuration, insert the record of the second abnormal task into the alarm operation table, mark the alarm operation table status as initialized, and change the monitoring operation table status to task abnormality, registered;

[0083] Step S304. The expected alarm time of the second abnormal task is calculated according to the alarm configuration, including: if the alarm configuration is urgent, obtaining the current system time as the expected alarm time; if the alarm configuration is non-urgent, if the current system time is working time, using the current system time as the expected alarm time, otherwise obtaining the most recent working time as the expected alarm time;

[0084] Step S305: Change the status of the second normal task in the monitoring operation table to a normal task, no alarm required.

[0085] In the embodiment of the present invention, the tasks and batch information of the tasks with the status of initialization on the current day are obtained, and the batch operation table is searched for records of the batch tasks. The batch operation table is usually used to record the execution status of the batch tasks, including the task start time, end time, status and other information. If there is a record, it means that the task has been executed, and if there is no record, it means that the task has not been executed or recorded.

[0086] The task status usually includes: not started, in progress, completed, error, etc. The batch status exception hit is successful when any of the following conditions occurs, including:

[0087] Case 1: In the monitoring operation table, this record is monitored as task start, but the actual status of the task is not started;

[0088] Case 2: This record in the monitoring operation table monitors that the task is completed, but the actual status of the task is any status other than completed.

[0089] Step S4: Obtain the third task with the status of initialization from the alarm operation table, send alarm information, and change the status in the alarm operation table to alarmed.

[0090] Preferably, since the actual alarm time may have a certain delay relative to the monitoring time, in order to improve the experience, before the alarm is executed, it is necessary to query again whether the task batch to be alarmed has been called up / executed at the time of the alarm. If so, a prompt alarm is initiated: delayed calling up / delayed completion; otherwise, an alarm is initiated: not called up yet / not completed yet.

[0091] In the embodiment of the present invention, an alarm is initiated according to the alarm mode preset by the system, including active notification and passive query. Active notification refers to sending a message to the alarm platform after framing; passive query refers to printing an alarm log with keywords, which is captured by the keyword screening of the log center, etc.

[0092] Preferably, when the number of accumulated alarms exceeds the threshold, the alarm function is initiated in advance. Set the threshold for the number of non-urgent alarms in advance, and check the number of non-urgent alarms in the alarm operation table each time the alarm process is executed. If it exceeds the threshold, all alarms are output immediately, and the alarm record status in the alarm operation table is set to alarmed.

[0093] In addition, for other information that requires periodic inspection and alarm, an alarm is initiated according to the system preset alarm method. Other information that requires periodic inspection and alarm includes but is not limited to the current file handle number threshold alarm, database connection pool threshold alarm, thread pool threshold alarm, CPU usage threshold alarm, storage space usage threshold alarm and other resource usage.

[0094] The application can track the progress and execution results of tasks through a state management framework, and can use the state fields in a database table to represent different states of tasks. The state management framework helps manage and monitor tasks.

[0095] The present application can also utilize logging and message queues (such as Kafka) to achieve information capture and notification, and the information may include task status changes, alarm information, etc.

[0096] The present invention performs customized demand analysis on monitoring and alarm respectively, and realizes separate control of "monitoring" and "alarm". In the banking and financial scenario, monitoring alarm is a layer of protection for the system operation status. Monitoring alarm can ensure the reliability of the system, and support monitoring today to find a fault and alarm tomorrow; one monitoring and multiple alarms; multiple monitoring of multiple problems and alarms together, etc.

[0097] Another aspect of the present invention also relates to a batch processing task timeout monitoring and alarm system, the structure of which is as follows: Figure 2 As shown, including:

[0098] The startup module is used to deploy monitoring and alarm programs on cluster nodes, use the database to implement distributed locks, start polling tasks for successfully competed distributed nodes, and enter the monitoring and alarm process;

[0099] The pre-monitoring and delay monitoring module is used to obtain the first task from the timed batch monitoring alarm configuration table and insert the first task into the monitoring operation table or the delay monitoring table;

[0100] A monitoring and pre-alarm module, used for obtaining a second task with a status of initialization on the day in the monitoring operation table, obtaining a second abnormal task, and inserting a record of the second abnormal task into the alarm operation table;

[0101] The alarm module is used to obtain the third task with the status of initialization from the alarm operation table, send alarm information, and change the status in the alarm operation table to alarmed.

[0102] By using this system, the above-mentioned operation processing method can be executed and the corresponding technical effects can be achieved.

[0103] The present invention also relates to Figure 3 The monitoring and alarm program deployment shown in the figure can support both single-point deployment and cluster deployment, and is usually deployed together with a batch execution application as a functional module. The monitoring and alarm program deployed on all nodes in the cluster is consistent with the alarm configuration, and deployment nodes can be easily added or deleted to achieve expansion or reduction.

[0104] The scheduled batch monitoring alarm configuration table, online batch monitoring alarm configuration table, monitoring operation table, delay monitoring table, and alarm operation table involved in the present invention are all stored in a database, usually sharing the database with the scheduling configuration, and can be accessed by any module deployed in the cluster.

[0105] The present invention also relates to Figure 4 The flowchart of the embodiment method of using the monitoring alarm program to monitor the database is shown. The monitoring alarm program described in the present invention is executed in a timed manner. Since the timed scheduling request of the scheduling node cannot ensure that it can reach the execution node, the monitoring alarm program is called up by each node in the batch execution in a timed polling manner. The execution frequency of all nodes is consistent. For example, each node self-schedules the monitoring alarm program once every 15 minutes. Each node deployed in the cluster can independently complete all the functions of monitoring and alarming. However, it is necessary to ensure the accuracy of monitoring and alarming, and also to ensure that one problem cannot be alarmed multiple times, that is, each node in the cluster competes for the execution right of this monitoring and alarming. Therefore, a database library table is used to implement a distributed lock, and the node successfully inserted into the library table continues to execute the monitoring alarm, and the remaining nodes end the polling execution this time.

[0106] The traditional database monitoring script is deployed on the machine where the database is located. As a supplementary means of monitoring and alarming, it monitors the number of data in the distributed lock table at the same frequency. If the number of monitored items each time is 1 plus the number of monitored items last time, it means that at least one node has executed the monitoring and alarming program in this cycle, otherwise the alarm is an error. This further ensures the stable operation of the monitoring and alarming program.

[0107] An embodiment of the present invention also provides a computer-readable storage medium capable of implementing all the steps in the method in the above embodiment, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, all the steps in the method in the above embodiment are implemented.

[0108] An embodiment of the present invention also provides an electronic device for executing the above method. As an implementation device of the method, the electronic device has at least a processor and a memory, and in particular, the memory stores data and related computer programs required for executing the method, such as program instructions, etc., and the processor calls the data and program in the memory to execute all steps of the implementation method and obtains the corresponding technical effect.

[0109] Preferably, the electronic device may include a bus architecture, the bus may include any number of interconnected buses and bridges, and the bus will include various circuits linked together by one or more processors and memories. The bus may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are all well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the receiver and transmitter. The receiver and transmitter may be the same component, i.e., a transceiver, providing a unit for communicating with various other systems over a transmission medium. The processor is responsible for managing the bus and general processing, while the memory may be used to store data used by the processor when performing operations.

[0110] In addition, the electronic device may further include components such as a communication module, an input unit, an audio processor, a display, and a power supply. The processor (or controller, operation control) used may include a microprocessor or other processor device and / or a logic device, which receives input and controls the operation of various components of the electronic device; the memory may be one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory, or other suitable devices, which may store the above-mentioned related data information, and may also store a program for executing related information, and the processor may execute the program stored in the memory to realize information storage or processing, etc.; the input unit is used to provide input to the processor, for example, it may be a key or a touch input device; the power supply is used to provide power to the electronic device; the display is used to display display objects such as images and text, for example, it may be an LCD display. The communication module is a transmitter / receiver that sends and receives signals via an antenna. The communication module (transmitter / receiver) is coupled to the processor to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal. Based on different communication technologies, multiple communication modules may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless local area network module, etc. The communication module (transmitter / receiver) is also coupled to a speaker and a microphone via an audio processor to provide an audio output via the speaker and receive an audio input from the microphone, thereby realizing a common telecommunication function. The audio processor may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor is also coupled to a central processor, so that recording can be performed on the machine through the microphone, and the sound stored on the machine can be played through the speaker.

[0111] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0112] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or multiple boxes.

[0113] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction system, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0114] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps of the functions specified in a box or multiple boxes. Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they know the basic creative concept. Therefore, the attached claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0115] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A batch task timeout monitoring and alarm method, characterized in that: include: S1. Deploy monitoring and alarm programs on cluster nodes, use the database to implement distributed locks, start polling tasks for successfully competed distributed nodes, and enter the monitoring and alarm process; S2. Get the first task from the scheduled batch monitoring alarm configuration table, and insert the first task into the monitoring operation table or the delay monitoring table; S3. Get the second task whose status is initialized on the day of the monitoring operation table, get the second abnormal task, and insert the record of the second abnormal task into the alarm operation table; S4. Obtain the third task with the status of initialization from the alarm operation table, send alarm information, and change the status in the alarm operation table to alarmed.

2. The method according to claim 1, characterized in that The obtaining of the first task from the timing batch monitoring alarm configuration table and inserting the first task into the monitoring operation table or the delay monitoring table includes: Get the current system time; Obtain the first task whose task start time or task end time is within the current polling coverage period from the scheduled batch monitoring alarm configuration table; Traversing the first task; Calculating a scheduling date for the first task; Verify whether the first task should be called up on the day according to the scheduling configuration. If the verification fails, continue to traverse the first task; if the verification passes, calculate the expected execution date of the first task and insert it into the monitoring operation table or the delayed monitoring table according to whether the execution is postponed.

3. The method according to claim 2, characterized in that The expected execution date includes: daily execution and execution postponed to working days during holidays; the calculation of the expected execution date of the first task and inserting the monitoring operation table or the delayed monitoring table according to whether the execution is postponed include: If the first task is to be executed daily, inserting the first task into the monitoring operation table and setting the monitoring state to initialization; If the first task is postponed to a working day due to a holiday, if the current day is a holiday, the first task will be inserted into the delay monitoring table and the monitoring status will be set to delayed monitoring; if the current day is a working day, the first task will be inserted into the monitoring operation table and the monitoring status will be set to initialization.

4. The method according to claims 1 to 3, characterized in that: The method also includes: monitoring the first task in a redundant compensation manner, traversing the tasks in all scheduled batch monitoring alarm configuration tables of the day before the current polling period, traversing the tasks in all scheduled batch monitoring alarm configuration tables of the previous day, querying the monitoring operation table and the delay monitoring table, and inserting unrecorded tasks into the monitoring operation table and the delay monitoring table according to the scheduling configuration.

5. The method according to claim 1, characterized in that The step of obtaining a second task whose status on the day in the monitoring operation table is initialized, obtaining a second abnormal task, and inserting a record of the second abnormal task into the alarm operation table includes: Get the current system time; Obtain the second task whose status is initialized on the day in the monitoring operation table, query whether the second task and its batch information have records in the batch operation table, mark the second task with no records or with records and abnormal batch status as the second abnormal task, and mark the second tasks in other situations as the second normal tasks; Calculate the expected alarm time of the second abnormal task according to the alarm configuration, insert the record of the second abnormal task into the alarm operation table, mark the state of the alarm operation table as initialized, and change the state of the monitoring operation table to task abnormality, registered; The calculating the expected alarm time of the second abnormal task according to the alarm configuration includes: if the alarm configuration is urgent, obtaining the current system time as the expected alarm time; if the alarm configuration is non-urgent, if the current system time is working time, using the current system time as the expected alarm time, otherwise obtaining the most recent working time as the expected alarm time; Change the status of the second normal task in the monitoring operation table to normal task, no alarm required.

6. The method according to claim 5, characterized in that The batch processing status exception is any of the following situations, which can be used to determine that the batch processing status exception hit is successful, including: This record in the monitoring operation table monitors the task start monitoring, and the task status is not started; This record in the monitoring operation table monitors task completion monitoring, and the task status is any status other than completed.

7. A batch task timeout monitoring and alarm system, characterized in that: include: The startup module is used to deploy monitoring and alarm programs on cluster nodes, use the database to implement distributed locks, start polling tasks for successfully competed distributed nodes, and enter the monitoring and alarm process; The pre-monitoring and delay monitoring module is used to obtain the first task from the timed batch monitoring alarm configuration table and insert the first task into the monitoring operation table or the delay monitoring table; A monitoring and pre-alarm module, used for obtaining a second task with a status of initialization on the day in the monitoring operation table, obtaining a second abnormal task, and inserting a record of the second abnormal task into the alarm operation table; The alarm module is used to obtain the third task with the status of initialization from the alarm operation table, send alarm information, and change the status in the alarm operation table to alarmed.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store program instructions; The processor is configured to execute the method according to any one of claims 1 to 6 by calling program instructions.

10. A computer program product comprising a computer program and / or instructions, characterized in that: When the computer program and / or the instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.