A min-heap based task timeout monitoring system and method

Abstract

The present application relates to the technical field of task monitoring, and discloses a task timeout monitoring system and method based on a minimum heap, which comprises a task monitor module and a member module connected with the task monitor module, wherein the task monitor module is internally provided with a timer instance and a member queue loaded with information of each running task, and the task monitor module comprises the member module as the timer instance and for realizing a minimum quad heap. The present application can ensure accurate timeout monitoring during high-density task operation, and can realize the minimum quad heap based on the queue and the timer provided by the programming language sdk, so as to perform timing inspection on the top member of the heap and monitor the timeout of all task operations.

Description

Technical Field

[0001] This invention relates to the field of task monitoring technology, specifically a task timeout monitoring system and method based on a min-heap. Background Technology

[0002] With the rapid development of technologies such as big data, artificial intelligence, and blockchain, and the accompanying surge in various business-related computing tasks, these tasks consume various hardware and software resources for a period of time. Therefore, each task needs to define its resource usage and timeout before execution. Upon reaching the timeout, the task is forcibly terminated and the occupied resources are released. Traditionally, at the start of a task's execution, it is assembled into a container such as a set (list / set) or a dictionary (map / dict). Then, at very short intervals (e.g., 1 second), the timeout values ​​of each task in the container are checked to determine if the task has reached its timeout and forcibly terminated.

[0003] The above approach involves periodically checking all running tasks at regular intervals and verifying their timeouts one by one. This approach has the following drawbacks:

[0004] 1. It is impossible to scientifically provide a reasonable check interval. If the interval is too small, it will increase the additional consumption of hardware and software resources. If the interval is too large, it will not be able to accurately cover tasks with timeouts shorter than the check interval.

[0005] 2. Full traversal of all running tasks takes more time as there are more tasks. This can result in the next check interval arriving before the current check is finished, or it may fail to accurately cover tasks with short timeouts (the task's timeout has already expired when it is reached).

[0006] 3. Tasks that have been running recently may not be checked in time and will need to wait until the next check interval.

[0007] 4. It is inconvenient to manage tasks and operations during operation, etc.

[0008] In summary, to address the existing problems, we propose a task timeout monitoring system and method based on a min-heap. Summary of the Invention

[0009] The purpose of this invention is to provide a task timeout monitoring system and method based on a min-heap. The task monitor is implemented using a quad-heap based on a queue and a timer. The next most recently timed-out task information is placed at the top of the heap and the timer is refreshed to that time. When the timer expires, only the task information at the top of the heap is checked, and the min-heap sorting is automatically adjusted. This solves the problems of inaccurate monitoring time and excessively long time consumption for each check caused by task operation based on global traversal.

[0010] To achieve the above objectives, the present invention provides the following technical solution: a task timeout monitoring system based on a min-heap, comprising: a task monitor module and a member module connected to the task monitor module, wherein the task monitor module has a built-in timer instance from the programming language SDK and a member queue containing information on each running task, and the task monitor module includes a member module as a timer instance and implementing a min-heap, a private function module for implementing private function computation, and an external function module for implementing external function computation.

[0011] In a preferred embodiment of the present invention, the member sub-modules of the task monitor module include:

[0012] The timer private member module is a timer instance, which is activated when the target time is reached after the target time is set.

[0013] The memberQueue private member module is a queue composed of members that encapsulate task and job information, and is used to implement a minimum quad heap.

[0014] In a preferred embodiment of the present invention, the private function calculation module includes:

[0015] The delMemberWithIndex private function unit is used to delete the member at the specified index from the memberQueue.

[0016] delMember0 is a private function unit used to remove the top member from the memberQueue.

[0017] The adjustMembers private function unit is used to adjust the min-quad heap.

[0018] In a preferred embodiment of the present invention, the external function module includes:

[0019] The AddMember external function unit is used to add new members to the memberQueue. This function refreshes the target timer set by the monitor and calls the adjustMembers function to readjust the min-heap.

[0020] RemoveMember is an external function unit used to remove a specified member from the memberQueue. This function refreshes the target timer set by the monitor and calls the adjustMembers function to readjust the min-heap.

[0021] The CheckMembers function unit is used to check whether the members defined by the minimum quadruple heap of memberQueue have reached their timeout period. This function needs to be called once when the program starts, and subsequently a daemon thread will monitor the timeout period of the top member of the heap through the private member timer of monitor.

[0022] In a preferred embodiment of the present invention, the member module includes:

[0023] The ID is a private member subunit, which is used to identify the member based on a UUID.

[0024] The Index is a private member subunit used to identify the index of the member in the monitor's memberQueue. This value will be changed accordingly when the monitor adjusts the min-heap.

[0025] The task is a private member subunit that references the task's job information entity.

[0026] The `period` is a private member subunit representing the timeout period defined for the task.

[0027] The callback is a private member subunit used to activate a callback function when the member reaches its timeout period.

[0028] A task timeout monitoring method based on a min-heap includes the following steps:

[0029] Step S1: Add a new task;

[0030] Step S2: Delete the new task;

[0031] Step S3: Refresh the timer and adjust the min-quad heap;

[0032] Step S4: Start the monitor to perform monitoring tasks;

[0033] Step S5: Perform a restart process after a code crash or process interruption.

[0034] As a preferred embodiment of the present invention, the detailed steps of step S1: adding a new task are as follows:

[0035] Step S1.1 First, the AddMember function is called through the task monitor module;

[0036] Step S1.2: Compare the timeout time of the member with the timeout time of the top member in the monitor's memberQueue, and reset the monitor's timer by taking the timeout time closest to the current time.

[0037] Step S1.3 then appends the member to memberQueue and calls adjustMembers to adjust the minimum quad heap.

[0038] As a preferred embodiment of the present invention, the detailed steps of step S2: deleting the new task are as follows;

[0039] Step S2.1: Based on the member's ID, first locate the member in the monitor's memQueue and delete it;

[0040] Step S2.2 calls adjustMembers again to readjust the min-quad heap;

[0041] Step S2.3 Finally, obtain the timeout of the adjusted minimum quad heap top member and reset the monitor's timer target time.

[0042] As a preferred embodiment of the present invention, step S4: starting the monitor for monitoring work is detailed as follows:

[0043] Step S4.1 When the program or process starts, it calls CheckMembers to start the monitor's daemon thread to load the timeout of the monitor's minimum quad heap top member, which is used to reset the timer's timeout.

[0044] Step S4.2 When the timer expires, the top member of the heap is retrieved and it is determined whether the callback function needs to be executed. Then, the min-quadruple heap is adjusted, and the timeout time of the new top member of the heap is used to reset the timer time to wait for the next timer to expire.

[0045] Step S4.3 Repeat the above steps.

[0046] In a preferred embodiment of the present invention, in step S5: restarting after code crash or process interruption, when the program crashes or the process is interrupted, the monitor will be restarted based on the member information backed up in WAL when the process is restarted, and the CheckMembers function will be called to enter step S4 above to start normal monitoring.

[0047] Compared with the prior art, the beneficial effects of the present invention are:

[0048] 1. This invention can ensure accurate timeout monitoring during high-intensity task operations.

[0049] 2. This invention implements a minimum quad-heap based on a queue and uses the timer provided by the programming language SDK to perform periodic checks on the top member of the heap to monitor all task timeouts.

[0050] 3. By monitoring only the top member of the heap and maintaining a min-heap and resetting the timer with the timeout of the top member each time a member is added or deleted, we only need to monitor the timeout task closest to the current time at the top of the heap. This eliminates the need to traverse all task information, greatly speeding up the monitoring and checking timeout and enabling accurate monitoring of the timeout of each task. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention.

[0052] Figure 1 This is a system structure diagram of the present invention.

[0053] Figure 2 Add a task flowchart for this invention.

[0054] Figure 3 This is a flowchart of the deletion task of the present invention.

[0055] Figure 4 This is a flowchart illustrating the working principle of the present invention.

[0056] Figure 5 This is a backup diagram of the code restart for this invention. Detailed Implementation

[0057] To make the technical problems to be solved, the technical solutions, and the beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0058] This invention provides a task timeout monitoring system based on a min-heap, comprising: a task monitor module and a member module connected to the task monitor module. The task monitor module has a built-in timer instance from the programming language SDK and a member queue containing information about each running task. The task monitor module includes a member module that serves as a timer instance and implements a min-heap, a private function module for implementing private function computation, and an external function module for implementing external function computation.

[0059] Furthermore, the member sub-modules of the task monitor module include:

[0060] The timer private member module is a timer instance, which is activated when the target time is reached after the target time is set.

[0061] The memberQueue private member module is a queue composed of members that encapsulate task and job information, and is used to implement a minimum quad heap.

[0062] Furthermore, the private function computation module includes:

[0063] The delMemberWithIndex private function unit is used to delete the member at the specified index from the memberQueue.

[0064] delMember0 is a private function unit used to remove the top member from the memberQueue.

[0065] The adjustMembers private function unit is used to adjust the min-quad heap.

[0066] Furthermore, the external function module includes:

[0067] AddMember is an external function unit used to add new members to the memberQueue. This function refreshes the target timer set by the monitor and calls the adjustMembers() function to readjust the min-heap.

[0068] RemoveMember is an external function unit used to remove a specified member from the memberQueue. This function refreshes the target timer set by the monitor and calls the adjustMembers function to readjust the min-heap.

[0069] The CheckMembers function unit is used to check whether the members defined by the minimum quadruple heap of memberQueue have reached their timeout period. This function needs to be called once when the program starts, and subsequently a daemon thread will monitor the timeout period of the top member of the heap through the private member timer of monitor.

[0070] Furthermore, the member module includes:

[0071] The ID is a private member subunit, which is used to identify the member based on a UUID.

[0072] The Index is a private member subunit used to identify the index of the member in the monitor's memberQueue. This value will be changed accordingly when the monitor adjusts the min-heap.

[0073] The task is a private member subunit that references the task's job information entity.

[0074] The `period` is a private member subunit representing the timeout period defined for the task.

[0075] The callback is a private member subunit used to activate a callback function when the member reaches its timeout period.

[0076] A task timeout monitoring method based on a min-heap includes the following steps:

[0077] Step S1: Add a new task;

[0078] Step S2: Delete the new task;

[0079] Step S3: Refresh the timer and adjust the min-quad heap;

[0080] Step S4: Start the monitor to perform monitoring tasks;

[0081] Step S5: Perform a restart process after a code crash or process interruption.

[0082] Furthermore, the detailed steps for step S1: adding a new task are as follows:

[0083] Step S1.1 First, the AddMember function is called through the task monitor module;

[0084] Step S1.2: Compare the timeout time of the member with the timeout time of the top member in the monitor's memberQueue, and reset the monitor's timer by taking the timeout time closest to the current time.

[0085] Step S1.3 then appends the member to memberQueue and calls adjustMembers to adjust the minimum quad heap.

[0086] Furthermore, the detailed steps of step S2: deleting the new task are as follows;

[0087] Step S2.1: Based on the member's ID, first locate the member in the monitor's memQueue and delete it;

[0088] Step S2.2 calls adjustMembers again to readjust the min-quad heap;

[0089] Step S2.3 Finally, obtain the timeout of the adjusted minimum quad heap top member and reset the monitor's timer target time.

[0090] Furthermore, step S4: Start the monitor to perform monitoring tasks. The detailed steps are as follows:

[0091] Step S4.1 When the program or process starts, it calls CheckMembers to start the monitor's daemon thread to load the timeout of the monitor's minimum quad heap top member, which is used to reset the timer's timeout.

[0092] Step S4.2 When the timer expires, the top member of the heap is retrieved and it is determined whether the callback function needs to be executed. Then, the min-quadruple heap is adjusted, and the timeout time of the new top member of the heap is used to reset the timer time to wait for the next timer to expire.

[0093] Step S4.3 Repeat the above steps.

[0094] Furthermore, in step S5: restarting after code crash or process interruption, when the program crashes or the process is interrupted, the monitor will be restarted based on the member information backed up in WAL when the process is restarted, and the CheckMembers function will be called to enter step S4 above to start normal monitoring.

[0095] Example 1: Please refer to Figure 1 The monitor includes a built-in timer instance from the programming language SDK and a member queue containing information about each running task. In addition, it implements private functions such as delMemberWithIndex, delMember0, and adjustMembers, as well as public functions such as AddMember, RemoveMember, and CheckMembers.

[0096] Their functions are as follows:

[0097] The timer private member module is a timer instance that will be activated when the target time is reached after the target time has been set.

[0098] The memberQueue private member module is a queue composed of members that encapsulate task and job information, and is used to implement a minimum quad heap.

[0099] The delMemberWithIndex private function unit is used to remove the member at the specified index from the memberQueue.

[0100] delMember0 is a private function unit used to remove the top member from the memberQueue.

[0101] The adjustMembers private function unit is used to adjust the min-quad heap.

[0102] The AddMember external function unit is used to add new members to the memberQueue. This function refreshes the target timer setting of the monitor and calls the adjustMembers function to readjust the minimum quad heap.

[0103] The RemoveMember external function unit is used to remove a specified member from the memberQueue. This function refreshes the target timer set by the monitor and calls the adjustMembers function to readjust the min-heap.

[0104] The CheckMembers function unit is used to check whether the members defined by the minimum quadruple heap of memberQueue have reached their timeout period. This function needs to be called once when the program starts, and subsequently a daemon thread will monitor the timeout period of the top member of the heap through the private member timer of monitor.

[0105] The members in the member list are:

[0106] The ID is a private member subunit, which is used to identify the member based on a UUID.

[0107] The Index is a private member subunit used to identify the index of the member in the monitor's memberQueue. This value will be changed accordingly when the monitor adjusts the min-heap.

[0108] The task is a private member subunit that references the task's job information entity.

[0109] The `period` is a private member subunit representing the timeout period defined for the task.

[0110] The callback is a private member subunit used to activate a callback function when the member reaches its timeout period.

[0111] Example 2: Please refer to Figure 2-5 A task timeout monitoring method based on a min-heap includes the following steps:

[0112] Step S1: Add a new task, please refer to Figure 2 In this embodiment, the task is encapsulated as a member instance. After calling the AddMember function, the timeout time of the member is compared with the timeout time of the member at the top of the heap in the monitor's memberQueue. The timeout time closest to the current time is taken to reset the monitor's timer. Then the member is added to the memberQueue and adjustMembers is called to adjust the minimum quad heap.

[0113] Step S2: Delete the new task. Please refer to [link / reference]. Figure 3 Based on the member's ID, first find the member in the monitor's memQueue and delete it. Then, call adjustMembers to readjust the minimum quad heap. Finally, get the timeout of the member at the top of the adjusted minimum quad heap and reset the monitor's timer target time.

[0114] Step S3: Refresh the timer and adjust the min-quad heap. Please refer to [link / reference]. Figures 2-3 Adding or deleting a member requires adjusting the heap and refreshing the timer's target timeout based on the timeout of the member at the top of the min-quadruple heap.

[0115] Step S4: Start the monitor to perform monitoring tasks. Please refer to [link / reference]. Figure 4 When a program or process starts, it calls CheckMembers to start the monitor's daemon thread to load the timeout of the top member of the monitor's min-heap. This is used to reset the timer's timeout. When the timer expires, it retrieves the top member of the heap and determines whether the member needs to execute a callback function. Then, it adjusts the min-heap and uses the new timeout of the top member to reset the timer's timeout, waiting for the next timer to expire. This process repeats continuously, allowing the monitor to focus on a single top member, thus simplifying the inaccurate and time-consuming monitoring caused by the traditional method of iterating through all tasks and checking timeouts.

[0116] Step S5: Perform a restart process after a code crash or process interruption. Please refer to [link to relevant documentation]. Figure 5 Each of the above operations (adding a member and deleting a member) will record the corresponding WAL (pre-write log, a common practice in the industry for data recovery, which will not be elaborated here). When the program crashes or the process is interrupted, when the process is restarted, the monitor will be restarted based on the member information backed up in the WAL and the CheckMembers() function will be called to enter step S4 above and start normal monitoring.

[0117] For example, the processor fetches instructions one by one from the memory, analyzes the instructions, and then performs the corresponding operations according to the instructions, generating a series of control commands to enable the various parts of the computer to act automatically, continuously and in coordination, forming an organic whole, realizing program input, data input, and calculation and output of results. The arithmetic or logical operations generated in this process are all performed by the arithmetic unit; the memory includes a read-only memory (ROM), which is used to store computer programs, and the memory is provided with external protection devices.

[0118] For example, a computer program can be divided into one or more modules, one or more of which are stored in memory and executed by a processor to perform the present invention. The one or more modules can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in a terminal device.

[0119] Those skilled in the art will understand that the above description of the service equipment is merely an example and does not constitute a limitation on the terminal equipment. It may include more or fewer components than described above, or a combination of certain components, or different components, such as input / output devices, network access devices, buses, etc.

[0120] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. This processor is the control center of the terminal device, connecting various parts of the user terminal via various interfaces and lines.

[0121] The aforementioned memory can be used to store computer programs and / or modules. The aforementioned processor implements various functions of the aforementioned terminal device by running or executing the computer programs and / or modules stored in the memory, and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as information collection template display function, product information publishing function, etc.); the data storage area may store data created based on the use of the berth status display system (such as product information collection templates corresponding to different product types, product information that different product providers need to publish, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0122] If the modules / units integrated into the terminal device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the modules / units in the systems of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the functions of the various system embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0123] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0124] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A task timeout monitoring system based on a min-heap, characterized in that, include: The task monitor module and the member module connected to the task monitor module are provided. The task monitor module has a built-in timer instance from the programming language SDK and a member queue containing information about each running task. The task monitor module includes a member module that serves as a timer instance and implements a minimum quad heap, a private function computation module for implementing private function computation, and an external function module for implementing external function computation. The member sub-modules of the task monitor module include: The timer private member module is a timer instance, which is activated when the target time is reached after the target time is set. The memberQueue private member module is a queue composed of members that encapsulate task and job information, and is used to implement a minimum quad heap. The private function calculation module includes: The delMemberWithIndex private function unit is used to delete the member at the specified index from the memberQueue. delMember0 is a private function unit used to remove the top member from the memberQueue. The adjustMembers private function unit is used to adjust the min-quad heap; The external function module includes: The AddMember external function unit is used to add new members to the memberQueue. This function refreshes the target timer set by the monitor and calls the adjustMembers function to readjust the min-heap. RemoveMember is an external function unit used to remove a specified member from the memberQueue. This function refreshes the target timer set by the monitor and calls the adjustMembers function to readjust the min-heap. The `CheckMembers` function unit is used to check whether the member at the top of the min-heap of `memberQueue` has reached its timeout period. This function needs to be called once when the program starts, and subsequently a daemon thread will monitor the timeout period of the top member of the heap through the private member `timer` of `monitor`. The member module includes: Id is a private member subunit, which is used to identify a member based on a UUID; The Index is a private member subunit used to identify the index of the member in the monitor's memberQueue. This index value will be changed accordingly when the monitor adjusts the min-heap. The task is a private member subunit that references the task's job information entity. The period is a private member subunit containing the timeout period defined for the task. A callback private member subunit, used to activate a callback function when the member reaches its timeout period; a task timeout monitoring method based on a min-heap, characterized by the following steps: Step S1: Add a new task; Step S2: Delete the new task; Step S3: Refresh the timer and adjust the min-quad heap; Step S4: Start the monitor to perform monitoring tasks; Step S5: Restart the process after a code crash or process interruption; the detailed steps of step S1: adding a new task are as follows: Step S1.1 First, the AddMember function is called through the task monitor module; Step S1.2: Compare the timeout time of the member with the timeout time of the top member in the monitor's memberQueue, and reset the monitor's timer by taking the timeout time closest to the current time. Step S1.3 then appends the member to memberQueue and calls adjustMembers to adjust the minimum quad heap.

2. The task timeout monitoring system based on a min-heap according to claim 1, characterized in that, The detailed steps for step S2: deleting the new task are as follows; Step S2.1: Based on the member's ID, first locate the member in the monitor's memberQueue and delete it; Step S2.2 calls adjustMembers again to readjust the min-quad heap; Step S2.3 Finally, obtain the timeout of the adjusted minimum quad heap top member and reset the monitor's timer target time.

3. A task timeout monitoring system based on a min-heap according to claim 2, characterized in that... Step S4: Start the monitor to perform monitoring tasks. Detailed steps are as follows: Step S4.1 When the program or process starts, it calls CheckMembers to start the monitor's daemon thread to load the timeout of the monitor's minimum quad heap top member, which is used to reset the timer's target time. Step S4.2 When the timer expires, the top member of the heap is retrieved and it is determined whether the callback function needs to be executed. Then, adjustMembers is called to adjust the min-quadruple heap, and the timeout time of the new top member of the heap is used to reset the timer time to wait for the next timer to expire. Step S4.3 Repeat the above steps.

4. The task timeout monitoring system based on a min-heap according to claim 3, characterized in that, In step S5: restarting after code crash or process interruption, when the program crashes or the process is interrupted, the monitor will be restarted based on the member information backed up in WAL when the process is restarted, and the CheckMembers function will be called to enter step S4 above to start normal monitoring.

Images