Timing task processing method and device, equipment and medium
By creating a scheduler instance in a distributed system, determining whether a task is within the execution capacity, and executing scheduled tasks in parallel on multiple physical nodes, the problem of scheduling scheduled tasks in distributed scenarios is solved, achieving efficient and flexible task scheduling.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA TELECOM CLOUD TECH CO LTD
- Filing Date
- 2025-12-03
- Publication Date
- 2026-04-24
AI Technical Summary
In distributed scenarios, scheduling timed tasks is difficult, mainly due to differences in hardware configuration and network environment among nodes, making effective scheduling challenging.
By creating a scheduler instance, it is determined whether the scheduled task is within the execution capability of the scheduler instance, and coroutines are started on multiple target physical nodes to execute the scheduled task, thus achieving flexible and efficient task scheduling.
It improves the scheduling efficiency and flexibility of scheduled tasks, avoids blocking the main loop of the scheduler instance during task execution, and achieves high-concurrency task execution.
Smart Images
Figure CN121918952A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, device, and medium for processing timed tasks. Background Technology
[0002] Currently, in distributed scenarios, large-scale model training often involves using a large number of physical machines for online training.
[0003] In existing technologies, various pre-training checks are performed on the physical machine before training a large model. These checks include basic checks on the server's network card, CPU (Central Processing Unit), memory, drivers, and graphics card, as well as RDMA (Remote Direct Memory Access) communication tests and communication library tests for simulation training. Some of these checks (such as basic server checks) need to be performed periodically to monitor the physical machine's status during training.
[0004] However, in distributed scenarios, scheduled tasks need to perform detection operations on multiple nodes (such as physical machines or servers), and the differences in hardware configuration and network environment of each node make it difficult to schedule scheduled tasks. Summary of the Invention
[0005] In view of the above problems, a method, apparatus, device, and medium for processing timed tasks are proposed to overcome or at least partially solve the above problems, including: A method for processing scheduled tasks, applied to a scheduler, wherein the scheduler executes scheduled tasks on physical nodes by creating scheduler instances, the method comprising: Retrieve the target scheduled task to be executed from the unassigned scheduler instance; Determine whether the target scheduled task is within the execution capability of the current scheduler instance; If the target scheduled task is within the execution capacity of the current scheduler instance, the target scheduled task is assigned to the current scheduler instance, and the current scheduler instance starts a coroutine to execute the target scheduled task on multiple target physical nodes.
[0006] Optionally, obtain the target scheduled task to be executed for which no scheduler instance has been allocated, including: From the database, determine that the scheduled tasks with an empty scheduler instance identifier field are scheduled tasks without an assigned scheduler instance, and from the scheduled tasks without an assigned scheduler instance, determine the scheduled task whose next execution time is closest to the current time as the target scheduled task. After assigning the target scheduled task to the current scheduler instance, the process further includes: Set the scheduler instance identifier field of the target scheduled task in the database to the scheduler instance identifier of the current scheduler instance.
[0007] Optionally, after starting a coroutine through the current scheduler instance to execute the target scheduled task on multiple target physical nodes, the method further includes: After the target scheduled task is completed, the scheduler instance identifier field of the target scheduled task in the database is set to empty.
[0008] Optionally, determining whether the target scheduled task is within the execution capability of the current scheduler instance includes: Determine a first number of target physical nodes that need to execute the target timed task; Determine the second number of physical nodes that execute the scheduled task through the current scheduler instance; Based on the first quantity and the second quantity, it is determined whether the target scheduled task is within the execution capacity of the current scheduler instance.
[0009] Optionally, determining whether the target scheduled task is within the execution capacity of the current scheduler instance based on the first quantity and the second quantity includes: Determine whether the sum of the first quantity and the second quantity is less than or equal to the upper limit of the quantity; wherein, the upper limit of the quantity is the upper limit of the number of physical nodes that can execute the scheduled task through the current scheduler instance; If the sum of the first quantity and the second quantity is less than or equal to the upper limit of the quantity, the target timed task is determined to be within the execution capability of the current scheduler instance.
[0010] Optionally, after starting a coroutine through the current scheduler instance to execute the target scheduled task on multiple target physical nodes, the method further includes: After the target timed task is completed, the first quantity is subtracted from the second quantity to update the second quantity.
[0011] Optionally, after starting a coroutine through the current scheduler instance to execute the target scheduled task on multiple target physical nodes, the method further includes: Obtain the task execution result of the target timed task; When the task execution result indicates that the task was executed successfully, the task type of the target timed task is determined. If the task type of the target timed task is a periodic task, the next run time of the target timed task is updated according to the next scheduling time of the target timed task. At the updated next run time, the target timed task is executed on multiple target physical nodes by starting a coroutine through the current scheduler instance.
[0012] Optionally, updating the next execution time of the target timed task based on its next scheduling time includes: Obtain the next scheduling time for the target timed task; If the next scheduling time is earlier than the current time, the next scheduling time is updated according to the execution interval of the target timed task until the updated next scheduling time is not earlier than the current time; wherein, the execution interval is the time interval between two consecutive executions of the target timed task; The next execution time of the target timed task is updated using the updated next scheduling time.
[0013] Optionally, it also includes: if the target timed task is a one-time task, deleting the relevant data of the target timed task.
[0014] Optionally, it further includes: when the task execution result indicates that the task execution has failed, updating the next run time of the target timed task through a retry mechanism, so that at the updated next run time, the target timed task is executed on multiple target physical nodes by starting a coroutine through the current scheduler instance.
[0015] Optionally, the next execution time of the target timed task is updated through a retry mechanism, including: Obtain the retry interval for the target timed task; Update the next run time of the target timed task based on the retry interval.
[0016] Optionally, the timed task is a task used to perform timed detection on multiple distributed physical nodes.
[0017] A processing apparatus for scheduled tasks, applied to a scheduler, the scheduler executing scheduled tasks on physical nodes by creating scheduler instances, the apparatus comprising: The target scheduled task acquisition module is used to acquire the target scheduled tasks to be executed that have not been assigned to a scheduler instance; The execution capability judgment module is used to determine whether the target timed task is within the execution capability range of the current scheduler instance; The target scheduled task execution module is used to assign the target scheduled task to the current scheduler instance if the target scheduled task is within the execution capability of the current scheduler instance, and to start a coroutine through the current scheduler instance to execute the target scheduled task on multiple target physical nodes.
[0018] An electronic device includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method described above.
[0019] A computer-readable storage medium on which a computer program is stored, which, when executed by a processor, implements the method described above.
[0020] The embodiments of the present invention have the following advantages: In this embodiment of the invention, by obtaining the target scheduled task to be executed that has not been allocated to a scheduler instance; determining whether the target scheduled task is within the execution capability range of the current scheduler instance; if the target scheduled task is within the execution capability range of the current scheduler instance, allocating the target scheduled task to the current scheduler instance, and starting a coroutine through the current scheduler instance to execute the target scheduled task on multiple target physical nodes, the invention achieves the allocation of scheduled tasks according to the execution capability of the scheduler instance, improving the flexible scheduling of scheduled tasks. By starting a coroutine through the scheduler instance to execute scheduled tasks in parallel on multiple target physical nodes, the actual execution of scheduled tasks is not blocked by the main loop of the scheduler instance, thereby achieving high concurrency and improving the scheduling efficiency of scheduled tasks. Attached Figure Description
[0021] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0022] Figure 1 This is a flowchart of the steps of a method for processing a timed task provided in some embodiments of the present invention; Figure 2 This is a flowchart illustrating how a scheduler instance cyclically executes a timed task, as provided in some embodiments of the present invention. Figure 3 This is a flowchart of the steps of a third method for processing a timed task provided in some embodiments of the present invention; Figure 4 This is a structural block diagram of a timed task processing device provided in some embodiments of the present invention. Detailed Implementation
[0023] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.
[0024] In related technologies, a detection task can be considered as an HTTP (Hypertext Transfer Protocol) request. The backend service can assemble the corresponding parameters and send them to the script proxy service for execution. After the execution is completed, the result is returned to the backend service through a callback.
[0025] Each detection task consists of a certain number of nodes. When an instance is executed, it starts a process for each node, and there is a limit to the number of processes that each instance can start. If scheduled tasks need to be executed, a scheduled task scheduling mechanism is also required to receive requests for management, calculate the next execution time of each task, and call the script proxy service for execution.
[0026] Based on this, considering the robustness of the program, the upper limit of a single script proxy service, and the distributed scenario, this embodiment of the invention proposes to create a scheduler instance through a scheduler, and determine whether the scheduled task is within the execution capability range of the scheduler instance, so as to flexibly and efficiently execute the scheduled task on the physical node.
[0027] The present invention will be further described below with reference to the accompanying drawings: Reference Figure 1 The diagram illustrates a flowchart of a method for processing a timed task according to some embodiments of the present invention. The method is applied to a scheduler, which executes a timed task on a physical node by creating a scheduler instance. The timed task is a task for performing timed detection on multiple distributed physical nodes.
[0028] As examples, physical nodes can be servers, computer devices, or other entities with computing capabilities. These physical nodes can be distributed across different geographical locations and can form a distributed system through network connections.
[0029] In practical applications, scheduler instances can be created (i.e., the scheduler can be started) on multiple physical nodes, and the scheduler instances can be used to perform timed detection tasks on the physical nodes.
[0030] As examples, all scheduler instances can connect to a shared database that maintains a scheduled task table; this table stores all scheduled tasks to be executed.
[0031] The shared database can also maintain a scheduler instance table, which can store information about all scheduler instances, such as scheduler instance identifier and heartbeat time. Each scheduler instance can periodically update its own heartbeat in the scheduler instance table. During the update, the scheduler instance table can also be traversed to delete scheduler instances that have timed out and have not been updated.
[0032] In the above embodiments, by creating distributed scheduler instances and periodically updating heartbeats, the stability and reliability of the system can be guaranteed when the scheduler instance is abnormal. For example, when a physical node reaches its running limit, the scheduler instance can be blocked to prevent it from acquiring new scheduled tasks, thereby giving other idle scheduler instances the opportunity to acquire scheduled tasks.
[0033] Specifically, it may include the following steps: Step 101: Obtain the target scheduled task to be executed for any unassigned scheduler instance.
[0034] In practical applications, the scheduler instance can retrieve the scheduled tasks that have not been assigned to a scheduler instance by querying the scheduled task table in the database. Then, it can filter out the scheduled task whose next execution time is closest to the current time from the scheduled task that has not been assigned to a scheduler instance and use it as the target scheduled task.
[0035] In some embodiments of the present invention, obtaining the target scheduled task to be executed from the unallocated scheduler instance includes: determining from the database that the scheduled task with an empty scheduler instance identifier field is the scheduled task with the unallocated scheduler instance, and determining from the unallocated scheduler instance the scheduled task whose next execution time is closest to the current time as the target scheduled task.
[0036] As examples, each scheduled task can be associated with a scheduler instance identifier field (such as instance_id) to indicate that the scheduled task has been assigned to a scheduler instance.
[0037] In some examples, if the scheduler instance identifier field associated with the scheduled task is empty (e.g., instance_0), it means that the scheduled task has not been assigned to a scheduler instance. The next execution time of the scheduled task can be determined based on the task attribute information of the scheduled task. Combined with the current time, the scheduled task whose next execution time is closest to the current time can be selected as the target scheduled task.
[0038] As examples, the attribute information of a scheduled task may include multiple attribute fields, each of which may contain different information, such as task name, task type (periodic task or one-time task), run interval, retry interval, next scheduling time, etc.
[0039] For example, the task attribute information of a scheduled task may include: oneTime (indicating a one-time task), Immediate (indicating that the task will be executed immediately), firstRun (indicating the first time the task will run), interval (indicating the running interval), nextRun (indicating the next running time), and nextSched (indicating the next scheduling time). Among them, immediate and firstRun are mutually exclusive.
[0040] The next scheduled time (nextSched) refers to the fixed normal execution time of the scheduled task (e.g., every Wednesday at 9 AM), which remains unchanged at the original interval. The next run time (nextRun) is the actual execution time of the task, which is usually consistent with nextSched. If the task fails and retry is allowed, nextRun will adjust according to the retry interval (not the original interval), and after a successful retry, it will return to normal scheduling according to nextSched.
[0041] In some examples, the task attribute information of the scheduled task can be combined to create the following six states of the scheduled task. When data is initially inserted, the values of nextRun and nextSched can be calculated: 1. oneTime + immediate + (firstRun and interval are empty) One-time task is executed immediately, nextRun = nextSched = current time.
[0042] 2. oneTime+firstRun (interval and immediate are empty) A one-time task is executed at a specified time point, nextRun=nextSched=firstRun.
[0043] 3. oneTime+interval (firstRun and immediate are empty) One-time task is executed after a time delay, nextRun=nextSched=current time+interval.
[0044] 4. `immediate + interval` (with `firstRun` and `oneTime` empty) executes periodic tasks immediately, and `nextRun` = `nextSched` = the current time.
[0045] 5. interval + firstRun (oneTime and immediate are empty) Periodic tasks start execution at a specified time point, nextRun = nextSched = firstRun.
[0046] 6. interval (oneTime, immediate, and firstRun are empty) Periodic tasks are executed after a delay time interval, nextRun = nextSched = current time + interval.
[0047] In the above embodiments, task attribute information can be combined to form scheduled tasks that support 6 different scenarios, and nextRun and nextSched are added to deal with time updates in abnormal scenarios, which can enrich the types of scheduled tasks supported.
[0048] Step 102: Determine whether the target scheduled task is within the execution capability of the current scheduler instance.
[0049] As examples, scheduler instances compete with each other through a competition mechanism to determine the scheduler instance that will handle the target scheduled task as the current scheduler instance.
[0050] After obtaining the target scheduled task to be executed from the unassigned scheduler instance, it can be determined whether to assign the target scheduled task to the current scheduler instance for scheduling based on the execution capacity of the current scheduler instance.
[0051] In some embodiments of the present invention, determining whether the target scheduled task is within the execution capability range of the current scheduler instance includes: Sub-step 11: Determine the first number of target physical nodes that need to execute the target timed task.
[0052] After determining the target scheduled task, the initial number of target physical nodes that need to execute the target scheduled task can be determined based on the node attribute information carried by the target scheduled task.
[0053] As examples, the node attribute information carried by a scheduled task can be used to determine multiple target physical nodes that need to execute the scheduled task.
[0054] The node attribute information can be a list, containing the identifiers or addresses of all physical nodes that need to execute the scheduled task. By counting the number of nodes in this list, the total number of target physical nodes that need to execute the scheduled task (i.e., the first number) can be obtained.
[0055] For example, if the node attribute information carried by the target timed task indicates that the multiple target physical nodes that need to execute the target timed task are A, B, and C, then the first quantity is 3.
[0056] Sub-step 12: Determine the second number of physical nodes that will execute the scheduled task through the current scheduler instance.
[0057] In practical applications, the total number of physical nodes to which the current scheduler instance has been assigned scheduled tasks can be counted, thus obtaining the second number of physical nodes on which the current scheduler instance is executing scheduled tasks. This second number reflects the load status of the current scheduler instance, i.e., how many physical nodes' scheduled tasks the current scheduler instance is currently processing.
[0058] For example, if the current scheduler instance has assigned the scheduled task P to physical nodes D, E, F, and G for execution, then the second number of physical nodes on which the current scheduler instance executes the scheduled task is 4.
[0059] Sub-step 13: Based on the first quantity and the second quantity, determine whether the target timed task is within the execution capability of the current scheduler instance.
[0060] In practical applications, if a scheduled task requires execution on multiple physical nodes, each scheduler instance will start a process for each physical node when allocating the scheduled task. The maximum number of physical nodes each scheduler instance can allocate for executing scheduled tasks is limited; there is a maximum number. Therefore, we can determine whether the target scheduled task is within the execution capacity of the current scheduler instance by comparing the sum of the first and second numbers with the maximum number that the current scheduler instance can execute.
[0061] In some embodiments of the present invention, determining whether the target scheduled task is within the execution capability range of the current scheduler instance based on the first quantity and the second quantity includes: Determine whether the sum of the first quantity and the second quantity is less than or equal to the upper limit of the quantity; wherein, the upper limit of the quantity is the upper limit of the number of physical nodes that can execute the scheduled task through the current scheduler instance; if the sum of the first quantity and the second quantity is less than or equal to the upper limit of the quantity, determine that the target scheduled task is within the execution capability of the current scheduler instance.
[0062] In practical applications, the first quantity and the second quantity can be summed to obtain the total value of the first quantity and the second quantity. Then, this total value can be compared with the upper limit of the number of physical nodes that the current scheduler instance can execute scheduled tasks.
[0063] If the sum is less than or equal to the maximum number of tasks, it means that the current scheduler instance has enough physical nodes to execute the target scheduled task, and the target scheduled task is within the execution capacity of the current scheduler instance. Conversely, if the sum is greater than the maximum number of tasks, it means that the current scheduler instance has reached the upper limit of its execution capacity and cannot allocate more scheduled tasks, and the target scheduled task is outside the execution capacity of the current scheduler instance.
[0064] Step 103: If the target scheduled task is within the execution capability of the current scheduler instance, the target scheduled task is assigned to the current scheduler instance, and the current scheduler instance starts a coroutine to execute the target scheduled task on multiple target physical nodes.
[0065] As examples, starting a coroutine refers to the current scheduler instance creating an independent, asynchronous logical flow within its process to prevent the actual execution of the scheduled task from blocking the scheduler instance's main loop, thereby achieving high concurrency.
[0066] In practical applications, if it is determined that the target scheduled task is within the execution capability of the current scheduler instance, the current scheduler instance can assign the target scheduled task to itself, and then the current scheduler instance will start a coroutine to execute the target scheduled task on multiple target physical nodes.
[0067] For example, when faced with multiple server nodes distributed in different geographical locations, a coroutine can simultaneously send execution instructions to these physical nodes to make them execute the target scheduled task.
[0068] As examples, once the scheduler instance starts, it can continuously load scheduled tasks from the shared database into the main loop, and execute the scheduled tasks in parallel on multiple target physical nodes by starting coroutines.
[0069] like Figure 2 The process of executing the scheduler instance in a loop can include the following steps: S1. Retrieve the CronJob (scheduled task) with an empty instance ID and the smallest nextRun value, lock it, and update the instance ID to the ID of the current CronInstance (scheduler instance).
[0070] S2 checks the result of S1. If the acquisition fails, it blocks for a fixed interval and then enters the next loop; if the acquisition succeeds, it enters S3.
[0071] S3: Determine the nodeCount (second quantity) of the scheduler instance itself + the number of nodes of the CronJob task (first quantity). Check if the upper limit of concurrent nodes of the task has been reached (nodeMaxCount). If the upper limit is exceeded, block and wait for other goroutines to be released before starting to run. If the upper limit has not been reached, update nodeCount = nodeCount + the number of nodes of the CronJob task, and continue to execute S4.
[0072] S4, start the coroutine to execute the task.
[0073] S5, the main loop ends and the next loop begins (S1 is executed).
[0074] S4-1 executes a scheduled task and retrieves the task execution result.
[0075] S4-2 determines whether the task execution result of S4-1 was successful. If it fails, the nextRun (next running time) is calculated through the retry mechanism, and the process jumps to S4-5. If it is successful, the process proceeds to S4-3.
[0076] S4-3: Determine if the scheduled task is a one-time task. If so, delete the data and proceed to S4-6; if it is a periodic task, proceed to S4-4.
[0077] S4-4 Calculate the next run time. If nextSched is earlier than the current time, repeat the execution of nextSched=nextSched+interval until nextSched is greater than the current time, and set nextRun=nextSched. This step is to compensate for the problem that the current time exceeds nextSched due to the previous blocking.
[0078] S4-5, update nodeCount = nodeCount - the number of nodes in the CronJob task, set the instance ID in CronJob to empty and update it back to the database.
[0079] S4-6, the sixth step of the coroutine ends.
[0080] In some embodiments of the present invention, after assigning the target scheduled task to the current scheduler instance, the method further includes: setting the scheduler instance identifier field of the target scheduled task in the database to the scheduler instance identifier of the current scheduler instance.
[0081] In practical applications, after assigning the target scheduled task to the current scheduler instance, the scheduler instance identifier field (such as instance_0) corresponding to the target scheduled task in the database can be updated to the unique identifier of the current scheduler instance (such as instance_001) to establish the binding relationship between the target scheduled task and the current scheduler instance.
[0082] In some embodiments of the present invention, after starting a coroutine through the current scheduler instance to execute the target timed task on multiple target physical nodes, the method further includes: after the target timed task is executed, setting the scheduler instance identifier field of the target timed task in the database to null.
[0083] In practical applications, after the target scheduled task has been executed, in order to reassign the scheduled task or perform other related processing, the scheduler instance identifier field corresponding to the target scheduled task in the database can be reset to empty (such as instance_0), so that the scheduled task is in an unassigned scheduler instance state and can be obtained and executed by other scheduler instances with execution capabilities.
[0084] In some embodiments of the present invention, after starting a coroutine through the current scheduler instance to execute the target timed task on multiple target physical nodes, the method further includes: after the target timed task is executed, subtracting the first quantity from the second quantity to update the second quantity.
[0085] In practical applications, once the target scheduled task has been completed, it indicates that the current scheduler instance has finished processing all target physical nodes associated with that target scheduled task. The first number (i.e., the total number of target physical nodes involved in the target scheduled task) can be subtracted from the second number (i.e., the total number of physical nodes for which the current scheduler instance is executing scheduled tasks) to update the total number of physical nodes that the current scheduler instance is actually processing scheduled tasks.
[0086] In some embodiments of the present invention, after starting a coroutine through the current scheduler instance to execute the target timed task on multiple target physical nodes, the method further includes: Obtain the task execution result of the target scheduled task; when the task execution result indicates that the task was executed successfully, determine the task type of the target scheduled task, and if the task type of the target scheduled task is a periodic task, update the next run time of the target scheduled task according to the next scheduling time of the target scheduled task, so that at the updated next run time, the target scheduled task can be executed on multiple target physical nodes by starting a coroutine through the current scheduler instance.
[0087] In practical applications, after a coroutine executes a target timed task on multiple target physical nodes, it can obtain the task execution result of the target timed task, which may include whether the execution was successful or failed.
[0088] As examples, the task type and next scheduling time (nextSched) can be determined based on the attribute information of the target scheduled task. If the target scheduled task is a periodic task and is in a normal scheduling process (such as first execution, after successful retry), its next run time (nextRun) is updated to the value of the next scheduling time (nextSched), ensuring that the task returns to the preset cycle.
[0089] In some embodiments of the present invention, updating the next execution time of the target timed task according to the next scheduling time of the target timed task includes: Obtain the next scheduling time of the target timed task; if the next scheduling time is earlier than the current time, update the next scheduling time according to the running interval of the target timed task until the updated next scheduling time is not earlier than the current time; wherein, the running interval is the time interval between two consecutive executions of the target timed task; use the updated next scheduling time to update the next running time of the target timed task.
[0090] As examples, the run interval is a fixed time difference between two consecutive executions of a target scheduled task under normal execution conditions.
[0091] For example, if the target scheduled task is to synchronize data every hour, and the last two successful executions were at 10:30 on October 1st and 11:30 on October 1st, then the execution interval is 60 minutes.
[0092] In some examples, the next scheduling time (nextSched) can be obtained from the attribute information of the target scheduled task; by comparing nextSched with the current time, it can be determined whether there is a scheduling time that has expired (i.e., earlier than the current time).
[0093] If the time has expired, iteratively calculate the new scheduling time according to the run interval of the target scheduled task (e.g., if the current time is 10:00, the original nextSched is 9:00, and the run interval is 1 hour, then update the next scheduling time to 11:00); until the updated nextSched is no earlier than the current time; use the finally calibrated nextSched to overwrite and update the next run time (nextRun) of the target scheduled task, ensuring that the task is executed according to the calibrated time.
[0094] For example, the original next scheduling time (nextSched) is read from the task attributes of the target scheduled task as 08:00 on October 1st; it is determined that this time is earlier than the current time (09:30 on October 2nd), so it needs to be calibrated; calculated according to the 24-hour running interval, the first update is to 08:00 on October 2nd (still earlier than the current time), and the next update is to 08:00 on October 3rd (not earlier than the current time); using the calibrated nextSched (08:00 on October 3rd), the next run time (nextRun) of the target scheduled task is updated to 08:00 on October 3rd, and the task will finally be executed at that time.
[0095] In some embodiments of the present invention, the method further includes: deleting the relevant data of the target timed task when the task type of the target timed task is a one-time task.
[0096] In practical applications, if the target scheduled task is a one-time task, after the task is completed, the relevant data of the one-time scheduled task, such as task attribute information, node attribute information, and scheduler instance identifier field, can be deleted from the shared database.
[0097] In some embodiments of the present invention, the method further includes: when the task execution result indicates that the task execution has failed, updating the next run time of the target timed task through a retry mechanism, so that at the updated next run time, the target timed task is executed on multiple target physical nodes by starting a coroutine through the current scheduler instance.
[0098] In practical applications, if the task execution result indicates that the task execution failed, and the target scheduled task allows retry, the next run time of the target scheduled task can be updated according to the retry interval. After the update is completed, the current scheduler instance will start the coroutine again at the new next run time and re-execute the target scheduled task on multiple target physical nodes.
[0099] The updated next run time may no longer follow the original next scheduling time (nextSched) setting, but will be postponed according to the retry interval.
[0100] In some embodiments of the present invention, updating the next execution time of the target timed task through a retry mechanism includes: obtaining the retry interval of the target timed task; and updating the next execution time of the target timed task according to the retry interval.
[0101] As examples, the retry interval can be preset or determined by the system default configuration. After obtaining the specific value of the retry interval, the next execution time of the target scheduled task after the update can be calculated by adding the retry interval to the current time point when the current task failed.
[0102] For example, if the retry interval is set to 5 minutes and the current task failed at 10:00, then the next run time after the update will be 10:05.
[0103] In this embodiment of the invention, by obtaining the target scheduled task to be executed that has not been allocated to a scheduler instance; determining whether the target scheduled task is within the execution capability range of the current scheduler instance; if the target scheduled task is within the execution capability range of the current scheduler instance, allocating the target scheduled task to the current scheduler instance, and starting a coroutine through the current scheduler instance to execute the target scheduled task on multiple target physical nodes, the invention achieves the allocation of scheduled tasks according to the execution capability of the scheduler instance, improving the flexible scheduling of scheduled tasks. By starting a coroutine through the scheduler instance to execute scheduled tasks in parallel on multiple target physical nodes, the actual execution of scheduled tasks is not blocked by the main loop of the scheduler instance, thereby achieving high concurrency and improving the scheduling efficiency of scheduled tasks.
[0104] Reference Figure 3 The diagram illustrates a flowchart of another method for processing a timed task provided by some embodiments of the present invention. This method is applied to a scheduler, which executes a timed task on a physical node by creating a scheduler instance. The timed task is a task for performing timed detection on multiple distributed physical nodes.
[0105] Specifically, it may include the following steps: Step 301: From the database, determine that the scheduled tasks with an empty scheduler instance identifier field are scheduled tasks without an assigned scheduler instance, and from the scheduled tasks without an assigned scheduler instance, determine the scheduled task whose next execution time is closest to the current time as the target scheduled task.
[0106] Step 302: Determine the first number of target physical nodes that need to execute the target timed task.
[0107] Step 303: Determine the second number of physical nodes that will execute the scheduled task through the current scheduler instance.
[0108] Step 304: Based on the first quantity and the second quantity, determine whether the target timed task is within the execution capability of the current scheduler instance.
[0109] Step 305: If the target scheduled task is within the execution capability of the current scheduler instance, the target scheduled task is assigned to the current scheduler instance, and the current scheduler instance starts a coroutine to execute the target scheduled task on multiple target physical nodes.
[0110] In this embodiment of the invention, by obtaining the target scheduled task to be executed that has not been allocated to a scheduler instance; determining whether the target scheduled task is within the execution capability range of the current scheduler instance; if the target scheduled task is within the execution capability range of the current scheduler instance, allocating the target scheduled task to the current scheduler instance, and starting a coroutine through the current scheduler instance to execute the target scheduled task on multiple target physical nodes, the invention achieves the allocation of scheduled tasks according to the execution capability of the scheduler instance, improving the flexible scheduling of scheduled tasks. By starting a coroutine through the scheduler instance to execute scheduled tasks in parallel on multiple target physical nodes, the actual execution of scheduled tasks is not blocked by the main loop of the scheduler instance, thereby achieving high concurrency and improving the scheduling efficiency of scheduled tasks.
[0111] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.
[0112] Reference Figure 4 The diagram shows a schematic of a timed task processing device provided in some embodiments of the present invention, which is applied to a scheduler. The scheduler executes timed tasks on physical nodes by creating scheduler instances.
[0113] Specifically, it can include the following modules: The target scheduled task acquisition module 401 is used to acquire the target scheduled tasks to be executed that have not been allocated a scheduler instance; The execution capability judgment module 402 is used to determine whether the target timed task is within the execution capability range of the current scheduler instance; The target scheduled task execution module 403 is used to allocate the target scheduled task to the current scheduler instance if the target scheduled task is within the execution capability of the current scheduler instance, and to start a coroutine through the current scheduler instance to execute the target scheduled task on multiple target physical nodes.
[0114] In some embodiments of the present invention, the target timed task acquisition module 401 includes: The target scheduled task determination submodule is used to determine from the database that scheduled tasks with an empty scheduler instance identifier field are scheduled tasks without an assigned scheduler instance, and to determine the scheduled task with the next execution time closest to the current time from the scheduled tasks without an assigned scheduler instance as the target scheduled task.
[0115] In some embodiments of the present invention, the apparatus further includes: The first instance identifier setting module is used to set the scheduler instance identifier field of the target timed task in the database to the scheduler instance identifier of the current scheduler instance.
[0116] In some embodiments of the present invention, the apparatus further includes: The second instance identifier setting module sets the scheduler instance identifier field of the target scheduled task in the database to empty after the target scheduled task is executed.
[0117] In some embodiments of the present invention, the execution capability determination module 402 includes: The first quantity determination submodule is used to determine the first quantity of multiple target physical nodes that need to execute the target timed task; The second quantity determination submodule is used to determine the second number of physical nodes that execute the scheduled task through the current scheduler instance; The target scheduled task determination submodule is used to determine whether the target scheduled task is within the execution capacity of the current scheduler instance based on the first quantity and the second quantity.
[0118] In some embodiments of the present invention, the target timed task determination submodule is used for: Determine whether the sum of the first quantity and the second quantity is less than or equal to the upper limit of the quantity; wherein, the upper limit of the quantity is the upper limit of the number of physical nodes that can execute the scheduled task through the current scheduler instance; If the sum of the first quantity and the second quantity is less than or equal to the upper limit of the quantity, the target timed task is determined to be within the execution capability of the current scheduler instance.
[0119] In some embodiments of the present invention, the apparatus further includes: The second quantity update module is used to subtract the first quantity from the second quantity after the target timed task is completed, so as to update the second quantity.
[0120] In some embodiments of the present invention, the apparatus further includes: The task execution result determination module is used to obtain the task execution result of the target timed task; The runtime update module is used to determine the task type of the target scheduled task when the task execution result indicates that the task was executed successfully, and if the task type of the target scheduled task is a periodic task, update the next runtime of the target scheduled task according to the next scheduling time of the target scheduled task, so that the target scheduled task can be executed on multiple target physical nodes by starting a coroutine through the current scheduler instance at the updated next runtime.
[0121] In some embodiments of the present invention, the runtime update module is used for: Obtain the next scheduling time for the target timed task; If the next scheduling time is earlier than the current time, the next scheduling time is updated according to the execution interval of the target timed task until the updated next scheduling time is not earlier than the current time; wherein, the execution interval is the time interval between two consecutive executions of the target timed task; The next execution time of the target timed task is updated using the updated next scheduling time.
[0122] In some embodiments of the present invention, the apparatus further includes: The related data deletion module is used to delete the related data of the target timed task when the task type of the target timed task is a one-time task.
[0123] In some embodiments of the present invention, the apparatus further includes: The retry module is used to update the next run time of the target scheduled task through a retry mechanism when the task execution result indicates that the task execution has failed. In order to execute the target scheduled task on multiple target physical nodes through the coroutine started by the current scheduler instance at the updated next run time.
[0124] In some embodiments of the present invention, the retry module is used for: Obtain the retry interval for the target timed task; Update the next run time of the target timed task based on the retry interval.
[0125] In some embodiments of the present invention, the timed task is a task used to perform timed detection on multiple distributed physical nodes.
[0126] Some embodiments of the present invention also provide an electronic device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method described above.
[0127] Some embodiments of the present invention also provide a computer-readable storage medium on which a computer program is stored, and which, when executed by a processor, implements the method described above.
[0128] Some embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the method described above.
[0129] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.
[0130] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.
[0131] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0132] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0133] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0134] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0136] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.
[0137] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the aforementioned element.
[0138] The above provides a detailed description of a method, apparatus, device, and medium for processing timed tasks. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A method for processing timed tasks, characterized in that, Applied to a scheduler that executes scheduled tasks on physical nodes by creating scheduler instances, the method includes: Retrieve the target scheduled task to be executed from the unassigned scheduler instance; Determine whether the target scheduled task is within the execution capability of the current scheduler instance; If the target scheduled task is within the execution capacity of the current scheduler instance, the target scheduled task is assigned to the current scheduler instance, and the current scheduler instance starts a coroutine to execute the target scheduled task on multiple target physical nodes.
2. The method according to claim 1, characterized in that, Retrieve the target scheduled tasks to be executed for unassigned scheduler instances, including: From the database, determine that the scheduled tasks with an empty scheduler instance identifier field are scheduled tasks without an assigned scheduler instance, and from the scheduled tasks without an assigned scheduler instance, determine the scheduled task whose next execution time is closest to the current time as the target scheduled task. After assigning the target scheduled task to the current scheduler instance, the process further includes: Set the scheduler instance identifier field of the target scheduled task in the database to the scheduler instance identifier of the current scheduler instance.
3. The method according to claim 2, characterized in that, After starting a coroutine through the current scheduler instance to execute the target scheduled task on multiple target physical nodes, the process further includes: After the target scheduled task is completed, the scheduler instance identifier field of the target scheduled task in the database is set to empty.
4. The method according to claim 1, characterized in that, Determining whether the target scheduled task is within the execution capability of the current scheduler instance includes: Determine a first number of target physical nodes that need to execute the target timed task; Determine the second number of physical nodes that execute the scheduled task through the current scheduler instance; Based on the first quantity and the second quantity, it is determined whether the target scheduled task is within the execution capacity of the current scheduler instance.
5. The method according to claim 4, characterized in that, Based on the first quantity and the second quantity, determining whether the target scheduled task is within the execution capacity of the current scheduler instance includes: Determine whether the sum of the first quantity and the second quantity is less than or equal to the upper limit of the quantity; wherein, the upper limit of the quantity is the upper limit of the number of physical nodes that can execute the scheduled task through the current scheduler instance; If the sum of the first quantity and the second quantity is less than or equal to the upper limit of the quantity, the target timed task is determined to be within the execution capability of the current scheduler instance.
6. The method according to claim 4, characterized in that, After starting a coroutine through the current scheduler instance to execute the target scheduled task on multiple target physical nodes, the process further includes: After the target timed task is completed, the first quantity is subtracted from the second quantity to update the second quantity.
7. The method according to any one of claims 1-6, characterized in that, After starting a coroutine through the current scheduler instance to execute the target scheduled task on multiple target physical nodes, the process further includes: Obtain the task execution result of the target timed task; When the task execution result indicates that the task was executed successfully, the task type of the target timed task is determined. If the task type of the target timed task is a periodic task, the next run time of the target timed task is updated according to the next scheduling time of the target timed task. At the updated next run time, the target timed task is executed on multiple target physical nodes by starting a coroutine through the current scheduler instance.
8. The method according to claim 7, characterized in that, Update the next execution time of the target scheduled task based on its next scheduling time, including: Obtain the next scheduling time for the target timed task; If the next scheduling time is earlier than the current time, the next scheduling time is updated according to the execution interval of the target timed task until the updated next scheduling time is not earlier than the current time; wherein, the execution interval is the time interval between two consecutive executions of the target timed task; The next execution time of the target timed task is updated using the updated next scheduling time.
9. The method according to claim 7, characterized in that, Also includes: If the target timed task is a one-time task, delete the relevant data for the target timed task.
10. The method according to claim 7, characterized in that, Also includes: When the task execution result indicates that the task execution has failed, the next run time of the target scheduled task is updated through a retry mechanism so that the target scheduled task can be executed on multiple target physical nodes by starting a coroutine through the current scheduler instance at the updated next run time.
11. The method according to claim 10, characterized in that, The next execution time of the target scheduled task is updated through a retry mechanism, including: Obtain the retry interval for the target timed task; Update the next run time of the target timed task based on the retry interval.
12. The method according to claim 1, characterized in that, The scheduled task is used to perform timed detection on multiple distributed physical nodes.
13. A processing device for timed tasks, characterized in that, Applied to a scheduler that executes scheduled tasks on physical nodes by creating scheduler instances, the apparatus includes: The target scheduled task acquisition module is used to acquire the target scheduled tasks to be executed that have not been assigned to a scheduler instance; The execution capability judgment module is used to determine whether the target timed task is within the execution capability range of the current scheduler instance; The target scheduled task execution module is used to assign the target scheduled task to the current scheduler instance if the target scheduled task is within the execution capability of the current scheduler instance, and to start a coroutine through the current scheduler instance to execute the target scheduled task on multiple target physical nodes.
14. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method as described in any one of claims 1 to 12.
15. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the method as described in any one of claims 1 to 12.