Task processing method and device, equipment and storage medium
Through the combination of distributed scheduling and delay queues, the problem that timing tasks cannot be processed on time in the prior art is solved, and tasks are processed without delay are achieved, with high availability and high performance, and are suitable for business scenarios with high time accuracy requirements.
Patent Information
- Application Number
- CN202311182893.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-13
- Publication Date
- 2025-03-14
AI Technical Summary
The time interval of timing tasks in the prior art makes it impossible to process expired tasks on time, especially in business scenarios with high time accuracy requirements, and when the amount of data is large, scanning pressure increases, making it difficult to ensure time accuracy.
Through the combination of distributed scheduling and delay queue, a set of pending expired execution tasks is obtained, and distributed scheduling is performed based on the task execution time and preset time interval. The first expired execution task that each task process needs to execute is determined, and it is added to the delay queue, so that the task is processed without delay and the on-time expiration is achieved.
It realizes tasks without delay, has high availability and high performance, and can handle expired tasks on time in business scenarios with high time accuracy requirements, reducing scanning pressure.
Smart Images
Figure CN119621256A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to computer technology, and in particular to a task processing method, apparatus, device and storage medium. Background Art
[0002] With the rapid development of computer technology, we often encounter some business scenarios with time limits. For example, the gifted limited-time consumption amount needs to be cleared after expiration, or the order needs to be closed and inventory needs to be increased due to unpaid orders after the expiration date. Some tasks need to become invalid immediately at the expiration time and may be accompanied by the refresh of related data. Therefore, it is necessary to handle the tasks that are due to be executed as punctually as possible.
[0003] At present, usually a scheduled task is started, tasks that need to be processed due are scanned according to a set time interval, and corresponding due operations are performed after the due date is confirmed. However, in the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:
[0004] Since scheduled tasks have a certain time interval, such as once every 2 seconds or once every 1 minute, it is possible that the task cannot be executed when it expires, which is not suitable for business scenarios with high time accuracy requirements. Although the task timing interval can be shortened, all business data must be queried for each scan, which greatly increases the scanning pressure, especially when the amount of data is too large, and time accuracy cannot be guaranteed. Summary of the invention
[0005] The embodiments of the present invention provide a task processing method, apparatus, device and storage medium to achieve processing of tasks without delay, achieve on-time expiration, and have high availability and high performance.
[0006] In a first aspect, an embodiment of the present invention provides a task processing method, including:
[0007] Get the collection of pending due execution tasks;
[0008] Based on the task execution time corresponding to each due execution task in the due execution task set and the preset time interval, the due execution task set is distributedly scheduled, and the first due execution task that each task process in the distributed scheduling needs to execute within the preset time interval is determined;
[0009] Adding the first due execution task corresponding to each task process to the delay queue corresponding to the task process;
[0010] The second due execution task currently due is taken out from the delay queue, and the due operation corresponding to the second due execution task is executed.
[0011] In a second aspect, an embodiment of the present invention further provides a task processing device, including:
[0012] A task set acquisition module is used to acquire a set of pending due execution tasks;
[0013] A distributed scheduling module, configured to perform distributed scheduling on the set of overdue execution tasks based on the task execution time corresponding to each overdue execution task in the set of overdue execution tasks and a preset time interval, and determine the first overdue execution task that each task process in the distributed scheduling needs to execute within the preset time interval;
[0014] A task adding module, used for adding the first due execution task corresponding to each task process to a delay queue corresponding to the task process;
[0015] The queue processing module is used to take out the second due execution task that is currently due from the delay queue, and execute the due operation corresponding to the second due execution task.
[0016] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:
[0017] one or more processors;
[0018] A memory for storing one or more programs;
[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement the task processing method provided by any embodiment of the present invention.
[0020] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the task processing method provided by any embodiment of the present invention is implemented.
[0021] One embodiment of the above invention has the following advantages or beneficial effects:
[0022] By performing distributed scheduling on the due execution task set based on the task execution time corresponding to each due execution task in the due execution task set and the preset time interval, the first due execution task that each task process in the distributed scheduling needs to execute within the preset time interval is determined, so that distributed deployment can be achieved by using distributed scheduling, a large number of tasks are dispersed, and the first due execution task that is about to expire is added to the delay queue, the second due execution task that is currently expiring is taken out from the delay queue, and the expiration operation corresponding to the second due execution task is executed, so that the delay queue can be used to process a small number of dispersed tasks without delay, so as to achieve on-time expiration. By using highly available distributed scheduling and delay queues with no delay characteristics, tasks can be processed without delay, with high availability and high performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0024] Figure 1 is a flow chart of a task processing method provided by an embodiment of the present invention;
[0025] Figure 2 is a flow chart of another task processing method provided by an embodiment of the present invention;
[0026] Figure 3 is a flowchart of another task processing method provided by an embodiment of the present invention;
[0027] Figure 4 is a structural diagram of a task processing device provided by an embodiment of the present invention;
[0028] Figure 5 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0029] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.
[0030] Figure 1This is a flowchart of a task processing method provided by an embodiment of the present invention. This embodiment is applicable to the case of processing a large number of due execution tasks, especially to the case of executing corresponding due operations when the business expires in a business scenario with a time limit. The method can be executed by a task processing device, which can be implemented by software and / or hardware and integrated in an electronic device. Figure 1 As shown, the method specifically comprises the following steps:
[0031] S110: Obtain a set of pending due execution tasks.
[0032] Among them, the due execution task set may include multiple due execution tasks to be processed. Due execution tasks refer to tasks that need to be processed and executed when the specified time is reached, such as invalidation processing and refreshing related data when it expires. For example, due execution tasks can refer to tasks that clear the gifted limited-time consumption amount when it expires; due execution tasks can also refer to tasks that close orders and increase inventory when orders are not paid after timeout; due execution tasks can also refer to tasks that automatically resign when day-settled workers have been employed for 18 hours. Among them, day-settled workers are paid one day's salary for one day of work, and leave on the same day they join the job (here it refers to non-natural days, it can cross days), starting from the time of punching in. If it exceeds 18 hours, the employee needs to resign automatically, and the job resources are released to recruit new day-settled workers. The due task execution in the due execution task set can be tasks for the same business scenario or tasks for different business scenarios.
[0033] Specifically, the due execution tasks that need to be processed can be generated in real time in a heterogeneous manner. For example, each time a limited-time consumption amount is given away, a pending due execution task can be generated. Alternatively, each time a day worker clocks in, a pending due execution task can be generated. The pending due execution task set can be scanned and obtained at regular intervals. The amount of task data in each pending due execution task set is large, so it is necessary to solve the problem of a large amount of data expiring on time without delay. For example, in a business scenario where day workers automatically leave when their due date expires, nearly 100,000 due execution tasks can be generated in one day.
[0034] S120: Distribute the overdue execution task set based on the task execution time corresponding to each overdue execution task in the overdue execution task set and the preset time interval, and determine the first overdue execution task that each task process in the distributed scheduling needs to execute within the preset time interval.
[0035] Among them, the task execution time may refer to the time when the due task is executed, that is, the expiration time. For example, in a business scenario where daily workers automatically resign upon expiration, the task execution time may refer to the time obtained by adding 18 hours to the clock-in time. The preset time interval may be the maximum time interval between the current time and the expiration time, which is set in advance based on business needs, that is, the maximum remaining time from the current time to the expiration time. If the number of tasks is small, for example, there are several thousand tasks to be processed within 10 minutes, the preset time interval can be appropriately increased to retrieve more tasks.
[0036] Distributed scheduling can be performed by deploying multiple machines and starting one or more independent task processes on each machine to perform distributed task scheduling, so as to schedule all tasks to multiple task processes for concurrent execution, so as to disperse the task volume and improve processing efficiency. Distributed scheduling can be distributed scheduling of any framework. For example, distributed scheduling can be but not limited to Elastic-Job. The task process in Elastic-Job can refer to job sharding. Distributed scheduling can be periodic task scheduling according to a preset time. The preset time interval needs to be greater than the scheduling time interval during distributed scheduling. For example, the preset time interval is set to 1 minute, and the scheduling time interval during distributed scheduling can be set to 10 seconds, that is, task scheduling is performed every 10 seconds. The first due execution task can refer to an due execution task that is to be executed within a preset time interval. For example, if the preset time interval is 1 minute, the due execution task that is to be executed within a maximum of 1 minute can be determined as the first due execution task.
[0037] Specifically, the set of expired tasks can be distributedly scheduled to determine the task scheduling range of each task process, and based on the task execution time corresponding to each expired task within the task scheduling range, it is detected whether each expired task will expire within a preset time interval, and the expired task that will expire within a preset time interval is taken out as the first expired task, so that each task process can obtain the first expired task that is about to expire from the set of expired tasks, thereby realizing distributed deployment. It should be noted that if there is an expired task whose task execution time is earlier than the current time due to an abnormal situation, the expired task can be directly used as the first expired task so that the corresponding expiration operation can be performed immediately afterwards.
[0038] It should be noted that if each task process obtains a task that has just expired, it will be impossible to meet the time accuracy requirement due to the scheduling time interval as in the existing method. By obtaining tasks that will expire within a preset time interval, it is possible to avoid the situation where the task should be executed but is not executed before the scheduling time due to the scheduling time interval, thereby improving the time control accuracy.
[0039] S130: Add the first due execution task corresponding to each task process to the delay queue corresponding to the task process.
[0040] Different tasks in the delay queue can correspond to different delay times, that is, the tasks can be taken out for execution only when the specified deadline is reached. For example, the delay queue can be but not limited to the DelayQueue in the Java tool, or a delay queue implemented based on the time wheel algorithm. The delay queue has the characteristic of pure memory operation without delay, so that the punctuality of expiration can be guaranteed. Each task process can correspond to a delay queue.
[0041] Specifically, each task process may add the first due execution task to the corresponding delay queue, so as to perform more precise time control by utilizing the pure memory operation of the delay queue.
[0042] Exemplarily, step S130 may include: determining the remaining execution time corresponding to the first due execution task based on the current time and the task execution time corresponding to the first due execution task; adding the first due execution task corresponding to each task process and the remaining execution time corresponding to the first due execution task to the corresponding delay queue.
[0043] Specifically, the time difference obtained by subtracting the current time from the task execution time corresponding to the first due execution task can be used as the remaining execution time corresponding to the first due execution task, that is, the time that the delay queue needs to delay execution. If the task execution time is earlier than the current time, the remaining execution time is determined to be 0 so that it can be executed immediately without delay. By adding each first due execution task and the corresponding remaining execution time corresponding to each task process to the corresponding delay queue, the delay queue can release the corresponding task immediately when the corresponding remaining execution time is delayed, thereby meeting the time requirement of no delay.
[0044] S140: Take out the second due execution task that is currently due from the delay queue, and execute the due operation corresponding to the second due execution task.
[0045] Among them, the second due execution task may refer to the first due execution task in the delay queue that has reached the current task execution time. The due operation corresponding to the second due execution task may refer to the operation that needs to be performed when the specified time is reached, that is, the operation of expiring the limited-time business. For example, the due operation may refer to the operation of clearing the gifted limited-time consumption amount to make the gifted consumption amount expire and unusable. The due operation may also refer to the operation of closing overdue and unpaid orders and increasing inventory. The due operation may also refer to the operation of automatically resigning daily workers.
[0046] Specifically, when each task process detects that the current time reaches the task execution time corresponding to the first due execution task in the delay queue, it indicates that the first due execution task has expired. The first due execution task can be taken out of the delay queue as the second due execution task, and the expiration operation corresponding to the second due execution task can be executed, so that the time-limited service can be expired on time.
[0047] Exemplarily, step S140 may include: taking out the second due execution task that is currently due from the delay queue based on the remaining execution time. Specifically, after delaying the remaining execution time corresponding to each first due execution task, taking out the second due execution tasks that are due in sequence from the delay queue and performing the corresponding expiration operations, thereby achieving on-time expiration.
[0048] It should be noted that the amount of tasks to be processed that are due is large and cannot be processed using only the delay queue. If all tasks are added to the delay queue, the delay queue runs in memory. Once a shutdown or restart occurs, the data will be lost. At this time, all the data needs to be written in, and a full query operation needs to be performed. This query time is too long and the time accuracy cannot be effectively guaranteed. This embodiment can query out the tasks that are about to expire by utilizing distributed scheduling, and add them to the delay queue with higher time accuracy for processing, so that the amount of tasks can be reduced and dispersed, which can avoid the problem of the scheduling time interval of distributed scheduling, and can also utilize the delay-free characteristics of the pure memory operation of the delay queue to ensure the punctuality of expiration. At the same time, the distributed scheduling only queries a small number of tasks (i.e., tasks that are about to expire) each time, which will not affect the performance of the distributed scheduling, thereby ensuring high performance.
[0049] In addition, distributed scheduling itself has high availability. When the distributed scheduling registration center senses that the machine is down, although the tasks that have not been executed in the delay queue will disappear, the downtime task process will be transferred to another surviving machine during the next task scheduling, so that the tasks that were not executed in time due to the downtime can be loaded into the task process of another machine again during the next task scheduling, thereby ensuring that the tasks that have not been executed due to the downtime can continue to be executed, achieving high availability. The preset time interval (such as 1 minute) needs to be greater than the scheduling time interval (such as 10 seconds), so that the task can be taken out again during the next scheduling, and the task execution time has not been reached, thus ensuring high availability.
[0050] The technical solution of this embodiment performs distributed scheduling on the due execution task set based on the task execution time corresponding to each due execution task in the due execution task set and the preset time interval, and determines the first due execution task that each task process in the distributed scheduling needs to execute within the preset time interval, so that distributed deployment can be achieved by using distributed scheduling, a large number of tasks are dispersed, and the first due execution task that is about to expire is added to the delay queue, the second due execution task that is currently expiring is taken out from the delay queue, and the expiration operation corresponding to the second due execution task is executed, so that the delay queue can be used to process a small number of dispersed tasks without delay, so as to achieve timely expiration. By using highly available distributed scheduling and a delay queue with no delay characteristics, tasks can be processed without delay, with high availability and high performance.
[0051] On the basis of the above technical solution, after step S140, the following may be included: removing the second due execution task from the due execution task set; or changing the task status of the second due execution task from a pending status to a processed status.
[0052] Specifically, after the second due execution task is completed, the second due execution task can be removed from the due execution task set to avoid rescheduling the completed task, and it is also ensured that multiple scheduling can be performed before the task is not completed. Alternatively, the task state of the completed second due execution task is changed from a pending state to a processed state, so that only the due execution tasks in the unprocessed state are distributedly scheduled, thereby ensuring high availability of task processing.
[0053] Figure 2 This is a flowchart of another task processing method provided by an embodiment of the present invention. Based on the above embodiments, this embodiment describes the specific process of distributed scheduling in detail. The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here.
[0054] See also Figure 2 Another task processing method provided in this embodiment specifically includes the following steps:
[0055] S210: Obtain a set of pending due execution tasks.
[0056] The set of due execution tasks can be represented in the form of a list. For example, in the business scenario of automatic resignation upon expiration of daily wages, the due execution task table is shown in Table 1:
[0057] Table 1 Daily work expiration automatic resignation task list
[0058] Daily wage ID Number of task executions Task Status Task number Task execution time 10001 0 0 0 2023-7-17 17:48:08 10002 0 0 1 2023-7-17 21:21:45 10003 0 0 2 2023-7-17 12:22:25 …… …… …… …… ……
[0059] Each row in Table 1 represents an expired execution task. The daily worker ID is used to uniquely represent the identification information of the daily worker so as to distinguish different daily workers. The number of task executions may refer to the number of times the task has been executed currently, which is used to control the number of failed retries of the task when the task fails to execute due to timeout, system abnormality, etc. For example, if the maximum number of task executions configured is 6, each time the task fails to execute, the number of task executions corresponding to the task is increased by 1. When the number of task executions is 6, the task will not be scheduled and executed, thereby avoiding the situation where some tasks always fail and execute, occupying resources. The task status may include an unprocessed state or a processed state, wherein the unprocessed state may be represented by the number "0". The processed state may be represented by the number "1". The task sequence number may be the task identification information represented by a number. The task execution time may refer to the expiration time of the task. For example, the task execution time may be the time obtained by adding 18 hours to the clock-in time.
[0060] S220: Divide the set of due execution tasks into tasks, and determine the due execution tasks that can be scheduled for each task process.
[0061] Specifically, each due execution task in the due execution task set can be allocated as a due execution task that can be scheduled by each task process in a polling manner. For example, if the due execution task set includes 10 tasks and there are 3 task processes, the due execution tasks that can be scheduled by task process 1 are: task 1, task 4, task 7 and task 10; the due execution tasks that can be scheduled by task process 2 are: task 2, task 5 and task 8; the due execution tasks that can be scheduled by task process 3 are: task 3, task 6 and task 9.
[0062] Exemplarily, step S220 may include: dividing the due execution task set based on the number of task processes, the process number corresponding to each task process, and the task sequence number corresponding to each due execution task in the due execution task set, and determining the due execution tasks that can be scheduled for each task process.
[0063] Among them, the process number corresponding to each task process can be process identification information represented by numbers. For example, if there are 16 task processes, the process numbers corresponding to the 16 task processes are 0 to 15 respectively. The task serial numbers corresponding to the overdue execution tasks can be based on a preset serial number range, and the overdue execution tasks are cyclically allocated to disperse the tasks. For example, if the task needs to be divided into 64 parts, the preset serial number range can be set to 0 to 63. At this time, the task serial number corresponding to the first overdue execution task is 0, and the task serial number corresponding to the second overdue execution task is 1. And so on, the task serial number corresponding to the 64th overdue execution task is 63, and the task serial number corresponding to the 65th overdue execution task becomes 0 again, thereby realizing the cyclic allocation of task serial numbers.
[0064] Specifically, based on the process number corresponding to each task process and the task sequence number corresponding to each overdue execution task in the overdue execution task set, the overdue execution task set can be divided into the same number of task processes, and each of the divided tasks can be used as an overdue execution task that can be scheduled by each task process, thereby realizing distributed scheduling.
[0065] Exemplarily, step S220 specifically includes: dividing the task sequence number corresponding to each overdue execution task in the overdue execution task set by the number of task processes to obtain the remainder corresponding to each overdue execution task; based on the remainder corresponding to each overdue execution task and the process number corresponding to each task process, determining the overdue execution tasks that can be scheduled for each task process.
[0066] Specifically, the task number corresponding to each due execution task in the due execution task set can be divided by the number of task processes by the modulo method to obtain the remainder corresponding to each due execution task, and the due execution task whose remainder is the current task process is determined as the due execution task that can be scheduled by the current task process. For example, the task number corresponding to the due execution task ranges from 0 to 63, and the current task process is the first task process, that is, the process number is 0. The task number corresponding to each due execution task (a value from 0 to 63) is divided by the number of task processes (that is, 16). The task number with a remainder of 0 is the task number that can be scheduled by the first task process, that is, the task numbers that can be scheduled by the first task process are 0, 16, 32 and 48. At this time, the due execution tasks with task numbers 0, 16, 32 and 48 in the due execution task set are due execution tasks that can be scheduled by the first task process.
[0067] S230: Determine a first due execution task that each task process needs to execute within the preset time interval based on the task execution time corresponding to the schedulable due execution task and the preset time interval.
[0068] Specifically, for each task process, it is detected whether the task execution time corresponding to each schedulable due execution task will expire within a preset time interval. If so, the due execution task is used as the first due execution task that needs to be executed within the preset time interval.
[0069] Exemplarily, step S230 may include: adding the current time and the preset time interval to obtain the target time; comparing the task execution time corresponding to the due execution task that can be scheduled by the current task process with the target time; determining the due execution task that is less than or equal to the target time as the first due execution task that the current task process needs to execute within the preset time interval. By taking out only the due execution tasks that are about to expire by the task process, it is possible to avoid the time accuracy that cannot meet the expiration requirement due to the scheduling time interval.
[0070] S240: Add the first due execution task corresponding to each task process to the delay queue corresponding to the task process.
[0071] S250: Take out the second due execution task that is currently due from the delay queue, and execute the due operation corresponding to the second due execution task.
[0072] The technical solution of this embodiment determines the schedulable due execution tasks for each task process by dividing the due execution task set, and determines the first due execution task that each task process needs to execute within the preset time interval based on the task execution time corresponding to the schedulable due execution tasks and the preset time interval. Therefore, during distributed scheduling, only the due execution tasks that are about to expire are scheduled, avoiding the failure to meet the time accuracy required for expiration due to the scheduling time interval.
[0073] Figure 3 A flowchart of another task processing method provided by an embodiment of the present invention. Based on the above embodiments, this embodiment optimizes the step of "adding the first due execution task corresponding to each task process to the delay queue corresponding to the task process". The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here.
[0074] See also Figure 3 Another task processing method provided in this embodiment specifically includes the following steps:
[0075] S310: Obtain a set of pending due execution tasks.
[0076] S320: Based on the task execution time corresponding to each overdue task in the overdue task set and the preset time interval, the overdue task set is distributedly scheduled, and the first overdue task that each task process in the distributed scheduling needs to execute within the preset time interval is determined.
[0077] S330: Obtain an anti-duplicate task set corresponding to the current task process, where the anti-duplicate task set includes: tasks that have not yet been processed and are in a current delay queue corresponding to the current task process.
[0078] Among them, the anti-duplicate task set can only be used to store non-duplicate tasks. For example, the anti-duplicate task set can be but is not limited to the HashSet in the Java tool. Each task process corresponds to an anti-duplicate task set to store the tasks that have not been processed yet in the delay queue corresponding to the task process.
[0079] It should be noted that due tasks are removed or their status is changed only after they are executed to mark the task as completed. Before the task is executed, it is always in the collection, so that the same task may be scheduled multiple times during distributed scheduling. Therefore, it is necessary to use the anti-duplicate task collection to deduplicate and avoid repeatedly adding the same task to the delay queue.
[0080] Specifically, each task process may be used as the current task process, and tasks may be added to the corresponding delay queue by executing steps S330 - S350 .
[0081] S340: Based on the first due execution task and the anti-duplicate task set corresponding to the current task process obtained in the current scheduling, determine the first due execution task that does not exist in the anti-duplicate task set.
[0082] Specifically, the first due execution task corresponding to the current task process obtained by the current scheduling can be stored in the anti-duplicate task set, and the first due execution task that can be successfully stored is used as the first due execution task that does not exist in the anti-duplicate task set. The first due execution task that fails to be stored is used as the first due execution task that already exists in the anti-duplicate task set.
[0083] S350: Add the first due execution task that does not exist in the anti-duplicate task set to the current delay queue corresponding to the current task process.
[0084] Specifically, the first due execution task that has not been added among the first due execution tasks corresponding to the current task process obtained in the current scheduling, that is, the first due execution task that does not exist in the anti-duplicate task set, is added to the current delay queue corresponding to the current task process, thereby avoiding repeated addition of the same task to the delay queue, and further avoiding multiple executions of the same task, thereby improving resource utilization.
[0085] S360: Take out the second due execution task that is currently due from the delay queue, and execute the due operation corresponding to the second due execution task.
[0086] Exemplarily, after the second due execution task is completed, it needs to be removed from the anti-duplicate task set to update the anti-duplicate task set so that the anti-duplicate task set only stores tasks that have not yet been processed, thereby ensuring the accuracy of anti-duplicate processing.
[0087] The technical solution of this embodiment determines the first due execution task that does not exist in the anti-duplicate task set based on the first due execution task corresponding to the current task process obtained in the current scheduling and the anti-duplicate task set, and adds the first due execution task that does not exist in the anti-duplicate task set to the current delay queue corresponding to the current task process, thereby avoiding repeated addition of the same task to the delay queue, thereby avoiding multiple executions of the same task, and improving resource utilization.
[0088] The following is an embodiment of a task processing device provided by an embodiment of the present invention. The device and the task processing methods of the above embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the task processing device, reference can be made to the embodiment of the above task processing method.
[0089] Figure 4 This is a schematic diagram of the structure of a task processing device provided by an embodiment of the present invention. This embodiment is applicable to the case of processing a large number of due execution tasks, especially to the case of executing corresponding due operations when the business expires in a business scenario with time limits. Figure 4 As shown, the device specifically includes: a task set acquisition module 410, a distributed scheduling module 420, a task adding module 430 and a queue processing module 440.
[0090] Among them, the task set acquisition module 410 is used to obtain the set of overdue execution tasks to be processed; the distributed scheduling module 420 is used to perform distributed scheduling on the overdue execution task set based on the task execution time corresponding to each overdue execution task in the overdue execution task set and the preset time interval, and determine the first overdue execution task that each task process in the distributed scheduling needs to execute within the preset time interval; the task adding module 430 is used to add the first overdue execution task corresponding to each task process to the delay queue corresponding to the task process; the queue processing module 440 is used to take out the second overdue execution task that is currently due from the delay queue, and execute the expiration operation corresponding to the second overdue execution task.
[0091] The technical solution of this embodiment performs distributed scheduling on the due execution task set based on the task execution time corresponding to each due execution task in the due execution task set and the preset time interval, and determines the first due execution task that each task process in the distributed scheduling needs to execute within the preset time interval, so that distributed deployment can be achieved by using distributed scheduling, a large number of tasks are dispersed, and the first due execution task that is about to expire is added to the delay queue, the second due execution task that is currently expiring is taken out from the delay queue, and the expiration operation corresponding to the second due execution task is executed, so that the delay queue can be used to process a small number of dispersed tasks without delay, so as to achieve timely expiration. By using highly available distributed scheduling and a delay queue with no delay characteristics, tasks can be processed without delay, with high availability and high performance.
[0092] Optionally, the distributed scheduling module 420 includes:
[0093] A task division unit, used to divide the set of overdue execution tasks and determine the overdue execution tasks that can be scheduled for each task process;
[0094] The distributed scheduling unit is used to determine the first due execution task that each task process needs to execute within the preset time interval based on the task execution time corresponding to the schedulable due execution task and the preset time interval.
[0095] Optionally, the task division unit is specifically used to:
[0096] Based on the number of task processes, the process number corresponding to each task process, and the task sequence number corresponding to each due execution task in the due execution task set, the due execution task set is divided into tasks to determine the due execution tasks that can be scheduled for each task process.
[0097] Optionally, the task division unit is specifically used to:
[0098] Divide the task sequence number corresponding to each due execution task in the due execution task set by the number of task processes to obtain the remainder corresponding to each due execution task;
[0099] Based on the remainder corresponding to each due execution task and the process number corresponding to each task process, the due execution tasks that can be scheduled for each task process are determined.
[0100] Optionally, the distributed scheduling unit is specifically used to:
[0101] Add the current time and the preset time interval to obtain the target time; compare the task execution time corresponding to the due execution task that can be scheduled by the current task process with the target time; determine the due execution task that is less than or equal to the target time as the first due execution task that the current task process needs to execute within the preset time interval.
[0102] Optionally, the task adding module 430 is specifically used to: determine the remaining execution time corresponding to the first due execution task based on the current time and the task execution time corresponding to the first due execution task; add the first due execution task corresponding to each task process and the remaining execution time corresponding to the first due execution task to the corresponding delay queue;
[0103] The queue processing module 440 is specifically configured to: based on the remaining execution time, take out the second due execution task that is currently due from the delay queue.
[0104] Optionally, the task adding module 430 is specifically used to:
[0105] Obtaining an anti-duplicate task set corresponding to the current task process, the anti-duplicate task set comprising: tasks that have not yet been processed and are in a current delay queue corresponding to the current task process;
[0106] Based on the first due execution task corresponding to the current task process obtained in the current scheduling and the anti-duplicate task set, determine the first due execution task that does not exist in the anti-duplicate task set;
[0107] The first due execution task that does not exist in the anti-duplicate task set is added to the current delay queue corresponding to the current task process.
[0108] Optionally, the device further comprises:
[0109] The task processing module is used to remove the second due execution task from the due execution task set after executing the due operation corresponding to the second due execution task; or change the task status of the second due execution task from a pending status to a processed status.
[0110] The task processing device provided in the embodiment of the present invention can execute the task processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the task processing method.
[0111] It is worth noting that in the embodiment of the above-mentioned task processing device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.
[0112] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 5 A block diagram of an exemplary electronic device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 5 The electronic device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0113] like Figure 5 As shown, the electronic device 12 is in the form of a general purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).
[0114] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.
[0115] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.
[0116] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 5 not shown, usually called a "hard drive"). Although Figure 5 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.
[0117] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.
[0118] The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 12, and / or may communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the electronic device 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0119] The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, for example, implementing a task processing method provided by the embodiment of the present invention, the method comprising:
[0120] Get the collection of pending due execution tasks;
[0121] Based on the task execution time corresponding to each due execution task in the due execution task set and the preset time interval, the due execution task set is distributedly scheduled, and the first due execution task that each task process in the distributed scheduling needs to execute within the preset time interval is determined;
[0122] Adding the first due execution task corresponding to each task process to the delay queue corresponding to the task process;
[0123] The second due execution task currently due is taken out from the delay queue, and the due operation corresponding to the second due execution task is executed.
[0124] Of course, those skilled in the art can understand that the processor can also implement the technical solution of the task processing method provided by any embodiment of the present invention.
[0125] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the task processing method provided in any embodiment of the present invention are implemented. The method includes:
[0126] Get the collection of pending due execution tasks;
[0127] Based on the task execution time corresponding to each due execution task in the due execution task set and the preset time interval, the due execution task set is distributedly scheduled, and the first due execution task that each task process in the distributed scheduling needs to execute within the preset time interval is determined;
[0128] Adding the first due execution task corresponding to each task process to the delay queue corresponding to the task process;
[0129] The second due execution task currently due is taken out from the delay queue, and the due operation corresponding to the second due execution task is executed.
[0130] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.
[0131] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0132] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
[0133] Computer program code for performing the operations of the present invention may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0134] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.
[0135] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims
1. A task processing method, characterized in that: include: Get the collection of pending due execution tasks; Based on the task execution time corresponding to each due execution task in the due execution task set and the preset time interval, the due execution task set is distributedly scheduled, and the first due execution task that each task process in the distributed scheduling needs to execute within the preset time interval is determined; Adding the first due execution task corresponding to each task process to the delay queue corresponding to the task process; The second due execution task currently due is taken out from the delay queue, and the due operation corresponding to the second due execution task is executed.
2. The method according to claim 1, characterized in that The method of performing distributed scheduling on the set of overdue execution tasks based on the task execution time corresponding to each overdue execution task in the set of overdue execution tasks and the preset time interval, and determining the first overdue execution task that each task process in the distributed scheduling needs to execute within the preset time interval includes: Dividing the set of overdue execution tasks into tasks, and determining the overdue execution tasks that can be scheduled for each task process; Based on the task execution time corresponding to the schedulable due execution task and the preset time interval, a first due execution task that each task process needs to execute within the preset time interval is determined.
3. The method according to claim 2, characterized in that The step of dividing the set of due execution tasks to determine the due execution tasks that can be scheduled for each task process includes: Based on the number of task processes, the process number corresponding to each task process, and the task sequence number corresponding to each due execution task in the due execution task set, the due execution task set is divided into tasks to determine the due execution tasks that can be scheduled for each task process.
4. The method according to claim 3, characterized in that The method divides the due execution task set into tasks based on the number of task processes, the process number corresponding to each task process, and the task sequence number corresponding to each due execution task in the due execution task set, and determines the due execution tasks that can be scheduled for each task process, including: Divide the task sequence number corresponding to each due execution task in the due execution task set by the number of task processes to obtain the remainder corresponding to each due execution task; Based on the remainder corresponding to each due execution task and the process number corresponding to each task process, the due execution tasks that can be scheduled for each task process are determined.
5. The method according to claim 2, characterized in that: The determining, based on the task execution time corresponding to the schedulable due execution task and the preset time interval, the first due execution task that each task process needs to execute within the preset time interval includes: Add the current time to the preset time interval to obtain the target time; Compare the task execution time corresponding to the due execution task that can be scheduled by the current task process with the target time; The due execution task whose time is less than or equal to the target time is determined as the first due execution task that the current task process needs to execute within the preset time interval.
6. The method according to claim 1, characterized in that The adding the first due execution task corresponding to each task process to the delay queue corresponding to the task process includes: Determine the remaining execution time corresponding to the first due execution task based on the current time and the task execution time corresponding to the first due execution task; Adding the first due execution task corresponding to each task process and the remaining execution time corresponding to the first due execution task to the corresponding delay queue; The taking out the currently due second due execution task from the delay queue includes: Based on the remaining execution time, the second due execution task that is currently due is taken out from the delay queue.
7. The method according to claim 1, characterized in that The adding the first due execution task corresponding to each task process to the delay queue corresponding to the task process includes: Obtaining an anti-duplicate task set corresponding to the current task process, the anti-duplicate task set comprising: tasks that have not yet been processed and are in a current delay queue corresponding to the current task process; Based on the first due execution task corresponding to the current task process obtained in the current scheduling and the anti-duplicate task set, determine the first due execution task that does not exist in the anti-duplicate task set; The first due execution task that does not exist in the anti-duplicate task set is added to the current delay queue corresponding to the current task process.
8. The method according to any one of claims 1 to 7, characterized in that: After executing the expiration operation corresponding to the second expiration execution task, the method further includes: remove the second due execution task from the due execution task set; or, The task status of the second due execution task is changed from a pending status to a processed status.
9. A task processing device, characterized in that: include: A task set acquisition module is used to acquire a set of pending due execution tasks; A distributed scheduling module, configured to perform distributed scheduling on the set of overdue execution tasks based on the task execution time corresponding to each overdue execution task in the set of overdue execution tasks and a preset time interval, and determine the first overdue execution task that each task process in the distributed scheduling needs to execute within the preset time interval; A task adding module, used for adding the first due execution task corresponding to each task process to a delay queue corresponding to the task process; The queue processing module is used to take out the second due execution task that is currently due from the delay queue, and execute the due operation corresponding to the second due execution task.
10. An electronic device, characterized in that: The electronic device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the task processing method as described in any one of claims 1 to 8.
11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the task processing method as described in any one of claims 1 to 8 is implemented.