A task scheduling method, device and medium

Abstract

The present invention discloses a task scheduling method, device and medium, which are applicable to the field of data processing technology. By dividing the task into multiple standard tasks to be scheduled, and splitting into multiple execution windows at the same time, executing the current standard task to be scheduled in one execution window and then executing the next standard task to be scheduled, that is, executing the scheduled tasks in series, recording the waiting time during the scheduling process to realize timely perception of the pressure of task execution on the downstream, if the waiting time reaches the maximum execution time, it means that the downstream pressure is large, abandoning the current standard task to be scheduled and dynamically adjusting the scheduling configuration parameters so as to obtain the next standard task to be scheduled in the subsequent current execution window. Avoid the problem of downstream overload and server failure to work normally under normal circumstances, and improve the stability of concurrent scheduling of large batches of tasks.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a task scheduling method, device and medium. Background Art

[0002] When processing data, the input data generally contains multiple items, and there is no human interaction in the processing process. A computer program executes a series of tasks based on a batch of input to realize large-scale task scheduling and processing.

[0003] Normally, large batches of tasks are generally scheduled at one time (upstream system) to threads for asynchronous processing (downstream system). The results returned by the downstream system may be scheduled to time out and cause downstream overload. The upstream system is unaware of the call results and continues to request the downstream system at the previous rate and concurrency, causing a vicious cycle and possibly an "avalanche effect". If the returned results are scheduled to be interrupted, the subsequent tasks will fail to return. The above two situations result in poor stability of the scheduled tasks, causing the server to fail to work normally or even crash.

[0004] Therefore, it is urgent for those skilled in the art to find a method for scheduling tasks. Summary of the invention

[0005] The purpose of the present invention is to provide a task scheduling method, device and medium to improve the stability of the scheduled tasks.

[0006] In order to solve the above technical problems, the present invention provides a task scheduling method, comprising:

[0007] Obtain the scheduling configuration parameters within the current execution window so that the scheduling thread can obtain the current standard tasks to be scheduled according to the scheduling configuration parameters;

[0008] Put the standard tasks to be scheduled into the execution queue for scheduling so that the scheduling thread can wait for all the return results of the working threads in the thread pool;

[0009] Record the waiting time of the scheduling thread and determine whether the waiting time reaches the maximum execution time of the standard scheduled task;

[0010] If it is reached, the standard task to be scheduled is abandoned, the scheduling configuration parameters are adjusted and the process returns to the step of obtaining the scheduling configuration parameters in the current execution window.

[0011] Preferably, the standard to-be-scheduled tasks are placed into the execution queue for scheduling, including:

[0012] Record the start execution time of the standard scheduled tasks in the execution queue;

[0013] Determine whether the execution queue is empty;

[0014] If it is not empty, determine whether the start execution time is within the executable time of the current execution window;

[0015] If it is within the executable time, the standard scheduled tasks assigned to the working thread will be executed concurrently so that the scheduling thread can wait for all the return results of the working thread.

[0016] Preferably, obtaining the scheduling configuration parameters within the current execution window includes:

[0017] Determine whether there are tasks to be scheduled within the current execution window;

[0018] If yes, get the scheduling configuration parameters.

[0019] Preferably, when the waiting time does not reach the maximum execution time of the standard to-be-scheduled task, the method further includes:

[0020] When the waiting time does not reach the maximum execution time, the control scheduling thread obtains the current standard task to be scheduled according to the scheduling configuration parameters so as to enter the step of putting the standard task to be scheduled into the execution queue for scheduling processing.

[0021] Preferably, when the waiting time reaches the maximum execution time of the standard to-be-scheduled task, the method further includes:

[0022] Set the executable time of the current execution window to 0.

[0023] Preferably, after abandoning the standard to-be-scheduled task, the method further includes:

[0024] Treat the abandoned standard to-be-scheduled tasks as failed tasks;

[0025] Place the failed task in the next execution window for execution.

[0026] Preferably, the scheduling configuration parameters include the task volume, maximum execution time, executable time and number of working threads of the standard task to be scheduled.

[0027] In order to solve the above technical problems, the present invention further provides a task scheduling device, comprising:

[0028] An acquisition module is used to acquire the scheduling configuration parameters in the current execution window so that the scheduling thread can acquire the current standard task to be scheduled according to the scheduling configuration parameters;

[0029] The scheduling module is used to put the standard tasks to be scheduled into the execution queue for scheduling so that the scheduling thread waits for all the return results of the working threads in the thread pool;

[0030] A judgment module is used to record the waiting time of the scheduling thread and judge whether the waiting time reaches the maximum execution time of the standard scheduled task. If so, the abandonment module is triggered;

[0031] The abandonment module is used to abandon the standard to-be-scheduled task, adjust the scheduling configuration parameters and return to the step of obtaining the scheduling configuration parameters in the current execution window.

[0032] In order to solve the above technical problems, the present invention further provides a task scheduling device, comprising:

[0033] Memory for storing computer programs;

[0034] A processor is used to implement the steps of the above-mentioned task scheduling method when executing a computer program.

[0035] In order to solve the above technical problems, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the scheduling method for the above tasks are implemented.

[0036] The present invention provides a task scheduling method, which obtains the scheduling configuration parameters in the current execution window and then obtains the current standard task to be scheduled, puts the standard task to be scheduled into the execution queue for scheduling processing so that the scheduling thread waits for all the return results of the working thread; records the waiting time of the scheduling thread and determines whether the waiting time reaches the maximum execution time of the standard task to be scheduled; if it reaches, it means that the downstream pressure is too large, and the standard task to be scheduled is abandoned, and the scheduling configuration parameters are dynamically adjusted so that the next standard task to be scheduled is obtained in the subsequent current execution window. The method divides a large batch of tasks into multiple standard tasks to be scheduled, and splits them into multiple execution windows at the same time, executes the current standard task to be scheduled in an execution window and then executes the next standard task to be scheduled, that is, serially executes the scheduling tasks, records the waiting time during the scheduling processing to realize timely perception of the pressure of the task execution on the downstream, if the waiting time reaches the maximum execution time, it means that the downstream pressure is large, abandons the current standard task to be scheduled and adjusts the scheduling configuration parameters. Avoid the problems of downstream overload and server failure to work normally under normal circumstances, and improve the stability of concurrent scheduling of large batches of tasks.

[0037] In addition, the present invention also provides a task scheduling device and medium, which have the same beneficial effects as the above-mentioned task scheduling method. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A flowchart of a task scheduling method provided by an embodiment of the present invention;

[0040] Figure 2 A structural diagram of a task scheduling device provided by an embodiment of the present invention;

[0041] Figure 3 A structural diagram of another task scheduling device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] The core of the present invention is to provide a task scheduling method, device and medium to improve the stability of the scheduled tasks.

[0044] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0045] It should be noted that in a software system, there are two types of tasks that need to be run at a fixed time, one is executed once at a fixed time interval, and the other is triggered once when the local time reaches a certain hour. Usually, the timing of a task is scheduled once after it is triggered. The task scheduling method provided by the present invention uses the execution window as the unit for the entire process. By configuring a scheduled task that is executed once at a fixed time interval, a large task is split into a fixed time period for completion, and the time period between two scheduled task trigger points is the execution window.

[0046] Figure 1 A flowchart of a task scheduling method provided by an embodiment of the present invention, such as Figure 1 As shown, the method includes:

[0047] S11: Obtain the scheduling configuration parameters in the current execution window so that the scheduling thread can obtain the current standard task to be scheduled according to the scheduling configuration parameters.

[0048] It is understandable that the duration of the current execution window is the time interval at which the task is triggered. The specific value of the duration is not limited, but it should not be too long. Too long will cause temporary failed task scheduling starvation. The so-called task scheduling starvation refers to tasks that could have been successfully scheduled but have not been successfully scheduled for a long time due to various reasons. Too long is not conducive to the instability caused by scheduling failure and the rapid adjustment of key pressure parameters. Of course, it should not be too short, which will affect the task throughput. It is set according to the actual business scenario, generally within 10 minutes to 1 hour.

[0049] Specifically, when a task is triggered, at the time point when an execution window starts, the scheduling configuration parameters in the current execution window are obtained, and the latest scheduling configuration parameters are updated if there are tasks to be scheduled in the current execution window. The latest scheduling configuration parameters are obtained and updated in the current execution window if the scheduling configuration parameters are adjusted in the previous execution window to apply to the tasks to be scheduled in the current window.

[0050] The scheduling configuration parameters may include the task amount of a standard unit of scheduled tasks, the maximum execution time of the standard scheduled tasks, the executable time in the execution window, the number of threads required for the current scheduled tasks, etc. Multiple standard scheduled tasks can be executed in one execution window, and the scheduled tasks are divided into multiple standard scheduled tasks according to the scheduling configuration parameters, and the number of standard scheduled tasks that can be executed in the current execution window is allocated according to the executable time in the execution window.

[0051] During the executable time in the execution window, the scheduling thread obtains the current standard tasks to be scheduled for subsequent concurrent processing. The scheduling thread evenly distributes the obtained standard tasks to be scheduled to the working threads according to the number of working threads for concurrent processing.

[0052] S12: Put the standard to-be-scheduled tasks into the execution queue for scheduling so that the scheduling thread waits for all the return results of the working threads in the thread pool.

[0053] In step S11, the acquired standard tasks to be scheduled are placed in the execution queue so that the working thread can obtain the assigned standard tasks to be scheduled for scheduling. The scheduling thread needs to check the current execution queue and the time to start execution in the window. At the same time, the scheduling thread acts as a management thread to monitor and allocate resources of the working thread, and waits for the progress of the working thread in processing the standard tasks to be scheduled and all returned results.

[0054] Specifically, the scheduling process needs to record the current start execution time so that the waiting time for all returned results can be known later. The worker thread places the tasks in the execution queue in the worker thread for asynchronous processing based on the currently configured number of concurrent threads.

[0055] It can be understood that since the event of the scheduling thread is the return result for all working threads, there are multiple return results, including the result of the working thread successfully completing the progress, the result of the unfinished assigned standard scheduling task as the result of failure, and the result that was not returned as the result of failure. The above results are recorded as subsequent status in a database that specializes in storing result information for users to view.

[0056] S13: Record the waiting time of the scheduling thread and determine whether the waiting time reaches the maximum execution time of the standard task to be scheduled. If so, proceed to step S14.

[0057] When the working thread processes a task, the central processing unit (CPU) records the waiting time of the scheduling thread, and adds the waiting time from the current start execution time. The working thread will execute for a period of time. Since the scheduling thread needs a time limit to wait for all the working threads to return, that is, the maximum execution time of the standard task to be scheduled, the default system is 2-3 times the time required to execute a standard unit task under normal circumstances. It is set according to actual conditions, and the present invention does not make any requirements here.

[0058] Determine whether the waiting time reaches the maximum execution time of the standard task to be scheduled. If it reaches, it means timeout, the system is currently busy, and its downstream pressure is high. If it does not reach, it means that the current standard task to be scheduled is scheduled within the executable time, and all its return results are returned. The current standard task to be scheduled is completed, and the next standard task to be scheduled is obtained within the current execution window for scheduling.

[0059] S14: Abandon the standard to-be-scheduled task, adjust the scheduling configuration parameters and return to the step of obtaining the scheduling configuration parameters in the current execution window.

[0060] In step S13, when the waiting time reaches the maximum execution time, it indicates a timeout, the returned results are not all returned or the returned results include the result of the failed task processing in the working thread, then the current standard task to be scheduled fails, the waiting time is abandoned, the current standard task to be scheduled is directly abandoned, and the configuration parameters are adjusted at the same time, such as reducing the number of working threads to reduce concurrency, effectively reducing the system load. At the same time, return to step S11 to obtain the scheduling configuration parameters in the current execution window so as to obtain the next standard task to be scheduled according to the latest scheduling configuration parameters.

[0061] The present invention provides a task scheduling method, which obtains the scheduling configuration parameters in the current execution window and then obtains the current standard task to be scheduled, puts the standard task to be scheduled into the execution queue for scheduling processing so that the scheduling thread waits for all the return results of the working thread; records the waiting time of the scheduling thread and determines whether the waiting time reaches the maximum execution time of the standard task to be scheduled; if it reaches, it means that the downstream pressure is too large, and the standard task to be scheduled is abandoned, and the scheduling configuration parameters are dynamically adjusted so that the next standard task to be scheduled is obtained in the subsequent current execution window. The method divides a large batch of tasks into multiple standard tasks to be scheduled, and splits them into multiple execution windows at the same time, executes the current standard task to be scheduled in an execution window and then executes the next standard task to be scheduled, that is, serially executes the scheduling tasks, records the waiting time during the scheduling processing to realize timely perception of the pressure of the task execution on the downstream, if the waiting time reaches the maximum execution time, it means that the downstream pressure is large, abandons the current standard task to be scheduled and adjusts the scheduling configuration parameters. Avoid the problems of downstream overload and server failure to work normally under normal circumstances, and improve the stability of concurrent scheduling of large batches of tasks.

[0062] On the basis of the above embodiment, the step S12 of placing the standard to-be-scheduled task into the execution queue for scheduling includes:

[0063] Record the start execution time of the standard scheduled tasks in the execution queue;

[0064] Determine whether the execution queue is empty;

[0065] If it is not empty, determine whether the start execution time is within the executable time of the current execution window;

[0066] If it is within the executable time, the standard scheduled tasks assigned to the working thread will be executed concurrently so that the scheduling thread can wait for all the return results of the working thread.

[0067] Record the start execution time of the standard pending tasks and determine whether the execution queue is empty. If it is not empty, it means that the standard pending tasks have been stored in the execution queue and determine whether the start execution time is within the executable time of the current execution window. If it is within the executable time, the standard pending tasks in the execution queue can be handed over to the configured working thread for processing, and the standard pending tasks that have been assigned to the working thread need to be executed concurrently. When the working thread processes the standard pending tasks, the scheduling thread waits for all the return results of the working thread. Since the execution time of the working thread is long, the scheduling thread can process asynchronously, query and obtain the next standard pending task.

[0068] This embodiment provides a method for scheduling standard tasks to be scheduled, scheduling tasks smoothly and efficiently, and improving concurrent scheduling capabilities.

[0069] Based on the above embodiment, the step S11 of obtaining the scheduling configuration parameters in the current execution window includes:

[0070] Determine whether there are tasks to be scheduled within the current execution window;

[0071] If yes, get the scheduling configuration parameters.

[0072] When the current execution window is triggered, it starts to determine whether there are any scheduled tasks to be executed in the current execution window. If not, it will immediately return to wait for the next execution window to start. If so, it will obtain the updated scheduling configuration parameters to the current execution window, and then subsequently obtain the current standard scheduled tasks according to the scheduling configuration parameters.

[0073] The present embodiment provides for obtaining the scheduling configuration parameters within the current execution window, so that the subsequent scheduling thread can obtain the current standard task to be scheduled according to the scheduling configuration parameters. If there is no task to be scheduled, the next execution window is obtained, thereby improving the scheduling processing efficiency and shortening the task processing time.

[0074] On the basis of the above embodiment, when the waiting time does not reach the maximum execution time of the standard to-be-scheduled task, the method further includes:

[0075] When the waiting time does not reach the maximum execution time, the control scheduling thread obtains the current standard task to be scheduled according to the scheduling configuration parameters so as to enter the step of putting the standard task to be scheduled into the execution queue for scheduling processing.

[0076] It can be understood that the maximum execution time is the maximum execution time allowed for a standard task to be scheduled to be handed over to a worker thread for concurrent execution. If the maximum execution time is exceeded, it is considered that the downstream system is under too much pressure. If it is not exceeded, the current standard task to be scheduled completes the scheduling process on time, and the scheduling thread continues to be controlled within the current execution window to obtain the current standard task to be scheduled according to the scheduling configuration parameters, which refers to the next standard task to be scheduled.

[0077] Specifically, the scheduling thread obtains the next standard task to be scheduled according to the scheduling configuration parameters. The scheduling configuration parameters here are the scheduling configuration parameters before the last standard task to be scheduled is processed and are not adjusted. The scheduling configuration parameters can only be adjusted when the waiting time reaches the maximum execution time.

[0078] This embodiment provides that when the waiting time does not reach the maximum execution time, the next standard task to be scheduled is obtained to implement serial execution of scheduled tasks, so that the current execution window can schedule as many standard tasks to be scheduled as possible.

[0079] Based on the above embodiment, when the waiting time reaches the maximum execution time of the scheduled task, the method further includes:

[0080] Set the executable time of the current execution window to 0.

[0081] When the waiting time reaches the maximum execution time, the timeout occurs and the current standard task to be scheduled fails. When the current standard task to be scheduled is abandoned, the executable time of the current execution window is set to 0. For example, the current execution window can execute 5 standard tasks to be scheduled, but the third standard task to be scheduled times out during execution. The executable time of the current execution window is set to 0. When the next execution window starts, the remaining 3rd to 5th standard tasks to be scheduled in the current window are executed.

[0082] This embodiment provides setting the executable time of the current execution window to 0, placing the remaining standard tasks to be scheduled in the next execution window for execution, so that there is a period of idle time from the current execution window to the beginning of the next execution window, which can alleviate system busyness and buffer the pressure of the downstream system.

[0083] Specifically, in the above embodiment, when the waiting time reaches the maximum execution time of the standard task to be scheduled, the standard task to be scheduled needs to be abandoned, and thereafter, the method further includes:

[0084] Treat the abandoned standard to-be-scheduled tasks as failed tasks;

[0085] Place the failed task in the next execution window for execution.

[0086] It is understandable that task scheduling starvation refers to tasks that could have been scheduled successfully, but have not been scheduled successfully for a long time due to various reasons. Usually, due to system reasons, the failed task needs to wait until the entire batch of tasks to be scheduled is scheduled before it can be scheduled again. However, most tasks are scheduled in sequence, so scheduling starvation will occur when the failed task is scheduled again, which is unfair to other successfully scheduled tasks.

[0087] Tasks are scheduled in order within the current execution window. Standard tasks to be scheduled that fail due to system reasons will not be retried within the current execution window. Abandoned standard tasks to be scheduled will be treated as failed tasks, and the execution status of failed tasks will be set to pending. To ensure that the scheduling starvation time is executed within one execution window at most, the number of retry attempts can be updated when the failed task is retried. For failed tasks within the current execution window, the number of retry attempts is updated minus 1. When executing in the next execution window, its scheduling thread checks the execution status of the failed task. If the execution status is executable and the failed task has a retry count greater than 0, it will be connected to the execution queue in the next execution window.

[0088] This embodiment provides that the abandoned standard to-be-scheduled tasks are treated as failed tasks, and the failed tasks are placed in the next execution window for execution, thereby reducing the scheduling starvation time of temporarily executing failed tasks due to system reasons and ensuring the stability of concurrent task scheduling.

[0089] Based on the above embodiment, the current standard task to be scheduled is obtained according to the scheduling configuration parameters, wherein the scheduling configuration parameters specifically include the task amount, maximum execution time, executable time and number of working threads of the standard task to be scheduled.

[0090] Specifically, the task volume of standard tasks to be scheduled refers to the amount of tasks that meet the scheduling status obtained by the scheduling thread, which is configurable and greater than or equal to 1; the maximum execution time refers to the maximum execution time allowed for the scheduling thread to schedule the task volume of standard tasks to be scheduled and hand it over to the worker thread for concurrent execution. If this time is exceeded, it is considered that the downstream system pressure is large. The executable time within the execution window is the time that can be used to actually schedule tasks within each execution window, which is configurable, where the maximum execution time ≤ executable time ≤ (length of the execution window - maximum execution time). When scheduling fails, adjust the scheduling configuration parameters, and the number of worker threads is reduced to reduce the load on the concurrently effective lower system.

[0091] It is understandable that the scheduling configuration parameters are set according to the specific business scenario. For example, the task volume of the standard scheduled task is 100, and the maximum execution time is 10s. If the execution window duration is 10 minutes, the executable time within the execution window can be set to 580s, and the number of working threads is 10, etc.

[0092] The scheduling configuration parameters may also include setting the number of retries, the minimum execution time of standard tasks to be scheduled, etc., which are not specifically required by the present invention.

[0093] The scheduling configuration parameters provided in this embodiment are set, and then the current standard tasks to be scheduled are obtained according to the scheduling configuration parameters, and the concurrency is dynamically adjusted to achieve the goal of smooth and efficient scheduling of tasks.

[0094] The above describes in detail various embodiments corresponding to the task scheduling method. On this basis, the present invention also discloses a task scheduling device corresponding to the above method. Figure 2 The structure diagram of a task scheduling device provided by an embodiment of the present invention. Figure 2 As shown, the task scheduling device includes:

[0095] An acquisition module 11 is used to acquire the scheduling configuration parameters in the current execution window so that the scheduling thread can acquire the current standard task to be scheduled according to the scheduling configuration parameters;

[0096] The scheduling module 12 is used to put the standard tasks to be scheduled into the execution queue for scheduling so that the scheduling thread waits for all the return results of the working threads in the thread pool;

[0097] The judgment module 13 is used to record the waiting time of the scheduling thread and judge whether the waiting time reaches the maximum execution time of the standard scheduled task. If so, the abandonment module 14 is triggered;

[0098] The abandonment module 14 is used to abandon the standard task to be scheduled, adjust the scheduling configuration parameters and return to the step of obtaining the scheduling configuration parameters in the current execution window.

[0099] Since the embodiments of the device part correspond to the above embodiments, please refer to the description of the embodiments of the device part for the embodiments of the device part, and will not be repeated here.

[0100] The present invention provides a task scheduling device, which obtains the scheduling configuration parameters in the current execution window and then obtains the current standard task to be scheduled, puts the standard task to be scheduled into the execution queue for scheduling processing so that the scheduling thread waits for all the return results of the working thread; records the waiting time of the scheduling thread and determines whether the waiting time reaches the maximum execution time of the standard task to be scheduled; if it reaches, it means that the downstream pressure is too large, and the standard task to be scheduled is abandoned, and the scheduling configuration parameters are dynamically adjusted so that the next standard task to be scheduled is obtained in the subsequent current execution window. The device divides a large batch of tasks into multiple standard tasks to be scheduled and splits them into multiple execution windows at the same time, executes the current standard task to be scheduled in an execution window and then executes the next standard task to be scheduled, that is, executes the scheduling tasks serially, records the waiting time during the scheduling processing to realize timely perception of the pressure of the task execution on the downstream, if the waiting time reaches the maximum execution time, it means that the downstream pressure is large, abandons the current standard task to be scheduled and adjusts the scheduling configuration parameters. Avoid the problems of downstream overload and server failure to work normally under normal circumstances, and improve the stability of concurrent scheduling of large batches of tasks.

[0101] Figure 3 A structural diagram of another task scheduling device provided by an embodiment of the present invention, such as Figure 3 As shown, the device comprises:

[0102] A memory 21, used for storing computer programs;

[0103] The processor 22 is used to implement the steps of the task scheduling method when executing a computer program.

[0104] The task scheduling device provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer, or a desktop computer.

[0105] Among them, the processor 22 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 22 can be implemented in at least one hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 22 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU; the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 22 may be integrated with a graphics processing unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 22 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0106] The memory 21 may include one or more computer-readable storage media, which may be non-transitory. The memory 21 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 21 is at least used to store the following computer program 211, wherein, after the computer program is loaded and executed by the processor 22, it can implement the relevant steps of the scheduling method of the task disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 21 may also include an operating system 212 and data 213, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 212 may include Windows, Unix, Linux, etc. The data 213 may include, but is not limited to, data involved in the scheduling method of the task, etc.

[0107] In some embodiments, the task scheduling device may further include a display screen 23 , an input / output interface 24 , a communication interface 25 , a power supply 26 , and a communication bus 27 .

[0108] Those skilled in the art can understand that Figure 3 The structure shown in the figure does not constitute a limitation on the task scheduling device, and may include more or fewer components than those shown in the figure.

[0109] The processor 22 implements the task scheduling method provided by any of the above embodiments by calling the instructions stored in the memory 21 .

[0110] The present invention provides a task scheduling device, which obtains the scheduling configuration parameters in the current execution window and then obtains the current standard task to be scheduled, puts the standard task to be scheduled into the execution queue for scheduling processing so that the scheduling thread waits for all the return results of the working thread; records the waiting time of the scheduling thread and determines whether the waiting time reaches the maximum execution time of the standard task to be scheduled; if it reaches, it means that the downstream pressure is too large, and the standard task to be scheduled is abandoned, and the scheduling configuration parameters are dynamically adjusted so that the next standard task to be scheduled is obtained in the subsequent current execution window. The device divides a large batch of tasks into multiple standard tasks to be scheduled and splits them into multiple execution windows at the same time, executes the current standard task to be scheduled in an execution window and then executes the next standard task to be scheduled, that is, serially executes the scheduling tasks, records the waiting time during the scheduling processing to realize timely perception of the pressure of the task execution on the downstream, if the waiting time reaches the maximum execution time, it means that the downstream pressure is large, abandons the current standard task to be scheduled and adjusts the scheduling configuration parameters. Avoid the problems of downstream overload and server failure to work normally under normal circumstances, and improve the stability of concurrent scheduling of large batches of tasks.

[0111] Furthermore, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by the processor 22, the steps of the scheduling method for the above-mentioned tasks are implemented.

[0112] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0113] For an introduction to a computer-readable storage medium provided by the present invention, please refer to the above method embodiment, and the present invention will not be repeated here. It has the same beneficial effects as the above task scheduling method.

[0114] The above is a detailed introduction to a task scheduling method, device and medium provided by the present invention. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referenced to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

[0115] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A task scheduling method, characterized in that: include: Obtaining the scheduling configuration parameters within the current execution window so that the scheduling thread obtains the current standard task to be scheduled according to the scheduling configuration parameters; wherein the scheduling configuration parameters include the task amount, maximum execution time, executable time, set number of retries, minimum execution time of the standard task to be scheduled, and the number of working threads; the executable time is less than or equal to the difference between the duration of the execution window and the maximum executable time, and is greater than or equal to the maximum executable time; Put the standard to-be-scheduled tasks into the execution queue for scheduling processing so that the scheduling thread waits for all return results of the working threads in the thread pool; wherein the scheduling processing is to record the current start execution time, and the working thread places the tasks in the execution queue in the working thread for asynchronous processing according to the currently configured number of concurrent threads; Recording the waiting time of the scheduling thread and determining whether the waiting time reaches the maximum execution time of the standard task to be scheduled; If it is reached, the standard task to be scheduled is abandoned, the scheduling configuration parameters are adjusted and the process returns to the step of obtaining the scheduling configuration parameters in the current execution window; Correspondingly, placing the standard to-be-scheduled task into an execution queue for scheduling processing includes: Record the start execution time of the standard to-be-scheduled task in the execution queue; Determine whether the execution queue is empty; If it is not empty, determining whether the start execution time is within the executable time of the current execution window; If it is within the executable time, concurrently executing the standard to-be-scheduled tasks assigned to the working thread so that the scheduling thread can wait for all the return results of the working thread; Correspondingly, the obtaining of the scheduling configuration parameters in the current execution window includes: Determine whether there is a task to be scheduled within the current execution window; If yes, obtaining the scheduling configuration parameters; Correspondingly, after abandoning the standard to-be-scheduled task, the method further includes: Treat the abandoned standard to-be-scheduled task as a failed task; The failed task is placed in the next execution window for execution; wherein, when the failed task is retried, the number of retry attempts is updated, and for a failed task in the current execution window, the number of retry attempts is updated and reduced by 1. When executing in the next execution window, the scheduling thread checks the execution status of the failed task. If the execution status is executable and the failed task has a retry count greater than 0, it is placed in the execution queue in the next execution window.

2. The task scheduling method according to claim 1, characterized in that: When the waiting time does not reach the maximum execution time of the standard task to be scheduled, the method further includes: When the waiting time does not reach the maximum execution time, the scheduling thread is controlled to obtain the current standard task to be scheduled according to the scheduling configuration parameters so as to enter the step of putting the standard task to be scheduled into the execution queue for scheduling processing.

3. The task scheduling method according to claim 1, characterized in that: When the waiting time reaches the maximum execution time of the standard task to be scheduled, the method further includes: The executable time of the current execution window is set to 0.

4. A task scheduling device, characterized in that: include: An acquisition module is used to acquire the scheduling configuration parameters in the current execution window so that the scheduling thread acquires the current standard task to be scheduled according to the scheduling configuration parameters; wherein the scheduling configuration parameters include the task amount, maximum execution time, executable time, set number of retries, minimum execution time of the standard task to be scheduled and the number of working threads; the executable time is less than or equal to the difference between the duration of the execution window and the maximum executable time, and is greater than or equal to the maximum executable time; The scheduling module is used to put the standard to-be-scheduled tasks into the execution queue for scheduling processing so that the scheduling thread waits for all the return results of the working threads in the thread pool; wherein the scheduling processing is to record the current start execution time, and the working thread places the tasks in the execution queue in the working thread for asynchronous processing according to the currently configured number of concurrent threads; A judgment module, used for recording the waiting time of the scheduling thread and judging whether the waiting time reaches the maximum execution time of the standard task to be scheduled, and if so, triggering the abandonment module; The abandonment module is used to abandon the standard task to be scheduled, adjust the scheduling configuration parameters and return to the step of obtaining the scheduling configuration parameters in the current execution window; Correspondingly, placing the standard to-be-scheduled task into an execution queue for scheduling processing includes: Record the start execution time of the standard to-be-scheduled task in the execution queue; Determine whether the execution queue is empty; If it is not empty, determining whether the start execution time is within the executable time of the current execution window; If it is within the executable time, concurrently executing the standard to-be-scheduled tasks assigned to the working thread so that the scheduling thread can wait for all the return results of the working thread; Correspondingly, the obtaining of the scheduling configuration parameters in the current execution window includes: Determine whether there is a task to be scheduled within the current execution window; If yes, obtaining the scheduling configuration parameters; Correspondingly, after abandoning the standard to-be-scheduled task, the method further includes: Treat the abandoned standard to-be-scheduled task as a failed task; The failed task is placed in the next execution window for execution; wherein, when the failed task is retried, the number of retry attempts is updated, and for a failed task in the current execution window, the number of retry attempts is updated and reduced by 1. When executing in the next execution window, the scheduling thread checks the execution status of the failed task. If the execution status is executable and the failed task has a retry count greater than 0, it is placed in the execution queue in the next execution window.

5. A task scheduling device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the task scheduling method as described in any one of claims 1 to 3 when executing the computer program.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the task scheduling method according to any one of claims 1 to 3 are implemented.

Images