Task scheduling method, device, electronic device and storage medium

Abstract

The present application discloses a task scheduling method, device, electronic device and storage medium. The method includes: obtaining a first task set; the first task set includes at least one task in a first state; the first state represents a state to be scheduled; generating at least one target task set based on the first task set according to a first configuration parameter; the first configuration parameter represents the set number of tasks; the target task set represents a task set with the number of tasks of the first configuration parameter; loading the at least one target task set into a first queue; the first queue is a queue composed of task sets to be executed; executing each target task set in at least one target task set in the first queue.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a task scheduling method, device, electronic device and storage medium. Background Art

[0002] In the related technologies, as the complexity of business application systems increases, the requirements for distributed processing of system batch tasks become more complex and changeable. There are scheduling mechanisms that can realize distributed scheduling and processing of batch tasks, and there are scheduling mechanisms that can provide distributed coordinated scheduling of tasks. However, these scheduling mechanisms have the problem of low scheduling efficiency when scheduling tasks, and cannot achieve balanced scheduling for large batches of tasks. Summary of the invention

[0003] In view of this, embodiments of the present application provide a task scheduling method, device, electronic device and storage medium to at least solve the problem of low efficiency of batch task scheduling in related technologies.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides a scene detection method, the method comprising:

[0006] Acquire a first task set; the first task set includes at least one task in a first state; the first state represents a state to be scheduled;

[0007] According to a first configuration parameter, based on the first task set, at least one target task set is generated; the first configuration parameter represents the number of tasks set; the target task set represents the task set with the number of tasks of the first configuration parameter;

[0008] Loading the at least one target task set into a first queue; the first queue is a queue consisting of task sets to be executed;

[0009] Each target task set of at least one target task set in the first queue is executed.

[0010] The present application also provides a task scheduling device, including:

[0011] An acquisition unit, configured to acquire a first task set; the first task set includes at least one task in a first state; the first state represents a state to be scheduled;

[0012] A generating unit, configured to generate at least one target task set based on the first task set according to a first configuration parameter; the first configuration parameter represents the number of tasks set; the target task set represents a task set whose number of tasks is the first configuration parameter;

[0013] A loading unit, used to load the at least one target task set into a first queue; the first queue is a queue composed of task sets to be executed;

[0014] An execution unit is used to execute each target task set in at least one target task set in the first queue.

[0015] The embodiment of the present application also provides an electronic device, a processor and a memory for storing a computer program that can be run on the processor.

[0016] Wherein, the processor is used to execute the steps of any of the above methods when running the computer program.

[0017] An embodiment of the present application further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0018] In an embodiment of the present invention, by obtaining the number of tasks in a task set, the tasks in the task set are grouped into a target task set that meets the set number of tasks, and the tasks are executed in units of a target task set. This allows batches of tasks to be standardized and efficient scheduling of tasks to be achieved, thereby enabling balanced scheduling of large batches of tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of the implementation flow of a task scheduling method provided in one embodiment of the present application;

[0020] Figure 2 A schematic diagram of an implementation flow of a task scheduling method provided in yet another embodiment of the present application;

[0021] Figure 3 A schematic diagram of an implementation flow of a task scheduling method provided in yet another embodiment of the present application;

[0022] Figure 4 A schematic diagram of an implementation flow of a task scheduling method provided in yet another embodiment of the present application;

[0023] Figure 5 A schematic diagram of an implementation flow of a task scheduling method provided in yet another embodiment of the present application;

[0024] Figure 6 A schematic diagram of different states of a task in a task scheduling process provided by an embodiment of the present application;

[0025] Figure 7 A task scheduling process diagram provided for an application embodiment of the present application

[0026] Figure 8A schematic diagram of a task scheduling architecture provided in an embodiment of the present application;

[0027] Fig. 9 A schematic diagram of a distributed cluster deployment provided in an embodiment of the present application;

[0028] Fig.10 A schematic diagram of the structure of a task scheduling device provided in one embodiment of the present application;

[0029] Fig.11 A schematic diagram of the hardware structure of an electronic device is provided for one embodiment of the present application. DETAILED DESCRIPTION

[0030] The present application is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0031] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0032] It should be noted that the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0033] In addition, in the embodiments of the present application, "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. In the embodiments of the present application, "at least one" means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, and can mean including any one or more elements selected from the set consisting of A, B, and C.

[0034] In an embodiment of the present application, a task scheduling method is provided. Figure 1 FIG. 1 is a flow chart of a task scheduling method according to an embodiment of the present application. Figure 1 As shown, the method includes:

[0035] S101: Acquire a first task set; the first task set includes at least one task in a first state; the first state represents a state to be scheduled.

[0036] Here, a first task set consisting of batch tasks is obtained, and the first task set contains at least one task in a first state, wherein the first state represents a state to be scheduled, that is, the tasks in the first task set have not been scheduled, and the tasks in the first task set are unrelated tasks. In practical applications, the first task set can be obtained through the corresponding interface, wherein the first task set supports task sets consisting of a mixture of different types of tasks, and also supports task sets consisting of a large number of single-type tasks, and also supports task sets consisting of a single task. In the process of obtaining the first task set, the task information of each task in the first task set can also be obtained, wherein the task information includes at least: a globally unique task ID, a task name, a task type, etc.

[0037] In one embodiment, if Figure 2 As shown, obtaining the first task set includes:

[0038] S201: Determine a second parameter of the first queue at second intervals; the second parameter represents the number of target task sets in the first queue.

[0039] Here, since the target task set in the first queue is the task set to be executed, the second parameter of the first queue is determined every second time, and the number of target task sets in the first queue can be known regularly, and the scheduling of the current task can be further determined according to the second parameter in the first queue. In practical applications, the second time can be pre-configured, for example, the second time can be set to 3 seconds by default, so the first queue will be detected every 3 seconds to determine the second parameter of the first queue.

[0040] S202: When the second parameter is less than or equal to a third configuration parameter, obtain a first task set; the third configuration parameter is the number of set target task sets.

[0041] Here, when the second parameter is less than or equal to the third configuration parameter, it indicates that the number of target task sets in the current first queue is small. Since the target task set in the first queue is a task set to be executed, when the target task set in the first queue is too small, the execution of all target task sets in the first queue can be completed in a short time, resulting in the current task processing state being an idle state, which is not conducive to improving the execution efficiency of the task. Therefore, it is necessary to obtain the first task set and schedule a batch of new tasks as soon as possible to ensure the processing efficiency of the task. In practical applications, the third configuration parameter can be set to half of the threshold of the target task set that can be stored in the first queue. For example, the threshold for storing the target task set configured for the first queue is 100, and the third parameter can be set to 50. The number of tasks of the rescheduled first task set can also be configured in advance, wherein the number of tasks of the rescheduled first task set can be configured with reference to the first configuration parameter. For example, the first configuration parameter is 100, then when re-acquiring the first task set, the task of 5 times the first configuration parameter can be obtained by default as the first task set, and the re-acquired first task set contains 500 tasks to be scheduled.

[0042] S102: Generate at least one target task set according to a first configuration parameter and based on the first task set; the first configuration parameter represents the set number of tasks; the target task set represents a task set whose number of tasks is the first configuration parameter.

[0043] Here, the target task set represents a task set whose number of tasks is the first configuration parameter, that is, the target task set is composed of tasks whose number of tasks is the first configuration parameter. In practical applications, when the tasks of the target task set are executed, the number of tasks in the target task set can affect the processing efficiency of the target task set. For example, when the number of tasks in the target task set is small, the processing efficiency of the target task set will be reduced within the same time. When the number of tasks in the target task set is large, the processing time of the target task set will be too long. Therefore, by limiting the number of tasks of the target task set and limiting the number of tasks of the target task set to a relatively appropriate range, the processing efficiency of the task set can be improved. After obtaining the first task set, according to the first configuration parameter, the tasks whose task quantity is the first configuration parameter are obtained from the first task set, and the target task set is combined, so as to complete the task scheduling while improving the processing efficiency of the task set. In practical applications, the first configuration parameter can be pre-configured, or it can be adjusted according to the processing status of the task in the actual process. In the embodiment of the present application, the first configuration parameter can be defaulted to 50, that is, the target task set is composed of 50 tasks, then 50 tasks can be extracted from the first task set, and the 50 tasks extracted are composed of the target task set. In practical applications, the generated target task set will be stored in the first queue and wait for execution. When the first queue is full of target task sets, the generated target task set cannot be stored in the first queue. At this time, the scheduling of the first task set can be suspended, and the first task set can be stored in the setting database. When there is a successful execution of the target task set in the first queue, there will be an idle position in the first queue, and the target task set can continue to be stored in the first queue, thereby extracting the first task set stored in the setting database and continuing to schedule the first task set.

[0044] In one embodiment, if Figure 3 As shown, generating at least one target task set based on the first task set according to the first configuration parameter includes:

[0045] S301: Divide the first task set into at least one second task set according to the task type of each task in the first task set; the second task set is a task set consisting of tasks of the same task type in the first task set.

[0046] Here, the tasks in the target task set all belong to the same task type, so that the processing efficiency of the target task set can be improved when the target task set is executed. Therefore, when generating the target task set based on the first task set, the tasks of the first task set are first divided into at least one second task set according to the task type, wherein the second task set is composed of tasks with the same task type in the first task set.

[0047] S302: Generate at least one target task set according to a size relationship between a first parameter of each second task set in at least one of the second task sets and the first configuration parameter; the first parameter represents the number of tasks.

[0048] Here, the target task set is composed of a certain number of tasks of the same task type. Therefore, based on the second task set composed of tasks of the same task type, it is necessary to determine whether the first parameter of the second task set satisfies the first configuration parameter, wherein the first parameter of the second task set represents the number of tasks in the second task set. Through the size relationship between the first parameter of the second task set and the first configuration parameter, the second task set is processed accordingly, and a target task set with the number of tasks being the first configuration parameter is generated based on the second task set.

[0049] In one embodiment, if Figure 4 As shown, when generating at least one target task set according to the size relationship between the first parameter of each second task set in at least one second task set and the first configuration parameter, the method includes:

[0050] S401: When a first parameter of the second task set is equal to the first configuration parameter, determining the second task set as the target task set.

[0051] Here, when the first parameter of the second task set is the same as the first configuration parameter, for example, the first configuration parameter is 50, when the first parameter of the second task set is also 50, it indicates that the second task set meets the set number of tasks, and the second task set is identified as the target task set.

[0052] S402: When the first parameter of the second task set is greater than the first configuration parameter, extract tasks whose number of tasks is the first configuration parameter from the second task set to generate at least one of the target task sets.

[0053] Here, when the first parameter of the second task set is greater than the first configuration parameter, tasks with a task quantity equal to the first configuration parameter can be extracted from the second task set to form a target task set. For example, when the first configuration parameter is 50 and the second task set is 100, 50 tasks can be extracted from the second task set respectively to form two target task sets.

[0054] S403: When the first parameter of the second task set is less than the first configuration parameter, add the tasks in the second task set to a third task set with the same task type as the second task set within the first time, and determine the third task set as the target task set if the set conditions are met; the third task set represents a task set in the second state and whose first parameter is less than the first configuration parameter; the second state represents a scheduled state.

[0055] Here, there are two sources for the second task set whose first parameter is less than the first configuration parameter. One source is that the first parameter of the divided second task set is less than the first configuration parameter. For example, the first configuration parameter is 50, and the number of tasks of task type A in the first task set is 2. Then the first parameter of the second task set composed of task type A is less than 50. The other source is that the first parameter of the second task set after the number of extracted tasks is the first configuration parameter is less than the first configuration parameter. For example, the first configuration parameter is 50, and the first parameter of the second task set is 51. Since the first parameter of the second task set is greater than the first configuration parameter, 50 tasks are extracted from the second task set to form the target task set. After the extracted task, there is 1 task left in the second task set. At this time, the first parameter of the second task set is less than the first configuration parameter. When the first parameter of the second task set is less than the first configuration parameter, the target task set cannot be generated by the second task set alone, and the tasks in the second task set are added to the third task set within the first time. The task type of the third task set is the same as the task type of the second task set, and the tasks in the third task set are in a scheduled state, wherein the first time is the time allowed to add tasks to the third task set, and the tasks in the second task set are added to the third task set whose number of tasks does not meet the first configuration parameter. In practical applications, the third task set can be a second task set with the same task type and a first parameter less than the first configuration parameter in the last task scheduling. When the third set meets the set conditions, the third set is determined as the target task set, wherein the set conditions are conditions for identifying the third set as the target set. In practical applications, a task accumulation thread can be created to continuously monitor the task accumulation of each third task set, and the target task set is output when the third task set meets the set conditions. Furthermore, after the tasks of the first task set are divided into at least one second task set according to the task type, the scheduling status of each task in the first task set can be adjusted from an unscheduled state to a scheduled state, and the scheduling duration of each task can also be determined, so that the scheduling status of the task can be understood through the scheduling status and scheduling duration of the task.

[0056] In one embodiment, the setting condition includes any one of the following:

[0057] The target task set does not exist in the first queue;

[0058] There is at least one target task set in the first queue, and the first parameter of the third task set is greater than or equal to the first configuration parameter;

[0059] There is at least one target task set in the first queue, and the first time is greater than or equal to a second configuration parameter; the second configuration parameter is a set duration.

[0060] Here, for the third task set whose first parameter is less than the first configuration parameter, it is necessary to accumulate tasks for the third task set and then generate a target task set. In the process of accumulating tasks for the third task set, it is impossible to execute each task in the third task set. Therefore, it is necessary to impose certain restrictions on the task accumulation of the third task set to avoid being unable to process the tasks of the third task set for a long time. In actual applications, the task accumulation of the third task set is limited by setting conditions. Specifically, by limiting the first parameter and the first time of the third task set, it is determined that the third task set can become the target task set. In actual applications, the target task set will be put into the first queue, and the target task set will be executed in order according to the order of the target task sets in the first queue. When the target task set does not exist in the first queue, it means that it is currently in an idle state and there is no task set to be executed. At this time, the third task set can be determined as the target task set, and the execution of the task set can be completed as soon as possible. In this case, the third task set does not exist on the task accumulation thread. Therefore, tasks can be directly accumulated for the second task set A whose first parameter is less than the first configuration parameter. Since the second task set A has completed the scheduling of tasks in the second task set A when accumulating tasks, the second task set A is in the second state. For the tasks in the second task set B whose first parameter is less than the first configuration parameter in the next task scheduling process, the second task set A is equivalent to the third task set. When the task types of the second task set A and the second task set B are the same, the tasks in the second task set B can be added to the second task set A. When there is a target task set in the first queue, there is time for the third task set to accumulate tasks. While accumulating tasks for the third task set, the execution of the target task set can be maintained. When the third task set makes the first parameter greater than or equal to the first configuration parameter by accumulating tasks, the third task set is determined as the target task set. In addition, when there is a target task set in the first queue, the second configuration parameter is not reached in the first time, and the first parameter of the third task set does not reach the first configuration threshold, it means that after the tasks in the second task set are merged into the third task set, the first parameter of the third task set is still less than the first configuration threshold, then tasks can continue to be accumulated for the third task set. When the first time for the third task set to accumulate tasks is greater than or equal to the second configuration parameter, the third task set is identified as the target task set. In practical applications, the second configuration parameter can be pre-configured or configured according to the task scheduling situation. In the embodiment of the present application, the default value of the second configuration parameter is 50 milliseconds. In practical applications, a logical decision table for the target task set is generated, as shown in Table 1.

[0061] Table 1

[0062]

[0063] In one embodiment, when generating at least one target task set based on the first task set according to the magnitude relationship between the first parameter of the first task set and the first configuration parameter, the method further includes:

[0064] The scheduling parameters of each task of at least one task in the first task set are stored in a setting database; the setting database is used to store task information of the task; and the scheduling parameters at least include the scheduling time.

[0065] Here, after the target task set is generated, the scheduling status of each task in the corresponding target task set will change, and the scheduling parameters in the scheduling process can also be obtained, and the scheduling parameters of each task can be stored in the setting database. For example, the scheduling parameters can include the scheduling status and scheduling duration of the task. The scheduled status of each task in the target task set can be stored in the setting database, and the scheduling duration of each task in the target task set can also be stored in the setting database as another scheduling parameter, so that the timed-out tasks can be extracted and rescheduled according to the scheduling parameters of the tasks stored in the setting database.

[0066] S103: Load the at least one target task set into a first queue; the first queue is a queue composed of task sets to be executed.

[0067] Here, at least one generated target task set is recorded in the first queue. At least one target task set is stored in the first queue. The target task set in the first queue is the target task set to be executed. In actual applications, there may be a large number of tasks that need to be executed, and the resources for executing the target task set are limited. Therefore, a part of the target task set will be executed first. After the execution is completed, other target task sets that need to be executed will be loaded for execution. Therefore, in order to ensure the orderly execution of the target task set, the target task set can be loaded into the first queue, so that the task set to be executed can be accurately known. In addition, for the target task set stored in the first queue, the different target task sets are sorted according to the time when the target task set enters the first queue, so that the target task set to be executed first and the target task set to be executed later can be determined, and the target task set to be loaded into the first queue.

[0068] In one embodiment, if Figure 5 As shown, the at least one target task set is loaded into the first queue, and the method further includes:

[0069] S501: Acquire a fourth task set every third time; the fourth task set is a set of at least one task in the setting information library whose scheduling time has timed out or whose execution status has failed; the third time is determined based on the number of task execution failures.

[0070] Here, in the process of scheduling or executing tasks, there may be tasks that have timed out or failed to execute. These target tasks need to be compensated for so that they can be rescheduled and re-executed, such as Figure 6 As shown, Figure 6 A schematic diagram showing different states of tasks in the task scheduling process is shown. Unscheduled tasks are loaded, scheduled, and the state of the task changes to the scheduled state. The scheduled tasks that have not timed out are executed. According to the execution results, the tasks can be divided into successfully executed tasks and failed tasks. Compensatory scheduling is performed for the scheduled tasks that have timed out and the tasks that have failed to execute. The fourth task set is obtained from the setting information base at a third time interval. Specifically, the setting information base is used to store task information of tasks, so that at least one task whose scheduling time has timed out or whose execution status has failed can be extracted from the setting information base to form the fourth task set. In actual applications, the setting information base at least stores the scheduling duration and execution status of the task. In addition, the process information of task execution processing such as the scheduling status of the task, the task receiving time, the task execution completion time, and the number of execution failures can also be stored. Among them, the scheduling timeout time can be configured in advance. For example, the task scheduling timeout time can be configured to 5 minutes, and the task with a scheduling time exceeding 5 minutes is determined as a scheduling timeout task. Through the task information stored in the setting information base, the tasks that need to be rescheduled and executed are extracted to form the fourth task set. In actual applications, the number of tasks in the fourth task set can be the first configuration parameter. In actual applications, the third time is determined based on the number of execution failures of the task. The third time is extended by the number of execution failures of the task. The extended third time is twice the third time before the extension. For example, the third time is configured to 30 seconds. When the number of execution failures of the task is 2, the corresponding third time is 60 seconds.

[0071] S502: Generate a target task set corresponding to the fourth task set according to a set rule.

[0072] Here, according to the set rules, a target task set corresponding to the fourth task set is generated. Specifically, the tasks in the fourth task set are divided into different task sets according to the task type, and then the relationship between the number of tasks in each task set and the first configuration parameter is used. Referring to the processing rules corresponding to the relationship between the number of tasks and the first configuration parameter in Table 1, the task set is processed accordingly to generate the target task set corresponding to the fourth task set.

[0073] S503: Loading the target task set corresponding to the fourth task set into the first queue.

[0074] Here, after the target task set corresponding to the fourth task set is generated based on the fourth task set, the target task set corresponding to the fourth task set is loaded into the first queue, so that the task that has timed out of scheduling is executed, and the task that failed to execute is executed again. In actual applications, when the target task set corresponding to the fourth task set is loaded into the first queue, it is not considered whether the number of target task sets in the first queue exceeds the number threshold configured for storing target task sets for the first queue, so as to ensure that the tasks that have timed out of scheduling or failed to execute can be loaded and scheduled for processing on a regular basis. In actual applications, when the target task set corresponding to the fourth task set is loaded into the first queue, if the number of target task sets in the first queue is less than the number threshold configured for storing target task sets, the next batch of fourth task sets can continue to be loaded.

[0075] S104: Execute each target task set in at least one target task set in the first queue.

[0076] Here, according to the order of the target task set in the first queue, the target task set in the first queue is executed in sequence. In practical applications, a target task execution thread pool can be set, and the number of threads can be configured in advance. In the present embodiment, the number of threads can be configured to be 10. A target task set is obtained from the first queue, and the task execution thread is handed over to perform idempotent execution processing, wherein, by performing idempotent execution processing on the task, it can be ensured that the application effect of the task being successfully executed multiple times is the same as that of successfully executing once, and it can also be ensured that the application effect of the task being successfully executed again after the execution failure is the same as that of the first successful execution, thereby ensuring the consistency and integrity of the business data. In practical applications, after obtaining the target task set, the execution of the target task set is realized on the corresponding thread by encapsulating the task execution parameter as a list parameter, when the thread pool has no idle thread, a new thread is released after the thread completes the execution of the standard task set, and the target task set in the first queue can be loaded immediately. In the case that there is no target task set in the first queue, the first queue continues to be monitored, and when the target task set appears in the first queue, the new target task set in the first queue is loaded and executed, thereby improving the execution efficiency of the task.

[0077] In one embodiment, after executing each target task set in at least one target task set in the first queue, the method further includes:

[0078] The execution parameters of each target task set in the target task set are stored in the setting database; the execution parameters at least include the execution status and the number of task execution failures.

[0079] Here, after a target task set is completed, the execution parameters of each task in the target task set are stored in a setting database based on the execution status of each task in the target task set. For example, the execution parameters may include the execution status of the task and the number of task execution failures. The execution status of the task can be used to determine whether the task is successfully executed, and the number of execution failures can be used to determine the execution status of the task after the execution failure. Therefore, the failed tasks can be rescheduled and executed based on the execution parameters of the tasks in the setting database.

[0080] In the above embodiment, the task set that needs to be scheduled is divided into a target task set consisting of a set number of tasks, so that the number of tasks in each target task set is the same, the scheduled target task set is loaded into the queue to be executed, and the target task sets in the queue to be executed are executed in sequence, thereby realizing the scheduling of large batches of tasks and improving the efficiency of task scheduling. Moreover, the number of tasks set in the target task set is the number of tasks that the system can complete within a reasonable time segment, thereby improving the overall processing efficiency of the task set and supporting better balanced scheduling.

[0081] This application also provides an application embodiment, such as Figure 7 As shown, Figure 7 A schematic diagram of a task scheduling process is shown.

[0082] S701: Acquire a first task set, where the first task set is a set consisting of batch tasks.

[0083] S702: Divide the first task set into at least one second task set according to the task type.

[0084] S703: Determine whether the first queue is full of target task sets. The first queue is used to store target task sets, which are task sets to be executed and represent a set of tasks whose number of tasks is a first configuration parameter.

[0085] S704: When the first queue is not full of target task sets, the target task set generated based on the first task set is stored in the first queue.

[0086] S705: Extract the target task set in the first queue for execution.

[0087] S706: Storing the execution parameters of each task in the target task set in a setting database.

[0088] S707: When the first queue is full of the target task set, the first task set is stored in the database.

[0089] S708: When the first queue is not full of the target task set, re-read the first task set.

[0090] S709: Load the tasks that have timed out and failed to execute in the setting database, and reschedule these tasks.

[0091] The present application also provides a task scheduling architecture diagram, such as Figure 8 As shown, in Figure 8 In the task scheduling architecture diagram, the task receiving unit, task dynamic batching unit, task coordination scheduling unit, task execution unit and task compensation scheduling unit are included. The task receiving unit is responsible for the reception, storage and feedback management of batch tasks of the business application system. The task dynamic batching unit is responsible for dividing the large batch of tasks received into several small batch tasks when receiving large batch tasks, as well as the batch management of the target tasks accumulated by small batch tasks. The task coordination scheduling unit is responsible for balancing and coordinating the backlog of batch tasks received by a business application system to be executed in other business application systems. The task execution unit is responsible for the efficient idempotent execution of the target task set and the management of task status through multi-threading technology. The task compensation scheduling unit is responsible for rescheduling tasks that have timed out or failed to execute. In actual applications, the task scheduling system and the business that initiates batch task execution can be deployed in the same process, and supports distributed cluster deployment, such as Fig. 9 As shown, Fig. 9 A distributed cluster deployment diagram is shown, which can ensure the stability, fault tolerance and high concurrency of the system.

[0092] To implement the method of the embodiment of the present application, the embodiment of the present application also provides a task scheduling device, such as Fig.10 As shown, the device comprises:

[0093] An acquisition unit 1001 is used to acquire a first task set; the first task set includes at least one task in a first state; the first state represents a state to be scheduled;

[0094] A generating unit 1002 is configured to generate at least one target task set based on the first task set according to a first configuration parameter; the first configuration parameter represents the number of set tasks; the target task set represents a task set whose number of tasks is the first configuration parameter;

[0095] The loading unit 1003 is used to load the at least one target task set into a first queue; the first queue is a queue composed of task sets to be executed;

[0096] The execution unit 1004 is configured to execute each target task set in at least one target task set in the first queue.

[0097] In one embodiment, when generating at least one target task set based on the first task set according to the first configuration parameter, the generating unit 1002 is configured to:

[0098] According to the task type of each task in the first task set, the first task set is divided into at least one second task set; the second task set is a task set composed of tasks of the same task type in the first task set;

[0099] At least one target task set is generated according to the magnitude relationship between a first parameter of each second task set in at least one of the second task sets and the first configuration parameter; the first parameter represents the number of tasks.

[0100] In one embodiment, when generating at least one target task set based on the first task set according to the first configuration parameter, the generating unit 1002 is configured to:

[0101] When the first parameter of the second task set is equal to the first configuration parameter, determining the second task set as the target task set;

[0102] When the first parameter of the second task set is greater than the first configuration parameter, extracting tasks whose number of tasks is the first configuration parameter from the second task set to generate at least one of the target task sets;

[0103] When the first parameter of the second task set is less than the first configuration parameter, the tasks in the second task set are added to a third task set with the same task type as the second task set within the first time, and the third task set is determined as the target task set if the set conditions are met; the third task set represents a task set in a second state and whose first parameter is less than the first configuration parameter; the second state represents a scheduled state.

[0104] In one embodiment, the setting condition includes any one of the following:

[0105] The target task set does not exist in the first queue;

[0106] There is at least one target task set in the first queue, and the first parameter of the third task set is greater than or equal to the first configuration parameter;

[0107] There is at least one target task set in the first queue, and the first time is greater than or equal to a second configuration parameter; the second configuration parameter is a set duration.

[0108] In one embodiment, when acquiring the first task set, the acquiring unit 1001 is used to:

[0109] Determine a second parameter of the first queue at a second time interval; the second parameter represents the number of target task sets in the first queue;

[0110] When the second parameter is less than or equal to a third configuration parameter, a first task set is acquired; the third configuration parameter is the number of set target task sets.

[0111] In one embodiment, when executing each target task set in at least one target task set in the first queue, the execution unit 1004 is configured to:

[0112] In an idle thread, idempotent execution is performed on each of the target task sets in at least one of the target task sets in the first queue.

[0113] In one embodiment, when the loading unit 1003 loads the at least one target task set into the first queue, the apparatus is further configured to:

[0114] Obtaining a fourth task set every third time; the fourth task set is a set of at least one task in the setting information library whose scheduling time has timed out or whose execution status has failed; the third time is determined based on the number of task execution failures;

[0115] According to the set rules, a target task set corresponding to the fourth task set is generated;

[0116] The target task set corresponding to the fourth task set is loaded into the first queue.

[0117] In one embodiment, when the generating unit 1002 generates at least one target task set based on the first task set according to the size relationship between the first parameter of the first task set and the first configuration parameter, the device is further configured to:

[0118] storing the scheduling parameters of each task of at least one task in the first task set in a setting database; the setting database is used to store task information of the task; the scheduling parameters at least include the scheduling time;

[0119] After the execution unit 1004 executes each target task set in at least one target task set in the first queue, the device is further configured to:

[0120] The execution parameters of each target task set in the target task set are stored in the setting database; the execution parameters at least include the execution status and the number of task execution failures.

[0121] In actual application, the acquisition unit 1001, the generation unit 1002, the loading unit 1003, and the execution unit 1004 can be implemented by a processor in the task scheduling device. Of course, the processor needs to run the program stored in the memory to implement the functions of the above program modules.

[0122] It should be noted that the above Fig.10 The task scheduling device provided in the embodiment only uses the division of the above-mentioned program modules as an example when performing task scheduling. In actual applications, the above-mentioned processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the task scheduling device provided in the above embodiment and the task scheduling method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0123] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiment of the present application, the embodiment of the present application further provides an electronic device, Fig.11 Schematic diagram of the hardware structure of the electronic device of the present application embodiment. Fig.11 As shown, the electronic equipment includes:

[0124] Communication interface 1, capable of exchanging information with other devices such as network devices;

[0125] The processor 2 is connected to the communication interface 1 to implement information exchange with other devices and is used to execute the task scheduling method provided by one or more technical solutions when running a computer program. The computer program is stored in the memory 3.

[0126] Of course, in actual application, the various components in the electronic device are coupled together through the bus system 4. It can be understood that the bus system 4 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 4 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Fig.11 Various buses are labeled as bus system 4.

[0127] The memory 3 in the embodiment of the present application is used to store various types of data to support the operation of the electronic device. Examples of such data include: any computer program used to operate on the electronic device.

[0128] It can be understood that the memory 3 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 3 described in the embodiments of the present application is intended to include but is not limited to these and any other suitable types of memories.

[0129] The method disclosed in the above embodiment of the present application can be applied to the processor 2, or implemented by the processor 2. The processor 2 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 2 or the instruction in the form of software. The above processor 2 may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 2 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in the memory 3, and the processor 2 reads the program in the memory 3 and completes the steps of the above method in combination with its hardware.

[0130] When the processor 2 executes the program, the corresponding processes in the various methods of the embodiments of the present application are implemented, which will not be repeated here for the sake of brevity.

[0131] In an exemplary embodiment, the present application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, a memory 3 storing a computer program, and the computer program can be executed by a processor 2 to complete the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface storage, optical disk, or CD-ROM.

[0132] In several embodiments provided in the present application, it should be understood that the disclosed devices, electronic devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0133] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0134] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0135] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium, which, when executed, executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, disks or optical disks.

[0136] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0137] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A task scheduling method, characterized in that: include: Get the first task set; The first task set includes at least one task in a first state; The first state represents a waiting-to-be-scheduled state; Generating at least one target task set based on the first task set according to a first configuration parameter; the first configuration parameter represents the number of set tasks; The target task set represents a task set whose number of tasks is the first configuration parameter; Loading the at least one target task set into a first queue; the first queue is a queue consisting of task sets to be executed; executing each target task set in at least one target task set in the first queue; The step of generating at least one target task set based on the first task set according to the first configuration parameter includes: According to the task type of each task in the first task set, the first task set is divided into at least one second task set; the second task set is a task set composed of tasks of the same task type in the first task set; Generate at least one target task set according to the magnitude relationship between the first parameter of each second task set in at least one of the second task sets and the first configuration parameter; the first parameter represents the number of tasks; Wherein, when generating at least one target task set according to the magnitude relationship between the first parameter of each second task set in at least one of the second task sets and the first configuration parameter, the method includes: When the first parameter of the second task set is less than the first configuration parameter, the tasks in the second task set are added to a third task set with the same task type as the second task set within the first time, and the third task set is determined as the target task set if the set conditions are met; the third task set represents a task set in a second state and whose first parameter is less than the first configuration parameter; the second state represents a scheduled state.

2. The method according to claim 1, characterized in that The method further comprises: When the first parameter of the second task set is equal to the first configuration parameter, determining the second task set as the target task set; When the first parameter of the second task set is greater than the first configuration parameter, tasks whose number of tasks is the first configuration parameter are extracted from the second task set to generate at least one of the target task sets.

3. The method according to claim 2, characterized in that The setting conditions include any of the following: The target task set does not exist in the first queue; There is at least one target task set in the first queue, and the first parameter of the third task set is greater than or equal to the first configuration parameter; There is at least one target task set in the first queue, and the first time is greater than or equal to a second configuration parameter; the second configuration parameter is a set duration.

4. The method according to claim 1, characterized in that The obtaining of the first task set includes: Determine a second parameter of the first queue at a second time interval; the second parameter represents the number of target task sets in the first queue; When the second parameter is less than or equal to a third configuration parameter, a first task set is acquired; the third configuration parameter is the number of set target task sets.

5. The method according to claim 1, characterized in that The executing each target task set of at least one target task set in the first queue includes: In an idle thread, idempotent execution is performed on each of the target task sets in at least one of the target task sets in the first queue.

6. The method according to claim 1, characterized in that The method further comprises: loading the at least one target task set into the first queue; Obtaining a fourth task set every third time; the fourth task set is a set of at least one task whose scheduling time has timed out or whose execution status has failed in the setting information base; the third time is determined based on the number of task execution failures; According to the set rules, a target task set corresponding to the fourth task set is generated; The target task set corresponding to the fourth task set is loaded into the first queue.

7. The method according to claim 1, characterized in that When generating at least one target task set based on the first task set according to the size relationship between the first parameter of the first task set and the first configuration parameter, the method further includes: storing the scheduling parameters of each task of at least one task in the first task set in a setting database; the setting database is used to store task information of the task; the scheduling parameters at least include the scheduling time; After executing each target task set in at least one target task set in the first queue, the method further includes: The execution parameters of each target task set in the target task set are stored in the setting database; the execution parameters at least include the execution status and the number of task execution failures.

8. A task scheduling device, characterized in that: include: An acquisition unit, used for acquiring a first task set; The first task set includes at least one task in a first state; The first state represents a waiting-to-be-scheduled state; A generating unit, configured to generate at least one target task set based on the first task set according to a first configuration parameter; the first configuration parameter represents a set number of tasks; The target task set represents a task set whose number of tasks is the first configuration parameter; A loading unit, configured to load the at least one target task set into a first queue; The first queue is a queue consisting of a set of tasks to be executed; An execution unit, configured to execute each target task set in at least one target task set in the first queue; The generating unit is further used to divide the first task set into at least one second task set according to the task type of each task in the first task set; the second task set is a task set composed of tasks of the same task type in the first task set; at least one target task set is generated according to the size relationship between the first parameter of each second task set in at least one second task set and the first configuration parameter; the first parameter represents the number of tasks; The generating unit is further configured to, when the first parameter of the second task set is less than the first configuration parameter, add the tasks in the second task set to a third task set of the same task type as the second task set within a first time, and determine the third task set as the target task set if a set condition is met; the third task set represents a task set in the second state and whose first parameter is less than the first configuration parameter; The second state represents a scheduled state.

9. An electronic device, characterized in that: include: a processor and a memory for storing a computer program capable of being executed on the processor, Wherein, when the processor is used to run the computer program, it executes the steps of the method described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images