Method and device for scheduling push task

Abstract

The present invention discloses a method and device for scheduling push tasks, and relates to the field of computer technology. A specific implementation of the method includes: testing and running all push tasks, and regularly obtaining the cluster residual throughput of the target cluster that receives push data during the test; determining the execution order of all push tasks according to the task parameters of each push task and the cluster residual throughput obtained regularly; and pushing the push data to the target cluster according to the execution order. This implementation automatically coordinates the execution order of all push tasks on a cluster by testing and running all push tasks, so as to achieve the purpose of efficiently utilizing the throughput of the entire cluster.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for scheduling a number-pushing task. Background Art

[0002] The push task is used to push offline data to the cluster. In order to efficiently utilize cluster functions, a cluster may carry index pushes for multiple projects. When billions of data are pushed to the cluster, cluster throughput will be a bottleneck. In the existing technology, the utilization rate of ES cluster throughput is usually improved by increasing the cluster write speed and processing push tasks serially or in parallel. However, this method cannot improve the utilization rate of the entire cluster throughput when the cluster throughput is limited. Summary of the invention

[0003] In view of this, an embodiment of the present invention provides a method and device for scheduling push tasks, which automatically coordinates the execution order of all push tasks on a cluster by testing and running all push tasks, thereby achieving the purpose of efficiently utilizing the throughput of the entire cluster.

[0004] To achieve the above object, according to one aspect of an embodiment of the present invention, a method for scheduling a push task is provided, comprising:

[0005] The test runs all push tasks and periodically obtains the remaining cluster throughput of the target cluster that receives the pushed data during the test.

[0006] Determine the execution order of all the push tasks according to the task parameters of each push task and the cluster remaining throughput obtained at regular intervals;

[0007] The pushed data is pushed to the target cluster according to the execution order.

[0008] Optionally, the task parameters include: task occupancy throughput and task priority;

[0009] According to the task parameters of each push task and the cluster remaining throughput obtained at regular intervals, the execution order of all push tasks is determined, including:

[0010] After obtaining the remaining throughput of the cluster each time, the waiting push tasks whose task occupied throughput is less than or equal to the remaining throughput of the cluster are sorted in descending order of task priority to obtain a sorting result;

[0011] The execution order of all the push tasks is determined according to the sorting result corresponding to the cluster remaining throughput obtained each time.

[0012] Optionally, the task parameters also include: push duration and start time;

[0013] After sorting the waiting push tasks whose task occupied throughput is less than or equal to the remaining throughput of the cluster in order of task priority from high to low, the method further includes:

[0014] Determine whether there are waiting push tasks with the same task priority in the sorted task sequence; if not, use the task sequence as the sorting result; if so, sort the waiting push tasks with the same task priority in the task sequence again in order of push duration from short to long;

[0015] Determine whether there are waiting-pushing tasks with the same pushing time in the re-sorted task sequence; if so, sort the waiting-pushing tasks with the same pushing time in the re-sorted task sequence in order from front to back of the start time to obtain the sorting result; otherwise, use the second task sequence as the sorting result.

[0016] Optionally, after determining the execution order of all the push number tasks, the method further includes: judging whether the execution order of all the push number tasks satisfies a first adjustment condition; if so, adjusting the execution order of all the push number tasks according to the following steps:

[0017] Adjust the task priority of at least one waiting push task to be the same as the task priority of the previous push task, and then retest and run all the push tasks to determine the new execution order of all the push tasks;

[0018] Determine whether the total execution time of the push-number tasks corresponding to the new execution order is greater than the total execution time of the push-number tasks corresponding to the execution order; if so, roll back the adjusted task priority of the waiting push-number tasks, and do not adjust the execution order of all the push-number tasks; otherwise, use the new execution order as the execution order of all the push-number tasks;

[0019] Among them, the first adjustment condition includes: there is an idle period on the push-number time axis corresponding to the execution order, and there is a waiting push-number task in the idle period; the idle period refers to the period when the cluster remaining throughput of the target cluster is equal to zero; the previous push-number task refers to the push-number task that was last executed before the idle period.

[0020] Optionally, adjusting the task priority of at least one waiting push task to be the same as the task priority of the previous push task includes:

[0021] Determine the waiting task whose priority is to be adjusted according to the cluster maximum throughput of the target cluster and the task occupied throughput of each waiting task on the push timeline;

[0022] The task priority of the waiting task whose priority is to be adjusted is adjusted to be the same as the task priority of the preceding task, and the cluster remaining throughput of the target cluster after adjustment is updated.

[0023] Optionally, after adjusting the task priority of at least one waiting push task to be the same as the task priority of the previous push task, the method further includes:

[0024] It is determined whether the cluster remaining throughput of the target cluster after adjustment is greater than zero; if so, the task occupied throughput of the adjusted waiting push task is increased to make the cluster remaining throughput of the target cluster zero.

[0025] Optionally, after determining the execution order of all the push number tasks, the method further includes: judging whether the execution order of all the push number tasks satisfies a second adjustment condition; if so, adjusting the execution order of all the push number tasks according to the following steps:

[0026] Adjust the task priority of at least one waiting push task whose task occupied throughput is less than or equal to the remaining throughput of the cluster to be the same as the task priority of the push task being executed, and then retest and run all the push tasks to determine a new execution order of all the push tasks;

[0027] Determine whether the total execution time of the push-number tasks corresponding to the new execution order is greater than the total execution time of the push-number tasks corresponding to the execution order; if so, roll back the adjusted task priority of the waiting push-number tasks, and do not adjust the execution order of all the push-number tasks; otherwise, use the new execution order as the execution order of all the push-number tasks;

[0028] Among them, the second adjustment condition includes: there is an incomplete period on the push timeline corresponding to the execution order, and there is a waiting push task in the incomplete period whose task occupation throughput is less than or equal to the remaining throughput of the cluster; the incomplete period refers to the period when the remaining cluster throughput of the target cluster is greater than zero and less than the maximum throughput of the cluster.

[0029] Optionally, adjusting the task priority of at least one waiting push task whose task occupied throughput is less than or equal to the remaining throughput of the cluster to be the same as the task priority of the push task being executed includes:

[0030] Determine the waiting task whose priority is to be adjusted according to the remaining throughput of the cluster and the task occupied throughput of each waiting task on the push timeline;

[0031] The task priority of the waiting task whose priority is to be adjusted is adjusted to be the same as the task priority of the push task being executed, and the cluster remaining throughput of the target cluster after the adjustment is updated.

[0032] Optionally, after adjusting the task priority of at least one waiting push task whose task occupied throughput is less than or equal to the remaining throughput of the cluster to be the same as the task priority of the push task being executed, the method further includes:

[0033] It is determined whether the cluster remaining throughput of the target cluster after adjustment is greater than zero; if so, the task occupied throughput of the adjusted waiting push task is increased to make the cluster remaining throughput of the target cluster zero.

[0034] According to a second aspect of an embodiment of the present invention, a device for scheduling a push task is provided, comprising:

[0035] The acquisition module tests and runs all push tasks, and periodically obtains the remaining cluster throughput of the target cluster that receives the pushed data during the test;

[0036] A determination module determines the execution order of all push tasks according to the task parameters of each push task and the cluster remaining throughput obtained at regular intervals;

[0037] A push module pushes the push data to the target cluster according to the execution order.

[0038] Optionally, the task parameters include: task occupancy throughput and task priority;

[0039] The determination module determines the execution order of all push tasks according to the task parameters of each push task and the cluster remaining throughput obtained at regular intervals, including:

[0040] After obtaining the remaining throughput of the cluster each time, the waiting push tasks whose task occupied throughput is less than or equal to the remaining throughput of the cluster are sorted in descending order of task priority to obtain a sorting result;

[0041] The execution order of all the push tasks is determined according to the sorting result corresponding to the cluster remaining throughput obtained each time.

[0042] Optionally, the task parameters also include: push duration and start time;

[0043] The determination module is further used to: sort the waiting push tasks whose task occupied throughput is less than or equal to the remaining throughput of the cluster in order of task priority from high to low,

[0044] Determine whether there are waiting push tasks with the same task priority in the sorted task sequence; if not, use the task sequence as the sorting result; if so, sort the waiting push tasks with the same task priority in the task sequence again in order of push duration from short to long;

[0045] Determine whether there are waiting-pushing tasks with the same pushing time in the re-sorted task sequence; if so, sort the waiting-pushing tasks with the same pushing time in the re-sorted task sequence in order from front to back of the start time to obtain the sorting result; otherwise, use the second task sequence as the sorting result.

[0046] Optionally, the determination module is further used to: after determining the execution order of all the push number tasks, determine whether the execution order of all the push number tasks meets the first adjustment condition; if so, adjust the execution order of all the push number tasks according to the following steps:

[0047] Adjust the task priority of at least one waiting push task to be the same as the task priority of the previous push task, and then retest and run all the push tasks to determine the new execution order of all the push tasks;

[0048] Determine whether the total execution time of the push-number tasks corresponding to the new execution order is greater than the total execution time of the push-number tasks corresponding to the execution order; if so, roll back the adjusted task priority of the waiting push-number tasks, and do not adjust the execution order of all the push-number tasks; otherwise, use the new execution order as the execution order of all the push-number tasks;

[0049] Among them, the first adjustment condition includes: there is an idle period on the push-number time axis corresponding to the execution order, and there is a waiting push-number task in the idle period; the idle period refers to the period when the cluster remaining throughput of the target cluster is equal to zero; the previous push-number task refers to the push-number task that was last executed before the idle period.

[0050] Optionally, the determining module adjusts the task priority of at least one waiting push task to be the same as the task priority of the previous push task, including:

[0051] Determine the waiting task whose priority is to be adjusted according to the cluster maximum throughput of the target cluster and the task occupied throughput of each waiting task on the push timeline;

[0052] The task priority of the waiting task whose priority is to be adjusted is adjusted to be the same as the task priority of the preceding task, and the cluster remaining throughput of the target cluster after adjustment is updated.

[0053] Optionally, the determining module is further configured to: adjust the task priority of at least one waiting push task to be the same as the task priority of the previous push task,

[0054] It is determined whether the cluster remaining throughput of the target cluster after adjustment is greater than zero; if so, the task occupied throughput of the adjusted waiting push task is increased to make the cluster remaining throughput of the target cluster zero.

[0055] Optionally, the determination module is further used to: after determining the execution order of all the push number tasks, determine whether the execution order of all the push number tasks meets the second adjustment condition; if so, adjust the execution order of all the push number tasks according to the following steps:

[0056] Adjust the task priority of at least one waiting push task whose task occupied throughput is less than or equal to the remaining throughput of the cluster to be the same as the task priority of the push task being executed, and then retest and run all the push tasks to determine a new execution order of all the push tasks;

[0057] Determine whether the total execution time of the push-number tasks corresponding to the new execution order is greater than the total execution time of the push-number tasks corresponding to the execution order; if so, roll back the adjusted task priority of the waiting push-number tasks, and do not adjust the execution order of all the push-number tasks; otherwise, use the new execution order as the execution order of all the push-number tasks;

[0058] Among them, the second adjustment condition includes: there is an incomplete period on the push timeline corresponding to the execution order, and there is a waiting push task in the incomplete period whose task occupation throughput is less than or equal to the remaining throughput of the cluster; the incomplete period refers to the period when the remaining cluster throughput of the target cluster is greater than zero and less than the maximum throughput of the cluster.

[0059] Optionally, the determining module adjusts the task priority of at least one waiting push task whose task occupied throughput is less than or equal to the remaining throughput of the cluster to be the same as the task priority of the push task being executed, including:

[0060] Determine the waiting task whose priority is to be adjusted according to the remaining throughput of the cluster and the task occupied throughput of each waiting task on the push timeline;

[0061] The task priority of the waiting task whose priority is to be adjusted is adjusted to be the same as the task priority of the push task being executed, and the cluster remaining throughput of the target cluster after the adjustment is updated.

[0062] Optionally, the determining module is further configured to: adjust the task priority of at least one waiting push task whose task occupied throughput is less than or equal to the remaining throughput of the cluster to be the same as the task priority of the push task being executed,

[0063] It is determined whether the cluster remaining throughput of the target cluster after adjustment is greater than zero; if so, the task occupied throughput of the adjusted waiting push task is increased to make the cluster remaining throughput of the target cluster zero.

[0064] According to a third aspect of an embodiment of the present invention, there is provided an electronic device for scheduling push task, comprising:

[0065] one or more processors;

[0066] a storage device for storing one or more programs,

[0067] When the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the first aspect of the embodiment of the present invention.

[0068] According to a fourth aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method provided by the first aspect of the embodiment of the present invention is implemented.

[0069] One embodiment of the above invention has the following advantages or beneficial effects: by testing and running all push tasks to automatically coordinate the execution order of all push tasks on a cluster, the throughput of the entire cluster can be efficiently utilized. By detecting the execution status of each push task on the entire push timeline and dynamically adjusting task parameters such as the task priority and task occupied throughput of the push task, the utilization rate of the throughput of the entire cluster can be further improved.

[0070] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific implementation examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0072] Figure 1 Schematic diagram of the main process of the method for scheduling push task in an embodiment of the present invention;

[0073] Figure 2 It is a flowchart of a method for scheduling push task in an optional embodiment of the present invention;

[0074] Figure 3 It is a flowchart of adjusting the task parameters of the push task based on the idle period in an optional embodiment of the present invention;

[0075] Figure 4 It is a flowchart of adjusting the task parameters of the push task based on the incomplete time period in an optional embodiment of the present invention;

[0076] Figure 5 It is a schematic diagram of main modules of the device for scheduling push task according to an embodiment of the present invention;

[0077] Figure 6 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0078] Figure 7 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0079] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0080] According to one aspect of an embodiment of the present invention, a method for scheduling push task is provided.

[0081] Figure 1 FIG. 1 is a schematic diagram of the main process of the method for scheduling push task in an embodiment of the present invention. Figure 1 As shown, the method for scheduling push task includes: step S101, step S102 and step S103.

[0082] In step S101 and step S102, all push tasks are run through a test to determine the execution order of all push tasks. Specifically, in step S101, all push tasks are run through a test to periodically obtain the cluster residual throughput of the target cluster that receives the push data during the test. In step S102, the execution order of all push tasks is determined based on the task parameters of each push task and the cluster residual throughput obtained periodically. In step S103, the push data is pushed to the target cluster according to the execution order.

[0083] Exemplarily, assuming that when the test runs all push tasks, the cluster remaining throughput of the target cluster receiving the push data during the test is obtained every 1 minute, and it is obtained twice in total. After the cluster remaining throughput of the target cluster is obtained for the first time, the execution order of all push tasks is determined to be T1, T2, T3, T4, T5 according to the task parameters of each push task and the cluster remaining throughput obtained for the first time. When the push task T 2 is completed, the cluster remaining throughput of the target cluster is obtained for the second time, and the execution order of all unexecuted push tasks is determined to be T3, T5, T4 according to the task parameters of each push task and the cluster remaining throughput obtained for the second time. Then the execution order of all push tasks obtained after the test run is T1, T2, T3, T5, T4.

[0084] The content and quantity of task parameters can be selectively set according to actual conditions. For example, if the task parameters include task occupied throughput, when determining the execution order of all push tasks according to the task parameters of each push task and the cluster remaining throughput obtained periodically, all push tasks are sorted in order of task occupancy from high to low or from low to high.

[0085] Optionally, the task parameters include: task occupied throughput and task priority. According to the task parameters of each push task and the cluster remaining throughput obtained periodically, the execution order of all push tasks is determined, including: after each acquisition of the cluster remaining throughput, the waiting push tasks whose task occupied throughput is less than or equal to the cluster remaining throughput are sorted in order from high to low according to the task priority to obtain the sorting result; according to the sorting result corresponding to the cluster remaining throughput obtained each time, the execution order of all push tasks is determined.

[0086] The execution order of all push tasks is determined according to the task priority from high to low, which can ensure that more important or core push data is pushed to the target cluster first.

[0087] Optionally, the task parameters also include: push duration and start time. After sorting the waiting push tasks whose task occupancy throughput is less than or equal to the remaining throughput of the cluster in order of task priority from high to low, it also includes: judging whether there are waiting push tasks with the same task priority in the sorted task sequence; if not, taking the task sequence as the sorting result; if so, sorting the waiting push tasks with the same task priority in the task sequence again in order of push duration from short to long; judging whether there are waiting push tasks with the same push duration in the re-sorted task sequence; if so, sorting the waiting push tasks with the same push duration in the re-sorted task sequence in order of start time from front to back to obtain the sorting result; otherwise, taking the second task sequence as the sorting result.

[0088] Figure 2 FIG. 1 is a flow chart of a method for scheduling a task in a preferred embodiment of the present invention. Figure 2 The optional embodiment shows that the cluster maximum throughput of the target cluster and the task parameters such as the task occupied throughput and push duration of each push task are first tested, and the task parameters of each push task are added so as to determine the execution order of the push task later. In this embodiment, the task parameters of each push task can be dynamically detected and optimized, and the modified task parameters are added so as to adjust the execution order of the push task according to the modified parameters. For each push task, first detect whether the current cluster remaining throughput of the target cluster is less than the task occupied throughput of the push task. If so, it indicates that the current environment is not suitable for executing the push task, and wait and loop to detect whether there is an upstream task to be executed; if not, detect whether there is a high-priority push task that has not been executed. If so, it indicates that there is a more important or core push task to be executed at present, and the push task is not executed at this time, and wait and loop to detect whether there is an upstream task to be executed; if not, detect whether there is a push task with a shorter push duration among the push tasks with the same task priority as the push task. If so, the push task is not executed, and wait and loop to detect whether there is an upstream task to be executed; if not, execute the push task.

[0089] In this embodiment, among multiple push tasks with the same task priority, the push tasks with shorter push duration are processed preferentially, which can ensure that the concurrency of processing the push tasks is increased and the utilization rate of the overall cluster throughput is improved.

[0090] The embodiment of the present invention automatically coordinates the execution order of all the push tasks on a cluster by testing and running all the push tasks, thereby achieving the purpose of efficiently utilizing the throughput of the entire cluster.

[0091] In some optional embodiments, after determining the execution order of all the push number tasks, the method further includes: determining whether the execution order of all the push number tasks satisfies a first adjustment condition; if so, adjusting the execution order of all the push number tasks according to the following steps:

[0092] Adjust the task priority of at least one waiting push task to be the same as the task priority of the previous push task, and then retest and run all the push tasks to determine the new execution order of all the push tasks;

[0093] Determine whether the total execution time of the push-number tasks corresponding to the new execution order is greater than the total execution time of the push-number tasks corresponding to the execution order; if so, roll back the adjusted task priority of the waiting push-number tasks, and do not adjust the execution order of all the push-number tasks; otherwise, use the new execution order as the execution order of all the push-number tasks;

[0094] Among them, the first adjustment condition includes: there is an idle period on the push-number time axis corresponding to the execution order, and there is a waiting push-number task in the idle period; the idle period refers to the period when the cluster remaining throughput of the target cluster is equal to zero; the previous push-number task refers to the push-number task that was last executed before the idle period.

[0095] For example, assuming that there is an idle period on the push time axis, the task priority of the last push task executed before the idle period is A, and there are multiple waiting push tasks in the idle period, then the task priority of the waiting push task with the earliest start time or the highest task priority or the shortest push duration after the idle period is adjusted to A, so that the waiting push task is executed in the idle period. This embodiment can make full use of the throughput of the target cluster in the idle period and improve the overall cluster throughput utilization of the target cluster in this push cycle.

[0096] Optionally, the task priority of at least one waiting push task is adjusted to be the same as the task priority of the previous push task, including: determining the waiting task whose priority is to be adjusted based on the cluster maximum throughput of the target cluster and the task occupied throughput of each waiting push task on the push timeline; adjusting the task priority of the waiting task whose priority is to be adjusted to be the same as the task priority of the previous push task, and updating the cluster remaining throughput of the target cluster after the adjustment.

[0097] When there are many tasks waiting to push data in the idle period, the waiting tasks whose priorities are to be adjusted can be screened in the order of start time from early to late, task priority from high to low, or push duration from short to long. For example, assuming that the maximum throughput of the cluster is 10, there are 4 tasks waiting to push data in the idle period in the order of priority from high to low, and the corresponding task occupancy throughputs are 3, 3, 3, and 4 respectively. Then the task priorities of the first three tasks waiting to push data are adjusted to be the same as the task priority of the previous task.

[0098] Optionally, after adjusting the task priority of at least one waiting push task to be the same as the task priority of the previous push task, it also includes: determining whether the cluster remaining throughput of the target cluster after the adjustment is greater than zero; if so, increasing the task occupied throughput of the adjusted waiting push task so that the cluster remaining throughput of the target cluster is zero.

[0099] When adjusting the task occupied throughput of the adjusted waiting push tasks, the remaining cluster throughput of the target cluster at the time of adjustment can be evenly allocated to the adjusted waiting push tasks, or only allocated to some of the adjusted waiting push tasks. The throughput allocated to each adjusted waiting push task can be the same or different. Figure 3 FIG. 1 is a flow chart of adjusting the task parameters of the push task based on the idle period in an optional embodiment of the present invention. Figure 3 As shown, first check whether there is an idle period on the push timeline. If not, the process ends; if so, check whether there is a waiting push task in the idle period. If not, the process ends; if so, increase the priority and task occupancy throughput of the waiting push task. Then retest and run all push tasks to determine the total push task execution time of all push tasks after adjustment. Determine whether the total push task execution time of all push tasks after the adjustment becomes longer, that is, whether the total push task execution time of all push tasks after the adjustment is greater than the total push task execution time of all push tasks before the adjustment. If so, roll back the task priority and task status throughput of the waiting push task, and do not adjust the task parameters of the push task.

[0100] For example, assuming that the maximum cluster throughput is 10, there are 4 waiting push tasks in the idle period in order of priority from high to low, and the corresponding task occupied throughputs are 3, 3, 3, and 4 respectively. After the task priorities of the first three waiting push tasks are adjusted to be the same as the task priorities of the previous push tasks, the cluster remaining throughput of the target cluster is greater than zero. At this time, the remaining 1 throughput can be added to any push task with the adjusted task priority. This embodiment can maximize the use of the cluster throughput of the target cluster and improve the utilization rate of the overall cluster throughput by increasing the task occupied throughput of the adjusted waiting push tasks so that the cluster remaining throughput of the target cluster is zero.

[0101] In some other optional embodiments, after determining the execution order of all the push number tasks, the method further includes: judging whether the execution order of all the push number tasks satisfies a second adjustment condition; if so, adjusting the execution order of all the push number tasks according to the following steps:

[0102] Adjust the task priority of at least one waiting push task whose task occupied throughput is less than or equal to the remaining throughput of the cluster to be the same as the task priority of the push task being executed, and then retest and run all the push tasks to determine a new execution order of all the push tasks;

[0103] Determine whether the total execution time of the push-number tasks corresponding to the new execution order is greater than the total execution time of the push-number tasks corresponding to the execution order; if so, roll back the adjusted task priority of the waiting push-number tasks, and do not adjust the execution order of all the push-number tasks; otherwise, use the new execution order as the execution order of all the push-number tasks;

[0104] Among them, the second adjustment condition includes: there is an incomplete period on the push timeline corresponding to the execution order, and there is a waiting push task in the incomplete period whose task occupation throughput is less than or equal to the remaining throughput of the cluster; the incomplete period refers to the period when the remaining cluster throughput of the target cluster is greater than zero and less than the maximum throughput of the cluster.

[0105] For example, assuming that there is an incomplete period on the push time axis, the task priority of the push task being executed in the incomplete period is A, and there are multiple waiting push tasks in the incomplete period, then the task priority of the waiting push task with the earliest start time or the highest task priority or the shortest push duration after the incomplete period is adjusted to A, so that the waiting push task can be executed in the incomplete period. This embodiment can make full use of the remaining throughput of the target cluster in the incomplete period, and improve the overall cluster throughput utilization of the target cluster in this push cycle.

[0106] Optionally, the task priority of at least one waiting push task whose task occupied throughput is less than or equal to the remaining throughput of the cluster is adjusted to be the same as the task priority of the push task being executed, including: determining the waiting task whose priority is to be adjusted based on the remaining throughput of the cluster and the task occupied throughput of each waiting push task on the push timeline; adjusting the task priority of the waiting task whose priority is to be adjusted to be the same as the task priority of the push task being executed, and updating the cluster remaining throughput of the target cluster after the adjustment.

[0107] When there are a large number of waiting tasks in the partially filled period, the waiting tasks whose priorities are to be adjusted can be screened in the order of start time from early to late, task priority from high to low, or push duration from short to long. For example, assuming that the maximum throughput of the cluster is 10, and the remaining throughput of the cluster is 3 in a 15-minute period, then the 15-minute period is an partially filled period. If there is a waiting task with a task occupancy throughput of 3 in the partially filled period, the task priority of the waiting task can be adjusted to the same as the task priority of the currently executing push task.

[0108] Optionally, after adjusting the task priority of at least one waiting push task whose task occupied throughput is less than or equal to the remaining throughput of the cluster to be the same as the task priority of the push task being executed, it also includes: determining whether the cluster remaining throughput of the target cluster after the adjustment is greater than zero; if so, increasing the adjusted task occupied throughput of the waiting push task so that the cluster remaining throughput of the target cluster is zero.

[0109] When adjusting the task occupied throughput of the adjusted waiting push tasks, the remaining cluster throughput of the target cluster at the time of adjustment can be evenly allocated to the adjusted waiting push tasks, or only allocated to some of the adjusted waiting push tasks. The throughput allocated to each adjusted waiting push task can be the same or different. Figure 4 FIG. 1 is a flow chart of adjusting the task parameters of the push task based on the incomplete time period in an optional embodiment of the present invention. Figure 4 As shown, first check whether there is an incomplete period on the push timeline. If not, the process ends; if so, check whether there is a waiting push task in the idle period. If not, the process ends; if so, increase the priority and task occupancy throughput of the waiting push task. Then retest and run all push tasks to determine the total push task execution time of all push tasks after adjustment. Determine whether the total push task execution time of all push tasks becomes longer after the adjustment, that is, whether the total push task execution time of all push tasks after the adjustment is greater than the total push task execution time of all push tasks before the adjustment. If so, roll back the task priority and task status throughput of the waiting push task, and do not adjust the task parameters of the push task.

[0110] Exemplarily, assuming that the maximum cluster throughput is 10, and there is a 15-minute period in which the cluster remaining throughput is 5, then the 15-minute period is an underutilized period. If there are waiting push tasks with task occupancy throughputs of 1 and 2 respectively in the underutilized period, then after the task priorities of the two waiting push tasks are adjusted to be the same as the task priority of the currently executing push task, the cluster remaining throughput of the target cluster is 2, which is greater than zero. At this time, the remaining 2 throughputs can be added to the two push tasks with adjusted task priorities, that is, the task occupancy throughputs of the two push tasks with adjusted task priorities are 2 and 3 respectively. This embodiment maximizes the utilization of the cluster throughput of the target cluster and improves the utilization rate of the overall cluster throughput by increasing the task occupancy throughput of the adjusted waiting push tasks so that the cluster remaining throughput of the target cluster is zero.

[0111] If you write push data to the cluster by batch submission and multithreading, and then increase the write speed when the throughput is full, the pushed data will be discarded because it cannot be processed by the cluster in time, resulting in data loss. If you increase cluster throughput by optimizing hardware, the cost of cluster machines will increase, and when the hardware is optimized, you will still encounter write bottlenecks if you continue to increase the amount of push data. If you increase the cluster write speed by increasing the refresh interval, reducing the number of replicas, etc., you will sacrifice search real-time performance and data reliability. If you increase the cluster write speed by serial processing of push tasks, the failure or delay of some pre-tasks will cause the entire cluster push time to be delayed. If you increase the cluster write speed by parallel processing of push tasks, because the amount of push data for each push task is different, using a low concurrency will result in a long push time, and the throughput of the entire cluster cannot be efficiently utilized. When using a high concurrency, because the upstream tasks of the index end at different times, if too many push tasks are executed at the same time, the concurrency will be too large, and the data pushed to the cluster will be lost. If you add a lost data retry mechanism, not only will the cluster throughput not be improved, but too many lost data retries will increase the load of the entire cluster.

[0112] In the actual production process, the goal is not to pursue the ultimate writing speed, but to maximize the cluster throughput utilization under limited hardware equipment conditions and under certain business scenarios. The embodiment of the present invention automatically coordinates the execution order of all push tasks on a cluster by testing and running all push tasks, thereby achieving the purpose of efficiently utilizing the throughput of the entire cluster. By detecting the execution status of each push task on the entire push timeline and dynamically adjusting task parameters such as the task priority and task occupancy throughput of the push task, the utilization of the entire cluster throughput can be further improved.

[0113] It should be noted that the cluster mentioned in the embodiment of the present invention may be other clusters besides ES (ElasticSearch, a search server based on Lucene), such as MySQL (a relational database management system) cluster, HBase (a distributed, column-oriented open source database) cluster, etc.

[0114] According to a second aspect of an embodiment of the present invention, a device for implementing the above method is provided.

[0115] Figure 5 FIG. 1 is a schematic diagram of the main modules of the apparatus for scheduling push task according to an embodiment of the present invention. Figure 5 As shown, the device 500 for scheduling push task includes:

[0116] Acquisition module 501 tests and runs all push tasks, and periodically obtains the remaining cluster throughput of the target cluster that receives the push data during the test;

[0117] A determination module 502 determines the execution order of all push tasks according to the task parameters of each push task and the cluster remaining throughput obtained at regular intervals;

[0118] The push module 503 pushes the push data to the target cluster according to the execution order.

[0119] Optionally, the task parameters include: task occupancy throughput and task priority;

[0120] The determination module determines the execution order of all push tasks according to the task parameters of each push task and the cluster remaining throughput obtained at regular intervals, including:

[0121] After obtaining the remaining throughput of the cluster each time, the waiting push tasks whose task occupied throughput is less than or equal to the remaining throughput of the cluster are sorted in descending order of task priority to obtain a sorting result;

[0122] The execution order of all the push tasks is determined according to the sorting result corresponding to the cluster remaining throughput obtained each time.

[0123] Optionally, the task parameters also include: push duration and start time;

[0124] The determination module is further used to: sort the waiting push tasks whose task occupied throughput is less than or equal to the remaining throughput of the cluster in order of task priority from high to low,

[0125] Determine whether there are waiting push tasks with the same task priority in the sorted task sequence; if not, use the task sequence as the sorting result; if so, sort the waiting push tasks with the same task priority in the task sequence again in order of push duration from short to long;

[0126] Determine whether there are waiting-pushing tasks with the same pushing time in the re-sorted task sequence; if so, sort the waiting-pushing tasks with the same pushing time in the re-sorted task sequence in order from front to back of the start time to obtain the sorting result; otherwise, use the second task sequence as the sorting result.

[0127] Optionally, the determination module is further used to: after determining the execution order of all the push number tasks, determine whether the execution order of all the push number tasks meets the first adjustment condition; if so, adjust the execution order of all the push number tasks according to the following steps:

[0128] Adjust the task priority of at least one waiting push task to be the same as the task priority of the previous push task, and then retest and run all the push tasks to determine the new execution order of all the push tasks;

[0129] Determine whether the total execution time of the push-number tasks corresponding to the new execution order is greater than the total execution time of the push-number tasks corresponding to the execution order; if so, roll back the adjusted task priority of the waiting push-number tasks, and do not adjust the execution order of all the push-number tasks; otherwise, use the new execution order as the execution order of all the push-number tasks;

[0130] Among them, the first adjustment condition includes: there is an idle period on the push-number time axis corresponding to the execution order, and there is a waiting push-number task in the idle period; the idle period refers to the period when the cluster remaining throughput of the target cluster is equal to zero; the previous push-number task refers to the push-number task that was last executed before the idle period.

[0131] Optionally, the determining module adjusts the task priority of at least one waiting push task to be the same as the task priority of the previous push task, including:

[0132] Determine the waiting task whose priority is to be adjusted according to the cluster maximum throughput of the target cluster and the task occupied throughput of each waiting task on the push timeline;

[0133] The task priority of the waiting task whose priority is to be adjusted is adjusted to be the same as the task priority of the preceding task, and the cluster remaining throughput of the target cluster after adjustment is updated.

[0134] Optionally, the determining module is further configured to: adjust the task priority of at least one waiting push task to be the same as the task priority of the previous push task,

[0135] It is determined whether the cluster remaining throughput of the target cluster after adjustment is greater than zero; if so, the task occupied throughput of the adjusted waiting push task is increased to make the cluster remaining throughput of the target cluster zero.

[0136] Optionally, the determination module is further used to: after determining the execution order of all the push number tasks, determine whether the execution order of all the push number tasks meets the second adjustment condition; if so, adjust the execution order of all the push number tasks according to the following steps:

[0137] Adjust the task priority of at least one waiting push task whose task occupied throughput is less than or equal to the remaining throughput of the cluster to be the same as the task priority of the push task being executed, and then retest and run all the push tasks to determine a new execution order of all the push tasks;

[0138] Determine whether the total execution time of the push-number tasks corresponding to the new execution order is greater than the total execution time of the push-number tasks corresponding to the execution order; if so, roll back the adjusted task priority of the waiting push-number tasks, and do not adjust the execution order of all the push-number tasks; otherwise, use the new execution order as the execution order of all the push-number tasks;

[0139] Among them, the second adjustment condition includes: there is an incomplete period on the push timeline corresponding to the execution order, and there is a waiting push task in the incomplete period whose task occupation throughput is less than or equal to the remaining throughput of the cluster; the incomplete period refers to the period when the remaining cluster throughput of the target cluster is greater than zero and less than the maximum throughput of the cluster.

[0140] Optionally, the determining module adjusts the task priority of at least one waiting push task whose task occupied throughput is less than or equal to the remaining throughput of the cluster to be the same as the task priority of the push task being executed, including:

[0141] Determine the waiting task whose priority is to be adjusted according to the remaining throughput of the cluster and the task occupied throughput of each waiting task on the push timeline;

[0142] The task priority of the waiting task whose priority is to be adjusted is adjusted to be the same as the task priority of the push task being executed, and the cluster remaining throughput of the target cluster after the adjustment is updated.

[0143] Optionally, the determining module is further configured to: adjust the task priority of at least one waiting push task whose task occupied throughput is less than or equal to the remaining throughput of the cluster to be the same as the task priority of the push task being executed,

[0144] It is determined whether the cluster remaining throughput of the target cluster after adjustment is greater than zero; if so, the task occupied throughput of the adjusted waiting push task is increased to make the cluster remaining throughput of the target cluster zero.

[0145] According to a third aspect of an embodiment of the present invention, there is provided an electronic device for scheduling push task, comprising:

[0146] one or more processors;

[0147] a storage device for storing one or more programs,

[0148] When the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the first aspect of the embodiment of the present invention.

[0149] According to a fourth aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method provided by the first aspect of the embodiment of the present invention is implemented.

[0150] Figure 6 An exemplary system architecture 600 is shown to which the method or apparatus for scheduling push-number tasks according to the embodiments of the present invention may be applied.

[0151] like Figure 6 As shown, system architecture 600 may include terminal devices 601, 602, 603, network 604 and server 605. Network 604 is used to provide a medium for communication links between terminal devices 601, 602, 603 and server 605. Network 604 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0152] Users can use terminal devices 601, 602, and 603 to interact with server 605 through network 604 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 601, 602, and 603, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0153] The terminal devices 601 , 602 , and 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers, etc.

[0154] The server 605 may be a server that provides various services, such as a backend management server (only an example) that provides support for shopping websites browsed by users using the terminal devices 601, 602, and 603. The backend management server may analyze and process the received data such as product information query requests, and feed back the processing results (such as target push information, product information - only an example) to the terminal device.

[0155] It should be noted that the method for scheduling push-number tasks provided in the embodiment of the present invention is generally executed by the server 605 , and accordingly, the device for scheduling push-number tasks is generally arranged in the server 605 .

[0156] It should be understood that Figure 6 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0157] Reference below Figure 7 , which shows a schematic diagram of the structure of a computer system 700 of a terminal device suitable for implementing an embodiment of the present invention. Figure 7 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0158] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0159] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage section 708 as needed.

[0160] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above-mentioned functions defined in the system of the present invention are executed.

[0161] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0162] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0163] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be set in a processor, for example, may be described as: a processor comprising: an acquisition module, which tests and runs all push tasks, and periodically obtains the remaining cluster throughput of the target cluster that receives the push data during the test; a determination module, which determines the execution order of all push tasks according to the task parameters of each push task and the remaining cluster throughput obtained periodically; and a push module, which pushes the push data to the target cluster according to the execution order. Among them, the names of these modules do not constitute a limitation on the module itself in some cases. For example, the acquisition module may also be described as "a module that pushes the push data to the target cluster according to the execution order".

[0164] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: testing and running all push tasks, and periodically obtaining the cluster remaining throughput of the target cluster receiving the push data during the test; determining the execution order of all push tasks according to the task parameters of each push task and the cluster remaining throughput obtained periodically; and pushing the push data to the target cluster according to the execution order.

[0165] According to the technical solution of the embodiment of the present invention, by testing and running all push tasks to automatically coordinate the execution order of all push tasks on a cluster, the purpose of efficiently utilizing the throughput of the entire cluster can be achieved. By detecting the execution status of each push task on the entire push timeline and dynamically adjusting task parameters such as the task priority and task occupancy throughput of the push task, the utilization rate of the throughput of the entire cluster can be further improved.

[0166] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for scheduling push task, characterized in that: include: The test runs all push tasks and periodically obtains the remaining cluster throughput of the target cluster that receives the pushed data during the test. Determine the execution order of all the push tasks according to the task parameters of each push task and the cluster remaining throughput obtained at regular intervals; Determine whether the execution order of all the push-number tasks meets the first adjustment condition; if so, adjust the execution order of all the push-number tasks according to the following steps: adjust the task priority of at least one waiting push-number task to be the same as the task priority of the previous push-number task, and then retest and run all the push-number tasks to determine the new execution order of all the push-number tasks; determine whether the total execution time of the push-number tasks corresponding to the new execution order is greater than the total execution time of the push-number tasks corresponding to the execution order; If so, roll back the adjusted task priority of the waiting push task, and do not adjust the execution order of all the push tasks; otherwise, use the new execution order as the execution order of all the push tasks; the first adjustment condition includes: there is an idle period on the push time axis corresponding to the execution order, and there is a waiting push task in the idle period; the idle period refers to the period when the cluster remaining throughput of the target cluster is zero; the previous push task refers to the push task that was last executed before the idle period; The pushed data is pushed to the target cluster according to the execution order.

2. The method according to claim 1, characterized in that The task parameters include: task occupancy throughput and task priority; According to the task parameters of each push task and the cluster remaining throughput obtained at regular intervals, the execution order of all push tasks is determined, including: After obtaining the remaining throughput of the cluster each time, the waiting push tasks whose task occupied throughput is less than or equal to the remaining throughput of the cluster are sorted in descending order of task priority to obtain a sorting result; The execution order of all the push tasks is determined according to the sorting result corresponding to the cluster remaining throughput obtained each time.

3. The method according to claim 2, characterized in that The task parameters also include: push duration and start time; After sorting the waiting push tasks whose task occupied throughput is less than or equal to the remaining throughput of the cluster in order of task priority from high to low, the method further includes: Determine whether there are waiting push tasks with the same task priority in the sorted task sequence; if not, use the task sequence as the sorting result; if so, sort the waiting push tasks with the same task priority in the task sequence again in order of push duration from short to long; Determine whether there are waiting-pushing tasks with the same pushing time in the re-sorted task sequence; if so, sort the waiting-pushing tasks with the same pushing time in the re-sorted task sequence in order from front to back of the start time to obtain the sorting result; otherwise, use the second task sequence as the sorting result.

4. The method according to claim 2, characterized in that Adjust the task priority of at least one waiting push task to be the same as the task priority of the preceding push task, including: Determine the waiting task whose priority is to be adjusted according to the cluster maximum throughput of the target cluster and the task occupied throughput of each waiting task on the push timeline; The task priority of the waiting task whose priority is to be adjusted is adjusted to be the same as the task priority of the preceding task, and the cluster remaining throughput of the target cluster after adjustment is updated.

5. The method according to claim 1 or 4, characterized in that After adjusting the task priority of at least one waiting push task to be the same as the task priority of the preceding push task, the method further includes: It is determined whether the cluster remaining throughput of the target cluster after adjustment is greater than zero; if so, the task occupied throughput of the adjusted waiting push task is increased to make the cluster remaining throughput of the target cluster zero.

6. The method according to claim 2, characterized in that After determining the execution order of all the push number tasks, the method further includes: judging whether the execution order of all the push number tasks satisfies a second adjustment condition; if so, adjusting the execution order of all the push number tasks according to the following steps: Adjust the task priority of at least one waiting push task whose task occupied throughput is less than or equal to the remaining throughput of the cluster to be the same as the task priority of the push task being executed, and then retest and run all the push tasks to determine a new execution order of all the push tasks; Determine whether the total execution time of the push-number tasks corresponding to the new execution order is greater than the total execution time of the push-number tasks corresponding to the execution order; if so, roll back the adjusted task priority of the waiting push-number tasks, and do not adjust the execution order of all the push-number tasks; otherwise, use the new execution order as the execution order of all the push-number tasks; Among them, the second adjustment condition includes: there is an incomplete period on the push timeline corresponding to the execution order, and there is a waiting push task in the incomplete period whose task occupation throughput is less than or equal to the remaining throughput of the cluster; the incomplete period refers to the period when the remaining cluster throughput of the target cluster is greater than zero and less than the maximum throughput of the cluster.

7. The method according to claim 6, characterized in that The task priority of at least one waiting push task whose task occupied throughput is less than or equal to the remaining throughput of the cluster is adjusted to be the same as the task priority of the push task being executed, including: Determine the waiting task whose priority is to be adjusted according to the remaining throughput of the cluster and the task occupied throughput of each waiting task on the push timeline; The task priority of the waiting task whose priority is to be adjusted is adjusted to be the same as the task priority of the push task being executed, and the cluster remaining throughput of the target cluster after the adjustment is updated.

8. The method according to claim 6 or 7, characterized in that After adjusting the task priority of at least one waiting push task whose task occupied throughput is less than or equal to the remaining throughput of the cluster to be the same as the task priority of the push task being executed, the method further includes: It is determined whether the cluster remaining throughput of the target cluster after adjustment is greater than zero; if so, the task occupied throughput of the adjusted waiting push task is increased to make the cluster remaining throughput of the target cluster zero.

9. A device for scheduling push task, characterized in that: include: The acquisition module tests and runs all push tasks, and periodically obtains the remaining cluster throughput of the target cluster that receives the pushed data during the test; A determination module determines the execution order of all the push tasks according to the task parameters of each push task and the remaining throughput of the cluster obtained at regular intervals; determines whether the execution order of all the push tasks satisfies the first adjustment condition; if so, adjusts the execution order of all the push tasks according to the following steps: adjusts the task priority of at least one waiting push task to be the same as the task priority of the previous push task, and then retests and runs all the push tasks to determine the new execution order of all the push tasks; determines whether the total execution time of the push tasks corresponding to the new execution order is greater than the total execution time of the push tasks corresponding to the execution order; If so, roll back the adjusted task priority of the waiting push task, and do not adjust the execution order of all the push tasks; otherwise, use the new execution order as the execution order of all the push tasks; the first adjustment condition includes: there is an idle period on the push time axis corresponding to the execution order, and there is a waiting push task in the idle period; the idle period refers to the period when the cluster remaining throughput of the target cluster is zero; the previous push task refers to the push task that was last executed before the idle period; A push module pushes the push data to the target cluster according to the execution order.

10. An electronic device for scheduling push tasks, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

11. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images