A method and device for task scheduling

Abstract

The present invention discloses a method and device for task scheduling, and relates to the field of computer technology. A specific implementation of the method includes: determining a task executor, sending a workload calculation instruction to each task executor; receiving the result value fed back by each task executor, determining the time required for each task executor to feed back the result value, so as to calculate the work efficiency of each task executor; determining the amount of tasks to be processed, and performing task allocation according to the work efficiency of each task executor. This implementation method accurately evaluates the processing capacity of each task executor by means of proof of workload, and realizes intelligent task allocation according to its processing capacity and the amount of tasks to be processed, thereby speeding up the processing speed of tasks.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for task scheduling. Background Art

[0002] Existing distributed task scheduling systems, such as TBSchedule and Clover, all implement the function of evenly distributing tasks and can detect the number of currently valid executors to ensure that tasks can be correctly distributed. For example, TBSchedule uses QUEUEID to distinguish executors and the tasks to be processed by each executor, which can ensure that each executor gets the same number of tasks fairly.

[0003] In the process of implementing the present invention, the inventors found that the prior art has at least the following problems:

[0004] 1) Currently, although the executors of task allocation are manually assigned to machines with uniform performance, due to network jitter, the number of tasks, etc., the performance of each machine may not be exactly the same;

[0005] 2) The existing technology cannot intelligently allocate tasks based on the current status and processing power of each executor, resulting in executors with higher hardware configurations or idle time only being able to obtain the same number of tasks as other executors, and being unable to fully utilize their processing power, resulting in a waste of resources. Summary of the invention

[0006] In view of this, an embodiment of the present invention provides a method and device for task scheduling, which can at least solve the problem that the processing performance of each executor is not considered in the prior art, resulting in "high-performance machines are idle and low-performance machines cannot complete processing".

[0007] To achieve the above-mentioned purpose, according to one aspect of an embodiment of the present invention, a method for task scheduling is provided, including: determining a task executor, sending a workload calculation instruction to each task executor; receiving a result value fed back by each task executor, determining the time required for each task executor to feed back the result value, so as to calculate the work efficiency of each task executor; wherein the result value corresponds to the workload calculation instruction; determining the amount of tasks to be processed, and allocating the tasks according to the work efficiency of each task executor.

[0008] Optionally, determining the time required for each task executor to feedback the result value to calculate the work efficiency of each task executor includes: determining the time required for each task executor to feedback the result value, counting the total time for all task executors to feedback the result value, and calculating the proof-of-work value of each task executor; wherein, the proof-of-work value = total time of all task executors / time of a single task executor; based on the proof-of-work value of each task executor, determining the total proof-of-work value of all task executors, and calculating the work efficiency of each task executor; wherein, the work efficiency = proof-of-work value of a single task executor / total proof-of-work value of all task executors.

[0009] Optionally, receiving the result value fed back by each task executor and determining the time required for each task executor to feed back the result value also includes: receiving the result value fed back by each task executor, verifying the fed back result value based on predetermined verification rules, and eliminating task executors whose result values ​​do not comply with the verification rules; and determining the time required for each task executor to feed back the result value, filtering the determined time based on a predetermined time threshold, and eliminating task executors whose time exceeds the predetermined time threshold.

[0010] Optionally, determining the amount of tasks to be processed and allocating the tasks according to the work efficiency of each task executor includes: receiving the newly allocated amount of tasks, obtaining the currently unprocessed amount of tasks of each task executor, and obtaining the amount of tasks to be processed; determining the amount of tasks to be allocated to each task executor from the newly allocated amount of tasks according to the amount of tasks to be processed, the work efficiency of each task executor and the currently unprocessed amount of tasks.

[0011] Optionally, sending the workload calculation instruction to each task executor includes: sending the workload calculation instruction to each task executor when a predetermined update cycle is reached.

[0012] To achieve the above-mentioned purpose, according to another aspect of an embodiment of the present invention, a task scheduling device is provided, including: an instruction sending module, used to determine the task executor, and send a workload calculation instruction to each task executor; an efficiency calculation module, used to receive the result value fed back by each task executor, determine the time required for each task executor to feed back the result value, so as to calculate the work efficiency of each task executor; wherein the result value corresponds to the workload calculation instruction; and a task allocation module, used to determine the amount of tasks to be processed, and allocate the tasks according to the work efficiency of each task executor.

[0013] Optionally, an efficiency calculation module is used to: determine the time required for each task executor to feedback a result value, count the total time for all task executors to feedback the result value, and calculate the proof-of-work value of each task executor; wherein, the proof-of-work value = the total time of all task executors / the time of a single task executor; based on the proof-of-work value of each task executor, determine the total proof-of-work value of all task executors, and calculate the work efficiency of each task executor; wherein, the work efficiency = the proof-of-work value of a single task executor / the total proof-of-work value of all task executors.

[0014] Optionally, an efficiency calculation module is used to: receive the result value fed back by each task executor, verify the fed back result value based on predetermined verification rules, and eliminate the task executors whose result values ​​do not meet the verification rules; and determine the time required for each task executor to feed back the result value, filter the received time based on a predetermined time threshold, and eliminate the task executors whose time exceeds the predetermined time threshold.

[0015] Optionally, a task allocation module is used to: receive a newly allocated task volume, obtain the current unprocessed task volume of each task executor, and obtain the task volume to be processed; and determine the task volume allocated to each task executor from the newly allocated task volume based on the task volume to be processed, the work efficiency of each task executor, and the current unprocessed task volume.

[0016] Optionally, the instruction sending module is used to: send workload calculation instructions to each task executor when a predetermined update cycle is reached.

[0017] To achieve the above objective, according to another aspect of an embodiment of the present invention, an electronic device for task scheduling is provided.

[0018] The electronic device of an embodiment of the present invention includes: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement any of the task scheduling methods described above.

[0019] To achieve the above objective, according to another aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, any of the above-mentioned task scheduling methods is implemented.

[0020] According to the solution provided by the present invention, an embodiment of the above invention has the following advantages or beneficial effects: through the proof of work method, the processing capacity of each task executor is accurately evaluated, and intelligent task allocation is achieved based on its processing capacity and the amount of tasks to be processed, so as to achieve "those who are capable do more work" and achieve the purpose of rapid task processing.

[0021] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0023] Figure 1 This is a schematic diagram of the main flow of a task scheduling method according to an embodiment of the present invention;

[0024] Figure 2 This is a distribution diagram of the Master node and task executor under Zookeeper;

[0025] Figure 3 A schematic flow chart of an optional task scheduling method according to an embodiment of the present invention;

[0026] Figure 4 is a flowchart of another optional task scheduling method according to an embodiment of the present invention;

[0027] Figure 5 is a flowchart of a specific task scheduling method according to an embodiment of the present invention;

[0028] Figure 6 is a schematic diagram of main modules of a task scheduling device according to an embodiment of the present invention;

[0029] Figure 7 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0030] Figure 8 It is a schematic diagram of the structure of a computer system of a mobile device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0032] It should be noted that the embodiments of the present invention can be applied to task allocation scenarios, such as TBSchedule and Clover.

[0033] Distributed systems are prone to errors, especially when dealing with contention and synchronization. Existing solutions usually coordinate distributed systems based on reliable coordination systems, such as Zookeeper. The functions provided by Zookeeper include at least: configuration maintenance, name service, distributed synchronization, group service, etc.

[0034] Since Zookeeper is more mature and popular, the present invention is mainly implemented based on Zookeeper, and does not make any improvements to Zookeeper, but only uses some functions of Zookeeper. However, it should be noted that distributed components with the same functions can also be used.

[0035] The following are explanations of the terms involved in the present invention:

[0036] Proof-of-Work (PoW) is an economic countermeasure to service and resource abuse or service blocking attacks. It generally requires users to perform some time-consuming and complex calculations, and the answers can be quickly verified by the service provider, so that the time, equipment and energy consumed are used as guarantee costs to ensure that services and resources are used for real needs.

[0037] Time MS: The time taken by the task executor to calculate the proof of work, counted in milliseconds.

[0038] Executor: An application role that specifically executes tasks, consisting of hardware machines and software systems. In the present invention, executors are divided into Master node executors that assign tasks and task executors that process tasks.

[0039] Master node machine: an application role that performs task query and allocation, which can be merged or split with the executor code and consists of hardware machines and software systems.

[0040] Node (Master node or Executor node): Zookeeper terminology, a unit that can be used to store data in a distributed system. A node can be named Master or Executor, and data can be stored or subnodes can be created under the node. The overall structure is a tree.

[0041] See also Figure 1 , which shows a main flow chart of a task scheduling method provided by an embodiment of the present invention, comprising the following steps:

[0042] S101: Determine a task executor and send a workload calculation instruction to each task executor.

[0043] S102: receiving the result value fed back by each task executor, determining the time required for each task executor to feed back the result value, so as to calculate the work efficiency of each task executor; wherein the result value corresponds to the workload calculation instruction.

[0044] S103: Determine the amount of tasks to be processed, and distribute the tasks according to the work efficiency of each task executor.

[0045] The present invention mainly consists of two roles: the master node and the executor. These two roles are determined by the Master election in Zookeeper to ensure that there is only one master node in the system at the same time. In order to make the system run stably, the present invention is explained as always having a valid Master node.

[0046] To avoid subsequent name confusion, the Master node executor is referred to as the Master node; the executor for task processing under the Executor node is referred to as the task executor.

[0047] For step S101, the election of the master node can be at system startup or when each sub-node detects an abnormality in the master node. The registration request can be the identification information of each executor, such as the IP address of the executor. Specifically:

[0048] 1) When the system is started for the first time, the master node will be elected. At this time, Zookeeper will receive the master node registration request of each executor, determine the executor that successfully registers first as the master node Master, and the remaining executors that fail to elect the master node will register as child nodes under the Executor node so that the Master can assign tasks accordingly;

[0049] 2) After the system is started and the master node is elected, Zookeeper will regularly check whether the Master node machine is alive based on a predetermined detection cycle. If the survival status is abnormal (for example, downtime, network failure), a notification will be sent to all executors (including the abnormal Master node machine) to re-elect the master node Master.

[0050] In the case where the master node is abnormal and needs to be re-registered, the re-determined master node continues to take over the work of the previous master node, and its unfinished tasks can be allocated to other task executors for execution.

[0051] In addition, the Master node does not need to process tasks, so the registration request for the Master node does not need to include the workload value, but at least includes the IP address of the executor, which is convenient for the subsequent Master node to assign tasks to the executor subnodes.

[0052] For the registration of the task executor machine, the registration content is the executor's IP address and its initial workload PoW. And because the Executor's child nodes can register multiple, the possibility of registration failure is low when the communication is normal, but if it fails, the executor will not be assigned a task. In addition, if the executor detects that the registration is not successful, it will resend the registration request regularly to reduce resource waste.

[0053] See Figure 2 As shown in the figure, for the zookeeper coordination system, TASK_TYPE_1 is a task type node, which has a Master node and an Executor node. A new node can be added under the Master node, and the content is the IP address of the Master machine; multiple nodes can be registered under the Executor node, and the content is the IP address and PoW value of the Executor machine.

[0054] The Master node stores IP address information to distinguish which executor with which IP address is elected as the Master. The task executors of the remaining child nodes will periodically query the Master node content to distinguish their own responsibilities and re-elect. The re-election can be notified by the Master after detecting an abnormality in the Master node, or by the executor child node querying the Master node to disappear and then preempting it.

[0055] After the Master node is determined, the executor registered under the Master node obtains the task executor list by reading the identification information of the task executor registered under the Executor node. The Master node communicates with each task executor in the executor list based on the IP address to notify the proof of work calculation or task processing.

[0056] Furthermore, for the generated task executor list, since the number of task executors may increase or decrease at any time, in order to ensure the effectiveness of subsequent task processing, it is necessary to detect the number of currently effective task executors to ensure that tasks can be correctly allocated.

[0057] The workload calculation instructions sent by the Master node to each task executor can be sent in the form of content, such as "the first N bits of the Hash value after adding the variable through random number calculation are 0", which is a workload proof algorithm, where N can be set by the number of executor sub-nodes, and any data Hash value is different and completely random, for example:

[0058] 1) After “0” is calculated by MD5 (Message Digest Algorithm MD5), the hash value is “CFCD208495D565EF66E7DFF9F98764DA”;

[0059] 2) If you want to find a hash value with the first digit being 0, then add the numbers from 0 to 27 to get the hash value of 27: "02E74F10E0327AD868D138F2B4FDD6F0";

[0060] 3) If you want to find a hash value whose first two digits are 0, add it to 168 to get its hash value "006F52E9102A8D3BE2FE5614F42BA989".

[0061] For step S102, each task executor starts calculation after receiving the workload calculation instruction, and returns the calculation result value to the Master node. The result value here is the result value that meets the requirements of the workload proof algorithm, for example, the hash value is "3636638817772E42B59D74CFF571FBB3".

[0062] However, it should be noted that the task executor returns different result values ​​for different proof-of-work algorithms issued by the Master node, but they all meet the requirements of the proof-of-work algorithm. For example, in the above example, if the first two digits of the hash value starting from 0 are required, the task executor returns a value such as 168.

[0063] Furthermore, to improve the efficiency of task processing, before calculating the working efficiency of the task executor, the Master node can dynamically detect the result value returned by each task executor and the time consumed to screen the task executors. Specifically:

[0064] 1) Verify whether the result value returned by each task executor is correct: for example, if the first two digits of the Hash value are required to be 0, the verification is passed if the return value is 168; but if the return value is 169, the Hash value of 169 calculated by the master node is "3636638817772E42B59D74CFF571FBB3", which does not meet the requirements, the verification fails;

[0065] 2) Verify the time taken by each task executor to return the result value: For example, if the result value is not calculated for a certain period of time due to poor performance, network failure, downtime during calculation, etc., these task executors that do not return the result value can be eliminated.

[0066] The Master node calculates the work efficiency rate of each task executor based on the time returned by each task executor, which can also be called the allocation ratio. For example, 10 executors take 752ms in total - the first executor takes 10ms, the second takes 23ms, ..., and the last takes 10ms. By determining the ratio of the reciprocal value of the time to the sum of the reciprocal values ​​of each time, for task executor 1, its work efficiency rate is:

[0067] rate1=(1 / 10) / [(1 / 10)+(1 / 23)+……+(1 / 10)].

[0068] For step S103, when the Master node queries or receives tasks to be processed, it will intelligently allocate them according to the work efficiency of each task executor and the amount of tasks that are not being processed:

[0069] 1) In the initial startup phase of the system, the proof of work of each task executor is 0. At this time, the corresponding task volume can be directly determined based on its work efficiency; that is, work efficiency * total task volume;

[0070] 2) During the system startup phase, when new tasks are received, some task executors may have unfinished tasks. At this time, it is necessary to allocate tasks based on the number of tasks being processed. For example, after each task executor has executed the task for 10 seconds, the second task allocation begins, which is still based on work efficiency, that is, rate = (number of tasks being processed + number of newly allocated tasks) / total number of tasks, so the number of newly allocated tasks = total number of tasks * rate - number of tasks being processed.

[0071] Furthermore, some calculation result values ​​may not be integers, and the task allocation amount can be determined by rounding up or rounding down. There are many ways to allocate the remaining task amount (total task amount - allocated task amount):

[0072] 1) If the rounding-up method is adopted, the remaining task volume can be allocated to the task executor with the lowest work efficiency to improve the overall task processing efficiency;

[0073] 2) If the rounding down method is adopted, the remaining task volume can be allocated to the task executor with the highest work efficiency, which can also improve the overall task processing efficiency;

[0074] 3) There is no need to sort based on the performance of each machine. Instead, the remaining tasks are directly allocated in order according to the task executor list, and the remaining tasks are allocated to the last task executor in the table.

[0075] The method provided in the above embodiment provides a way of intelligently allocating tasks based on the task processing performance of the machine, so as to give full play to the processing capabilities of high-performance task executors, realize "more work for those who are capable", and thus improve the overall task processing speed.

[0076] See also Figure 3 , shows a schematic flow chart of an optional task scheduling method according to an embodiment of the present invention, comprising the following steps:

[0077] S301: Determine a task executor, and send a workload calculation instruction to each task executor.

[0078] S302: Receive the result value fed back by each task executor, determine the time required for each task executor to feed back the result value, count the total time for all task executors to feed back the result values, and calculate the proof-of-work value of each task executor; wherein, the proof-of-work value = total time of all task executors / time of a single task executor.

[0079] S303: Determine the total proof-of-work value of all task executors based on the proof-of-work value of each task executor, and calculate the work efficiency of each task executor; wherein, work efficiency = proof-of-work value of a single task executor / total proof-of-work value of all task executors.

[0080] S304: Determine the amount of tasks to be processed, and distribute the tasks according to the work efficiency of each task executor.

[0081] In the above implementation, steps S301 and S304 can be respectively referred to Figure 1 The description of steps S101 and S103 is not repeated here.

[0082] In the above implementation, for step S302, the Master node calculates the PoW value of each task executor according to the time returned by each task executor. For example, 10 executors take 752ms in total. The PoW value of each task executor is obtained by dividing 752ms by the time taken by each executor, that is:

[0083]

[0084] Among them, MS N represents the time taken by task executor N, M is the number of valid task executors, PoW N is the calculated proof-of-work value of task executor N.

[0085] Furthermore, for the time returned by each task executor, its PoW value can be maintained (updated and saved). For example, after the first calculation of task executor 1, its PoW value is 75.20, and the master node needs to save this value and use it as the basis for allocating task amounts; after obtaining the proof of work for the second time, its PoW value is 5.94, so the master node machine needs to modify 75.20 to 5.94.

[0086] Furthermore, the proof of work value can be updated according to a predetermined period, such as 10 minutes, 5 minutes, or other self-configured time. For example, the PoW value of task executor 1 is 75.20 in the first calculation, but 10 minutes later, due to network jitter or other work occupying the CPU, its availability rate decreases, so the PoW value obtained in the second calculation is 5.94, and the number of tasks assigned next time will be reduced.

[0087] For step S303, based on the obtained proof-of-work value, the corresponding work efficiency can be determined, namely:

[0088]

[0089] Among them, rate N is the working efficiency of task executor N, and ∑PoW is the accumulation of PoW values ​​of all task executors.

[0090] Similarly, during the use of the machine, there may be machine wear and tear, abnormal use, etc., which may reduce its work efficiency. Therefore, the work efficiency calculated each time is also different. The calculated work efficiency can also be saved and updated in a maintenance manner to facilitate subsequent inquiries and improve the accuracy of task allocation.

[0091] For each task executor, the obtained PoW value and the corresponding work efficiency can be generated into a table and stored in the node, as shown in Table 1 below:

[0092] Table 1 PoW and rate value examples

[0093]

[0094] It should be noted that the number of task executors may change at any time, and the workload calculation instructions will not be sent every time the number changes (to avoid excessive pressure on the server). For example, the number of effective task executors was 10 s ago, and the number became 20 10 s later, but the workload calculation instructions are once every 10 minutes.

[0095] When the number of task executors changes and the proof-of-work update cycle has not been reached, the corresponding work efficiency can be calculated directly based on the proof-of-work value of each task executor without relying on time, which is convenient for subsequent management and reduces server service pressure.

[0096] The method provided in the above embodiment specifically calculates the work efficiency of each task executor based on the proof of work method. If the number of task executors changes and the proof of work update cycle has not been reached, the corresponding work efficiency can be directly determined based on the proof of work value of each task executor.

[0097] See also Figure 4 , shows a schematic flow chart of an optional task scheduling method according to an embodiment of the present invention, comprising the following steps:

[0098] S401: Determine a task executor, and send a workload calculation instruction to each task executor.

[0099] S402: receiving the result value fed back by each task executor, determining the time required for each task executor to feed back the result value, so as to calculate the work efficiency of each task executor; wherein the result value corresponds to the workload calculation instruction.

[0100] S403: Receive the newly allocated task amount, obtain the currently unprocessed task amount of each task executor, and obtain the task amount to be processed.

[0101] S404: Determine the amount of tasks allocated to each task executor in the newly allocated amount of tasks according to the amount of tasks to be processed, the work efficiency of each task executor, and the amount of tasks that are not currently processed.

[0102] In the above implementation, step S401 can refer to Figure 1 For the description of step S101, see step S402. Figure 1 For the description of step S102, see Figure 3 The description of steps S302 and S303 is not repeated here.

[0103] In the above implementation, for steps S403 and S404, at the time of initial system startup, the amount of tasks being processed by each task executor is 0, so the amount of tasks can be allocated according to the work efficiency rate.

[0104] When the system is running and the task executor has started processing tasks, if a newly assigned task is received, each task executor needs to be assigned a different amount of new tasks due to the different amount of tasks being processed. The allocation strategy is still based on work efficiency, that is, the ratio of the task volume after trial allocation is equal to the rate value, so the newly assigned task volume = total task volume * rate - unprocessed task volume, and the total task volume = total unprocessed task volume + newly assigned task volume.

[0105] For example, referring to Table 2, taking task executor 1 as an example,

[0106] 1) For the first allocation, the initial workload is 0, so the amount of tasks allocated is 0.25*1000=250;

[0107] 2) After 10 seconds, there are still 34 tasks that have not been processed, but a new batch of tasks has arrived, and the total number is still 1000. At this time, based on the work efficiency of task executor 1, it is determined that the sum of the assigned tasks and the unfinished tasks still accounts for the total task volume of all task executors, that is, (34+x) / (1000+236)=0.25, and the calculated value is 309, which means that 309-34=275 tasks need to be assigned this time.

[0108] Table 2 Task allocation

[0109]

[0110] The method provided in the above embodiment provides a specific way to implement task allocation. Whether the task executor currently has unprocessed tasks, task allocation can be performed based on work efficiency to achieve intelligent processing of task allocation.

[0111] See also Figure 5 , shows a specific task scheduling method flow chart according to an embodiment of the present invention, including the following steps:

[0112] S501: When the system is started for the first time or an abnormality is detected in the current master node executor, a master node registration request transmitted by each executor is received, the executor that successfully registers first is selected as the master node executor, and the remaining executors are determined as task executors; wherein the registration request includes at least the identification information of the executor.

[0113] S502: When a predetermined update period is reached, a workload calculation instruction is sent to each task executor.

[0114] S503: Receive the result value fed back by each task executor, verify the fed back result value based on a predetermined verification rule, and remove the task executors whose result values ​​do not meet the verification rule.

[0115] S503': Determine the time required for each task executor to feed back a result value, filter the determined time based on a predetermined time threshold, and remove the task executors whose time exceeds the predetermined time threshold.

[0116] S504: Determine the time required for each task executor to feedback the result value, count the total time for all task executors to feedback the result value, and calculate the proof-of-work value of each task executor; wherein, the proof-of-work value = the total time of all task executors / the time of a single task executor.

[0117] S505: Determine the total proof-of-work value of all task executors based on the proof-of-work value of each task executor, and calculate the work efficiency of each task executor; wherein, work efficiency = proof-of-work value of a single task executor / total proof-of-work value of all task executors.

[0118] S506: Receive the newly allocated task amount, obtain the currently unprocessed task amount of each task executor, and obtain the task amount to be processed.

[0119] S507: Determine the amount of tasks allocated to each task executor in the newly allocated amount of tasks according to the amount of tasks to be processed, the work efficiency of each task executor, and the amount of tasks that are not currently processed.

[0120] In the above embodiment, steps S501 to S503' can be seen in Figure 1 The description of step S101 is shown in FIG. 1 ; steps S504 and S505 can be respectively referred to in Figure 3 The description of steps S302 and S303 shown in FIG. 1 and steps S506 and S507 can be respectively referred to in Figure 4 The description of steps S403 and S404 is not repeated here.

[0121] It should be noted that, when configuring the executor, the present invention does not require manual calculation of the processing capacity of each machine.

[0122] For machines that process the same type of tasks, usually machines with basically the same hardware configuration are configured as much as possible. However, in actual work, due to the increase in business needs, new machines are often added multiple times. For example, three machines may be enough one year ago, but one year later, 10 machines may be needed. Due to the lack of machine resources, the hardware configuration of each newly added machine may not be completely consistent with the previous one.

[0123] Currently, in order to ensure the consistency of machine performance and distribute a balanced workload in actual work, manual calculation configuration is often required, and the hardware is split and reorganized. For example, an 8-core CPU is virtualized into four 2-core CPUs to ensure consistent performance. The method shown in the present invention distributes tasks based on the work efficiency of the machine, and there is no need to manually calculate the processing capacity of each machine when configuring the machine, thereby achieving the effect of intelligent distribution.

[0124] The method provided in the embodiment of the present invention accurately evaluates the processing capacity of each task executor by means of proof of work, and realizes intelligent task allocation according to its processing capacity and the amount of tasks to be processed, so as to achieve rapid processing of tasks. In addition, by regularly updating the proof of work value and work efficiency, it avoids the situation where the task executor is abnormal and the task processing efficiency is low due to a large amount of assigned tasks.

[0125] See also Figure 6 , which shows a schematic diagram of main modules of a task scheduling device 600 provided in an embodiment of the present invention, including:

[0126] The instruction sending module 601 is used to determine the task executor and send the workload calculation instruction to each task executor;

[0127] The efficiency calculation module 602 is used to receive the result value fed back by each task executor, determine the time required for each task executor to feed back the result value, so as to calculate the work efficiency of each task executor; wherein the result value corresponds to the workload calculation instruction;

[0128] The task allocation module 603 is used to determine the amount of tasks to be processed and allocate the tasks according to the work efficiency of each task executor.

[0129] In the implementation device of the present invention, the efficiency calculation module 602 is used to:

[0130] Determine the time required for each task executor to feedback the result value, count the total time required for all task executors to feedback the result value, and calculate the proof of work value of each task executor; where proof of work value = total time of all task executors / time of a single task executor;

[0131] According to the proof-of-work value of each task executor, the total proof-of-work value of all task executors is determined, and the work efficiency of each task executor is calculated; where work efficiency = proof-of-work value of a single task executor / total proof-of-work value of all task executors.

[0132] In the implementation device of the present invention, the efficiency calculation module 602 is used to:

[0133] receiving the result value fed back by each task executor, verifying the fed back result value based on a predetermined verification rule, and eliminating the task executors whose result values ​​do not meet the verification rule; and

[0134] The time required for each task executor to feedback the result value is determined, the received time is filtered based on a predetermined time threshold, and the task executors whose time exceeds the predetermined time threshold are eliminated.

[0135] In the implementation device of the present invention, the task allocation module 603 is used to:

[0136] Receive the newly assigned task amount, obtain the current unprocessed task amount of each task executor, and obtain the task amount to be processed;

[0137] The amount of tasks allocated to each task executor in the newly allocated amount of tasks is determined according to the amount of tasks to be processed, the work efficiency of each task executor, and the amount of tasks that are not currently processed.

[0138] In the implementation device of the present invention, the instruction sending module 601 is used to send a workload calculation instruction to each task executor when a predetermined update cycle is reached.

[0139] In addition, the specific implementation content of the task scheduling device described in the embodiment of the present invention has been described in detail in the task scheduling method described above, so the repeated content will not be described again here.

[0140] The device provided by the embodiment of the present invention accurately evaluates the processing capacity of each task executor by means of proof of work, and realizes intelligent task allocation according to its processing capacity and the amount of tasks to be processed, so as to achieve rapid processing of tasks. In addition, by regularly updating the proof of work value and work efficiency, it avoids the situation where the task executor is abnormal and the task processing efficiency is low due to a large amount of assigned tasks.

[0141] See also Figure 7 An exemplary system architecture 700 is shown to which the task scheduling method or task scheduling device according to the embodiment of the present invention can be applied.

[0142] like Figure 7 As shown, system architecture 700 may include terminal devices 701, 702, 703, network 704 and server 705. Network 704 is used to provide a medium for communication links between terminal devices 701, 702, 703 and server 705. Network 704 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0143] Users can use terminal devices 701, 702, and 703 to interact with server 705 through network 704 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 701, 702, and 703, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0144] The terminal devices 701 , 702 , and 703 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0145] The server 705 may be a server that provides various services, such as a backend management server (only for example) that supports shopping websites browsed by users using the terminal devices 701, 702, and 703. The backend management server may analyze and process the received data such as product information query requests, and feed back the processing results (such as target push information, product information - only for example) to the terminal device.

[0146] It should be noted that the task scheduling method provided in the embodiment of the present invention is generally executed by the server 705 , and accordingly, the task scheduling device is generally disposed in the server 705 .

[0147] It should be understood that Figure 7 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0148] See also Figure 8 , which shows a schematic diagram of the structure of a computer system 800 of a terminal device suitable for implementing an embodiment of the present invention. Figure 8 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0149] like Figure 8 As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the system 800 are also stored. The CPU 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0150] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage section 808 as needed.

[0151] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the central processing unit (CPU) 801, the above-mentioned functions defined in the system of the present invention are executed.

[0152] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0153] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0154] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor. For example, they may be described as: a processor includes an instruction sending module, an efficiency calculation module, and a task allocation module. The names of these modules do not, in some cases, constitute limitations on the modules themselves. For example, the efficiency calculation module may also be described as a "work efficiency calculation module of a task executor."

[0155] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes:

[0156] Determine the task executor and send workload calculation instructions to each task executor;

[0157] receiving the result value fed back by each task executor, determining the time required for each task executor to feed back the result value, so as to calculate the working efficiency of each task executor; wherein the result value corresponds to the workload calculation instruction;

[0158] Determine the amount of tasks to be processed and distribute the tasks according to the work efficiency of each task executor.

[0159] According to the technical solution of the embodiment of the present invention, the processing capacity of each task executor is accurately evaluated by the proof of work method, and intelligent task allocation is realized according to its processing capacity and the amount of tasks to be processed, so as to achieve the effect of rapid task processing. In addition, by regularly updating the proof of work value, the situation where the task executor is abnormal and the task processing efficiency is low due to a large amount of assigned tasks is avoided.

[0160] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A task scheduling method, characterized in that: include: Determine the task executor and send workload calculation instructions to each task executor; receiving the result value fed back by each task executor, determining the time required for each task executor to feed back the result value, so as to calculate the working efficiency of each task executor; wherein the result value corresponds to the workload calculation instruction; Determine the amount of tasks to be processed and distribute the tasks according to the work efficiency of each task executor, including: Receive the newly allocated task volume, obtain the current unprocessed task volume of each task executor, and obtain the pending task volume; determine the task volume allocated to each task executor in the newly allocated task volume based on the pending task volume, the work efficiency of each task executor and the current unprocessed task volume.

2. The method according to claim 1, characterized in that Determining the time required for each task executor to feedback the result value to calculate the work efficiency of each task executor includes: Determine the time required for each task executor to feedback the result value, count the total time required for all task executors to feedback the result value, and calculate the proof of work value of each task executor; where proof of work value = total time of all task executors / time of a single task executor; According to the proof-of-work value of each task executor, the total proof-of-work value of all task executors is determined, and the work efficiency of each task executor is calculated; where work efficiency = proof-of-work value of a single task executor / total proof-of-work value of all task executors.

3. The method according to claim 1, characterized in that The receiving the result value fed back by each task executor and determining the time required for each task executor to feed back the result value further comprises: receiving the result value fed back by each task executor, verifying the fed back result value based on a predetermined verification rule, and eliminating the task executors whose result values ​​do not meet the verification rule; and The time required for each task executor to feed back a result value is determined, the determined time is screened based on a predetermined time threshold, and task executors whose time exceeds the predetermined time threshold are eliminated.

4. The method according to claim 1, characterized in that The sending of the workload calculation instruction to each task executor comprises: When a predetermined update period is reached, the workload calculation instruction is sent to each task executor.

5. A task scheduling device, characterized in that: include: An instruction sending module, used for determining a task executor and sending a workload calculation instruction to each task executor; an efficiency calculation module, used for receiving the result value fed back by each task executor, determining the time required for each task executor to feed back the result value, so as to calculate the working efficiency of each task executor; wherein the result value corresponds to the workload calculation instruction; The task allocation module is used to determine the amount of tasks to be processed and allocate the tasks according to the work efficiency of each task executor, including: Receive the newly allocated task volume, obtain the current unprocessed task volume of each task executor, and obtain the pending task volume; determine the task volume allocated to each task executor in the newly allocated task volume based on the pending task volume, the work efficiency of each task executor and the current unprocessed task volume.

6. The device according to claim 5, characterized in that The efficiency calculation module is used to: Determine the time required for each task executor to feedback the result value, count the total time required for all task executors to feedback the result value, and calculate the proof of work value of each task executor; where proof of work value = total time of all task executors / time of a single task executor; According to the proof-of-work value of each task executor, the total proof-of-work value of all task executors is determined, and the work efficiency of each task executor is calculated; where work efficiency = proof-of-work value of a single task executor / total proof-of-work value of all task executors.

7. The device according to claim 5, characterized in that The efficiency calculation module is used to: Receive the result value fed back by each task executor, verify the fed back result value based on a predetermined verification rule, and remove the task executors whose result values ​​do not meet the verification rule; as well as The time required for each task executor to feed back a result value is determined, the received time is screened based on a predetermined time threshold, and task executors whose time exceeds the predetermined time threshold are eliminated.

8. The device according to claim 5, characterized in that The instruction sending module is used to: When a predetermined update period is reached, the workload calculation instruction is sent to each task executor.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images