Task allocation methods, devices, storage media, and computer equipment

Abstract

The task allocation method, apparatus, storage medium, and computer equipment provided in this application, when there are tasks to be allocated, can first determine the priority of the tasks to be allocated, and statistically analyze the current working status of each working node in each computing center and the current queuing status of the task scheduling queues in each computing center. Then, based on the current working status of each working node in each computing center and the current queuing status of the task scheduling queues in each computing center, the estimated completion time of allocating the tasks to be allocated to each computing center and the task balance of each computing center can be determined. In this way, when allocating the tasks to be allocated, the target computing center for processing the tasks to be allocated can be determined based on the estimated completion time, task balance, and priority of each computing center, and the tasks to be allocated can be allocated to the target computing center. This can improve data processing efficiency and optimize resource utilization.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a task allocation method, apparatus, storage medium, and computer device. Background Technology

[0002] Currently, when allocating processing tasks across multiple computing centers, the task distribution center dynamically assigns tasks to the scheduling centers configured in each computing center based on different task types and the available resources of each computing center. The scheduling centers then distribute the processing tasks to several worker nodes within each computing center for processing. For example, when GPU resources are needed for de-identification computing tasks, the task distribution center can allocate these tasks to various computing centers.

[0003] In the current method of assigning data masking tasks, the task distribution center mainly creates data masking tasks in the corresponding computing center based on the storage address of the data to be masked. Since there are many types of data to be masked, and the processing time for each type of data varies greatly, and the processing capabilities of each computing center are also different due to resource differences, this can easily lead to a serious backlog of tasks and an imbalance in processing capabilities in a certain computing center, which in turn makes it impossible to complete the timely masking of business data and meet business objectives. Summary of the Invention

[0004] The purpose of this application is to at least solve one of the aforementioned technical defects, in particular the technical defect that the task allocation method in the prior art easily leads to a serious backlog of tasks and an imbalance in processing capacity in a certain computing center, which in turn makes it impossible to complete the timely desensitization of business data and meet business objectives.

[0005] This application provides a task allocation method, the method comprising:

[0006] Obtain the tasks to be assigned and their priorities;

[0007] Statistics on the current working status of each working node in each computing center and the current queuing status of the task scheduling queue in each computing center;

[0008] Based on the current working status of each working node in each computing center and the current queuing status of the task scheduling queue in each computing center, the estimated completion time for allocating the tasks to be assigned to each computing center and the task balance of each computing center are determined.

[0009] Based on the estimated completion time of each computing center, the task balance, and the priority of the tasks to be assigned, the target computing center for processing the tasks to be assigned is determined, and the tasks to be assigned are assigned to the target computing center.

[0010] Optionally, the step of statistically analyzing the current working status of each working node in each computing center includes:

[0011] For each computing center:

[0012] The statistics include the number of worker nodes in the computing center, the types of tasks being processed by the worker nodes currently processing tasks, the amount of tasks processed, the processing time, and the real-time weight of the computing center when processing different types of tasks. The real-time weight of the computing center when processing different types of tasks includes the real-time weight of the computing center when processing different types of tasks within the same computing center and the real-time weight of the computing center when processing different types of tasks in other computing centers.

[0013] Based on the real-time weights of the computing center when processing different types of tasks, the task type, task volume, and processing time of the work node currently processing the task, calculate the remaining processing time of the work node currently processing the task.

[0014] The number of worker nodes in the computing center and the remaining processing time of the worker node currently processing the task are taken as the current working status of each worker node in the computing center.

[0015] Optionally, the step of calculating the real-time weights of the computing center when processing different types of tasks includes:

[0016] Obtain the task type, actual workload, and actual processing time of the historical tasks processed by the computing center within a preset historical period;

[0017] For different types of historical tasks, the actual processing speed of the historical task is calculated based on the actual workload and actual processing time of the historical task.

[0018] The actual weights of different types of historical tasks are determined based on their actual processing speeds, and then compared with the test weights of different types of test tasks pre-configured through benchmark testing to obtain the comparison results.

[0019] The real-time weights of the computing center when processing different types of tasks are determined based on the comparison results.

[0020] Optionally, the process of configuring the test weights for the different types of test tasks includes:

[0021] Pre-configure different types of test tasks and determine the corresponding test task volume for each type of test task;

[0022] Benchmark tests were performed on different types of test tasks to obtain the test duration for each type of test task.

[0023] Calculate the test speed corresponding to each type of test task based on the test duration and test volume of each type of test task.

[0024] Configure the test weights for different types of test tasks according to the test speeds corresponding to different types of test tasks.

[0025] Optionally, calculating the remaining processing time of the task being processed by the worker node based on the real-time weight of the computing center when processing different types of tasks, the task type, task volume, and processing time of the worker node currently processing the task, includes:

[0026] The real-time weight of the task being processed by the work node is determined based on the real-time weight of the work node processing different types of tasks in the computing center and the task type of the task being processed by the work node currently processing the task.

[0027] The total processing time of the work node processing the task is calculated based on the real-time weight and task volume of the work node currently processing the task.

[0028] The remaining processing time of the task is obtained by subtracting the already processed time from the total processing time of the task being processed by the worker node.

[0029] Optionally, the step of statistically analyzing the current queuing status of the task scheduling queues in each computing center includes:

[0030] For each computing center:

[0031] The system collects statistics on the queue position, task type, priority, and number of tasks to be processed in the task scheduling queue of the computing center, as well as the real-time weight of the computing center when processing different types of tasks.

[0032] Based on the real-time weights of the computing center when processing different types of tasks, the task type and task quantity of each task to be processed, the total processing time of each task to be processed is calculated.

[0033] The queue position, priority, and total processing time of each task to be processed in the task scheduling queue are used as the current queuing status of the task scheduling queue.

[0034] Optionally, determining the estimated completion time for assigning the tasks to be assigned to each computing center based on the current working status of each working node in each computing center and the current queuing status of the task scheduling queues in each computing center includes:

[0035] For each computing center:

[0036] Based on the current working status of each working node in the computing center and the current queuing status of the task scheduling queue in the computing center, calculate the shortest waiting time to assign the task to be assigned to the computing center.

[0037] Obtain the real-time weights of the computing center when processing different types of tasks, as well as the task type and workload of the task to be assigned;

[0038] Based on the task type and workload of the task to be assigned, and the real-time weight of the computing center when processing different types of tasks, calculate the estimated processing time for assigning the task to be assigned to the computing center.

[0039] Based on the minimum waiting time and the estimated processing time, calculate the estimated completion time for assigning the task to the computing center.

[0040] Optionally, the current working status of each worker node in the computing center includes the number of worker nodes in the computing center and the remaining processing time of the worker node that is processing a task; the current queuing status of the task scheduling queue in the computing center includes the queue position, priority, and total processing time of each task to be processed in the task scheduling queue.

[0041] The step of calculating the shortest waiting time to assign the task to the computing center based on the current working status of each working node and the current queuing status of the task scheduling queue includes:

[0042] Determine the set of tasks in the task scheduling queue of the computing center that have a priority no lower than the priority of the task to be assigned, as well as the number and queue position of the tasks to be processed in the task set;

[0043] Based on the number of tasks to be processed in the task set and the number of worker nodes in the computing center, calculate the number of waiting rounds for the tasks to be assigned to the computing center.

[0044] Based on the total processing time and queue position of each task to be processed in the task set, calculate the estimated waiting time of the task to be assigned under the number of waiting rounds;

[0045] Based on the estimated waiting time of the task to be assigned under the number of waiting rounds and the remaining processing time of the work node currently processing the task in the computing center, calculate the shortest waiting time to assign the task to the computing center.

[0046] Optionally, the number of waiting rounds includes odd-numbered rounds and even-numbered rounds;

[0047] The step of calculating the estimated waiting time of the task to be assigned under the number of waiting rounds based on the total processing time and queue position of each task to be processed in the task set includes:

[0048] Based on the total processing time and queue position of each pending task in the task set, calculate the first processing time of the first pending task with the longest total processing time among the pending tasks processed by the computing node in the odd-numbered rounds.

[0049] And, based on the total processing time and queue position of each pending task in the task set, calculate the second processing time of the second pending task with the shortest total processing time among the pending tasks processed by the computing node in the even-numbered rounds.

[0050] The estimated waiting time for the task to be assigned is determined based on the first processing time and the second processing time, under the number of waiting rounds.

[0051] Optionally, the current working status of each worker node in each computing center includes the number of worker nodes in each computing center and the remaining processing time of the worker node that is processing the task; the current queuing status of the task scheduling queue in each computing center includes the total processing time of each pending task in the task scheduling queue.

[0052] The process of determining the task balance of each computing center based on the task completion status of each working node in each computing center and the queuing status of the task scheduling queues in each computing center includes:

[0053] For each computing center:

[0054] The total processing time of the computing center is calculated based on the remaining processing time of the work nodes that are currently processing tasks and the total processing time of each pending task in the task scheduling queue of the computing center.

[0055] Calculate the task balance of the computing center based on its total processing time and the number of worker nodes.

[0056] Optionally, the priority of the task to be assigned includes high priority, medium priority and low priority, and the task delay time of the task to be assigned is different for different priorities;

[0057] The process of determining the target computing center for processing the tasks to be assigned, based on the estimated completion time of each computing center, the task balance, and the priority of the tasks to be assigned, includes:

[0058] If the priority of the task to be assigned is high priority, then the computing center with the earliest expected completion time among all computing centers is selected as the target computing center for processing the task to be assigned.

[0059] If the priority of the task to be assigned is medium priority, then the target computing center for processing the task to be assigned is determined based on the task balance of the computing center where the task to be assigned is located.

[0060] If the priority of the task to be assigned is low, then the computing center with the smallest task balance among all computing centers is selected as the target computing center for processing the task to be assigned.

[0061] Optionally, determining the target computing center for processing the task to be assigned based on the task balance of the computing center where the task to be assigned is located includes:

[0062] Determine whether the task balance of the computing center where the task to be assigned is located does not exceed a preset balance threshold;

[0063] If so, the computing center where the task to be assigned is located will be the target computing center;

[0064] Otherwise, based on the task delay time of the task to be assigned, the expected completion time of each computing center, and the task balance, the computing center that meets the task delay time and has the smallest task balance among all computing centers is selected as the target computing center.

[0065] This application also provides a task allocation device, including:

[0066] The task acquisition module is used to acquire the tasks to be assigned and their priorities.

[0067] The status statistics module is used to collect statistics on the current working status of each working node in each computing center and the current queuing status of the task scheduling queue in each computing center.

[0068] The indicator determination module is used to determine the estimated completion time of the task to be assigned to each computing center and the task balance of each computing center based on the current working status of each working node in each computing center and the current queuing status of the task scheduling queue in each computing center.

[0069] The task allocation module is used to determine the target computing center for processing the task to be allocated based on the estimated completion time of each computing center, the task balance, and the priority of the task to be allocated, and to allocate the task to the target computing center.

[0070] This application also provides a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the task allocation method as described in any of the above embodiments.

[0071] This application also provides a computer device, including: one or more processors, and memory;

[0072] The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the task allocation method as described in any of the above embodiments.

[0073] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0074] The task allocation method, apparatus, storage medium, and computer equipment provided in this application, when there are tasks to be allocated, can first determine the priority of the tasks to be allocated, and statistically analyze the current working status of each working node in each computing center and the current queuing status of the task scheduling queues in each computing center. Then, based on the current working status of each working node in each computing center and the current queuing status of the task scheduling queues in each computing center, the estimated completion time of allocating the tasks to be allocated to each computing center and the task balance of each computing center can be determined. Thus, when allocating the tasks to be allocated, the target computing center for processing the tasks to be allocated can be determined based on the estimated completion time, task balance, and priority of each computing center, and the tasks to be allocated can be allocated to the target computing center. This not only distributes the tasks to the computing center that can complete the tasks the fastest, improving the timeliness of business data processing, but also takes into account the task balance of each computing center, thereby maintaining the task balance of each computing center as much as possible while meeting business objectives and optimizing resource utilization. Attached Figure Description

[0075] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0076] Figure 1 A flowchart illustrating a task allocation method provided in an embodiment of this application;

[0077] Figure 2 This is a schematic diagram of the structure of a task allocation system provided in an embodiment of this application;

[0078] Figure 3 This is a schematic diagram of the structure of a task allocation device provided in an embodiment of this application;

[0079] Figure 4 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0080] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0081] In existing methods of data anonymization task allocation, the task distribution center primarily creates anonymization tasks in the corresponding computing centers based on the storage address of the data to be anonymized. Since there are many types of data to be anonymized, and the processing time for each type varies significantly, and the processing capabilities of different computing centers differ due to resource variations, this can easily lead to severe task backlogs and unbalanced processing capabilities in certain computing centers, ultimately preventing timely anonymization of business data and failing to meet business objectives. Based on this, this application proposes the following technical solution, as detailed below:

[0082] In one embodiment, such as Figure 1 As shown, Figure 1 This is a flowchart illustrating a task allocation method provided in an embodiment of this application; this application provides a task allocation method, which may include:

[0083] S110: Get the tasks to be assigned and their priorities.

[0084] In this step, when assigning tasks, we can first obtain the tasks to be assigned and their priorities, and then dynamically assign the tasks to be assigned to the corresponding computing centers for processing based on the current status of each computing center.

[0085] It is understood that each computing center in this application may also be referred to as a computing cluster. A computing cluster is a computer system that is tightly integrated with a group of loosely connected computer software or hardware to perform computing tasks. In a sense, they can be regarded as a single computer. Individual computers in a computing cluster are usually called worker nodes and are typically connected via a local area network (LAN), but other possible connection methods also exist.

[0086] Each worker node in the computing center can process tasks and generate tasks to be assigned. For example, when a user wants to de-identify business data, they can send a de-identification task to a worker node in the computing center. When the worker node receives the de-identification task, it can send it to the task distribution center. Alternatively, when the task distribution center detects that a worker node has generated a de-identification task, it can also actively obtain the de-identification task. After obtaining the de-identification task, it can further obtain the priority corresponding to the de-identification task.

[0087] It should be noted that this application pre-sets a corresponding priority for each task to be assigned. The priority of different types of tasks to be assigned can be determined according to the business scenario of the task. For example, when the business scenario of a task to be assigned is used frequently in actual application, a higher priority can be configured for the task to be assigned. When the security level of the business scenario of a task to be assigned is high, a higher priority can also be configured for the task to be assigned. The specific settings can be made according to the actual situation, and no restrictions are imposed here.

[0088] S120: Statistics on the current working status of each working node in each computing center and the current queuing status of the task scheduling queue in each computing center.

[0089] In this step, after obtaining the tasks to be assigned and their priorities through S110, the current working status of each working node in each computing center and the current queuing status of the task scheduling queue in each computing center can be statistically analyzed. Then, the status of each computing center can be calculated in real time based on the current working status of each working node and the current queuing status of the task scheduling queue.

[0090] Indicatively, such as Figure 2 As shown, Figure 2 This is a schematic diagram of the structure of a task allocation system provided in an embodiment of this application; Figure 2 In this system, the task allocation system can obtain tasks to be allocated through the task distribution center and allocate the tasks to the corresponding computing centers. Each computing center is equipped with a task scheduler, which can count the current working status of each working node in each computing center and the current queuing status of the task scheduling queue in each computing center. The scheduler then sends the statistical results to the task distribution center so that the task distribution center can make dynamic allocations based on the statistical results.

[0091] Each computing center includes several working nodes, namely Figure 2The w1, w2, w3, w4, etc. in the table can be used to indicate the current working status of each working node, including whether the working node is processing a task, the task type, the amount of task, the processing time, and the remaining processing time. Each computing center includes at least one task scheduling queue, and the current queuing status of the task scheduling queue can include the number of tasks waiting to be processed in the queue, the priority of each task, the task type, the amount of task, the total processing time, and the queue position.

[0092] It is understood that the tasks to be assigned, tasks to be processed, and tasks being processed in this application may include multiple types. The priority of each type of task may be the same or different. The task volume and processing time of each type of task may also be the same or different. Here, the task volume refers to the data size of the task, such as 200MB, 100s, etc.

[0093] S130: Based on the current working status of each working node in each computing center and the current queuing status of the task scheduling queue in each computing center, determine the estimated completion time for allocating tasks to each computing center and the task balance of each computing center.

[0094] In this step, after S120 statistically analyzes the current working status of each working node in each computing center and the current queuing status of the task scheduling queue in each computing center, this application can determine the estimated completion time for allocating tasks to each computing center and the task balance of each computing center based on the current working status of each working node in each computing center and the current queuing status of the task scheduling queue in each computing center.

[0095] Specifically, after obtaining the current working status of each worker node in each computing center and the current queuing status of the task scheduling queue in each computing center, for each computing center, this application can calculate the remaining processing time of each worker node in processing tasks based on the current working status of each worker node in the computing center, and calculate the current queuing time based on the current queuing status of the task scheduling queue in the computing center. Then, based on the remaining processing time of each worker node in processing tasks and the current queuing time, the estimated completion time of the computing center in processing the task to be assigned can be determined. In this way, the estimated completion time of all computing centers in processing the task to be assigned can be obtained.

[0096] Next, this application can also calculate the load of the computing center based on the remaining processing time and current queuing time of each working node in the computing center, and use the load of the computing center as the task balance of the computing center.

[0097] S140: Based on the estimated completion time of each computing center, the task balance, and the priority of the tasks to be assigned, determine the target computing center for processing the tasks to be assigned, and assign the tasks to be assigned to the target computing center.

[0098] In this step, after determining the estimated completion time of the tasks to be assigned to each computing center and the task balance of each computing center through S130, this application can determine the target computing center for processing the tasks to be assigned based on the estimated completion time of each computing center, the task balance, and the priority of the tasks to be assigned, and then assign the tasks to the target computing center.

[0099] For example, when allocating tasks, this application not only considers the current task processing status of each computing center and calculates the estimated completion time and task balance of each computing center based on the current task processing status, but also considers the priority of the tasks to be allocated. As a result, high-priority tasks can be allocated to computing centers with shorter estimated completion times, while low-priority tasks can be allocated to computing centers with longer estimated completion times but lower task balance. This can prioritize resource utilization and improve task processing efficiency.

[0100] In the above embodiments, when there are tasks to be assigned, the priority of the task to be assigned can be determined first, and the current working status of each working node in each computing center and the current queuing status of the task scheduling queue in each computing center can be counted. Then, based on the current working status of each working node in each computing center and the current queuing status of the task scheduling queue in each computing center, the estimated completion time of the task to be assigned to each computing center and the task balance of each computing center can be determined. In this way, when assigning the task to be assigned, the target computing center for processing the task to be assigned can be determined based on the estimated completion time, task balance, and priority of each computing center, and the task to be assigned can be assigned to the target computing center. This can distribute the task to the computing center that can complete the task the fastest, improving the timeliness of business data processing, and also take into account the task balance of each computing center, thereby maintaining the task balance of each computing center as much as possible while meeting business objectives and optimizing resource utilization.

[0101] In one embodiment, S120 includes statistically analyzing the current working status of each working node in each computing center, including:

[0102] S121: For each computing center: Calculate the number of worker nodes in the computing center, the types of tasks being processed by the worker nodes currently processing tasks, the task volume, and the processing time, as well as the real-time weight of the computing center when processing different types of tasks; wherein, the real-time weight of the computing center when processing different types of tasks includes the real-time weight of the computing center when processing different types of tasks within the same computing center and the real-time weight of the computing center when processing different types of tasks in other computing centers.

[0103] S122: Calculate the remaining processing time of the task being processed by the work node of the currently processing task based on the real-time weight of the computing center when processing different types of tasks, the task type, task volume, and processing time of the work node of the currently processing task.

[0104] S123: The number of work nodes in the computing center and the remaining processing time of the work node currently processing the task are taken as the current working status of each work node in the computing center.

[0105] In this embodiment, when calculating the current working status of each worker node in a computing center, the following steps can be taken: First, the number of worker nodes in the computing center, the task type, task volume, and processing time of the worker node currently processing a task can be calculated, along with the real-time weight of the computing center when processing different types of tasks. Then, based on the real-time weight of the computing center when processing different types of tasks, the task type, task volume, and processing time of the worker node currently processing a task can be used to calculate the remaining processing time of the worker node currently processing a task. Finally, the number of worker nodes in the computing center and the remaining processing time of the worker node currently processing a task can be used as the current working status of each worker node in the computing center.

[0106] In this application, different task types correspond to different weights, and these weights can be dynamically changed according to the real-time processing capacity of the computing center. Therefore, when calculating the remaining processing time of a worker node that is currently processing a task, this application can first obtain the real-time weights of the computing center when processing different types of tasks, and then combine the task type, task volume, and processing time of the worker node that is currently processing a task to calculate the remaining processing time of the worker node that is currently processing a task. The remaining processing time obtained in this way takes into account the real-time processing capacity of the computing center, and is therefore more accurate.

[0107] Furthermore, the real-time weights of a computing center in this application when processing different types of tasks include both the real-time weights of the computing center when processing different types of tasks within its own computing center and the real-time weights of the computing center when processing different types of tasks in other computing centers. It is understood that while all tasks to be assigned in this application originate from computing centers, the computing center that ultimately processes the task is not necessarily the one that originally generated it. When a task is assigned to another computing center for processing, it needs to be transmitted across computing centers, thus generating a certain network transmission weight. Therefore, the real-time weights in this application include both the real-time weights of the computing center when processing different types of tasks within its own computing center and the real-time weights of the computing center when processing different types of tasks in other computing centers.

[0108] In one embodiment, calculating the real-time weights of the computing center when processing different types of tasks in S121 may include:

[0109] S1210: Obtain the task type, actual task volume, and actual processing time of the historical tasks processed by the computing center within a preset historical period.

[0110] S1211: For different types of historical tasks, calculate the actual processing speed of the historical task based on the actual workload and actual processing time.

[0111] S1212: Determine the actual weight of different types of historical tasks based on their actual processing speed, and compare the actual weight of different types of historical tasks with the test weights of different types of test tasks pre-configured through benchmark testing to obtain the comparison results.

[0112] S1213: Determine the real-time weight of the computing center when processing different types of tasks based on the comparison results.

[0113] In this embodiment, when calculating the real-time weights of a computing center when processing different types of tasks, the task type, actual workload, and actual processing time of the historical tasks processed by the computing center within a preset historical period can be obtained. For different types of historical tasks, the actual processing speed of the historical task is calculated based on the actual workload and actual processing time. The actual weights of different types of historical tasks are determined based on the actual processing speeds of different types of historical tasks. Then, the actual weights of different types of historical tasks can be compared with the test weights of different types of test tasks pre-configured through benchmark testing to obtain the comparison results. The real-time weights of the computing center when processing different types of tasks are determined based on the comparison results.

[0114] For example, when this application calculates that the actual processing speed of a certain computing center when processing a certain type of historical task is 100MB / s, the actual weight of that type of historical task can be set to 100MB:1, that is, every 100MB is converted into 1 second. In this way, when the same type of historical task is obtained, the actual processing time of the historical task can be calculated according to the task volume and the actual weight. Of course, if the historical task is a task generated by other computing centers, when calculating the actual weight of the historical task, the network transmission weight between other computing centers and this computing center in the historical period can also be considered, such as 50MB:1. In this way, the network transmission weight of the historical task can be calculated according to the task volume, and the network transmission weight can be combined with the actual weight of the historical task to obtain the final actual weight.

[0115] Furthermore, to obtain the real-time weights of each computing center when processing different types of tasks, this application can also compare the actual weights of different types of historical tasks with the test weights of different types of test tasks pre-configured through benchmark testing, and perform comparative calculations based on the comparison results to determine the final real-time weights. For example, if a computing center calculates a high actual weight for a certain type of historical task (100MB:5), while the pre-configured test weight for that type of test task is 100MB:2, it indicates that the computing center currently has a heavy computational load. In this case, the test weights can be adjusted according to the actual weights of that type of task to obtain the final real-time weights. Conversely, if a computing center calculates a low actual weight for a certain type of historical task (100MB:0.5), while the pre-configured test weight for that type of test task is 100MB:2, it indicates that the computing center currently has a light computational load. In this case, the test weights can also be adjusted according to the actual weights of that type of task to obtain the final real-time weights.

[0116] In one embodiment, the process of configuring test weights for different types of test tasks in S1212 may include:

[0117] S2120: Pre-configure different types of test tasks and determine the test task volume corresponding to each type of test task.

[0118] S2121: Benchmark tests are performed on different types of test tasks to obtain the test duration for each type of test task.

[0119] S2122: Calculate the test speed for each type of test task based on the test duration and test volume.

[0120] S2123: Configure the test weights for different types of test tasks according to the test speeds corresponding to different types of test tasks.

[0121] In this embodiment, when configuring test weights for different types of test tasks, different types of test tasks can be pre-configured, and the test task quantity for each type of test task can be determined. Then, benchmark tests can be performed on different types of test tasks to obtain the test duration for each type of test task. Then, the test duration for each type of test task is divided by the test task quantity to obtain the test speed for each type of test task. Finally, the test weights for different types of test tasks are configured according to the test speeds for different types of test tasks.

[0122] For example, in data anonymization scenarios, the main data types being anonymized are mbag, video, image, and zip compressed files. Since the size of each data type varies, the processing time differs. Therefore, this application can pre-benchmark tests on data of common sizes, assigning different test weights based on the processing time of various data types. For example, the test weight for mbag data could be 100MB:1, for video data 5secs:1, for image data image:1, and for zip data 10MB:1. Of course, when calculating the test weight for this test task, the network transmission weight between other computing centers and this computing center can also be considered, such as 50MB:1. This allows the network transmission weight of the test task to be calculated based on its workload, and the final test weight is obtained by combining this network transmission weight with the test weight of the test task. Here, 1 can be 1 second, 1 minute, or 1 hour. When the unit of the test weight is seconds (s) while the unit of the actual weight is minutes, appropriate conversion operations can be performed based on the different units.

[0123] In one embodiment, calculating the remaining processing time of the task being processed by the worker node in step S122, based on the real-time weight of the computing center when processing different types of tasks, the task type, task volume, and processing time of the worker node currently processing the task, may include:

[0124] S1221: Determine the real-time weight of the task being processed by the work node processing the task based on the real-time weight of the computing center when processing different types of tasks, and the task type of the task being processed by the work node processing the task.

[0125] S1222: Calculate the total processing time of the task being processed by the worker node processing the task based on the real-time weight and task volume of the task being processed.

[0126] S1223: Subtract the already processed time from the total processing time of the currently processing work node to obtain the remaining processing time of the currently processing work node.

[0127] In this embodiment, when calculating the remaining processing time of a work node that is currently processing a task, the real-time weight of the work node is first determined based on the real-time weight of the computing center when processing different types of tasks and the task type of the work node that is currently processing a task. Then, the total processing time of the work node is obtained by dividing the amount of the work node's task by the real-time weight. Finally, the remaining processing time of the work node is obtained by subtracting the processing time from the total processing time of the work node that is currently processing a task.

[0128] For example, when the real-time weight of a certain type of task is 50MB:1, in seconds, we can first obtain the task volume of that type of task. If the task volume is 200MB, then the total processing time of the task is 4 seconds. If the processing time of the worker node is 3 seconds, then the remaining processing time is 1 second.

[0129] In one embodiment, the process of S120, which involves calculating the current queuing status of the task scheduling queues of each computing center, may include:

[0130] S210: For each computing center: Calculate the queue position, task type, priority, and task quantity of each task to be processed in the task scheduling queue of the computing center, as well as the real-time weight of the computing center when processing different types of tasks.

[0131] S211: Calculate the total processing time for each task based on the real-time weights of the computing center when processing different types of tasks, the task type and workload of each task to be processed.

[0132] S212: The queue position, priority, and total processing time of each task to be processed in the task scheduling queue are taken as the current queuing status of the task scheduling queue.

[0133] In this embodiment, when calculating the current queuing status of the task scheduling queue of a certain computing center, the queue position, task type, priority, and task quantity of each task to be processed in the task scheduling queue of the computing center can be counted first, as well as the real-time weight of the computing center when processing different types of tasks. Then, based on the real-time weight of the computing center when processing different types of tasks, the task type and task quantity of each task to be processed, the total processing time of each task to be processed can be calculated. Finally, the queue position, priority, and total processing time of each task to be processed in the task scheduling queue are used as the current queuing status of the task scheduling queue.

[0134] Specifically, when calculating the total processing time for each pending task, this application can do so based on the real-time weight of the computing center when processing different types of tasks, the task type of each pending task, and the task volume. For example, if the real-time weight of the computing center corresponding to the task type of the pending task is 5 secs: 1 (in seconds), and the task volume of the pending task is 70 secs, then the total processing time for the pending task is 14 secs.

[0135] In one embodiment, determining the estimated completion time for assigning the tasks to be assigned to each computing center in step S130, based on the current working status of each working node in each computing center and the current queuing status of the task scheduling queues in each computing center, may include:

[0136] S131: For each computing center: Based on the current working status of each working node in the computing center and the current queuing status of the task scheduling queue in the computing center, calculate the shortest waiting time to assign the task to be assigned to the computing center.

[0137] S132: Obtain the real-time weights of the computing center when processing different types of tasks, as well as the task type and workload of the task to be assigned.

[0138] S133: Based on the task type and workload of the task to be assigned, and the real-time weight of the computing center when processing different types of tasks, calculate the estimated processing time for assigning the task to be assigned to the computing center.

[0139] S134: Calculate the estimated completion time for assigning the task to be assigned to the computing center based on the shortest waiting time and the estimated processing time.

[0140] In this embodiment, when calculating the estimated completion time for assigning a task to a computing center, the shortest waiting time for assigning the task to the computing center can be calculated first based on the current working status of each working node in the computing center and the current queuing status of the task scheduling queue of the computing center. Then, the real-time weight of the computing center when processing different types of tasks, as well as the task type and task quantity of the task to be assigned, are obtained. Based on the task type and task quantity of the task to be assigned, and the real-time weight of the computing center when processing different types of tasks, the estimated processing time for assigning the task to the computing center is calculated. Finally, the estimated completion time for assigning the task to the computing center can be obtained by adding the shortest waiting time to the estimated processing time.

[0141] In one embodiment, the current working status of each worker node in the computing center includes the number of worker nodes in the computing center and the remaining processing time of the worker node that is processing a task; the current queuing status of the task scheduling queue in the computing center includes the queue position, priority, and total processing time of each task to be processed in the task scheduling queue.

[0142] S131, based on the current working status of each working node in the computing center and the current queuing status of the task scheduling queue in the computing center, calculates the shortest waiting time for assigning the task to be assigned to the computing center, which may include:

[0143] S1310: Determine the set of tasks in the task scheduling queue of the computing center whose priority is not lower than the priority of the task to be assigned, as well as the number and queue position of the tasks to be processed in the task set.

[0144] S1311: Based on the number of tasks to be processed in the task set and the number of worker nodes in the computing center, calculate the number of waiting rounds for the tasks to be assigned to the computing center.

[0145] S1312: Based on the total processing time and queue position of each task to be processed in the task set, calculate the estimated waiting time of the task to be assigned under the number of waiting rounds.

[0146] S1313: Based on the estimated waiting time of the task to be assigned under the number of waiting rounds and the remaining processing time of the work node currently processing the task in the computing center, calculate the shortest waiting time to assign the task to be assigned to the computing center.

[0147] In this embodiment, when calculating the shortest waiting time to assign a task to a computing center, the following steps are taken: First, determine the set of tasks in the task scheduling queue of the computing center that have a priority no lower than that of the task to be assigned, as well as the number of tasks to be assigned and their queue positions. Then, divide the number of tasks to be assigned in the task set by the number of worker nodes in the computing center to obtain the number of waiting rounds for the task to be assigned to the computing center. Next, based on the total processing time and queue position of each task to be assigned in the task set, calculate the expected waiting time for the task to be assigned under the number of waiting rounds. Finally, add the expected waiting time for the task to be assigned under the number of waiting rounds to the remaining processing time of the worker node currently processing the task in the computing center to obtain the shortest waiting time for the task to be assigned to the computing center.

[0148] In one embodiment, the number of waiting rounds may include odd-numbered rounds and even-numbered rounds; S1312, based on the total processing time and queue position of each task to be processed in the task set, calculating the estimated waiting time of the task to be assigned under the number of waiting rounds may include:

[0149] S3120: Based on the total processing time and queue position of each pending task in the task set, calculate the first processing time of the first pending task with the longest total processing time among the pending tasks processed by the computing node in the odd-numbered rounds.

[0150] S3121: And, based on the total processing time and queue position of each pending task in the task set, calculate the second processing time of the second pending task with the shortest total processing time among the pending tasks processed by the computing node in the even-numbered rounds.

[0151] S3122: Determine the expected waiting time of the task to be assigned under the number of waiting rounds based on the first processing time and the second processing time.

[0152] In this embodiment, when calculating the expected waiting time of the task to be assigned under the corresponding number of waiting rounds, the first processing time of the first task to be assigned with the longest total processing time among the tasks to be processed in odd rounds and the second processing time of the second task to be assigned with the shortest total processing time among the tasks to be processed in even rounds can be calculated based on the total processing time and queue position of each task to be processed in the task set. Then, the first processing time and the second processing time can be added together to obtain the expected waiting time of the task to be assigned under the corresponding number of waiting rounds.

[0153] For example, if a computing center has 5 worker nodes and a total of 10 pending tasks in the task scheduling queue, with 4 pending tasks having a higher priority than the pending task, then when the pending task is scheduled to the computing center, it will be placed in the fifth position in the task scheduling queue. Based on the number of worker nodes in the computing center and the number of pending tasks with a higher priority than the pending task, the number of rounds the pending task needs to wait can be calculated to be 0 rounds. Round 0 is an even number of rounds, and there are no other odd number of rounds before it. Therefore, this application can directly calculate the processing time of the pending task with the shortest total processing time among the pending tasks processed in even number of rounds, and use this processing time as the expected waiting time of the pending task. When there are 2 work nodes and 6 pending tasks in the task scheduling queue with higher priority than pending tasks, it is necessary to wait 3 rounds before the pending tasks can be scheduled for allocation. In this case, it is necessary to calculate the first processing time of the first pending task with the longest total processing time among the pending tasks processed in odd rounds, and the second processing time of the second pending task with the shortest total processing time among the pending tasks processed in even rounds. The first processing time and the second processing time are then added together to obtain the estimated waiting time of the pending task in 3 rounds.

[0154] In one embodiment, the current working status of each worker node in each computing center includes the number of worker nodes in each computing center and the remaining processing time of the worker node that is processing a task; the current queuing status of the task scheduling queue in each computing center includes the total processing time of each pending task in the task scheduling queue.

[0155] S130 determines the task balance of each computing center based on the task completion status of each worker node in each computing center and the queuing status of the task scheduling queues in each computing center. This may include:

[0156] S301: For each computing center: Calculate the total processing time of the computing center based on the remaining processing time of the work nodes that are currently processing tasks in the computing center and the total processing time of each pending task in the task scheduling queue of the computing center.

[0157] S302: Calculate the task balance of the computing center based on the total processing time of the computing center and the number of worker nodes in the computing center.

[0158] In this embodiment, when calculating the task balance of a computing center, the total processing time of the computing center can be calculated based on the remaining processing time of the work nodes that are currently processing tasks in the computing center and the total processing time of each task to be processed in the task scheduling queue of the computing center. Then, the total processing time of the computing center is divided by the number of work nodes in the computing center to obtain the task balance of the computing center.

[0159] For example, if the remaining processing time of the worker nodes in a computing center is 10 minutes, and the total processing time of all pending tasks in the task scheduling queue of the computing center is 20 minutes, then the total processing time of the computing center is 30 minutes. If the number of worker nodes in the computing center is 5, then the task balance of the computing center is 6.

[0160] In one embodiment, the priority of the task to be assigned includes high priority, medium priority and low priority, and the task delay time is different for tasks with different priorities.

[0161] S140, based on the estimated completion time of each computing center, task balance, and the priority of the tasks to be assigned, determines the target computing center for processing the tasks to be assigned, which may include:

[0162] S141: If the priority of the task to be assigned is high priority, then the computing center with the earliest expected completion time among the computing centers is selected as the target computing center for processing the task to be assigned.

[0163] S142: If the priority of the task to be assigned is medium priority, then the target computing center for processing the task to be assigned is determined according to the task balance of the computing center where the task to be assigned is located.

[0164] S143: If the priority of the task to be assigned is low, then the computing center with the smallest task balance among all computing centers is selected as the target computing center for processing the task to be assigned.

[0165] In this embodiment, when determining the target computing center for processing a task based on the estimated completion time, task balance, and priority of the task to be assigned in each computing center, if the task to be assigned has a high priority, the computing center with the earliest estimated completion time among all computing centers can be selected as the target computing center for processing the task to be assigned; if the task to be assigned has a medium priority, the target computing center for processing the task to be assigned can be determined based on the task balance of the computing center where the task to be assigned is located; if the task to be assigned has a low priority, the computing center with the lowest task balance among all computing centers can be selected as the target computing center for processing the task to be assigned. This ensures both the timeliness of business data processing and improves resource utilization.

[0166] In this application, different priorities are pre-configured for different types of tasks to be assigned. The higher the priority, the shorter the task delay; the lower the priority, the longer the task delay. Therefore, when the priority of a task to be assigned is high, it needs to be assigned to the computing center that can complete the task earliest, i.e., the computing center with the earliest expected completion time among all computing centers is selected as the target computing center for processing the task to be assigned. When the priority of a task to be assigned is medium, it indicates that the task delay of the task to be assigned is moderate, and the task to be assigned to the computing center where the task to be assigned is located can be determined based on the task balance of the computing center where the task to be assigned is located. When the task delay of the task to be assigned is long, the computing center with the smallest task balance among all computing centers can be considered as the target computing center for processing the task to be assigned. This can make reasonable use of the resources of the computing center and improve the processing efficiency of the task to be assigned.

[0167] In one embodiment, determining the target computing center for processing the task to be assigned based on the task balance of the computing center where the task to be assigned is located in S142 may include:

[0168] S1421: Determine whether the task balance of the computing center where the task to be assigned is located does not exceed the preset balance threshold; if yes, execute S1422; otherwise, execute S1423.

[0169] S1422: The computing center where the task to be assigned is located is designated as the target computing center.

[0170] S1423: Based on the task delay time of the task to be assigned, the estimated completion time of each computing center, and the task balance, select the computing center that satisfies the task delay time and has the smallest task balance as the target computing center.

[0171] In this embodiment, when the priority of the task to be assigned is medium priority, when determining the target computing center for processing the task to be assigned, it can first be determined whether the task balance of the computing center where the task to be assigned is located does not exceed the preset balance threshold. If it does not exceed the threshold, the computing center where the task to be assigned is located is taken as the target computing center. If it exceeds the threshold, the computing center that meets the task delay time of the task to be assigned and has the smallest task balance among the computing centers can be selected as the target computing center based on the task delay time of the task to be assigned, the expected completion time of each computing center, and the task balance. This can balance resources and improve processing efficiency.

[0172] For example, when the priority of the task to be assigned is medium priority and the task delay time is 2 hours, the computing centers whose estimated completion time is no more than 2 hours can be selected first. Then, based on the task balance of the selected computing centers, the computing center with the lowest task balance can be selected as the target computing center. Furthermore, if there are more than two computing centers that can be used as the target computing center, any computing center can be selected, or the computing center that is closer to the computing center where the task to be assigned is located can be selected. The specific choice can be made according to the actual situation, and there are no restrictions here.

[0173] The task allocation apparatus provided in the embodiments of this application is described below. The task allocation apparatus described below and the task allocation method described above can be referred to in correspondence.

[0174] In one embodiment, such as Figure 3 As shown, Figure 3 This is a schematic diagram of a task allocation device provided in an embodiment of this application; this application also provides a task allocation device, which may include a task acquisition module 210, a situation statistics module 220, an indicator determination module 230, and a task allocation module 240, specifically including the following:

[0175] The task acquisition module 210 is used to acquire the tasks to be assigned and the priority of the tasks to be assigned.

[0176] The statistics module 220 is used to collect statistics on the current working status of each working node in each computing center and the current queuing status of the task scheduling queue in each computing center.

[0177] The indicator determination module 230 is used to determine the estimated completion time of the task to be assigned to each computing center and the task balance of each computing center based on the current working status of each working node in each computing center and the current queuing status of the task scheduling queue in each computing center.

[0178] The task allocation module 240 is used to determine the target computing center for processing the task to be allocated based on the estimated completion time of each computing center, the task balance, and the priority of the task to be allocated, and to allocate the task to the target computing center.

[0179] In the above embodiments, when there are tasks to be assigned, the priority of the task to be assigned can be determined first, and the current working status of each working node in each computing center and the current queuing status of the task scheduling queue in each computing center can be counted. Then, based on the current working status of each working node in each computing center and the current queuing status of the task scheduling queue in each computing center, the estimated completion time of the task to be assigned to each computing center and the task balance of each computing center can be determined. In this way, when assigning the task to be assigned, the target computing center for processing the task to be assigned can be determined based on the estimated completion time, task balance, and priority of each computing center, and the task to be assigned can be assigned to the target computing center. This can distribute the task to the computing center that can complete the task the fastest, improving the timeliness of business data processing, and also take into account the task balance of each computing center, thereby maintaining the task balance of each computing center as much as possible while meeting business objectives and optimizing resource utilization.

[0180] In one embodiment, this application also provides a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the task allocation method as described in any of the above embodiments.

[0181] In one embodiment, this application also provides a computer device, including: one or more processors, and memory.

[0182] The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the task allocation method as described in any of the above embodiments.

[0183] Indicatively, such as Figure 4 As shown, Figure 4 This is a schematic diagram of the internal structure of a computer device 300 provided in an embodiment of this application. The computer device 300 can be provided as a server. (Refer to...) Figure 4The computer device 300 includes a processing component 302, which further includes one or more processors, and memory resources represented by memory 301 for storing instructions, such as application programs, that can be executed by the processing component 302. The application programs stored in memory 301 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 302 is configured to execute instructions to perform the task allocation method of any of the above embodiments.

[0184] The computer device 300 may also include a power supply component 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305. The computer device 300 may operate on an operating system stored in memory 301, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or similar.

[0185] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0186] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0187] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0188] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A task allocation method, characterized in that, The method includes: Obtain the tasks to be assigned and their priorities; Statistics on the current working status of each working node in each computing center and the current queuing status of the task scheduling queue in each computing center; Based on the current working status of each working node in each computing center and the current queuing status of the task scheduling queue in each computing center, the estimated completion time for allocating the tasks to be assigned to each computing center and the task balance of each computing center are determined. Based on the estimated completion time of each computing center, the task balance, and the priority of the tasks to be assigned, the target computing center for processing the tasks to be assigned is determined, and the tasks to be assigned are assigned to the target computing center. The statistics on the current working status of each working node in each computing center include: For each computing center: The statistics include the number of worker nodes in the computing center, the types of tasks being processed by the worker nodes currently processing tasks, the amount of tasks processed, the processing time, and the real-time weight of the computing center when processing different types of tasks. The real-time weight of the computing center when processing different types of tasks includes the real-time weight of the computing center when processing different types of tasks within the same computing center and the real-time weight of the computing center when processing different types of tasks in other computing centers. Based on the real-time weights of the computing center when processing different types of tasks, the task type, task volume, and processing time of the work node currently processing the task, calculate the remaining processing time of the work node currently processing the task. The number of worker nodes in the computing center and the remaining processing time of the worker node currently processing the task are taken as the current working status of each worker node in the computing center. The statistics on the real-time weights of the computing center when processing different types of tasks include: Obtain the task type, actual workload, and actual processing time of the historical tasks processed by the computing center within a preset historical period; For different types of historical tasks, the actual processing speed of the historical task is calculated based on the actual workload and actual processing time of the historical task. The actual weights of different types of historical tasks are determined based on their actual processing speeds, and then compared with the test weights of different types of test tasks pre-configured through benchmark testing to obtain the comparison results. The real-time weights of the computing center when processing different types of tasks are determined based on the comparison results.

2. The task allocation method according to claim 1, characterized in that, The process of configuring test weights for the different types of test tasks includes: Pre-configure different types of test tasks and determine the corresponding test task volume for each type of test task; Benchmark tests were performed on different types of test tasks to obtain the test duration for each type of test task. Calculate the test speed corresponding to each type of test task based on the test duration and test volume of each type of test task. Configure the test weights for different types of test tasks according to the test speeds corresponding to different types of test tasks.

3. The task allocation method according to claim 1, characterized in that, The step of calculating the remaining processing time of the task being processed by the worker node based on the real-time weight of the computing center when processing different types of tasks, the task type, task volume, and processing time of the worker node currently processing the task, includes: The real-time weight of the task being processed by the work node is determined based on the real-time weight of the work node processing different types of tasks in the computing center and the task type of the task being processed by the work node currently processing the task. The total processing time of the work node processing the task is calculated by using the real-time weight and task volume of the work node currently processing the task. The remaining processing time of the task is obtained by subtracting the already processed time from the total processing time of the task being processed by the worker node.

4. The task allocation method according to claim 1, characterized in that, The statistics on the current queuing status of the task scheduling queues in each computing center include: For each computing center: The system collects statistics on the queue position, task type, priority, and number of tasks to be processed in the task scheduling queue of the computing center, as well as the real-time weights of the computing center when processing different types of tasks. Based on the real-time weights of the computing center when processing different types of tasks, the task type and task quantity of each task to be processed, the total processing time of each task to be processed is calculated. The queue position, priority, and total processing time of each task to be processed in the task scheduling queue are used as the current queuing status of the task scheduling queue.

5. The task allocation method according to any one of claims 1-4, characterized in that, The step of determining the estimated completion time for assigning the tasks to be assigned to each computing center based on the current working status of each working node in each computing center and the current queuing status of the task scheduling queue in each computing center includes: For each computing center: Based on the current working status of each working node in the computing center and the current queuing status of the task scheduling queue in the computing center, calculate the shortest waiting time to assign the task to be assigned to the computing center. Obtain the real-time weights of the computing center when processing different types of tasks, as well as the task type and workload of the task to be assigned; Based on the task type and workload of the task to be assigned, and the real-time weight of the computing center when processing different types of tasks, calculate the estimated processing time for assigning the task to be assigned to the computing center. Based on the minimum waiting time and the estimated processing time, calculate the estimated completion time for assigning the task to be assigned to the computing center.

6. The task allocation method according to claim 5, characterized in that, The current status of each worker node in the computing center includes the number of worker nodes in the computing center and the remaining processing time of the worker node that is processing a task; the current queuing status of the task scheduling queue in the computing center includes the queue position, priority, and total processing time of each task to be processed in the task scheduling queue. The step of calculating the shortest waiting time to assign the task to the computing center based on the current working status of each working node and the current queuing status of the task scheduling queue includes: Determine the set of tasks in the task scheduling queue of the computing center that have a priority no lower than the priority of the task to be assigned, as well as the number and queue position of the tasks to be processed in the task set; Based on the number of tasks to be processed in the task set and the number of worker nodes in the computing center, calculate the number of waiting rounds for the tasks to be assigned to the computing center. Based on the total processing time and queue position of each task to be processed in the task set, calculate the estimated waiting time of the task to be assigned under the number of waiting rounds; Based on the estimated waiting time of the task to be assigned under the number of waiting rounds and the remaining processing time of the work node currently processing the task in the computing center, calculate the shortest waiting time to assign the task to the computing center.

7. The task allocation method according to claim 6, characterized in that, The number of waiting rounds includes odd-numbered rounds and even-numbered rounds; The step of calculating the estimated waiting time of the task to be assigned under the number of waiting rounds based on the total processing time and queue position of each task to be processed in the task set includes: Based on the total processing time and queue position of each pending task in the task set, the first processing time of the first pending task with the longest total processing time among the pending tasks processed by the computing center in the odd-numbered rounds is calculated. In addition, based on the total processing time and queue position of each pending task in the task set, the second processing time of the second pending task with the shortest total processing time among the pending tasks processed by the computing center in the even-numbered rounds is calculated. The estimated waiting time for the task to be assigned is determined based on the first processing time and the second processing time, under the number of waiting rounds.

8. The task allocation method according to any one of claims 1-4, characterized in that, The current status of each worker node in each computing center includes the number of worker nodes in each computing center and the remaining processing time of the worker node that is currently processing a task; the current queuing status of the task scheduling queue in each computing center includes the total processing time of each pending task in the task scheduling queue. The process of determining the task balance of each computing center based on the task completion status of each working node in each computing center and the queuing status of the task scheduling queues in each computing center includes: For each computing center: The total processing time of the computing center is calculated based on the remaining processing time of the work nodes that are currently processing tasks and the total processing time of each pending task in the task scheduling queue of the computing center. Calculate the task balance of the computing center based on its total processing time and the number of worker nodes.

9. The task allocation method according to any one of claims 1-4, characterized in that, The priorities of the tasks to be assigned include high priority, medium priority and low priority, and the task delay time is different for tasks with different priorities. The process of determining the target computing center for processing the tasks to be assigned, based on the estimated completion time of each computing center, the task balance, and the priority of the tasks to be assigned, includes: If the priority of the task to be assigned is high priority, then the computing center with the earliest expected completion time among all computing centers is selected as the target computing center for processing the task to be assigned. If the priority of the task to be assigned is medium priority, then the target computing center for processing the task to be assigned is determined based on the task balance of the computing center where the task to be assigned is located. If the priority of the task to be assigned is low, then the computing center with the smallest task balance among all computing centers is selected as the target computing center for processing the task to be assigned.

10. The task allocation method according to claim 9, characterized in that, The step of determining the target computing center for processing the task to be assigned based on the task balance of the computing center where the task to be assigned is located includes: Determine whether the task balance of the computing center where the task to be assigned is located does not exceed a preset balance threshold; If so, the computing center where the task to be assigned is located will be the target computing center; Otherwise, based on the task delay time of the task to be assigned, the expected completion time of each computing center, and the task balance, the computing center that meets the task delay time and has the smallest task balance among all computing centers is selected as the target computing center.

11. A task allocation device, characterized in that, include: The task acquisition module is used to acquire the tasks to be assigned and their priorities. The status statistics module is used to collect statistics on the current working status of each working node in each computing center and the current queuing status of the task scheduling queue in each computing center. The indicator determination module is used to determine the estimated completion time of the task to be assigned to each computing center and the task balance of each computing center based on the current working status of each working node in each computing center and the current queuing status of the task scheduling queue in each computing center. The task allocation module is used to determine the target computing center for processing the task to be allocated based on the estimated completion time of each computing center, the task balance, and the priority of the task to be allocated, and to allocate the task to the target computing center. The status statistics module provides statistics on the current working status of each working node in each computing center, including: For each computing center: The statistics include the number of worker nodes in the computing center, the types of tasks being processed by the worker nodes currently processing tasks, the amount of tasks processed, the processing time, and the real-time weight of the computing center when processing different types of tasks. The real-time weight of the computing center when processing different types of tasks includes the real-time weight of the computing center when processing different types of tasks within the same computing center and the real-time weight of the computing center when processing different types of tasks in other computing centers. Based on the real-time weights of the computing center when processing different types of tasks, the task type, task volume, and processing time of the work node currently processing the task, calculate the remaining processing time of the work node currently processing the task. The number of worker nodes in the computing center and the remaining processing time of the worker node currently processing the task are taken as the current working status of each worker node in the computing center. The statistics on the real-time weights of the computing center when processing different types of tasks include: Obtain the task type, actual workload, and actual processing time of the historical tasks processed by the computing center within a preset historical period; For different types of historical tasks, the actual processing speed of the historical task is calculated based on the actual workload and actual processing time of the historical task. The actual weights of different types of historical tasks are determined based on their actual processing speeds, and then compared with the test weights of different types of test tasks pre-configured through benchmark testing to obtain the comparison results. The real-time weights of the computing center when processing different types of tasks are determined based on the comparison results.

12. A storage medium, characterized in that: The storage medium stores computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the task allocation method as described in any one of claims 1 to 10.

13. A computer device, characterized in that, include: One or more processors, and memory; The memory stores computer-readable instructions that, when executed by the one or more processors, perform the steps of the task allocation method as described in any one of claims 1 to 10.

Images