Task processing method and device, computer device, storage medium and program product
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- TSINGHUA UNIVERSITY
- Filing Date
- 2026-02-06
- Publication Date
- 2026-06-19
Smart Images
Figure CN122247989A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of task processing technology, and in particular to a task processing method, apparatus, computer equipment, storage medium, and program product. Background Technology
[0002] Secure multi-party computation (MPC) is a privacy-preserving computation technique that allows multiple parties who do not fully trust each other to collaboratively compute a function and obtain the output without revealing their private data. In practical applications, the performance of MPC technology depends on the quality of network communication.
[0003] However, the existing MPC technology has relatively low network communication efficiency, which leads to low computational efficiency. Summary of the Invention
[0004] Therefore, it is necessary to provide a task processing method, apparatus, computer equipment, storage medium, and program product that can improve the computing efficiency of MPC in response to the above-mentioned technical problems.
[0005] Firstly, this application provides a task processing method for coordinating nodes. The method includes:
[0006] The target number of parallel tasks is determined based on the task input size of the computing task to be executed and the minimum number of parallel tasks per computing node when the network bandwidth is saturated. The task input size is used to characterize at least one of the following: the magnitude, computational complexity, and resource requirements of the computing task.
[0007] Based on the target number of parallel tasks, the computation task is divided into multiple parallel subtasks;
[0008] Based on the preset multiple mapping methods and the target number of parallel tasks, the number of executable tasks corresponding to each mapping method is determined, and target parallel subtasks are allocated to each mapping method according to the number of executable tasks to execute the computing tasks. The task allocation ratios corresponding to different mapping methods are different.
[0009] In one embodiment, determining the target number of parallel tasks based on the task input size of the computational task to be executed and the minimum number of parallel tasks per computing node under network bandwidth saturation includes:
[0010] Obtain the unit communication volume coefficient of the logical role corresponding to each computing node. The communication volume coefficient includes the data transmission volume coefficient and the data reception volume coefficient of the logical role under the unit task input scale.
[0011] Based on the communication volume coefficients of each unit, determine the transmission time for each computing node to process the unit task input scale;
[0012] The target number of parallel tasks is determined based on the task input size, the minimum number of parallel tasks, and the transmission time.
[0013] In one embodiment, determining the target number of parallel tasks based on the task input size, the minimum number of parallel tasks, and the transmission time includes:
[0014] The task input size, the minimum number of each parallel task, the transmission time, and the preset aggregation cost function are input into a preset time optimization model to obtain the target number of parallel tasks output by the time optimization model.
[0015] In one embodiment, determining the number of executable tasks corresponding to each of the preset multiple mapping methods and the target number of parallel tasks includes:
[0016] For each of the mapping methods, the task allocation ratio corresponding to the mapping method is determined based on the task input scale and the bandwidth utilization of each computing node included in the mapping method. The mapping method includes a mapping relationship between multiple computing nodes and the logical roles corresponding to each computing node.
[0017] The product of the task allocation ratio and the target number of parallel tasks is used to determine the number of executable tasks corresponding to the mapping method.
[0018] In one embodiment, allocating target parallel subtasks to each of the mapping methods according to the number of each of the executable tasks includes:
[0019] Based on the load balancing strategy and the number of each executable task, establish a matching relationship between each parallel subtask and the mapping method corresponding to each number of executable tasks;
[0020] For each of the mapping methods, the target parallel subtasks corresponding to the matching relationship are allocated to each computing node corresponding to the mapping method, based on the matching relationship corresponding to the mapping method.
[0021] In one embodiment, the method further includes:
[0022] Obtain the average transmit bandwidth utilization and average receive bandwidth utilization of each computing node;
[0023] Determine the maximum value among the average transmit bandwidth utilization and average receive bandwidth utilization;
[0024] The minimum number of parallel tasks is determined based on the maximum value.
[0025] Secondly, this application also provides a task processing apparatus for coordinating nodes. The apparatus includes:
[0026] The first determining module is used to determine the target number of parallel tasks based on the task input scale of the computing task to be executed and the minimum number of parallel tasks of each computing node under network bandwidth saturation. The task input scale is used to characterize at least one of the following: the magnitude, computational complexity, and resource requirements of the computing task.
[0027] The partitioning module is used to divide the computing task into multiple parallel subtasks according to the target number of parallel tasks;
[0028] The execution module is used to determine the number of executable tasks corresponding to each of the preset multiple mapping methods and the target number of parallel tasks, and to allocate target parallel subtasks to each of the mapping methods according to the number of executable tasks to execute the computing tasks. The task allocation ratios are different for different mapping methods.
[0029] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described in the first aspect.
[0030] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.
[0031] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.
[0032] The aforementioned task processing method, apparatus, computer equipment, storage medium, and program product involve the following steps: First, the coordinating node determines the target number of parallel tasks based on the task input size of the computational task to be executed and the minimum number of parallel tasks that each computing node can handle when the network bandwidth is saturated. Then, based on the target number of parallel tasks, the computational task is divided into multiple parallel subtasks. Next, based on a variety of preset mapping methods and the target number of parallel tasks, the number of executable tasks corresponding to each mapping method is determined. Finally, target parallel subtasks are allocated to each mapping method according to the number of executable tasks to execute the computational task. Because the determination of the target number of parallel tasks combines the task input size and the minimum number of parallel tasks, the target number of parallel tasks matches the magnitude, computational complexity, and resource requirements of the computational task itself, while also fitting the network bandwidth capacity of each computing node. This ensures that the bandwidth resources of the computing nodes are fully utilized without bandwidth overload, guaranteeing the rationality and adaptability of the multiple parallel subtasks. At the same time, since different mapping methods have different task allocation ratios, using multiple mapping methods to execute computational tasks can avoid network bottlenecks, resource idleness, or communication congestion caused by unreasonable task partitioning and allocation strategies. This improves the parallel processing efficiency of multi-party secure computation tasks, achieves optimal matching between the parallelism of computational tasks and the network resources of nodes, and ultimately improves the computational efficiency of multi-party secure computation. Attached Figure Description
[0033] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0034] Figure 1 This is an application environment diagram of a task processing method in one embodiment;
[0035] Figure 2 This is a flowchart illustrating a task processing method in one embodiment;
[0036] Figure 3 This is a flowchart illustrating step 201 in one embodiment;
[0037] Figure 4 This is a flowchart illustrating step 203 in one embodiment;
[0038] Figure 5 This is a flowchart illustrating step 203 in another embodiment;
[0039] Figure 6 This is a flowchart illustrating the task processing method in another embodiment;
[0040] Figure 7 This is a structural block diagram of a task processing device in one embodiment;
[0041] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0042] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0043] In secure multi-party computation (MPC), the data for the computation task is typically divided into multiple task segments and distributed among the participating parties using different MPC techniques. Each participating party collaboratively executes secure protocols such as addition, multiplication, comparison, and transformation based on its assigned task segment to obtain the final computation result. These participating parties include different logical roles, such as data holders and computation participants. During the computation process, the participating parties need to exchange encrypted, privacy-preserving data over the network. Therefore, the performance of MPC applications depends on the number of communication rounds, the amount of data transmitted, and the network bandwidth utilization.
[0044] However, existing MPC technologies suffer from low network communication efficiency due to performance differences in communication modes among different participants, resulting in underutilization of network bandwidth and consequently, low computational efficiency. Therefore, this application proposes a task processing method to reduce task completion time and improve computational efficiency.
[0045] The task processing method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, the coordination node 102 communicates with the computing node 104 via a network. A data storage system can store the data that the computing node 104 needs to process. The data storage system can be integrated on the computing node 104 or placed in the cloud or on another network server. The coordination node 102 determines the target number of parallel tasks based on the task input size of the computing task to be executed and the minimum number of parallel tasks that each computing node can handle when the network bandwidth is saturated. The task input size is used to characterize at least one of the following: the magnitude, computational complexity, and resource requirements of the computing task. Based on the target number of parallel tasks, the coordination node 102 divides the computing task into multiple parallel subtasks. Based on multiple preset mapping methods and the target number of parallel tasks, it determines the number of executable tasks corresponding to each mapping method and allocates target parallel subtasks to each mapping method according to the number of executable tasks, so as to execute the computing task. Different mapping methods correspond to different task allocation ratios.
[0046] In one exemplary embodiment, such as Figure 2 As shown, a task processing method is provided, which is applied to Figure 1 Taking the coordination node in the example, the following steps are included:
[0047] Step 201: Determine the target number of parallel tasks based on the task input size of the computing task to be executed and the minimum number of parallel tasks for each computing node when the network bandwidth is saturated.
[0048] The task input scale characterizes at least one of the following: the magnitude of the computational task, its computational complexity, and its resource requirements. It should be noted that the task input scale is a quantitative description based on a quantitative analysis of the amount of data involved in the computation, the number of elements participating in the computation, and the resources required. Specifically, the magnitude of the computational task is a quantitative description of the scale of the task input data, the number of objects processed, and the overall size of the task. Computational complexity is a quantitative description of the difficulty of the task's computational logic, the total number of operations, and the intensity of computational resource consumption. Resource requirements refer to the amount of physical and / or logical resources needed to execute the task.
[0049] The minimum number of parallel tasks is the minimum number of parallel subtasks that must be allocated to a computing node when its network bandwidth utilization reaches a preset saturation threshold (e.g., 100% or close to the upper limit) to ensure that the node's bandwidth resources are fully utilized and can no longer accept additional tasks. It is understandable that if the number of parallel tasks is too low, the communication peaks of multiple tasks cannot be combined to fully utilize the bandwidth, resulting in insufficient actual throughput; conversely, if the number of parallel tasks is too high, additional aggregation overhead will occur during the task result merging stage, reducing overall execution efficiency. Therefore, it is necessary to determine the minimum number of parallel tasks for each computing node when the network bandwidth is saturated.
[0050] The target number of parallel tasks is the number of tasks actually executed in parallel, determined based on the computational tasks to be executed and the minimum number of parallel tasks per computing node. Different computational tasks correspond to different target numbers of parallel tasks.
[0051] In some embodiments, the target number of parallel tasks may be the optimal number of parallel tasks.
[0052] It should be noted that both the minimum number of parallel tasks and the target number of parallel tasks refer to the number of subtasks after the total task is broken down.
[0053] In this embodiment, the coordinating node can sum the minimum number of parallel tasks of each computing node when the network bandwidth is saturated to obtain the lower limit of the range of parallel execution tasks. Then, based on the task input scale and the preset subtask granularity, it determines the theoretical maximum number of parallel tasks that the computing task can be divided into. Finally, the larger value between the lower limit and the theoretical maximum number of parallel tasks is determined as the target number of parallel tasks.
[0054] Step 202: Divide the computation task into multiple parallel subtasks according to the target number of parallel tasks.
[0055] Parallel subtasks refer to dividing a computational task into several small, independently executable subtasks based on the target number of parallel tasks. Each subtask has approximately the same amount of data and no duplicate data.
[0056] It is understandable that the processing result of the computation task can be obtained by merging the processing results of each subtask.
[0057] In this embodiment, the coordinating node can divide the computing task into a number of data blocks of roughly equal size that are intended to be parallel tasks. The data, required resources, and processing objectives corresponding to each data block are determined as a subtask, and then multiple parallel subtasks are obtained.
[0058] Step 203: Based on the preset multiple mapping methods and the target number of parallel tasks, determine the number of executable tasks corresponding to each mapping method, and allocate target parallel subtasks to each mapping method according to the number of executable tasks to execute the computation tasks.
[0059] It's important to note that in MPC technology, the mapping between physical computing nodes and logical roles is typically fixed. However, different computing nodes have varying communication load capacities, leading to situations where some nodes are overloaded or others are idle. Therefore, logical roles can be decoupled from computing nodes, with different mapping methods used for different parallel tasks. This role-rotation mechanism can then evenly distribute the communication load, originally concentrated on certain computing nodes, across other nodes, thereby improving throughput.
[0060] In this embodiment, multiple mapping methods can be generated in advance based on the mapping relationship between logical roles and different physical entities, and the proportion of tasks that each logical role can handle in the total task volume. Different mapping methods correspond to different task allocation ratios. This allows for efficient task processing using multiple mapping methods.
[0061] The number of executable tasks refers to the maximum number of executable parallel subtasks that, under a certain preset mapping method, meet the task input scale of the computing task, the bandwidth utilization rate of the computing node, and do not exceed the target number of parallel tasks.
[0062] Among them, the target parallel subtask refers to the subtask selected from the multiple parallel subtasks and assigned to each mapping method.
[0063] In this embodiment, the coordinating node can first determine the upper limit of the number of tasks to be executed for each mapping method. Then, using the target number of parallel tasks as the global upper limit, it calculates the maximum number of tasks that the computing node can carry under each mapping method, i.e., the number of executable tasks. Next, based on the number of executable tasks corresponding to each mapping method, it selects the target subtasks that meet the mapping method from multiple parallel subtasks and assigns them to the mapping method. Then, it enables the computing nodes under each mapping method to execute the target subtasks in parallel.
[0064] In the above task processing method, the coordinating node first determines the target number of parallel tasks based on the task input size of the computing task to be executed and the minimum number of parallel tasks for each computing node when the network bandwidth is saturated. Then, based on the target number of parallel tasks, the computing task is divided into multiple parallel subtasks. Next, based on the preset multiple mapping methods and the target number of parallel tasks, the number of executable tasks corresponding to each mapping method is determined. Finally, target parallel subtasks are allocated to each mapping method according to the number of executable tasks to execute the computing task. Because the determination of the target number of parallel tasks combines the task input size and the minimum number of parallel tasks, the target number of parallel tasks matches the magnitude, computational complexity, and resource requirements of the computational task itself, while also fitting the network bandwidth capacity of each computing node. This ensures that the bandwidth resources of the computing nodes are fully utilized without bandwidth overload, guaranteeing the rationality and adaptability of the multiple parallel subtasks. At the same time, since different mapping methods have different task allocation ratios, using multiple mapping methods to execute computational tasks can avoid network bottlenecks, resource idleness, or communication congestion caused by unreasonable task partitioning and allocation strategies. This improves the parallel processing efficiency of multi-party secure computation tasks, achieves optimal matching between the parallelism of computational tasks and the network resources of nodes, and ultimately improves the computational efficiency of multi-party secure computation.
[0065] In one exemplary embodiment, such as Figure 3 As shown, this embodiment relates to the process by which a coordinating node determines the target number of parallel tasks based on the task input size of the computational task to be executed and the minimum number of parallel tasks for each computing node under network bandwidth saturation. Step 201 includes:
[0066] Step 301: Obtain the unit communication volume coefficient of the logical role corresponding to each computing node.
[0067] The communication volume coefficient includes the data transmission volume coefficient and data reception volume coefficient corresponding to the logical role under the unit task input scale.
[0068] The unit communication load coefficient is a quantified value used to quantify the communication load generated by a single parallel subtask in a certain logical role. The unit task input size refers to the amount of input data processed by a single standard subtask, which is a preset benchmark metric.
[0069] The transmit data volume coefficient refers to the total amount of uplink data transmitted by a logical role when processing a single standard parallel subtask with a unit task input size. The receive data volume coefficient refers to the total amount of downlink data received by a logical role when processing a single standard parallel subtask with a unit task input size.
[0070] In this embodiment, the coordinating node can obtain the total amount of data sent and received by each computing node under a unit task input scale when each computing node is in a different logical role from the historical execution log records of each computing node. Then, the ratio of the total amount of data sent to the number of parallel subtasks processed by the logical role is determined as the data sent coefficient, and the ratio of the total amount of data received to the number of parallel subtasks processed by the logical role is determined as the data received coefficient.
[0071] Step 302: Determine the transmission time for each computing node to process the unit task input scale based on the communication volume coefficient of each unit.
[0072] Transmission time refers to the time required for a computing node to transmit network communication data within its bandwidth limitations when processing a single standard parallel subtask of a unit task input size. Transmission time includes the transmission time for sending data and the transmission time for receiving data.
[0073] In this embodiment, for each computing node, the coordinating node can determine the transmission time for sending data when processing a unit task input scale by the ratio of the unit communication volume coefficient corresponding to the computing node to the upper limit of the uplink bandwidth corresponding to the computing node, and determine the transmission time for receiving data when processing a unit task input scale by the ratio of the unit communication volume coefficient corresponding to the computing node to the upper limit of the downlink bandwidth corresponding to the computing node.
[0074] Step 303: Determine the target number of parallel tasks based on the task input size, the minimum number of parallel tasks, and the transmission time.
[0075] In this embodiment, the coordinating node can sum the minimum number of parallel tasks and use the summation result as the lower limit of the number of parallel tasks. Then, it determines the upper limit of the number of parallel tasks based on the ratio of the task input size to the preset unit task input size. Next, within the range of the lower limit and the upper limit, combined with the transmission time, the number of parallel tasks that minimizes the overall transmission time is determined as the target number of parallel tasks.
[0076] In this embodiment, the coordinating section first obtains the unit communication volume coefficient of the logical role corresponding to each computing node. This communication volume coefficient includes the data transmission volume coefficient and data reception volume coefficient corresponding to the logical role under the unit task input scale. Then, based on each unit communication volume coefficient, the transmission time for each computing node to process a unit task input scale is determined. This can provide a standardized and quantifiable basis for transmission time calculation by accurately quantifying the bidirectional communication load of different logical roles under the unit task input scale. Then, based on the task input scale, the minimum number of parallel tasks, and the transmission time, the target number of parallel tasks is determined. This ensures that the target number of parallel tasks matches the scale, complexity, and resource requirements of the computing task itself, and also meets the bandwidth saturation constraints and transmission efficiency requirements of each node. Thus, the optimal matching between the parallelism of the computing task and the network resources of the nodes is achieved.
[0077] In an exemplary embodiment, this embodiment relates to the process by which a coordinating node determines the target number of parallel tasks based on the task input size, the minimum number of parallel tasks, and the transmission time. Step 303 above includes:
[0078] Input the task input size, the minimum number of parallel tasks, the transmission time, and the preset aggregation cost function into the preset time optimization model to obtain the target number of parallel tasks output by the time optimization model.
[0079] The time optimization model is a pre-defined mathematical programming optimization model. It takes the task input size, the minimum number of parallel tasks for each computing node, the transmission time of each computing node, and the aggregation cost function as input data, the number of parallel tasks as optimization variables, node task allocation constraints, task size adaptation constraints, and positive integer constraints as boundary conditions, and minimizing the aggregation cost function as the optimization objective. The model solves and outputs the target number of parallel tasks that satisfies all constraints and achieves the best overall system scheduling performance.
[0080] Among them, the aggregation cost function is a preset composite evaluation function used to comprehensively quantify the overall transmission time of each computing node, the load balancing degree between computing nodes, the bandwidth resource utilization efficiency, and the communication overhead.
[0081] In this embodiment, the time optimization model can be represented as t paral and t aggr,in:
[0082]
[0083]
[0084] Where l is the input size of the computation task, and m is the minimum number of parallel tasks. This represents the bandwidth utilization when using m tasks. This represents the minimum of the average transmit bandwidth utilization and the average receive bandwidth utilization across all nodes. Aggregation Time It is the time required to aggregate the results of all tasks. It is the execution time. It is the aggregation cost function.
[0085] In this embodiment, the coordinating node can take the task input size, the minimum number of parallel tasks for each computing node, the transmission time of each computing node, and the aggregation cost function as input data, substitute them into the time optimization model, solve for the result, and take the result as the target number of parallel tasks.
[0086] In this embodiment, the coordinating node inputs the task input scale, the minimum number of parallel tasks, the transmission time, and the preset aggregation cost function into a preset time optimization model. This allows the model to leverage the quantitative optimization mechanism of mathematical programming to comprehensively consider multiple optimization objectives, including the data processing scale requirements of the computational task itself, the minimum task allocation requirements under the bandwidth saturation constraints of each node, the differences in transmission efficiency between nodes, and the overall communication overhead and load balancing of the system. This avoids the one-sidedness and irrationality of traditional empirical or single-indicator methods for determining the number of parallel tasks. Furthermore, the model solution process, guided by minimizing the aggregation cost function, can automatically find the target number of parallel tasks that optimizes the overall transmission time and maximizes resource utilization while satisfying all constraints. This ensures the scientific rigor and adaptability of parallel subtask partitioning and node task allocation, thereby improving the parallel processing efficiency of multi-party secure computation tasks.
[0087] In one exemplary embodiment, such as Figure 4 As shown, this embodiment relates to the process by which a coordinating node determines the number of executable tasks corresponding to each mapping method based on multiple preset mapping methods and the target number of parallel tasks. Step 203 includes:
[0088] Step 401: For each mapping method, determine the task allocation ratio corresponding to the mapping method based on the task input scale and the bandwidth utilization of each computing node included in the mapping method.
[0089] The mapping method includes the mapping relationship between multiple computing nodes and the logical roles corresponding to each computing node. It is understandable that the logical role corresponding to the same computing node may be different in different mapping methods; therefore, the mapping relationships will also differ between different mapping methods.
[0090] The bandwidth utilization rate of each computing node refers to the ratio of the actual network communication bandwidth occupied by the computing node to its uplink / downlink bandwidth limit when the logical role bound to the computing node executes parallel subtasks under a certain mapping method. It is used to quantify the actual utilization of the node's network bandwidth resources. The bandwidth utilization rate includes both transmit bandwidth utilization and receive bandwidth utilization.
[0091] In this embodiment, the process of determining the task allocation ratio for each mapping method is similar. Taking one mapping method as an example, the process of determining the task allocation ratio corresponding to the mapping method will be described in an exemplary manner.
[0092] For example, the coordinating node can first determine the amount of tasks to be processed based on the task input scale, and then determine the bandwidth capacity margin of each computing node based on the bandwidth utilization rate and bandwidth utilization threshold of each computing node in the mapping method. Then, the bandwidth capacity margin is converted into the amount of executable tasks. Next, the sum of the executable tasks of each computing node is determined as the amount of executable tasks of the mapping method. Finally, the ratio of the amount of executable tasks of the mapping method to the amount of tasks to be processed is determined as the task allocation ratio corresponding to the mapping method.
[0093] Step 402: The product of the task allocation ratio and the target number of parallel tasks is used to determine the number of executable tasks corresponding to the mapping method.
[0094] In this embodiment, for each mapping method, the coordinating node can calculate the product of the task allocation ratio of the mapping method and the target number of parallel tasks to obtain the number of executable tasks corresponding to the mapping method, thereby obtaining the number of executable tasks corresponding to each mapping method in turn.
[0095] In this embodiment, for each mapping method, the coordinating node first determines the task allocation ratio corresponding to the mapping method based on the task input scale and the bandwidth utilization of each computing node included in the mapping method. This mapping method includes the mapping relationship between multiple computing nodes and the logical roles corresponding to each computing node, which enables the task allocation to be accurately matched with the bandwidth load status of the computing nodes, improving the accuracy of the task allocation ratio. Then, the product of the task allocation ratio and the target number of parallel tasks is determined as the number of executable tasks corresponding to the mapping method, which improves the adaptability between the number of executable tasks and the actual processing requirements of the computing tasks, thereby improving the rationality, balance and overall resource utilization efficiency of the mapping method in executing parallel computing tasks.
[0096] In one exemplary embodiment, such as Figure 5 As shown, this embodiment relates to the process by which the coordination node allocates target parallel subtasks to each mapping method according to the number of executable tasks. Step 203 above includes:
[0097] Step 501: Based on the load balancing strategy and the number of each executable task, establish a matching relationship between the mapping methods corresponding to each parallel subtask and the number of each executable task.
[0098] Among them, the load balancing strategy refers to the method in which, during the parallel scheduling of multi-party secure computing, the coordinating node reasonably distributes global parallel subtasks among various mapping methods and among computing nodes within the same mapping method, so that the task capacity of each mapping method matches the number of executable tasks.
[0099] It is understandable that each parallel subtask is matched with a mapping method.
[0100] In this embodiment, the coordinating node can allocate all parallel subtasks according to the load balancing strategy, with the number of executable tasks in each mapping method as the upper limit of allocation. All parallel subtasks are allocated proportionally and preferentially based on the size of the number of executable tasks in each mapping method, thereby allocating each parallel subtask to the corresponding mapping method and establishing a matching relationship between each parallel subtask and each mapping method.
[0101] Step 502: For each mapping method, according to the matching relationship corresponding to the mapping method, the target parallel subtask corresponding to the matching relationship is assigned to each computing node corresponding to the mapping method.
[0102] In this embodiment, taking each mapping method as an example, the coordinating node can assign the corresponding parallel subtasks in the matching relationship of the mapping method to the mapping method, so that each computing node included in the mapping method can receive each parallel subtask and execute the received parallel subtasks.
[0103] In this embodiment, the coordinating node first establishes a matching relationship between each parallel subtask and the mapping method corresponding to each number of executable tasks based on the load balancing strategy and the number of executable tasks. This ensures a high degree of compatibility between each parallel subtask and the mapping method, achieving load balancing among multiple mapping methods from the source of task allocation. Then, for each mapping method, the target parallel subtask corresponding to the matching relationship is quickly allocated to each computing node corresponding to the mapping method based on the matching relationship. This simplifies the task allocation execution process, improves allocation efficiency, and makes the computing load and network bandwidth utilization of the nodes more balanced, effectively reducing communication congestion and execution latency, thereby improving the overall parallel processing efficiency of multi-party secure computing tasks.
[0104] In one exemplary embodiment, such as Figure 6 As shown, this embodiment relates to the process by which the coordinating node determines the minimum number of parallel tasks. The method also includes:
[0105] Step 601: Obtain the average transmit bandwidth utilization and average receive bandwidth utilization of each computing node.
[0106] The average transmission bandwidth utilization rate refers to the ratio of the total amount of data actually transmitted by the computing node within the statistical period to the product of the computing node's uplink bandwidth limit and the duration of the statistical period. It is used to quantify the average uplink network bandwidth resource utilization of the computing node over a period of time.
[0107] The average receiving bandwidth utilization rate refers to the ratio of the total amount of data actually received by a computing node within a statistical period to the product of the computing node's uplink bandwidth limit and the duration of the statistical period. It is used to quantify the average occupancy of downlink network bandwidth resources by a computing node over a period of time.
[0108] In this embodiment, the coordinating node can obtain the total amount of data actually sent and the total amount of data actually received within the statistical period from the log records of each computing node according to the preset statistical period. Then, for each computing node, the maximum uplink bandwidth of the computing node is calculated by multiplying it by the statistical period. Next, the ratio between the total amount of data actually sent and the product is determined as the average transmission bandwidth utilization rate, and the ratio between the total amount of data actually received and the product is determined as the average reception bandwidth utilization rate.
[0109] Step 602: Determine the maximum value among the average transmit bandwidth utilization and the average receive bandwidth utilization.
[0110] It should be noted that, in multi-party secure computation scenarios, there is a significant asymmetry in the uplink and downlink communication load of computing nodes. By determining the maximum value, the bottleneck direction and the true constraint boundary of the bandwidth resource occupation of computing nodes within the statistical period can be accurately characterized. This avoids constraint deviations caused by judging based solely on the utilization rate of a single communication direction, thereby ensuring that the determined minimum number of parallel tasks simultaneously matches the secure uplink and downlink bandwidth carrying capacity of the nodes.
[0111] In this embodiment, the coordinating node can sort the average transmit bandwidth utilization and average receive bandwidth utilization of each computing node in order of magnitude, and determine the first one in the sorting result as the maximum value.
[0112] Step 603: Determine the minimum number of parallel tasks based on the maximum value.
[0113] In this embodiment, the coordinating node can compare the maximum value with a preset bandwidth security utilization threshold. For computing nodes whose maximum value reaches or exceeds the bandwidth security utilization threshold, the number of currently allocated tasks is directly determined as the minimum number of parallel tasks. For computing nodes whose maximum value is less than the bandwidth security utilization threshold, the calculation is performed based on the ratio of the current number of allocated tasks to the threshold and the maximum value. The result is then rounded down, and the larger value between the calculated result and the current number of allocated tasks is taken as the minimum number of parallel tasks for that node.
[0114] The process of determining the minimum number of parallel tasks can be expressed as follows:
[0115]
[0116]
[0117]
[0118] in, To minimize the number of parallel tasks, Let be the average transmission bandwidth utilization of the i-th computing node. Let be the average receiving bandwidth utilization of the i-th computing node, and n be the number of computing nodes.
[0119] In this embodiment, the coordinating node obtains the average transmit bandwidth utilization and average receive bandwidth utilization of each computing node, determines the maximum value among the average transmit bandwidth utilization and average receive bandwidth utilization, and then determines the minimum number of parallel tasks based on the maximum value. This can accurately locate the bottleneck direction in the uplink and downlink communication of the node, ensuring that the determined minimum number of parallel tasks can make full use of the node's bandwidth resources and avoid resource idleness. It can also effectively avoid problems such as uplink / downlink bandwidth overload and communication congestion caused by insufficient or excessive task allocation. This provides a unified and reliable bandwidth constraint basis for subsequent task allocation ratio determination, mapping method matching, and global load balancing, improving the rationality, security, and overall execution efficiency of parallel task scheduling.
[0120] To facilitate understanding by those skilled in the art, the task processing method provided in this application will be described in detail below. This method may include:
[0121] S1, obtain the average transmit bandwidth utilization and average receive bandwidth utilization of each computing node.
[0122] S2, determine the maximum value among the average transmit bandwidth utilization and average receive bandwidth utilization.
[0123] S3 determines the minimum number of parallel tasks per computing node when the network bandwidth is saturated, based on the maximum value.
[0124] S4, obtain the unit communication volume coefficient of the logical role corresponding to each computing node.
[0125] The communication volume coefficient includes the data transmission volume coefficient and data reception volume coefficient corresponding to the logical role under the unit task input scale.
[0126] S5 determines the transmission time for each computing node to process unit task input based on the communication volume coefficient of each unit.
[0127] S6. Input the task input size, the minimum number of parallel tasks, the transmission time, and the preset aggregation cost function into the preset time optimization model to obtain the target number of parallel tasks output by the time optimization model.
[0128] The task input scale is used to characterize at least one of the following: the magnitude of the computational task, the computational complexity, and the required resources.
[0129] S7 divides the computation task into multiple parallel subtasks based on the target number of parallel tasks.
[0130] S8 determines the task allocation ratio corresponding to each mapping method based on the task input size and the bandwidth utilization of each computing node included in the mapping method.
[0131] The mapping method includes the mapping relationship between multiple computing nodes and the logical roles corresponding to each computing node. Different mapping methods correspond to different task allocation ratios.
[0132] S9 determines the number of executable tasks corresponding to this mapping method by multiplying the task allocation ratio by the target number of parallel tasks.
[0133] S10: Based on the load balancing strategy and the number of each executable task, establish a matching relationship between each parallel subtask and the mapping method corresponding to the number of each executable task.
[0134] For each mapping method, S11 allocates the target parallel subtask corresponding to the mapping method to each computing node according to the matching relationship of the mapping method, so as to execute the computing task.
[0135] It should be noted that the descriptions in S1-S11 above can be found in the relevant descriptions in the above embodiments, and their effects are similar, so they will not be repeated here.
[0136] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0137] Based on the same inventive concept, this application also provides a task processing apparatus for implementing the task processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more task processing apparatus embodiments provided below can be found in the limitations of the task processing method described above, and will not be repeated here.
[0138] In one embodiment, such as Figure 7 As shown, a task processing apparatus is provided for coordinating nodes, comprising: a first determination module 701, a partitioning module 702, and an execution module 703, wherein:
[0139] The first determining module 701 is used to determine the target number of parallel tasks based on the task input size of the computing task to be executed and the minimum number of parallel tasks of each computing node under network bandwidth saturation. The task input size is used to characterize at least one of the following: the magnitude of the computing task, the computational complexity, and the required resources.
[0140] The partitioning module 702 is used to divide the computing task into multiple parallel subtasks according to the target number of parallel tasks;
[0141] The execution module 703 is used to determine the number of executable tasks corresponding to each mapping method based on the preset multiple mapping methods and the number of target parallel tasks, and to allocate target parallel subtasks to each mapping method according to the number of executable tasks to execute the computing tasks. The task allocation ratio is different for different mapping methods.
[0142] The task processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.
[0143] In one embodiment, the first determining module 701 includes:
[0144] The acquisition unit is used to acquire the unit communication volume coefficient of the logical role corresponding to each computing node. The communication volume coefficient includes the data transmission volume coefficient and the data reception volume coefficient of the logical role under the unit task input scale.
[0145] The first determining unit is used to determine the transmission time for each computing node to process a unit of task input based on the communication volume coefficient of each unit.
[0146] The second determining unit is used to determine the target number of parallel tasks based on the task input size, the minimum number of parallel tasks, and the transmission time.
[0147] The task processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.
[0148] In one embodiment, the second determining unit is specifically used for:
[0149] Input the task input size, the minimum number of parallel tasks, the transmission time, and the preset aggregation cost function into the preset time optimization model to obtain the target number of parallel tasks output by the time optimization model.
[0150] The task processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.
[0151] In one embodiment, the execution module 703 includes:
[0152] The third determining unit is used to determine the task allocation ratio corresponding to each mapping method based on the task input scale and the bandwidth utilization of each computing node included in the mapping method. The mapping method includes the mapping relationship between multiple computing nodes and the logical roles corresponding to each computing node.
[0153] The fourth determining unit is used to determine the number of executable tasks corresponding to the mapping method by multiplying the task allocation ratio by the target number of parallel tasks.
[0154] The task processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.
[0155] In one embodiment, the execution module 703 includes:
[0156] Establishment unit, used to establish the matching relationship between each parallel subtask and the mapping method corresponding to each number of executable tasks according to the load balancing strategy and the number of each executable task;
[0157] The allocation unit is used to allocate the target parallel subtasks corresponding to the mapping method to each computing node according to the matching relationship corresponding to the mapping method.
[0158] The task processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.
[0159] In one embodiment, the above-mentioned apparatus further includes:
[0160] The acquisition module is used to obtain the average transmit bandwidth utilization and average receive bandwidth utilization of each computing node;
[0161] The second determining module is used to determine the maximum value among the average transmit bandwidth utilization and the average receive bandwidth utilization;
[0162] The third determining module is used to determine the minimum number of parallel tasks based on the maximum value.
[0163] The task processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.
[0164] Each module in the aforementioned task processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.
[0165] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores execution data for computational tasks. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a task processing method.
[0166] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0167] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0168] The target number of parallel tasks is determined based on the task input size of the computing task to be executed and the minimum number of parallel tasks for each computing node under network bandwidth saturation. The task input size is used to characterize at least one of the following: the magnitude of the computing task, the computational complexity, and the required resources.
[0169] Based on the target number of parallel tasks, the computation task is divided into multiple parallel subtasks;
[0170] Based on the preset multiple mapping methods and the target number of parallel tasks, the number of executable tasks corresponding to each mapping method is determined, and target parallel subtasks are allocated to each mapping method according to the number of executable tasks to execute the computing tasks. The task allocation ratio is different for different mapping methods.
[0171] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0172] Obtain the unit communication volume coefficient of the logical role corresponding to each computing node. The communication volume coefficient includes the data transmission volume coefficient and data reception volume coefficient of the logical role under the unit task input scale.
[0173] Based on the communication volume coefficient of each unit, determine the transmission time for each computing node to process a unit of task input size;
[0174] The target number of parallel tasks is determined based on the task input size, the minimum number of parallel tasks, and the transmission time.
[0175] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0176] Input the task input size, the minimum number of parallel tasks, the transmission time, and the preset aggregation cost function into the preset time optimization model to obtain the target number of parallel tasks output by the time optimization model.
[0177] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0178] For each mapping method, the task allocation ratio corresponding to the mapping method is determined based on the task input scale and the bandwidth utilization of each computing node included in the mapping method. The mapping method includes the mapping relationship between multiple computing nodes and the logical roles corresponding to each computing node.
[0179] The product of the task allocation ratio and the target number of parallel tasks is used to determine the number of executable tasks corresponding to the mapping method.
[0180] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0181] Based on the load balancing strategy and the number of each executable task, establish a matching relationship between each parallel subtask and the mapping method corresponding to the number of each executable task;
[0182] For each mapping method, the target parallel subtasks corresponding to the mapping method are assigned to the computing nodes corresponding to the mapping method based on the matching relationship.
[0183] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0184] Obtain the average transmit bandwidth utilization and average receive bandwidth utilization of each computing node;
[0185] Determine the maximum value among the average transmit bandwidth utilization and average receive bandwidth utilization;
[0186] Determine the minimum number of parallel tasks based on the maximum value.
[0187] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:
[0188] The target number of parallel tasks is determined based on the task input size of the computing task to be executed and the minimum number of parallel tasks for each computing node under network bandwidth saturation. The task input size is used to characterize at least one of the following: the magnitude of the computing task, the computational complexity, and the required resources.
[0189] Based on the target number of parallel tasks, the computation task is divided into multiple parallel subtasks;
[0190] Based on the preset multiple mapping methods and the target number of parallel tasks, the number of executable tasks corresponding to each mapping method is determined, and target parallel subtasks are allocated to each mapping method according to the number of executable tasks to execute the computing tasks. The task allocation ratio is different for different mapping methods.
[0191] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0192] Obtain the unit communication volume coefficient of the logical role corresponding to each computing node. The communication volume coefficient includes the data transmission volume coefficient and data reception volume coefficient of the logical role under the unit task input scale.
[0193] Based on the communication volume coefficient of each unit, determine the transmission time for each computing node to process a unit of task input size;
[0194] The target number of parallel tasks is determined based on the task input size, the minimum number of parallel tasks, and the transmission time.
[0195] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0196] Input the task input size, the minimum number of parallel tasks, the transmission time, and the preset aggregation cost function into the preset time optimization model to obtain the target number of parallel tasks output by the time optimization model.
[0197] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0198] For each mapping method, the task allocation ratio corresponding to the mapping method is determined based on the task input scale and the bandwidth utilization of each computing node included in the mapping method. The mapping method includes the mapping relationship between multiple computing nodes and the logical roles corresponding to each computing node.
[0199] The product of the task allocation ratio and the target number of parallel tasks is used to determine the number of executable tasks corresponding to the mapping method.
[0200] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0201] Based on the load balancing strategy and the number of each executable task, establish a matching relationship between each parallel subtask and the mapping method corresponding to the number of each executable task;
[0202] For each mapping method, the target parallel subtasks corresponding to the mapping method are assigned to the computing nodes corresponding to the mapping method based on the matching relationship.
[0203] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0204] Obtain the average transmit bandwidth utilization and average receive bandwidth utilization of each computing node;
[0205] Determine the maximum value among the average transmit bandwidth utilization and average receive bandwidth utilization;
[0206] Determine the minimum number of parallel tasks based on the maximum value.
[0207] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:
[0208] The target number of parallel tasks is determined based on the task input size of the computing task to be executed and the minimum number of parallel tasks for each computing node under network bandwidth saturation. The task input size is used to characterize at least one of the following: the magnitude of the computing task, the computational complexity, and the required resources.
[0209] Based on the target number of parallel tasks, the computation task is divided into multiple parallel subtasks;
[0210] Based on the preset multiple mapping methods and the target number of parallel tasks, the number of executable tasks corresponding to each mapping method is determined, and target parallel subtasks are allocated to each mapping method according to the number of executable tasks to execute the computing tasks. The task allocation ratio is different for different mapping methods.
[0211] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0212] Obtain the unit communication volume coefficient of the logical role corresponding to each computing node. The communication volume coefficient includes the data transmission volume coefficient and data reception volume coefficient of the logical role under the unit task input scale.
[0213] Based on the communication volume coefficient of each unit, determine the transmission time for each computing node to process a unit of task input size;
[0214] The target number of parallel tasks is determined based on the task input size, the minimum number of parallel tasks, and the transmission time.
[0215] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0216] Input the task input size, the minimum number of parallel tasks, the transmission time, and the preset aggregation cost function into the preset time optimization model to obtain the target number of parallel tasks output by the time optimization model.
[0217] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0218] For each mapping method, the task allocation ratio corresponding to the mapping method is determined based on the task input scale and the bandwidth utilization of each computing node included in the mapping method. The mapping method includes the mapping relationship between multiple computing nodes and the logical roles corresponding to each computing node.
[0219] The product of the task allocation ratio and the target number of parallel tasks is used to determine the number of executable tasks corresponding to the mapping method.
[0220] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0221] Based on the load balancing strategy and the number of each executable task, establish a matching relationship between each parallel subtask and the mapping method corresponding to the number of each executable task;
[0222] For each mapping method, the target parallel subtasks corresponding to the mapping method are assigned to the computing nodes corresponding to the mapping method based on the matching relationship.
[0223] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0224] Obtain the average transmit bandwidth utilization and average receive bandwidth utilization of each computing node;
[0225] Determine the maximum value among the average transmit bandwidth utilization and average receive bandwidth utilization;
[0226] Determine the minimum number of parallel tasks based on the maximum value.
[0227] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.
[0228] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0229] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0230] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A task processing method, characterized in that, The method for coordinating nodes includes: The target number of parallel tasks is determined based on the task input size of the computing task to be executed and the minimum number of parallel tasks per computing node when the network bandwidth is saturated. The task input size is used to characterize at least one of the following: the magnitude, computational complexity, and resource requirements of the computing task. Based on the target number of parallel tasks, the computation task is divided into multiple parallel subtasks; Based on the preset multiple mapping methods and the target number of parallel tasks, the number of executable tasks corresponding to each mapping method is determined, and target parallel subtasks are allocated to each mapping method according to the number of executable tasks to execute the computing tasks. The task allocation ratios corresponding to different mapping methods are different.
2. The method according to claim 1, characterized in that, The determination of the target number of parallel tasks based on the task input size of the computational task to be executed and the minimum number of parallel tasks per computing node under network bandwidth saturation includes: Obtain the unit communication volume coefficient of the logical role corresponding to each computing node, wherein the communication volume coefficient includes the data transmission volume coefficient and the data reception volume coefficient corresponding to the logical role under the unit task input scale; Based on the unit communication volume coefficients, determine the transmission time for each computing node to process the unit task input scale; The target number of parallel tasks is determined based on the task input size, the minimum number of parallel tasks, and the transmission time.
3. The method according to claim 2, characterized in that, Determining the target number of parallel tasks based on the task input size, the minimum number of parallel tasks, and the transmission time includes: The task input size, the minimum number of each parallel task, the transmission time, and the preset aggregation cost function are input into a preset time optimization model to obtain the target number of parallel tasks output by the time optimization model.
4. The method according to any one of claims 1-3, characterized in that, The step of determining the number of executable tasks corresponding to each of the preset multiple mapping methods and the target number of parallel tasks includes: For each of the mapping methods, the task allocation ratio corresponding to the mapping method is determined based on the task input scale and the bandwidth utilization of each computing node included in the mapping method. The mapping method includes a mapping relationship between multiple computing nodes and the logical roles corresponding to each computing node. The product of the task allocation ratio and the target number of parallel tasks is used to determine the number of executable tasks corresponding to the mapping method.
5. The method according to any one of claims 1-3, characterized in that, The allocation of target parallel subtasks to each mapping method according to the number of each executable task includes: Based on the load balancing strategy and the number of each executable task, establish a matching relationship between each parallel subtask and the mapping method corresponding to each number of executable tasks; For each of the mapping methods, the target parallel subtasks corresponding to the matching relationship are allocated to each computing node corresponding to the mapping method according to the matching relationship of the mapping method.
6. The method according to claim 1, characterized in that, The method further includes: Obtain the average transmit bandwidth utilization and average receive bandwidth utilization of each computing node; Determine the maximum value among the average transmit bandwidth utilization and average receive bandwidth utilization; The minimum number of parallel tasks is determined based on the maximum value.
7. A task processing device, characterized in that, For coordinating nodes, the device includes: The first determining module is used to determine the target number of parallel tasks based on the task input scale of the computing task to be executed and the minimum number of parallel tasks of each computing node under network bandwidth saturation. The task input scale is used to characterize at least one of the following: the magnitude, computational complexity, and resource requirements of the computing task. A partitioning module is used to divide the computing task into multiple parallel subtasks based on the target number of parallel tasks. The execution module is used to determine the number of executable tasks corresponding to each of the preset multiple mapping methods and the target number of parallel tasks, and to allocate target parallel subtasks to each of the mapping methods according to the number of executable tasks to execute the computing tasks. The task allocation ratios are different for different mapping methods.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.