Task execution method, computer equipment, storage medium and program product

Abstract

The invention discloses a task execution method, computer equipment, a storage medium and a program product, and relates to the technical field of resource allocation, and a target request comprises task information and scheduling configuration information of multiple dimensions. After a target request is received, device allocation information is determined according to scheduling configuration information of a device parallel dimension, task information and resource device use information, and then resource allocation information is determined according to the task information, the device allocation information, and scheduling configuration information of a token prediction parallel dimension and a stage parallel dimension. And finally, scheduling configuration information, resource allocation information and task information of the parallel dimension are predicted according to the token, and an execution result is obtained and fed back to the target equipment. In the task execution process, equipment and resource allocation is performed from the scheduling configuration information of each dimension, and the task information is further combined during task execution, so that the influence of multiple dimensions on resource scheduling is considered, and the problem of low flexibility in the resource scheduling process is solved.

Description

Technical Field

[0001] The present application relates to the technical field of resource allocation, and in particular to a task execution method, a computer device, a storage medium, and a program product. Background Art

[0002] In recent years, the size of computer clusters has grown rapidly, and at the same time, the size of model parameters has also grown. Therefore, in order to ensure the task processing efficiency of the model, a pipeline parallel processing method is generally adopted to divide the processing stage of the model into multiple stages, and run multiple stages simultaneously in the computer cluster to improve processing efficiency.

[0003] However, simply allocating resources by dividing them into different stages has low flexibility. Summary of the invention

[0004] The present application provides a task execution method, apparatus, computer equipment, storage medium and program product to at least solve the problem of low resource scheduling flexibility in related technologies.

[0005] This application provides a task execution method, including: receiving a target request sent by a target device, wherein the target request includes a scheduling strategy and task information, the scheduling strategy includes scheduling configuration information of multiple dimensions, and the multiple dimensions include a device parallel dimension, a stage parallel dimension, and a token prediction parallel dimension; Determine device allocation information based on the scheduling configuration information of the device parallel dimension, task information, and pre-acquired resource device usage information; When the scheduling configuration information of the stage parallel dimension is used to indicate to execute the task in stages, the resource allocation information is determined according to the task information, the device allocation information, the scheduling configuration information of the token prediction parallel dimension, and the scheduling configuration information of the stage parallel dimension; According to the scheduling configuration information, resource allocation information, and task information of the token-predicted parallel dimension, the execution result is obtained and fed back to the target device.

[0006] The present application also provides a task execution device, comprising: A receiving module, configured to receive a target request sent by a target device, wherein the target request includes a scheduling strategy and task information, the scheduling strategy includes scheduling configuration information of multiple dimensions, and the multiple dimensions include a device parallel dimension, a stage parallel dimension, and a token prediction parallel dimension; A determination module, used to determine device allocation information according to the scheduling configuration information of the device parallel dimension, task information and pre-acquired resource device usage information; determine resource allocation information according to the task information, device allocation information, scheduling configuration information of the token prediction parallel dimension, and scheduling configuration information of the stage parallel dimension; The acquisition module is used to predict the scheduling configuration information, resource allocation information, and task information of the parallel dimension according to the token, obtain the execution result and feed it back to the target device.

[0007] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned task execution methods when executing the computer program.

[0008] The present application also provides a computer-readable storage medium, in which a computer program is stored, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned task execution methods are implemented.

[0009] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned task execution methods when executed by a processor.

[0010] Through the present application, in the process of resource scheduling, first, the device allocation information is determined based on the scheduling configuration information of the device parallel dimension, task information, and resource device usage information. Then, the resource allocation information can be determined based on the task information, the device allocation information determined in the previous step, the scheduling configuration information of the token prediction parallel dimension and the stage parallel dimension. In this way, in the resource scheduling process, by combining the scheduling configuration information of multiple dimensions of the device parallel dimension, the token prediction parallel dimension, and the stage parallel dimension, the problem of low flexibility caused by scheduling resources in a single dimension is avoided, and the flexibility of resource scheduling can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0012] Figure 1 A schematic diagram of the architecture of a computer cluster provided in an embodiment of the present application; Figure 2 A flowchart of a task execution method provided in an embodiment of the present application; Figure 3 A schematic diagram of a hierarchical scheduling framework provided in an embodiment of the present application; Figure 4 A functional schematic diagram of a central scheduler provided in an embodiment of the present application; Figure 5 A functional schematic diagram of a node scheduler provided in an embodiment of the present application; Figure 6A schematic diagram of a flow chart of a computing resource allocation operation provided in an embodiment of the present application; Figure 7 A schematic diagram of a flow chart of a memory resource allocation operation provided in an embodiment of the present application; Figure 8 A functional schematic diagram of a request scheduler provided in an embodiment of the present application; Fig. 9 A schematic diagram of a hierarchical scheduling strategy provided in an embodiment of the present application; Fig.10 A schematic diagram of another hierarchical scheduling framework provided in an embodiment of the present application; Fig.11 A schematic diagram of the structure of a task execution device provided in an embodiment of the present application; Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0014] It should be noted that, in the description of this application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0015] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0016] The embodiment of the present application provides a computer cluster, such as Figure 1As shown, the computer cluster may include multiple nodes, each of which may include a central processing unit (CPU) and a graphics processing unit (GPU). Among them, the node may be a bare metal server (Bare Metal Server). The central processing unit is used to parse the received request and perform corresponding resource scheduling operations. The graphics processor can be used as a resource device to load the model and use the model to perform tasks. For example, the inference result can be obtained after performing the inference task using the inference model. The client can communicate with the first node (any node) in the computer cluster, and then the first node determines whether to perform the task corresponding to the request from the client by itself according to the pre-set request processing mechanism, or send the request to other nodes, and the other nodes perform the task corresponding to the request from the client. Any node in the computer cluster can perform the task execution method provided in the embodiment of the present application, or multiple nodes in the computer cluster can cooperate with each other to perform the task execution method provided in the embodiment of the present application.

[0017] In the field of inference model technology, users can input task content (for example, a paragraph, a picture, etc.) on the client, and the client can generate a request based on the user input content and send it to the computer cluster. The computer cluster will execute the task and then feedback the result to the client. Since the execution of inference tasks requires a large amount of computing resources, in order to ensure the execution efficiency of inference tasks, the resources in the computer cluster can be scheduled in real time.

[0018] The embodiment of the present application provides a task execution method, such as Figure 2 As shown, the task execution method may include the following steps: Step S201, receiving a target request sent by a target device.

[0019] The target device may be a client or a first node. The target request may include a scheduling strategy and task information. The scheduling strategy may include scheduling configuration information of multiple dimensions, including a device parallel dimension, a stage parallel dimension, and a token prediction parallel dimension.

[0020] The scheduling configuration information of the device parallel dimension may include indication information of whether to perform distributed reasoning, and / or include the number of nodes, the number of devices, etc.

[0021] The scheduling configuration information of the stage parallel dimension may include information indicating whether to execute the task in stages, or, if the task is to be executed in stages, may also include resource allocation ratios of multiple stages. For example, in an inference task, a prefill stage and a decode stage may generally be included, and the scheduling configuration information of the stage parallel dimension may include a PD separation ratio, that is, the resource allocation ratios corresponding to the prefill stage and the decode stage, respectively.

[0022] The scheduling configuration information of the token prediction parallel dimension may include indication information of whether to perform multi-token prediction (Multi-Token Prediction, MTP).

[0023] The task information may include model parameters, task content, input length, task type, model identification, etc. For example, the model parameters may include model size and accuracy. The task content may be text content, pictures, or audio content. The input length may be the length of the task content. For example, in an inference task, the task type may include a long text type or a short text type. The input length of the task content of the long text type is greater than a preset length threshold, and the input length of the task content of the short text type is less than or equal to the preset length threshold.

[0024] Specifically, the user can select the corresponding model according to his / her needs in the client, and input the corresponding task content and the scheduling strategy. In this way, the client can generate a target request according to the model selected by the user, the input task content and the scheduling strategy, and send it to the first node in the computer cluster. The first node can parse the target request to obtain the task information and the scheduling strategy, or, according to the preset request processing mechanism, send the target request to other nodes, and the other nodes can parse the target request to obtain the task information and the scheduling strategy.

[0025] Step S202, determining device allocation information according to the scheduling configuration information of the device parallel dimension, the task information and the pre-acquired resource device usage information.

[0026] The resource device usage information may include the remaining resource amounts of multiple resource devices.

[0027] Specifically, the second node can update the resource usage of each resource device deployed on each node corresponding to the user in real time, that is, the resource device usage information can be obtained in real time. The second node can be any node in the computer cluster. For example, it can be the first node mentioned above, or it can be a node that receives the target request sent by the first node.

[0028] The second node can determine whether to perform distributed reasoning based on the indication information of distributed reasoning included in the scheduling configuration information of the device parallel dimension. If not, a resource to be used can be selected from multiple resource devices based on the model parameters and resource device usage information included in the task information. If yes, multiple resource devices to be used can be selected from multiple resource devices based on the model parameters and resource device usage information included in the task information, and the identification information of the resource devices to be used and the identification information of the node where each resource device to be used is located can be determined. Among them, the identification information of all the determined resource devices to be used and the identification information of the node together constitute the device allocation information.

[0029] Alternatively, the second node may determine whether to perform distributed reasoning based on the number of devices and the number of nodes included in the scheduling configuration information of the device parallel dimension. For example, when it is determined that the number of devices and the number of nodes are both the first preset value (for example, it may be 1), it is determined to perform distributed reasoning, or when it is determined that the number of devices or the number of nodes is not the first preset value, it is determined not to perform distributed reasoning. Furthermore, the second node may select a number of resource devices to be used that is equal to the number of devices from multiple resource devices in a corresponding manner (distributed reasoning or non-distributed reasoning) based on the number of devices and the number of nodes, the model parameters included in the task information, and the resource device usage information.

[0030] Step S203, determining resource allocation information according to the task information, the device allocation information, the scheduling configuration information of the token prediction parallel dimension, and the scheduling configuration information of the stage parallel dimension.

[0031] Specifically, the second node may determine whether the scheduling configuration information of the stage parallel dimension (specifically, the indication information of the staged execution task included therein) is used to indicate the staged execution task, and perform the resource allocation operation corresponding to the scheduling configuration information of the stage parallel dimension. Specifically, it can be divided into the following two cases: Case 1 may include the following steps: Step 1: When the scheduling configuration information of the stage parallel dimension is used to indicate to execute the task in stages, the resource allocation ratios of multiple stages are obtained.

[0032] Specifically, the second node may determine whether the scheduling configuration information of the stage parallel dimension includes resource allocation ratios of multiple stages. When it is determined that the scheduling configuration information of the stage parallel dimension includes resource allocation ratios of multiple stages, the resource allocation ratio of each stage may be extracted therefrom. Alternatively, when it is determined that the scheduling configuration information of the stage parallel dimension does not include resource allocation ratios, the resource allocation ratio of each stage corresponding to the task type may be obtained according to the task type in the task information.

[0033] For example, in an inference task, the multiple stages include a pre-filling stage and a decoding stage, and the task type is a long text type or a short text type. When the task type is a long text type, the resource allocation ratio of the pre-filling stage is greater than the resource allocation ratio of the decoding stage, or, when the task type is a short text type, the resource allocation ratio of the pre-filling stage is less than the resource allocation ratio of the decoding stage.

[0034] Step 2: Determine resource allocation information based on task information, device allocation information, resource allocation ratio of each stage, and scheduling configuration information of token prediction parallel dimension.

[0035] Resources may include multiple different types of resources, such as computing resources, memory resources, communication resources, etc. Accordingly, the second node may select multiple elements corresponding to the target resource type from the task information, device allocation information, resource allocation ratio of each stage, scheduling configuration information of token prediction parallel dimension, and scheduling configuration information of stage parallel dimension according to the allocation method of the target resource type, and perform allocation operation on the resources of the target resource type according to the multiple elements corresponding to the target resource type to determine the resource allocation information of the target resource type. The target resource type is any resource type.

[0036] For example, the allocation of different resources can be determined as follows: First, computing resources: Determine the computing resource allocation information based on the device allocation information and the resource allocation ratio of each stage.

[0037] Step 1: Count the first quantity of identification information of resource devices to be used included in the device allocation information.

[0038] Step 2: Determine the number of resource devices to be used in each stage according to the first number and the resource allocation ratio in each stage.

[0039] Step 3: Select resource devices to be used whose number is equal to the number of resource devices to be used in the first stage from the resource devices to be used as devices for executing the tasks in the first stage.

[0040] The first stage is any one of the multiple stages.

[0041] Step 4: Determine the allocation information of computing resources together with the identification information and the number of resource devices to be used in each stage.

[0042] Specifically, the second node may count the first number of identification information of the resource devices to be used included in the device allocation information, that is, count the first number of the resource devices to be used.

[0043] Further, when the resource allocation ratio is a percentage, the product of the first number and the resource allocation ratio of the first stage can be used to determine the number of resource devices to be used in the first stage, where the first stage is any one of the multiple stages. Then, the second node can select the same number of resource devices to be used as the number of resource devices to be used in the first stage from the resource devices to be used determined in step S202 as the devices for executing the tasks of the first stage. Finally, the second node can jointly determine the identification information and the number of resource devices to be used in each stage as the allocation information of the computing resources.

[0044] After determining the allocation information of computing resources, the second node can also perform initialization operations on the resource devices to be used. For example, in the inference task, multiple GPUs can be initialized as Prefill GPUs and the remaining GPUs as Decode GPUs according to the number of resource devices to be used in each stage. The initialization operation can include operations of marking GPUs, setting the GPU deployment environment, and deploying the corresponding model on the GPU.

[0045] In some optional implementations, in the process of selecting the equipment for the task of the first stage, the second node can determine the number of resource devices to be used on each node based on the identification information of each resource device to be used and the identification information of the node where it is located, sort each node according to the number of resource devices to be used on each node, and select resource devices to be used from one or more nodes that are equal to the number of resource devices to be used in the first stage as the equipment for performing the task of the first stage according to the sorting of each node. For example, the more nodes there are, the higher they are ranked, and resource devices to be used are selected from each node in turn according to the sorting from front to back until the number of resource devices to be used selected is equal to the number of resource devices to be used in the first stage, then the operation is stopped to determine the equipment for performing the task of the first stage. In this way, the probability that the resources to be used for performing the tasks of the same stage are located on the same node increases, which can reduce the number of cross-node communications and thus reduce the problem of resource waste.

[0046] Second, memory resources: Determine the allocation information of memory resources based on the scheduling configuration information of the token prediction parallel dimension, task information, and resource allocation ratio of each stage.

[0047] Step 1: Determine the amount of memory resources to be used based on the scheduling configuration information and model parameters of the token prediction parallel dimension.

[0048] Step 2: Determine the amount of memory resources to be used in each stage according to the amount of memory resources to be used and the resource allocation ratio of each stage.

[0049] Step 3: The amount of memory resources to be used in each stage is collectively determined as memory resource allocation information.

[0050] Specifically, the second node may determine whether to perform a multi-token prediction operation based on the indication information of the multi-token prediction included in the scheduling configuration information of the token prediction parallel dimension, and when the scheduling configuration information of the token prediction parallel dimension is used to indicate that a multi-token prediction is to be performed, the amount of memory resources to be used is determined based on the first preset multiple and the model parameters. Alternatively, when the scheduling configuration information of the token prediction parallel dimension is used to indicate that a multi-token prediction is not to be performed, the amount of memory resources to be used is determined based on the second preset multiple and the model parameters, and the first preset multiple is greater than the second preset multiple. For example, the first preset multiple may be 2, and the second preset multiple may be 1. Since multi-token prediction requires a secondary loading model operation in order to start Dual Pipe for bidirectional pipeline parallelism, more memory resources can be allocated in the memory resource allocation stage.

[0051] After determining the allocation information of memory resources, the second node can divide memory resources of the same size as the amount of memory resources to be used from its own memory resources or from the memory resources of all nodes where the resource devices to be used are located, and maintain the memory resources corresponding to different stages according to the amount of memory resources to be used in each stage. For example, in the inference task, the Prefill KV Cache and the Decode KV Cache are maintained separately.

[0052] After determining the allocation information of computing resources and memory resources, the second node can obtain the pre-trained model according to the model identifier, and load the pre-trained model onto the resource device to be used. For example, in an inference task, when multiple stages include a pre-filling stage and a decoding stage, the second node can load the pre-filling model onto the resource device to be used corresponding to the pre-filling stage, and load the decoding model onto the resource device to be used corresponding to the decoding stage.

[0053] Third, communication resources: Determine the allocation information of communication resources based on the device allocation information and the resource allocation ratio of each stage.

[0054] Step 1: Count the second quantity of identification information of the node where the resource device to be used is located included in the device allocation information.

[0055] Step 2: Determine the communication mode of each resource device to be used according to the first quantity and the second quantity.

[0056] Step 3: The communication mode of each resource device to be used and the resource allocation ratio in each stage are jointly determined as the allocation information of the communication resources.

[0057] Specifically, the second node may count the second number of identification information of the node where the resource device to be used is located included in the device allocation information. When it is determined that the first number or the second number is a first preset value (for example, the first preset value is 1), the communication mode of each resource device to be used is determined as an intra-node communication mode. Alternatively, when it is determined that the first number and the second number are not the first preset value, the communication mode of each resource to be used is determined as a cross-node communication mode. For example, the intra-node communication mode may be an NV Link communication mode, and the cross-node communication mode may be an InfiniBand (abbreviated as IB) communication mode. Finally, the second node may jointly determine the communication mode of each resource device to be used and the resource allocation ratio of each stage as the allocation information of communication resources.

[0058] Alternatively, the second node may also determine the total communication resource amount to be used of each node according to the number of resource devices to be used on each node and the preset communication resource amount. The second node may jointly determine the total communication resource amount to be used of each node, the resource allocation ratio of each stage, and the communication mode of each resource device to be used as the allocation mode of communication resources.

[0059] In some optional implementations, for any node where a resource device to be used is located (the third node is used as an example for illustration), if the third node includes resource devices to be used that perform tasks in the pre-filling stage and the decoding stage, the amount of communication resources in each stage on the third node can be obtained based on the third number of resource devices to be used that perform tasks in the pre-filling stage on the third node, the fourth number of resource devices to be used that perform tasks in the decoding stage, and the resource allocation ratio of each stage, as well as the total amount of communication resources to be used on the third node. The second node can jointly determine the amount of communication resources in each stage on each node and the communication mode of each resource device to be used as the allocation information of the communication resources. In this way, more detailed allocation information can be directly determined, the allocation speed of communication resources can be accelerated, and the flexibility of communication resource allocation can be improved.

[0060] After determining the allocation information of the communication resources, the second node can set the communication mode of each resource device to be used according to the communication mode of each resource device to be used included in the allocation information of the communication resources, and perform the allocation operation on the communication resources according to the resource allocation ratio of each stage included in the allocation information of the communication resources, so that each resource device to be used can share the parameter information in the model after communicating with other resource devices to be used in the process of loading the model to execute the task. Specifically, the second node can determine the total communication resource amount to be used of each node according to the number of resource devices to be used on each node and the preset communication resource amount, and then determine the communication resource amount to be used in each stage on each node according to the resource allocation ratio of each stage and the total communication resource amount to be used, and allocate the communication resources where each resource device to be used is located according to the resource allocation ratio of each stage. For example, in the inference task, the KV Cache communicator is initialized according to the PD allocation ratio, that is, initialized to the communication resources for the pre-filling stage and the communication resources for the decoding stage.

[0061] In some optional embodiments, the second node can determine whether the identification information of all nodes where the resource devices to be used are located is consistent. If so, it can be determined that the communication mode of the resource devices to be used is an intra-node communication mode. If not, it can be determined that each node includes only one resource device to be used. If so, it can be determined that the communication mode of the resource devices to be used on the node is a cross-node communication mode. If not, it can be determined that the communication mode of the resource devices to be used on the node includes an intra-node communication mode and a cross-node communication mode.

[0062] Case 2: When the scheduling configuration information of the stage parallel dimension is used to indicate that the task is not to be executed in stages, the second node can directly obtain the target model (for example, in the inference task, the target model can be a model including a pre-filling function and a decoding function) according to the model identifier included in the task information, and then load a target model on each resource device to be used, or load the target model on all resource devices to be used in a distributed manner. In addition, the second node can determine the amount of memory resources to be used according to the scheduling configuration information and model parameters of the token prediction parallel dimension (for specific processing, refer to the above case 1). The second node can divide out memory resources of the same size as the amount of memory resources to be used from its own memory resources, or from the memory resources of the nodes where all resource devices to be used are located, and initialize it as a KV Cache. In addition, the second node can count the first number of identification information of the resource devices to be used included in the device allocation information, and the second number of identification information of the nodes where the resource devices to be used are located included in the device allocation information, and determine the communication mode of each resource device to be used according to the first number and the second number. Finally, the communication mode of each resource device to be used can be jointly determined as the allocation information of the communication resources. In this way, without executing tasks in stages, resource allocation operations can be performed based on the token parallel dimension and the device allocation information determined by the device parallel dimension, which is highly flexible.

[0063] The computing resource allocation information, memory resource allocation information, and communication resource allocation information determined through the above operations together constitute the resource allocation information.

[0064] Step S204, obtaining the execution result and feeding it back to the target device according to the scheduling configuration information, resource allocation information, and task information of the token prediction parallel dimension.

[0065] Specifically, in addition to improving the task execution efficiency from the resource allocation dimension, the second node can also improve the task execution efficiency from the task division dimension. For example, when the target request is an inference request, executing the task corresponding to the target request may include: Step 1: when determining to divide the task content into blocks according to the input length and the preset pre-filling length, the block size is determined according to the input length and the number of resource devices to be used in the pre-filling stage.

[0066] Step 2: Divide the task content into multiple blocks according to the block size.

[0067] Step three, according to the identification information of the resource device to be used in the pre-filling stage, a block is allocated to a resource device to be used in the pre-filling stage to perform pre-filling processing on the block to obtain a pre-filling processing result corresponding to the block.

[0068] Step 4: Obtain a decoding model corresponding to the scheduling configuration information of the token prediction parallel dimension.

[0069] Step five, according to the identification information of the resource device to be used in the decoding stage, load the decoding model on the resource device to be used in the decoding stage, and use the decoding model to decode the pre-filled processing result corresponding to each block to obtain the execution result.

[0070] Specifically, the second node can determine whether the input length is greater than the preset prefill length (PrefillLength). If not, the task content can be determined as a chunk. If yes, it is determined that a chunk operation needs to be performed, and the ratio of the input length to the number of devices of the resource device to be used in the prefilling stage can be determined as the chunk size. Furthermore, the second node can chunk the task content according to the chunk size to obtain multiple chunks. The second node can allocate a chunk to a resource device to be used in the prefilling stage according to the identification information of the resource device to be used in the prefilling stage. In this way, the resource device to be used can perform a prefilling operation according to the loaded prefilling model and obtain the prefilling processing result corresponding to each chunk. When it is determined not to perform multi-token prediction according to the scheduling configuration information of the parallel dimension of token prediction, the second node can use the decoding model loaded on the resource device to be used in the decoding stage to process the prefilling processing result corresponding to each chunk and obtain the execution result. Alternatively, when it is determined to perform multi-token prediction according to the scheduling configuration information of the parallel dimension of token prediction, the second node can use the current training method to train multiple decoding models in real time, and deploy each decoding model to a resource device to be used in the decoding stage, and process the pre-filled processing results corresponding to each block to obtain the execution result. Finally, when the second node is the first node, the second node can send the execution result to the client, or, when the second node is not the first node, the second node can send the execution result to the first node, and the first node will feed back to the client.

[0071] Since multi-token prediction requires real-time training of multiple decoding models, this solution also obtains the corresponding decoding model based on the parallel dimension of the token, which is more flexible. In addition, in the case of multi-token prediction, it can meet the needs of decoding multiple words at the same time, improve decoding efficiency, and thus improve task execution efficiency.

[0072] In the process of executing the task, the second node can use the memory resources of the pre-filling stage to cache the model weights (for example, embedding and head) shared by the resource devices to be used in the pre-filling stage, and use the memory resources of the decoding stage to cache the model weights shared by the resource devices to be used in the decoding stage, so as to execute the tasks of each stage in parallel. In addition, the second node can use the communication resources of the pre-filling stage and the communication method of the resource devices to be used in the pre-filling stage to share the model weights in the memory resource cache of the pre-filling stage with other resource devices to be used in the pre-filling stage, and use the communication resources of the decoding stage and the communication method of the resource devices to be used in the decoding stage to share the model weights in the memory resource cache of the decoding stage with other resource devices to be used in the decoding stage.

[0073] In some optional implementations, in order to further improve the task execution efficiency, the second node can count the task execution efficiency during each task execution process, and record the task information, adjustment strategy, and task execution efficiency of each task, so that the following steps can be periodically performed: Step 1: Obtain the task execution efficiency corresponding to multiple tasks, as well as the task type and scheduling strategy corresponding to each task.

[0074] Step 2: Determine the scheduling strategy corresponding to the task with the highest execution efficiency among the tasks of the target task type as the scheduling strategy corresponding to the target task type.

[0075] The target task type is any task type.

[0076] Step three: Send each task type and the optimal scheduling strategy corresponding to each task type to the target device to instruct the target object to adjust the scheduling strategy.

[0077] The target object may be a user or a client where the user is located. The task corresponding to the target request may be one of the multiple tasks mentioned above.

[0078] In this way, when the target object needs to execute a task of the target task type, it can input the optimal scheduling strategy corresponding to the target task type, which can greatly improve the task execution efficiency.

[0079] In some optional implementations, in order to further enhance the flexibility of resource allocation, the second node may perform the following steps: Step 1: Monitor the processing progress of the pre-filling phase.

[0080] Step 2: When the monitored processing progress is equal to the preset processing progress, the resource ratio adjustment method corresponding to the preset processing progress and the latest task execution information are obtained.

[0081] The preset processing progress may include multiple ones. The resource ratio adjustment method includes the percentage of increase or decrease in each stage. The latest task execution information may include information of blocks that have not completed pre-filling processing and information of pre-filling processing results that have not completed decoding processing.

[0082] Step three: According to the resource ratio adjustment method corresponding to the preset processing progress, the resource allocation ratio of each stage is adjusted to obtain the updated resource allocation ratio of each stage.

[0083] Step 4: reallocate each type of resource according to the updated resource allocation ratio in each stage to obtain the resource allocation amount of each type in each stage.

[0084] Among them, the resource allocation amount of each stage corresponding to the computing resources is the number of resource devices to be used in each stage, the resource allocation amount of each stage corresponding to the memory resources is the amount of memory resources to be used in each stage, and the resource allocation amount of each stage corresponding to the communication resources is the amount of communication resources to be used in each stage.

[0085] Step 5: Continue to execute the task according to the latest task execution information and the resource allocation amount of each type at each stage to obtain the execution result.

[0086] Since in the process of executing the reasoning task, the task of the pre-filling stage is generally executed first, after obtaining some pre-processing results, the task of the pre-filling stage and the task of the decoding stage can be executed in parallel. As the task execution progresses, the resources required for the task of the pre-filling stage are getting less and less, and the resources required for the task of the decoding stage are getting more and more. Therefore, by real-time monitoring of the processing progress of the pre-filling stage, when the processing progress of the pre-filling stage reaches the preset processing progress, the resource allocation ratio of each stage can be adjusted according to the corresponding resource ratio adjustment method (for example, the resource allocation ratio of the pre-filling stage is reduced, and the resource allocation ratio of the decoding stage is increased), and the resource allocation amount of each stage under each type of resource is re-determined according to the re-determined resource allocation ratio of each stage. At the same time, the second node can use the resource allocation amount of each stage corresponding to each type to continue to execute the tasks of each stage. In this way, in the process of executing the task, the resources are scheduled in real time according to the actual load, which can improve the efficiency of task execution and have high flexibility.

[0087] In the task execution method in the embodiment of the present application, during the resource scheduling process, the device allocation information is first determined based on the scheduling configuration information of the device parallel dimension, the task information, and the resource device usage information. Then, the resource allocation information can be determined based on the task information, the device allocation information determined in the previous step, the scheduling configuration information of the token prediction parallel dimension, and the stage parallel dimension. In this way, during the resource scheduling process, by combining the scheduling configuration information of multiple dimensions of the device parallel dimension, the token prediction parallel dimension, and the stage parallel dimension, the problem of low flexibility caused by scheduling resources in a single dimension is avoided, and the flexibility of resource scheduling can be improved.

[0088] Moreover, when the related technologies allocate resources by dividing the stages, the resource ratio is usually fixed. However, in fact, when executing certain tasks, the stage division is not very useful for improving the efficiency of task execution, but will reduce the efficiency of task execution. That is, the stage division method is used to schedule resources and execute tasks in any case, which is less flexible. Therefore, this solution will only use the scheduling configuration information of the stage parallel dimension to determine the resource allocation information when executing tasks in stages, and this solution can obtain the corresponding resource allocation ratios of multiple stages according to the task type, that is, the task type is taken into account in resource scheduling, which is more flexible.

[0089] At present, some schemes use a preset block size to block the task content, while this scheme determines whether to block according to the actual situation. When it is determined to block, the block size is determined according to the number of resource devices to be used and the input length. In the blocking process, the scheduling configuration information of the device parallel dimension and the characteristics of the task information itself are combined. That is, the scheduling task execution process is considered from multiple aspects, which greatly improves the flexibility and task execution efficiency.

[0090] The above-mentioned task execution method is described below with a specific example. A hierarchical scheduling framework provided by an embodiment of the present application can be as follows Figure 3 As shown, it includes Central Scheduler, Node Scheduler, Request Scheduler, Compute Scheduler, Cache Scheduler, Communication Scheduler, Sequence Scheduler, PD Scheduler, and Token Scheduler. Each of the above schedulers is a software component deployed on each node of the computer cluster. In this example, the resource device is a GPU.

[0091] Among them, the central scheduler is the first-level scheduler, the node scheduler and request scheduler are the second-level schedulers, and the computing scheduler, cache scheduler, communication scheduler, sequence scheduler, pre-filled decoding scheduler, and word scheduler are the third-level schedulers. The first-level scheduler is responsible for directly managing the second-level scheduler and obtaining the allocation information generated by the third-level scheduler. The second-level scheduler is responsible for directly managing the third-level scheduler.

[0092] like Figure 4 As shown, the main functions of the central scheduler may include: initializing the node scheduler and the request scheduler, calling the node scheduler for scheduling, calling the request scheduler for scheduling, and executing the final scheduling result each time the user initiates an inference request.

[0093] like Figure 5 As shown in Figure 1, the main functions of the node scheduler include initializing the computing scheduler, cache scheduler, and communication scheduler; calling the computing scheduler to allocate computing resources; calling the cache scheduler to allocate memory resources; calling the communication scheduler to allocate communication resources; and returning the final scheduling result to the central scheduler. Figure 6 As shown, the computing scheduler can perform the computing resource allocation operation according to step S203, and determine the computing resource allocation information for recording. Figure 7 As shown, the cache scheduler can perform the allocation operation of the memory resource according to step S203, and determine the allocation information of the computing resource for recording.

[0094] like Figure 8 As shown, the main functions of the request scheduler may include initializing the sequence scheduler, pre-filling decoding scheduler, and word scheduler; calling the sequence scheduler to perform block operations on the task content; calling the pre-filling decoding scheduler to adjust the resource allocation ratio of the pre-filling stage and the decoding stage; calling the word scheduler to train the decoding model and return the final scheduling result to the central caller.

[0095] The operation of initializing the scheduler may include initializing configuration parameters in the caller, for example, initializing the configuration parameters to preset values.

[0096] In summary, under the above hierarchical scheduling framework, the hierarchical scheduling strategy can be as follows: Fig. 9 As shown in the figure, resource scheduling is performed from multiple dimensions such as computing resources (graph processors), memory resources (cache), communication resources, blocks, stages, tokens, etc., with high flexibility. In addition, multi-layer hierarchical scheduling is conducive to the management of different types of resources and the management of the task execution process.

[0097] In some optional implementations, the hierarchical scheduling framework may further include an execution module (a software component), such as Fig.10 As shown. The execution module may include a data generator (Data Generator), a model generator (ModelGenerator) and a data collector. Among them, the data generator can be used to generate virtual input data. The model generator can be used to generate virtual models, including the decoding model mentioned in the word scheduler. Furthermore, by copying the user model weights and the virtual model generated by the data generator from the cache, multiple decoding models can be trained. Then, the trained multiple decoding models can be used to decode multiple words at the same time, and after model reasoning (that is, executing the task), the final execution result is obtained. The data collector can be used to collect reasoning data (for example, task execution efficiency), parameter data (for example, task type, adjustment strategy, etc.), GPU data (for example, allocation information of computing resources), and communication data (for example, allocation data of communication resources) generated during the model reasoning process.

[0098] The data collector can determine the optimal scheduling strategy for each task type based on the collected data, which can be used in the subsequent task scheduling process to further improve the task execution efficiency and the rationality of resource allocation.

[0099] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0100] The embodiment of the present application also provides a task execution device, such as Fig.11 As shown, the device may include: The receiving module 1110 is used to receive a target request sent by a target device, wherein the target request includes a scheduling strategy and task information, the scheduling strategy includes scheduling configuration information of multiple dimensions, and the multiple dimensions include a device parallel dimension, a stage parallel dimension, and a token prediction parallel dimension; The determination module 1120 is used to determine the device allocation information according to the scheduling configuration information of the device parallel dimension, the task information and the pre-acquired resource device usage information; determine the resource allocation information according to the task information, the device allocation information, the scheduling configuration information of the token prediction parallel dimension, and the scheduling configuration information of the stage parallel dimension; The acquisition module 1130 is used to predict the scheduling configuration information, resource allocation information, and task information of the parallel dimension according to the token, obtain the execution result and feed it back to the target device.

[0101] In some optional implementations, the determination module 1120 is specifically configured to: When the scheduling configuration information of the stage parallel dimension is used to indicate to execute the task in stages, the resource allocation ratios of multiple stages are obtained; The resource allocation information is determined according to the task information, the device allocation information, the resource allocation ratio of each stage, and the scheduling configuration information of the token prediction parallel dimension.

[0102] In some optional implementations, the determination module 1120 is specifically configured to: Determine the allocation information of computing resources according to the device allocation information and the resource allocation ratio of each stage; Determine the allocation information of memory resources based on the scheduling configuration information of the token prediction parallel dimension, task information, and resource allocation ratio of each stage; Determine the allocation information of communication resources according to the device allocation information and the resource allocation ratio of each stage; Among them, the allocation information of computing resources, the allocation information of memory resources, and the allocation information of communication resources constitute the resource allocation information.

[0103] In some optional implementations, the device allocation information includes identification information of the resource device to be used and identification information of the node where the resource device to be used is located; the determination module 1120 is specifically used to: Counting a first quantity of identification information of resource devices to be used included in the device allocation information; Determine the number of resource devices to be used in each stage according to the first number and the resource allocation ratio in each stage; Selecting, from the resource devices to be used, resource devices of a number equal to the number of resource devices to be used in the first stage as devices for performing tasks in the first stage, wherein the first stage is any one of the multiple stages; The identification information and the number of resource devices to be used in each phase are jointly determined as the allocation information of the computing resources.

[0104] In some optional implementations, the task information includes model parameters; the determination module 1120 is specifically configured to: Determine the amount of memory resources to be used based on the scheduling configuration information and model parameters of the token prediction parallel dimension; Determine the amount of memory resources to be used in each stage according to the amount of memory resources to be used and the resource allocation ratio of each stage; The amount of memory resources to be used in each stage is collectively determined as allocation information of the memory resources.

[0105] In some optional implementations, the determination module 1120 is specifically configured to: Counting the second quantity of identification information of the node where the resource device to be used is located included in the device allocation information; Determine a communication mode of each resource device to be used according to the first quantity and the second quantity; The communication mode of each resource device to be used and the resource allocation ratio in each stage are jointly determined as the allocation information of the communication resources.

[0106] In some optional implementations, the multiple stages include a pre-filling stage and a decoding stage, and the task information includes task content and input length; the acquisition module 1130 is specifically used to: When determining to divide the task content into blocks according to the input length and the preset pre-filling length, the block size is determined according to the input length and the number of resource devices to be used in the pre-filling stage; Divide the task content into multiple blocks according to the block size; According to the identification information of the resource device to be used in the pre-filling stage, a block is allocated to a resource device to be used in the pre-filling stage to perform pre-filling processing on the block to obtain a pre-filling processing result corresponding to the block; Obtaining a decoding model corresponding to the scheduling configuration information of the token prediction parallel dimension; According to the identification information of the resource device to be used in the decoding stage, a decoding model is loaded on the resource device to be used in the decoding stage, and the decoding model is used to decode the pre-filled processing result corresponding to each block to obtain the execution result.

[0107] In some optional embodiments, the device further includes an adjustment module 1140, which is used to: Obtain the task execution efficiency corresponding to multiple tasks, as well as the task type and scheduling strategy corresponding to each task; Determine the scheduling strategy corresponding to the task with the highest task execution efficiency among the tasks of the target task type as the optimal scheduling strategy corresponding to the target task type, wherein the target task type is any task type; Each task type and the optimal scheduling strategy corresponding to each task type are sent to the target device to instruct the target object to adjust the scheduling strategy.

[0108] In some optional implementations, the determination module 1120 is specifically configured to: When the scheduling configuration information of the token prediction parallel dimension is used to indicate multi-token prediction, determining the amount of memory resources to be used according to the first preset multiple and the model parameters; or, When the scheduling configuration information of the token prediction parallel dimension is used to indicate that multi-token prediction is not performed, the amount of memory resources to be used is determined according to a second preset multiple and model parameters, wherein the first preset multiple is greater than the second preset multiple.

[0109] In some optional implementations, the determination module 1120 is specifically configured to: Determine whether the scheduling configuration information of the stage parallel dimension includes resource allocation ratios of multiple stages; When the scheduling configuration information for determining the stage parallel dimension includes resource allocation ratios of multiple stages, extracting the resource allocation ratio of each stage therefrom; or, When the scheduling configuration information for determining the stage parallel dimension does not include the resource allocation ratio, the resource allocation ratio of each stage corresponding to the task type is obtained according to the task type in the task information.

[0110] In some optional embodiments, the multiple stages include a pre-filling stage and a decoding stage, and the task type is a long text type or a short text type. When the task type is a long text type, the resource allocation ratio of the pre-filling stage is greater than the resource allocation ratio of the decoding stage, or, when the task type is a short text type, the resource allocation ratio of the pre-filling stage is less than the resource allocation ratio of the decoding stage.

[0111] In some optional implementations, the determination module 1120 is specifically configured to: When it is determined that the first number or the second number is the first preset value, determining the communication mode of each resource device to be used as an intra-node communication mode; Alternatively, when it is determined that the first number and the second number are not the first preset value, the communication mode of each resource to be used is determined as an inter-node communication mode.

[0112] The description of the features in the embodiment corresponding to the task execution device can refer to the relevant description of the embodiment corresponding to the task execution method, which will not be repeated here.

[0113] The embodiment of the present application also provides an electronic device, such as Fig.12 As shown, it may include a processor 10 and a memory 20, wherein a computer program is stored in the memory 20, and the processor 10 is configured to run the computer program to execute the steps in any of the above-mentioned task execution method embodiments. The electronic device may be the above-mentioned node.

[0114] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned task execution method embodiments when running.

[0115] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0116] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above-mentioned task execution method embodiments are implemented.

[0117] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned task execution method embodiments are implemented.

[0118] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0119] The above is a detailed introduction to a task execution method, device, computer equipment, storage medium and program product provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A task execution method, characterized in that: include: Receive a target request sent by a target device, wherein the target request includes a scheduling strategy and task information, the scheduling strategy includes scheduling configuration information of multiple dimensions, and the multiple dimensions include a device parallel dimension, a stage parallel dimension, and a token prediction parallel dimension; Determine device allocation information according to the scheduling configuration information of the device parallel dimension, the task information and the pre-acquired resource device usage information; Determine resource allocation information according to the task information, the device allocation information, the scheduling configuration information of the token prediction parallel dimension, and the scheduling configuration information of the stage parallel dimension; According to the scheduling configuration information of the token prediction parallel dimension, the resource allocation information, and the task information, an execution result is obtained and fed back to the target device.

2. The task execution method according to claim 1, characterized in that: Determining resource allocation information according to the task information, the device allocation information, the scheduling configuration information of the token prediction parallel dimension, and the scheduling configuration information of the stage parallel dimension includes: When the scheduling configuration information of the stage parallel dimension is used to indicate to execute the task in stages, obtaining resource allocation ratios of multiple stages; Resource allocation information is determined according to the task information, the device allocation information, the resource allocation ratio of each stage, and the scheduling configuration information of the token prediction parallel dimension.

3. The task execution method according to claim 2, characterized in that: Determining the resource allocation information according to the task information, the device allocation information, the resource allocation ratio of each stage, and the scheduling configuration information of the token prediction parallel dimension includes: Determining computing resource allocation information according to the device allocation information and the resource allocation ratio of each stage; Determine memory resource allocation information according to the scheduling configuration information of the token prediction parallel dimension, the task information and the resource allocation ratio of each stage; Determining communication resource allocation information according to the device allocation information and the resource allocation ratio in each stage; The allocation information of the computing resources, the allocation information of the memory resources, and the allocation information of the communication resources constitute the resource allocation information.

4. The task execution method according to claim 3, characterized in that: The device allocation information includes identification information of the resource device to be used and identification information of the node where the resource device to be used is located; the determining the allocation information of the computing resources according to the device allocation information and the resource allocation ratio of each stage includes: Counting a first quantity of identification information of resource devices to be used included in the device allocation information; Determine the number of resource devices to be used in each stage according to the first number and the resource allocation ratio of each stage; Selecting, from the resource devices to be used, a number of resource devices to be used that is equal to the number of resource devices to be used in the first stage as devices for performing the tasks of the first stage, wherein the first stage is any one of the multiple stages; The identification information and the number of resource devices to be used in each phase are jointly determined as the allocation information of the computing resources.

5. The task execution method according to claim 3, characterized in that: The task information includes model parameters; the scheduling configuration information of the token prediction parallel dimension, the task information and the resource allocation ratio of each stage are used to determine the allocation information of the memory resources, including: Determining the amount of memory resources to be used according to the scheduling configuration information of the token prediction parallel dimension and the model parameters; Determine the amount of memory resources to be used in each stage according to the amount of memory resources to be used and the resource allocation ratio in each stage; The amount of memory resources to be used in each stage is collectively determined as the allocation information of the memory resources.

6. The task execution method according to claim 4, characterized in that: The determining, according to the device allocation information and the resource allocation ratio of each stage, the communication resource allocation information comprises: Counting a second amount of identification information of the node where the resource device to be used is located included in the device allocation information; Determining a communication mode of each resource device to be used according to the first quantity and the second quantity; The communication mode of each resource device to be used and the resource allocation ratio in each stage are jointly determined as the allocation information of the communication resources.

7. The task execution method according to claim 4, characterized in that: The multiple stages include a pre-filling stage and a decoding stage, the task information includes task content and input length; the scheduling configuration information, the resource allocation information, and the task information predicted according to the token parallel dimension, obtaining the execution result and feeding it back to the target device, including: When determining to divide the task content into blocks according to the input length and the preset pre-filling length, determining the block size according to the input length and the number of resource devices to be used in the pre-filling stage; Dividing the task content into a plurality of blocks according to the block size; According to the identification information of the resource device to be used in the pre-filling stage, one of the blocks is allocated to one of the resource devices to be used in the pre-filling stage, so as to perform pre-filling processing on the block and obtain a pre-filling processing result corresponding to the block; Obtaining a decoding model corresponding to the scheduling configuration information of the token prediction parallel dimension; According to the identification information of the resource device to be used in the decoding stage, the decoding model is loaded on the resource device to be used in the decoding stage, and the pre-filled processing result corresponding to each block is decoded using the decoding model to obtain the execution result.

8. The task execution method according to any one of claims 1 to 6, characterized in that: The method further comprises: Obtaining the task execution efficiencies corresponding to the multiple tasks, as well as the task types and scheduling strategies corresponding to each of the tasks; Determine the scheduling strategy corresponding to the task with the highest task execution efficiency among the tasks of the target task type as the optimal scheduling strategy corresponding to the target task type, wherein the target task type is any task type; Each of the task types and the optimal scheduling strategy corresponding to each of the task types are sent to the target device to instruct the target object to adjust the scheduling strategy.

9. The task execution method according to claim 5, characterized in that: The determining the amount of memory resources to be used according to the scheduling configuration information of the token prediction parallel dimension and the model parameters includes: When the scheduling configuration information of the token prediction parallel dimension is used to indicate multi-token prediction, determining the amount of memory resources to be used according to the first preset multiple and the model parameters; or, When the scheduling configuration information of the token prediction parallel dimension is used to indicate that multi-token prediction is not performed, the amount of memory resources to be used is determined according to a second preset multiple and the model parameters, wherein the first preset multiple is greater than the second preset multiple.

10. The task execution method according to any one of claims 2 to 6, characterized in that: When the scheduling configuration information of the stage parallel dimension is used to indicate to execute the task in stages, obtaining resource allocation ratios of multiple stages includes: Determining whether the scheduling configuration information of the stage parallel dimension includes the resource allocation ratios of multiple stages; When it is determined that the scheduling configuration information of the stage parallel dimension includes the resource allocation ratios of multiple stages, extracting the resource allocation ratio of each stage therefrom; or, When it is determined that the scheduling configuration information of the stage parallel dimension does not include the resource allocation ratio, the resource allocation ratio of each stage corresponding to the task type is acquired according to the task type in the task information.

11. The task execution method according to claim 10, characterized in that: The multiple stages include a pre-filling stage and a decoding stage, and the task type is a long text type or a short text type. When the task type is a long text type, the resource allocation ratio of the pre-filling stage is greater than the resource allocation ratio of the decoding stage, or, when the task type is a short text type, the resource allocation ratio of the pre-filling stage is less than the resource allocation ratio of the decoding stage.

12. The task execution method according to claim 6, characterized in that: The determining, according to the first quantity and the second quantity, a communication mode of each resource device to be used includes: When it is determined that the first number or the second number is a first preset value, determining the communication mode of each of the resource devices to be used as an intra-node communication mode; Alternatively, when it is determined that the first number and the second number are not the first preset value, the communication mode of each of the resources to be used is determined to be an inter-node communication mode.

13. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the task execution method as claimed in any one of claims 1 to 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the task execution method according to any one of claims 1 to 12.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the task execution method according to any one of claims 1 to 12 are implemented.

Images