Task execution method and device, electronic equipment and storage medium

Abstract

The invention provides a task execution method, and relates to the technical field of artificial intelligence, in particular to the technical field of distributed computing, deep learning, large models, natural language processing, cloud computing and chips. According to the specific implementation scheme, according to first to-be-processed data and a first parallel strategy, a plurality of calculation units are used for executing a to-be-processed task, and a plurality of first execution results are obtained; according to the multiple first execution results, multiple computing units are used for executing multiple first to-be-executed sub-tasks of the fusion task respectively, a second execution result is obtained, and at least one first to-be-executed sub-task is executed according to a second parallel strategy; according to second to-be-processed data, the computing units are used for executing at least one second to-be-executed sub-task of the fusion task, a task execution result is obtained, and the second to-be-processed data are obtained according to the second execution result. The invention further provides a task execution device, electronic equipment and a storage medium.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to distributed computing, deep learning, large models, natural language processing, cloud computing and chip technology, and can be applied to large language model (LLM) service deployment, real-time natural language processing system, high-concurrency artificial intelligence (AI) reasoning service and other scenarios. More specifically, the present disclosure provides a task execution method, device, electronic device and storage medium. Background Art

[0002] With the development of artificial intelligence technology, the hardware resources required for reasoning or training of deep learning models are increasing. Distributed computing platforms can be used to implement reasoning or training of deep learning models. Distributed computing platforms can include multiple hardware devices, have high reliability and flexibility, can achieve efficient resource sharing, and can achieve large-scale data processing. During the reasoning or training process, in order to make full use of the computing resources of multiple hardware devices, a parallel strategy can be selected from a variety of parallel strategies to achieve parallel reasoning or training. Summary of the invention

[0003] The present disclosure provides a task execution method, apparatus, device and storage medium.

[0004] According to one aspect of the present disclosure, a task execution method is provided, the method comprising: according to first data to be processed and a first parallel strategy, using multiple computing units to respectively execute the tasks to be processed to obtain multiple first execution results; according to the multiple first execution results, using multiple computing units to respectively execute multiple first subtasks to be executed of a fusion task to obtain a second execution result, wherein at least one first subtask to be executed is executed according to the second parallel strategy; according to second data to be processed, using multiple computing units to respectively execute at least one second subtask to be executed of the fusion task to obtain a task execution result, wherein the second data to be processed is obtained according to the second execution result.

[0005] According to another aspect of the present disclosure, a task execution device is provided, which includes: a first execution module, which is used to use multiple computing units to respectively execute the tasks to be processed according to first data to be processed and a first parallel strategy, and obtain multiple first execution results; a second execution module, which is used to use multiple computing units to respectively execute multiple first subtasks to be executed of the fusion task according to the multiple first execution results, and obtain second execution results, wherein at least one first subtask to be executed is executed according to the second parallel strategy; a third execution module, which is used to use multiple computing units to respectively execute at least one second subtask to be executed of the fusion task according to the second data to be processed, and obtain a task execution result, wherein the second data to be processed is obtained according to the second execution result.

[0006] According to another aspect of the present disclosure, a task execution device is provided, including the apparatus provided by the present disclosure.

[0007] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided according to the present disclosure.

[0008] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided. The computer instructions are used to cause a computer to execute the method provided according to the present disclosure.

[0009] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method provided according to the present disclosure is implemented.

[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0012] Figure 1 is a schematic diagram of an exemplary system architecture to which a task execution method and apparatus can be applied according to an embodiment of the present disclosure;

[0013] Figure 2 is a flowchart of a task execution method according to an embodiment of the present disclosure;

[0014] Figure 3 is a schematic diagram of a fusion task according to an embodiment of the present disclosure;

[0015] Figure 4 is a schematic diagram of multiple tasks according to an embodiment of the present disclosure;

[0016] Figure 5 is a schematic diagram of a task execution method according to an embodiment of the present disclosure;

[0017] Figure 6 is a block diagram of a task execution device according to an embodiment of the present disclosure;

[0018] Figure 7 is a schematic block diagram of a task execution device according to an embodiment of the present disclosure; and

[0019] Figure 8 is a block diagram of an electronic device to which a task execution method can be applied according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0021] As mentioned above, a distributed computing platform can be used to implement reasoning or training of large language models. Reasoning of large language models can be performed based on static graphs. A static graph is a predefined computational graph. During compilation, the structure of the model can be generated based on the static graph so that the devices of the distributed computing platform can perform corresponding operations. Based on the static graph, a single parallel strategy can be used for reasoning. For example, the tensor parallel strategy can be used throughout the process. However, when processing long sequence inputs, the tensor parallel strategy often faces problems such as high memory pressure and low computational efficiency, making it difficult to fully utilize distributed computing resources.

[0022] In addition, in the process of reasoning based on static graphs, there are large differences in the computing characteristics and resource requirements of different reasoning stages. A single parallel strategy is difficult to balance efficiency and performance. For example, the reasoning stage of a large language model may include a prefill stage and a decoding stage. In the prefill stage, the word units in the input prompt can be processed in parallel. In the decoding stage, word units can be generated one by one in an autoregressive manner. There are large differences in the computing characteristics and resource requirements between the prefill stage and the decoding stage. Based on a single parallel strategy, it is difficult to optimize the performance of these two stages at the same time. In addition, when reasoning based on static graphs, it is difficult to arbitrarily change the network structure of the model, which makes it difficult to adapt to different parallel strategies.

[0023] Based on the static graph, the parallel strategies that can be used include: tensor parallel strategy and pipeline parallel strategy. Based on the tensor parallel strategy, the parameters of the model can be distributed on multiple computing devices. Each computing device can calculate some parameters. The tensor parallel strategy is used throughout the reasoning process. Tensor parallelism can effectively disperse the computing load, but when processing long sequence inputs, it will cause repeated calculations of some data and waste of storage resources. In addition, based on pipeline parallelism, different layers of the model can be assigned to different computing devices to form a processing pipeline. Pipeline parallelism can balance the computing load, but when processing long sequence inputs, it may introduce additional delays due to the filling and emptying process of the pipeline.

[0024] In addition to static graphs, dynamic graphs can also be used for reasoning. Based on dynamic graphs, dynamic model networking can be performed. That is, dynamic parallel strategy selection can be achieved in the reasoning stage. Various open source reasoning frameworks can perform reasoning based on dynamic graphs. However, these reasoning frameworks also do not implement different parallel strategies in the pre-filling stage and the decoding stage.

[0025] When performing reasoning on a large language model based on a static or dynamic graph, it is difficult to handle long sequence inputs or high concurrent requests using a single parallel strategy, which faces serious performance bottlenecks and is difficult to optimize the performance of the pre-filling stage and the decoding stage at the same time, resulting in low computational efficiency and high reasoning latency during the reasoning process. Based on static graphs, it is difficult to dynamically adjust the computational graph structure during the reasoning process, making it difficult to flexibly switch and combine multiple parallel strategies, and it is impossible to dynamically select the optimal parallel strategy based on the computational characteristics of different stages, limiting the model's adaptability scenarios and performance optimization space. In addition, the technical potential of dynamic graphs has not yet been fully tapped. Although dynamic graph technology theoretically supports dynamic networking, it is still relatively fixed in the choice of parallel strategies, and fails to fully tap the flexibility and adaptability of dynamic graph technology in the reasoning process.

[0026] Therefore, in order to fully improve the reasoning capability of distributed computing devices, the present disclosure provides a task execution method, which will be described below.

[0027] Figure 1 An exemplary system architecture to which the task execution method and apparatus according to an embodiment of the present disclosure can be applied is schematically shown.

[0028] It should be noted that Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0029] like Figure 1As shown, the system architecture 100 according to this embodiment may include a terminal device 101, a network 102, and a server cluster 103. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server cluster 103. The network 102 may also be used to provide a medium for a communication link within the server cluster 103. The network 102 may include various connection types, such as wired and / or wireless communication links, etc.

[0030] The user can use the terminal device 101 to interact with the server cluster 103 through the network 102 to receive or send messages, etc. For example, the terminal device 101 can send a request for training a deep learning model and / or a request for reasoning based on a deep learning model to the server cluster 103 through the network 102.

[0031] Various communication client applications may be installed on the terminal device 101, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients and / or social platform software (for example only).

[0032] The terminal device 101 may be any electronic device having a display screen and supporting web browsing, including but not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, and the like.

[0033] The server cluster 103 may be a server that provides various services, such as a background management server that provides support for requests sent by users using the terminal device 101 (only an example).

[0034] The server cluster 103 may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0035] The task execution method can be applied to a server cluster 103. The server cluster 103 includes multiple server nodes 1031, 1032, 1033, 1034, each of which includes one or more hardware computing units. The server cluster 103 can be used to implement a variety of parallel strategies for deep learning model reasoning or training.

[0036] It should be understood that Figure 1 The number of terminal devices, networks, and server nodes in the server cluster is only illustrative. Any number of terminal devices, networks, and server nodes may be provided as required.

[0037] It should be noted that the sequence numbers of the operations in the following method are only used as representations of the operations for the purpose of description, and should not be regarded as representing the execution order of the operations. Unless explicitly stated, the method does not need to be executed completely in the order shown.

[0038] Figure 2 is a flowchart of a task execution method according to an embodiment of the present disclosure.

[0039] like Figure 2 As shown, the method 200 may include operations S210 to S230.

[0040] In operation S210, based on the first data to be processed and the first parallel strategy, multiple computing units are used to respectively execute the tasks to be processed to obtain multiple first execution results.

[0041] In the embodiment of the present disclosure, the first data to be processed may be obtained according to input data, which may be various data such as text data, image data, and audio data.

[0042] In the embodiment of the present disclosure, the first parallel strategy may be various strategies such as the tensor parallel strategy and the pipeline parallel strategy mentioned above.

[0043] In the disclosed embodiment, the task to be processed may be an attention task, a convolution task, etc. The task to be processed may correspond to a node of a computational graph. The computational graph may be, for example, the above-mentioned static graph.

[0044] In the embodiments of the present disclosure, the plurality of computing units may be hardware computing units in one or more of the above-mentioned server nodes. The computing unit may be an artificial intelligence computing unit. The artificial intelligence computing unit may be various hardware computing units such as a general purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), and a neural network processing unit (NPU).

[0045] In operation S220, according to the plurality of first execution results, the plurality of computing units are used to respectively execute the plurality of first to-be-executed subtasks of the fusion task to obtain a second execution result.

[0046] In the disclosed embodiment, at least one first to-be-executed subtask is executed according to a second parallel strategy. The second parallel strategy may be different from the first parallel strategy. For example, if the first parallel strategy is a tensor parallel strategy, the second parallel strategy may be a sequence parallel strategy.

[0047] In the disclosed embodiment, the fusion task may correspond to a computing node at the graph level of the computing graph. The first subtask to be executed corresponds to a subnode within the computing node. Each computing unit may execute multiple first subtasks to be executed. The stage of executing the first subtask to be executed may be, for example, the above-mentioned pre-filling stage.

[0048] In operation S230, at least one second to-be-executed subtask of the fusion task is respectively executed by using a plurality of computing units according to the second to-be-processed data to obtain a task execution result.

[0049] In the embodiment of the present disclosure, the second data to be processed is obtained according to the second execution result. For example, the second execution result can be used as the second data to be processed, or the second execution result can be normalized to obtain the second data to be processed.

[0050] In the disclosed embodiment, as described above, the fusion task corresponds to a computing node at the graph level of the computing graph. The second subtask to be executed corresponds to a subnode within the computing node. Each computing unit can execute at least one second subtask to be executed. The stage of executing the second subtask to be executed can be, for example, the above-mentioned decoding stage.

[0051] Through the disclosed embodiments, a fusion task is added to multiple reasoning tasks or training tasks, so that the hardware computing device can execute the first subtask to be executed and the second subtask to be executed based on different parallel strategies, and multiple parallel strategies can coexist in the reasoning stage to fully utilize the computing characteristics of different stages of the model to allocate and utilize computing resources. For example, some computing resources can be saved in the pre-filling stage, and these computing resources can be used to improve the hardware execution efficiency in the decoding stage, and can also support more concurrent requests.

[0052] It can be understood that the above describes the method of the present disclosure, and the following will describe the fusion task of the present disclosure.

[0053] In some embodiments, the fusion task includes a plurality of first subtasks to be executed and at least one second subtask to be executed.

[0054] In some embodiments, the plurality of first tasks to be performed may include a reduce scatter subtask and an all gather subtask.

[0055] In some embodiments, according to multiple first execution results, using multiple computing units to respectively execute multiple first to-be-executed subtasks of the fusion task to obtain the second execution result includes: according to multiple first execution results, using multiple computing units to respectively execute the distributed protocol communication subtasks to obtain multiple distributed protocol communication results according to the second parallel strategy. According to multiple distributed protocol communication results, using multiple computing units to respectively execute the full collection communication subtasks to obtain the second execution result according to the second parallel strategy. For example, taking the second parallel strategy as a sequence parallel strategy as an example, according to multiple first execution results, multiple computing units can execute the distributed protocol communication subtasks. In the process of executing the distributed protocol communication subtasks, the first intermediate result can be obtained according to the obtained multiple first execution results. Next, the first intermediate result can be split in the sequence dimension to obtain multiple distributed protocol communication results. According to multiple distributed protocol communication results, multiple computing units can execute the full collection communication subtasks to combine in the sequence dimension to obtain the second execution result.

[0056] In some embodiments, the at least one second to-be-performed subtask may include an allreduce subtask.

[0057] In some embodiments, according to the second data to be processed, a task execution result can be obtained by using multiple computing units to respectively execute the full protocol communication subtask in at least one second subtask to be executed of the fusion task.

[0058] Through the embodiments of the present disclosure, multiple first subtasks to be executed include a distributed protocol communication subtask and a full collection communication subtask, and at least one second subtask to be executed includes a full protocol communication subtask. If multiple first subtasks to be executed and at least one second subtask to be executed are respectively executed based on the same data, the same result can be obtained. Thus, multiple first subtasks to be executed and multiple second subtasks to be executed can be fused into a fusion task, and then static graph reasoning or training can be performed using multiple computational graphs based on the fusion task. That is, multiple parallel strategies can be implemented based on the static graph to improve computing resource utilization and reduce resource overhead.

[0059] In an embodiment of the present disclosure, according to the second data to be processed, using multiple computing units to respectively execute the full protocol communication subtask in at least one second subtask to be executed of the fusion task, obtaining the task execution result may include: according to the second data to be processed and the first parallel strategy, using multiple computing units to respectively execute the tasks to be processed, multiple third execution results can be obtained. According to the multiple third execution results, the full protocol communication subtask can be executed by multiple computing units to obtain a fourth execution result. For example, taking the first parallel strategy as tensor parallel processing as an example, according to the multiple third execution results, the full protocol communication subtask is executed to obtain a fourth execution result.

[0060] It can be understood that the above describes the subtasks to be performed in the present disclosure, and the fusion tasks in the present disclosure will be further described below.

[0061] In some embodiments, the plurality of first subtasks to be performed may further include a first linear processing subtask, a first fusion subtask, and a first normalization subtask. The plurality of second subtasks to be performed may further include a second linear processing subtask, a second fusion subtask, and a second normalization subtask. Figure 3 Provide further explanation.

[0062] Figure 3 is a schematic diagram of a fusion task according to an embodiment of the present disclosure.

[0063] like Figure 3 As shown, the fusion task F30 includes a plurality of first subtasks to be executed and a plurality of second subtasks to be executed. The plurality of first subtasks to be executed include a first linear processing subtask 311, a distributed protocol communication subtask 312, a first fusion subtask 313, a first normalization subtask 314, and a full collection communication subtask 315. The plurality of second subtasks to be executed include a second linear processing subtask 321, a full protocol communication subtask 322, a second fusion subtask 323, and a first normalization subtask 324.

[0064] To further illustrate the principle of the fusion task, the following will take the execution of multiple first to-be-executed subtasks and multiple second to-be-executed subtasks based on the same data as an example to illustrate the fusion task. The same data may be the input data of the fusion task. The input data may be X, , indicating that the size (shape) of X can be B×S×H. The output data of the fusion task can be Y. , indicating that the scale of Y can be B×S×H. B represents the batch size. S represents the sequence length. H represents the hidden state dimension.

[0065] In the embodiment of the present disclosure, at least one of the plurality of first to-be-performed subtasks may be executed based on the second parallel strategy. For example, executing the first linear processing subtask 311 includes processing the input data X of the fusion task by the following formula:

[0066] (Formula 1)

[0067] W can be the weight corresponding to the first linear processing subtask, . It can be the bias corresponding to the first linear processing subtask, . L(X) can be the result of the first linear processing, Through the embodiment of the present disclosure, the first execution result can be adjusted by using linear processing to facilitate subsequent processing. Next, the decentralized protocol communication subtask 312 can be executed.

[0068] For another example, the second parallel strategy may be a sequence parallel strategy. According to the first linear processing result L(X), the distributed reduction communication subtask 312 may be executed to evenly divide the first linear processing result L(X) in the sequence dimension to obtain M distributed reduction communication results RS(L(X)). M is an integer greater than 1. M may be the number of computing units. According to the decentralized protocol communication result, the computing unit may execute the first fusion subtask 313 and the first normalization subtask 314 to obtain the first normalization result LN(RS(L(X))+b1′). b1' represents the bias corresponding to the first fusion subtask, which can indicate the data fused with the scatter-reduce communication result. Next, according to the M first normalization results, the full collection communication subtask 315 can be executed to recombine the data in the sequence dimension to obtain the full collection communication result AG(LN(RS(L(X))+b1')). The full collection communication result can be used as the above-mentioned output data Y.

[0069] It can be understood that the above describes a plurality of first subtasks to be executed, and the following describes a plurality of second subtasks to be executed.

[0070] In the embodiment of the present disclosure, at least one of the plurality of second to-be-performed subtasks may be executed based on the first parallel strategy. For example, the first parallel strategy may be a tensor parallel strategy, and executing the second linear processing subtask 321 includes processing the input data X of the fusion task by the following formula:

[0071] (Formula 2)

[0072] can be the weight corresponding to the second linear processing subtask, . It can be the bias corresponding to the second linear processing subtask, . j can be an integer less than or equal to M. The result may be processed for the second linear process. Next, the full protocol communication subtask 322 may be performed.

[0073] For another example, according to the M second linear processing results , the full protocol communication subtask 322 can be executed by the following formula to obtain the full protocol communication result AR(L(X)):

[0074] (Formula 3)

[0075] According to the full protocol communication result, the computing unit may execute the second fusion subtask 323 and the second normalization subtask 324 to obtain a second normalization result LN(AR(L(X))+b1). b1 represents the bias corresponding to the second fusion subtask. The second normalized result can be used as the output data Y.

[0076] It can be understood that when the input data of the fusion task is X, the second normalization result and the full-gather communication result can be the same. The reason is that the full-reduce communication operation is mathematically equivalent to the scatter-reduce operation followed by the full-gather operation:

[0077] (Formula 4)

[0078] In addition, the first normalization subtask and the second normalization subtask may be layer normalization subtasks. The layer normalization task may be performed on the feature dimension, so the results may be merged after the corresponding tasks are performed by M computing units respectively, that is:

[0079] (Formula 5)

[0080] It can be understood that the above description uses the example of multiple first subtasks to be executed and multiple second subtasks to be executed including linear processing subtasks, fusion subtasks and normalization subtasks to illustrate the present disclosure. However, the present disclosure is not limited to this, and multiple subtasks to be executed can be determined according to the calculation graph of the model, and the order of the subtasks to be executed can also be adjusted.

[0081] It can be understood that the above description describes the fusion task of the present disclosure in combination with the input data X, and the following description will describe the fusion task of the present disclosure in combination with different reasoning stages.

[0082] In an embodiment of the present disclosure, before executing a fusion task, the scale of the input data of the fusion task can be determined. For example, in the pre-filling stage, the scale of the input data of the fusion task is relatively large. In the decoding stage, the scale of the input data of the fusion task is relatively small. Therefore, if the scale of the input data of the fusion task is greater than a preset scale threshold, multiple first tasks to be executed can be executed based on the input data. Through the embodiment of the present disclosure, the subtasks to be executed can be determined according to the scale of the input data, and reasoning or training can be performed quickly and accurately to improve the efficiency of task execution. In addition, by switching based on the scale of the input data, the coexistence of multiple parallel strategies can be achieved on the basis of static graph reasoning.

[0083] In an embodiment of the present disclosure, based on multiple first execution results, using multiple computing units to respectively execute distributed protocol communication subtasks to obtain multiple distributed protocol communication results according to a second parallel strategy may include: based on multiple first execution results, using multiple computing units to respectively execute first linear processing subtasks to obtain multiple first linear processing results; based on multiple first linear processing results, using multiple computing units to respectively execute distributed protocol communication subtasks to obtain multiple distributed protocol communication results according to the second parallel strategy. For example, for each of the multiple computing units, each computing unit may execute the first linear processing subtask based on a first execution result to obtain a first linear processing result. Next, the multiple computing units may execute the distributed protocol communication subtask to split the first intermediate result obtained by combining the multiple first linear processing results into multiple distributed protocol communication results in the sequence dimension.

[0084] In the embodiment of the present disclosure, according to multiple distributed protocol communication results, using multiple computing units to respectively execute full collection communication subtasks to obtain the second execution result according to the second parallel strategy may include: according to the first data to be fused and the multiple distributed protocol communication results, using multiple computing units to respectively execute the first fusion subtask to obtain multiple first fusion results. According to the multiple first fusion results, using multiple computing units to respectively execute the first normalization subtask to obtain multiple first normalization results. According to the multiple first normalization results, using multiple computing units to respectively execute the full collection communication subtask to obtain the second execution result according to the second parallel strategy. For example, the first data to be fused may be obtained according to the input data of multiple tasks. The first fusion subtask may be, for example, a first addition operation subtask. Based on the bias item of the first fusion subtask, the computing unit may fuse part of the first data to be fused indicated by the bias item with the distributed protocol communication result to obtain the first fusion result. Next, according to the first fusion result, the computing unit may execute the first normalization subtask to obtain the first normalization result. After multiple computing units have completed the first fusion subtask and the first normalization subtask respectively, the full collection communication subtask can be performed using multiple computing units according to the multiple first normalization results to combine the multiple first normalization results into a second execution result in the sequence dimension.

[0085] It can be understood that the above describes a plurality of first subtasks to be executed, and the following describes a plurality of second subtasks to be executed.

[0086] In some embodiments, the method may further include: in response to determining that the size of the input data of the fusion task is less than or equal to a preset size threshold, switching the plurality of first to-be-executed subtasks of the fusion task to a plurality of second to-be-executed subtasks.

[0087] In an embodiment of the present disclosure, before executing a fusion task, the scale of the input data of the fusion task can be determined. For example, in the pre-filling stage, the scale of the input data of the fusion task is relatively large. In the decoding stage, the scale of the input data of the fusion task is relatively small. Therefore, if the scale of the input data of the fusion task is greater than a preset scale threshold, multiple first tasks to be executed can be executed based on the input data. At this time, the input data of the fusion task can be the above-mentioned multiple first execution results. If the scale of the input data of the fusion task is greater than the preset scale threshold, multiple second tasks to be executed can be executed based on the input data. At this time, the input data of the fusion task can be data obtained according to the second execution result.

[0088] In some embodiments, according to the plurality of third execution results, using the plurality of computing units to respectively execute at least one second to-be-executed subtask of the fusion task to obtain the fourth execution result includes: according to the plurality of third execution results, using the plurality of computing units to respectively execute the second linear processing subtask to obtain a plurality of third linear processing results. According to the plurality of third linear processing results, using the plurality of computing units to respectively execute the full protocol communication subtask to obtain a plurality of full protocol communication results according to the first parallel strategy. According to the second data to be fused and the plurality of full protocol communication results, using the plurality of computing units to respectively execute the second fusion subtask to obtain a plurality of second fusion results. According to the plurality of second fusion results, using the plurality of computing units to respectively execute the second normalization subtask to obtain the fourth execution result. For example, according to a third execution result, each computing unit in the plurality of computing units may execute a second linear processing task to obtain a second linear processing result. Next, the plurality of computing units may execute the full protocol communication subtask to fuse the plurality of second linear processing results into a full protocol communication result. Each computing unit may receive the full protocol communication result. Next, according to the second data to be fused and the full protocol communication result, the computing unit may execute the second fusion subtask to obtain the second fusion result. The second fusion task may be a second addition operation subtask. According to the second fusion result, the computing unit may execute the second normalization subtask to obtain a fourth execution result.

[0089] Through the embodiments of the present disclosure, multiple subtasks to be executed also include linear processing subtasks, fusion subtasks and normalization subtasks, which can be more suitable for reasoning or training of models built on the basis of the Transformer architecture, and help save the computing resources required for reasoning or selecting large models based on the Transformer architecture.

[0090] It can be understood that the above describes the fusion task, and the following will describe multiple tasks of the present disclosure.

[0091] In the embodiment of the present disclosure, the plurality of tasks may include tasks to be processed and fused tasks. The tasks to be processed may be of N levels, and the fused tasks may be of N levels. There may be at least one task of the i-th level of the N-level tasks to be processed. N may be an integer greater than or equal to 1. i is an integer greater than or equal to 1 and less than or equal to N. Figure 4 Through the embodiments of the present disclosure, when N is greater than 1, the distributed computing device can be used to fully obtain the information of the input data, so as to improve the accuracy of the task execution result, which helps to improve the user experience.

[0092] Figure 4 is a schematic diagram of multiple tasks according to one embodiment of the present disclosure.

[0093] like Figure 4 As shown, the plurality of tasks may include a first level to-be-processed task P41, a first level fused task F41, a second level to-be-processed task P421, a second level to-be-processed task P422, and a second level fused task F42. It can be understood that Figure 4 The 2-level pending tasks and 2-level fusion tasks shown are only examples. N can be any value greater than or equal to 1. It can also be understood that Figure 4 The multiple tasks shown correspond to nodes in multiple static graphs respectively.

[0094] In the embodiment of the present disclosure, the task to be processed may be various tasks such as an attention task, a linear processing task, and an activation task.

[0095] It can be understood that the above describes multiple tasks of the present disclosure, and the method of the present disclosure will be further described below.

[0096] Figure 5 is a schematic diagram of a task execution method according to an embodiment of the present disclosure.

[0097] like Figure 5 As shown, in the pre-filling stage, the multiple tasks executed by the multiple computing units include a normalization task ln50, a first-level task to be processed, a plurality of first subtasks to be executed of the first-level fusion task F51, a plurality of second-level tasks to be processed, and a plurality of second subtasks to be executed of the second-level fusion task F52. The first-level task to be processed may be an attention task P51. The plurality of first subtasks to be executed of the first-level fusion task F51 may include a first linear processing subtask 5111, a distributed protocol communication subtask 5112, a first fusion subtask 5113, a first normalization subtask 5114, and a full collection communication subtask 5115. The plurality of first subtasks to be executed of the second-level fusion task F52 may include a first linear processing subtask 5211, a distributed protocol communication subtask 5212, a first fusion subtask 5213, a first normalization subtask 5214, and a full collection communication subtask 5215.

[0098] In the embodiment of the present disclosure, the method may further include: according to the first input data, using a computing unit to perform a normalization task to obtain level 1 first data to be processed in the N levels of first data to be processed. For example, according to the first input data, using multiple computing units to respectively perform normalization tasks ln50, the level 1 first data to be processed can be obtained.

[0099] In the embodiment of the present disclosure, in some implementations of the above operation S210, according to the first data to be processed and the first parallel strategy, using multiple computing units to respectively execute the tasks to be processed, and obtaining multiple first execution results may include: according to the i-th level first data to be processed and the first parallel strategy, using multiple computing units to respectively execute at least one i-th level task to be processed, and obtaining multiple i-th level first execution results. Figure 5 As shown, the first level to be processed task is the attention task P51. According to the first level to be processed data and the first parallel strategy, multiple computing units can respectively execute the attention task P51 to obtain multiple first level first execution results. The first parallel strategy can be, for example, a tensor parallel strategy. The parameters of the attention tasks executed by different computing units are different, and the multiple first level first execution results obtained are also different.

[0100] In some implementations of the above operation S220, in the embodiments of the present disclosure, multiple computing units may be used to respectively execute multiple first to-be-executed subtasks of the first-level fusion task according to multiple first-level execution results. Figure 5 As shown, according to the multiple first-level first execution results, multiple computing units can be used to respectively execute the first linear processing subtask 5111 to obtain multiple first-level first linear processing results. According to the multiple first-level first linear processing results, multiple computing units can be used to execute the distributed protocol communication subtask 5112 to split the intermediate results obtained by combining the multiple first-level first linear processing results into multiple first-level distributed protocol communication results in the sequence dimension. According to the first-level distributed protocol communication results and the first data to be fused, the computing unit can be used to execute the first fusion subtask 5113.

[0101] In an embodiment of the present disclosure, the first data to be fused of the first level fusion task in the N level fusion tasks may be obtained based on the first input data. For example, the bias item of the first fusion subtask may indicate partial data in the first input data. The bias items of the first fusion subtasks of the multiple computing units may respectively indicate multiple partial data of the first input data as multiple first data to be fused. According to a distributed protocol communication result and a partial data, a first fusion result of the first level may be obtained by executing the first fusion subtask 5113 using the computing unit. Multiple computing units may respectively execute the first fusion subtask 5113 to obtain multiple first level first fusion results.

[0102] like Figure 5 As shown, according to the first fusion result of level 1, the computing unit can be used to execute the first normalization task 5114 to obtain the first normalization result of level 1. Multiple computing units can respectively execute the first normalization task 5114 to obtain multiple first normalization results of level 1. Next, according to multiple first normalization results of level 1, multiple computing units can execute the full collection communication subtask 5115 to obtain the second execution result of level 1. Thus, multiple first to-be-executed subtasks of the first-level fusion task are completed, and multiple second-level to-be-processed tasks can be executed below.

[0103] In the embodiment of the present disclosure, when i is greater than 1, at least one i-th level of tasks to be processed in the N-level tasks to be processed includes: an i-th level linear processing task and an i-th level activation task. Figure 5 As shown, the plurality of second-level tasks to be processed may include a linear processing task P521 and an activation task P522. The activation task may be implemented based on a Gaussian error linear unit (Gaussian error Linear unit, GeLu).

[0104] In the embodiment of the present disclosure, the first data to be processed at the i-th level is obtained according to the second execution result at the i-1th level. For example, the first data to be processed at the second level may be the second execution result at the first level.

[0105] In the disclosed embodiment, according to the first data to be processed at the i-th level and the first parallel strategy, using multiple computing units to respectively execute at least one task to be processed at the i-th level to obtain multiple first execution results at the i-th level may include: according to the first data to be processed at the i-th level and the first parallel strategy, using multiple computing units to respectively execute the linear processing tasks at the i-th level to obtain multiple third linear processing results at the i-th level. According to the multiple third linear processing results at the i-th level, using multiple computing units to execute the activation tasks at the i-th level to obtain multiple first execution results at the i-th level. Figure 5 As shown, according to the second execution result of the first level and the tensor parallel strategy, multiple computing units can be used to respectively execute the linear processing task P521 to obtain multiple third linear processing results of the second level. According to the multiple third linear processing results of the second level, multiple computing units can be used to execute the activation task P522 to obtain multiple first execution results of the second level.

[0106] Next, multiple first to-be-executed subtasks of the second-level fusion task can be executed. Based on multiple second-level first execution results, multiple computing units can be used to respectively execute the first linear processing subtask 5211 to obtain multiple second-level first linear processing results. Based on multiple second-level first linear processing results, multiple computing units can be used to execute the distributed protocol communication subtask 5212 to split the intermediate results obtained by combining multiple second-level first linear processing results into multiple second-level distributed protocol communication results in the sequence dimension. Based on the second-level distributed protocol communication results and the first data to be fused, the computing unit can be used to execute the first fusion subtask 5213.

[0107] In the disclosed embodiment, the first data to be fused of the nth level fusion task in the Nth level fusion task is obtained according to the first fusion result of the n-1th level. When N is greater than 1, n may be an integer greater than 1 and less than or equal to N. For example, according to a distributed protocol communication result and a first level fusion result, a computing unit may be used to execute the first fusion subtask 5213 to obtain a second level first fusion result. Multiple computing units may respectively execute the first fusion subtask 5213 to obtain multiple second level first fusion results.

[0108] like Figure 5 As shown, according to the first fusion result of the second level, the computing unit can be used to perform the first normalization task 5214 to obtain the first normalization result of the second level. Multiple computing units can respectively perform the first normalization task 5214 to obtain multiple first normalization results of the second level. Next, according to the multiple first normalization results of the second level, the multiple computing units can perform the full collection communication subtask 5215 to obtain the second execution result of the second level.

[0109] It can be understood that the pre-filling stage of the present disclosure is described above, and the method of the present disclosure will be further described below in conjunction with the decoding stage.

[0110] like Figure 5As shown, after executing the multiple first subtasks to be executed of the second-level fusion task F52, if the scale of the second-level second execution result is less than or equal to the preset scale threshold, the multiple first subtasks to be executed of the first-level fusion task F51 can be switched to multiple second subtasks to be executed. The multiple first subtasks to be executed of the second-level fusion task F52 can be switched to multiple second subtasks to be executed. The multiple second subtasks to be executed of the first-level fusion task F51 may include a second linear processing subtask 5121, a full protocol communication subtask 5122, a second fusion subtask 5123, and a second normalization subtask 5124. The multiple second subtasks to be executed of the second-level fusion task F52 may include a second linear processing subtask 5221, a full protocol communication subtask 5222, a second fusion subtask 5223, and a second normalization subtask 5224.

[0111] In some embodiments, the method may further include: performing a normalization task using a computing unit according to the second input data to obtain the first level second data to be processed in the N level second data to be processed. The second input data is obtained according to the second execution result. For example, the second level second execution result may be used as the second input data. According to the second level second execution result, a normalization task ln50 may be performed to obtain the first level second data to be processed.

[0112] In some embodiments, in some implementations of the above operation S230, according to the second data to be processed, using multiple computing units to respectively execute at least one second subtask to be executed of the fusion task, and obtaining the task execution result includes: according to the second data to be processed and the first parallel strategy, using multiple computing units to respectively execute the tasks to be processed, and obtaining multiple third execution results. For example, according to the i-th level second data to be processed and the first parallel strategy, using multiple computing units to respectively execute at least one i-th level task to be processed, multiple i-th level third execution results can be obtained. Figure 5 As shown, the first level to-be-processed task is the attention task P51. According to the first level second to-be-processed data and the tensor parallel strategy, multiple computing units can respectively execute the attention task P51 to obtain multiple first level third execution results. Based on the tensor parallel strategy, the parameters of the attention tasks executed by different computing units are different, and the obtained multiple first level third execution results are also different.

[0113] In some embodiments, in some implementations of the above operation S230, according to the second data to be processed, using multiple computing units to respectively execute at least one second subtask to be executed of the fusion task to obtain the task execution result may also include: according to multiple third execution results, using multiple computing units to respectively execute at least one second subtask to be executed of the fusion task to obtain a fourth execution result. For example, according to multiple third execution results, using multiple computing units to respectively execute the full protocol communication subtask to obtain the fourth execution result.

[0114] In an embodiment of the present disclosure, based on multiple third execution results, using multiple computing units to respectively execute at least one second subtask to be executed, and obtaining a fourth execution result includes: based on multiple third execution results, using multiple computing units to respectively execute the second linear processing subtask to obtain multiple third linear processing results. Based on the multiple third linear processing results, using multiple computing units to respectively execute full protocol communication subtasks to obtain multiple full protocol communication results according to the first parallel strategy. Based on the second data to be fused and multiple full protocol communication results, using multiple computing units to respectively execute the second fusion subtask to obtain multiple second fusion results. Based on the multiple second fusion results, using multiple computing units to respectively execute the second normalization subtask to obtain a fourth execution result.

[0115] like Figure 5 As shown, according to the multiple first-level first execution results, multiple computing units can be used to respectively execute the second linear processing subtask 5121 to obtain multiple first-level second linear processing results. According to the multiple first-level second linear processing results, multiple computing units can be used to execute the full protocol communication subtask 5122 to obtain multiple first-level full protocol communication results. According to the first-level full protocol communication results and the second data to be fused, the second fusion subtask 5123 can be executed by using the computing unit.

[0116] In the disclosed embodiment, the second data to be fused of the first-level fusion task in the N-level fusion task may be obtained based on the second input data. For example, the second input data may be, for example, the second execution result of the second level. According to the full protocol communication result and the second data to be fused, the second fusion subtask 5123 may be executed by the computing unit to obtain a first-level second fusion result. Multiple computing units may respectively execute the second fusion subtask 5123 to obtain multiple first-level second fusion results.

[0117] like Figure 5As shown, according to the second fusion result of level 1, the second normalization task 5124 can be executed by the computing unit to obtain the fourth execution result of level 1. Multiple computing units can respectively execute the second normalization task 5124 to obtain multiple fourth execution results of level 1. Thus, multiple second to-be-executed subtasks of the level 1 fusion task are completed, and multiple level 2 to-be-processed tasks can be executed below.

[0118] In the embodiment of the present disclosure, when i is greater than 1, at least one i-th level of tasks to be processed in the N-level tasks to be processed includes: an i-th level linear processing task and an i-th level activation task. Figure 5 As shown, the plurality of level 2 tasks to be processed may include a linear processing task P521 and an activation task P522.

[0119] In the embodiment of the present disclosure, the second data to be processed at the i-th level is obtained from the fourth execution result at the i-1th level. For example, the second data to be processed at the second level may be the fourth execution result at the first level.

[0120] In the embodiment of the present disclosure, according to the second data to be processed at the i-th level and the first parallel strategy, using multiple computing units to respectively execute at least one task to be processed at the i-th level, and obtaining multiple third execution results of the i-th level includes: according to the second data to be processed at the i-th level and the first parallel strategy, using multiple computing units to respectively execute the linear processing tasks at the i-th level, and obtaining multiple fourth linear processing results of the i-th level. According to the multiple fourth linear processing results of the i-th level, using multiple computing units to execute the activation tasks at the i-th level, and obtaining multiple third execution results of the i-th level. Figure 5 As shown, according to the fourth execution result of the first level and the tensor parallel strategy, multiple computing units can be used to respectively execute the linear processing task P521 to obtain multiple fourth linear processing results of the second level. According to the multiple fourth linear processing results of the second level, multiple computing units can be used to execute the activation task P522 to obtain multiple third execution results of the second level.

[0121] Next, multiple second to-be-executed subtasks of the second-level fusion task F52 may be executed. Based on multiple second-level third execution results, multiple computing units may be used to respectively execute second linear processing subtasks 5221 to obtain multiple second-level second linear processing results. Based on multiple second-level second linear processing results, multiple computing units may be used to execute full protocol communication subtasks 5222 to obtain multiple second-level full protocol communication results. Based on the second-level full protocol communication results and the second data to be fused, the computing unit may be used to execute the second fusion subtask 5223.

[0122] In the disclosed embodiment, the second to-be-fused data of the nth level fusion task in the Nth level fusion task is obtained based on the n-1th level second fusion result. For example, based on the full protocol communication result and a level 2 first fusion result, the second fusion subtask 5223 can be executed by the computing unit to obtain a level 2 second fusion result. Multiple computing units can respectively execute the second fusion subtask 5223 to obtain multiple level 2 second fusion results.

[0123] like Figure 5 As shown, according to the second fusion result of level 2, the computing unit can be used to perform the second normalization task 5224 to obtain the fourth execution result of level 2. Multiple computing units can respectively perform the second normalization task 5224 to obtain multiple fourth execution results of level 2.

[0124] In the embodiment of the present disclosure, in some implementations of the above operation S230, the task execution result can be obtained according to the fourth execution result. For example, the second-level fourth execution result can be used as the task execution result.

[0125] Through the embodiments of the present disclosure, the first parallel strategy and / or the second parallel strategy are adopted in different inference or training results, which provides a more flexible option for the deployment of large language models, and the method provided by the present disclosure can be applied in hardware devices of different scales. For example, the method provided by the present disclosure can be applied to edge computing devices and devices in large data centers.

[0126] Through the embodiments of the present disclosure, a variety of parallel strategies are used in the pre-filling stage, which can more efficiently and accurately process long sequence inputs, improve task execution efficiency in practical application scenarios (such as long document processing, multi-round conversations, etc.), and provide users with a smoother and more coherent interaction process.

[0127] It can be understood that the method of the present disclosure is described above, and the device of the present disclosure will be further described below.

[0128] Figure 6 is a block diagram of a task execution device according to an embodiment of the present disclosure.

[0129] like Figure 6 As shown, the apparatus 600 may include a first execution module 610 , a second execution module 620 and a third execution module 630 .

[0130] The first execution module 610 is used to use multiple computing units to respectively execute the tasks to be processed according to the first data to be processed and the first parallel strategy, so as to obtain multiple first execution results.

[0131] The second execution module 620 is used to use multiple computing units to respectively execute multiple first to-be-executed subtasks of the fusion task according to the multiple first execution results to obtain a second execution result. At least one first to-be-executed subtask is executed according to the second parallel strategy.

[0132] The third execution module 630 is used to use multiple computing units to respectively execute at least one second to-be-executed subtask of the fusion task according to the second to-be-processed data to obtain a task execution result. The second to-be-processed data is obtained according to the second execution result.

[0133] In some embodiments, the plurality of first subtasks to be executed include a distributed protocol communication subtask and a full collection communication subtask. The second execution module includes: a first execution submodule, for executing the distributed protocol communication subtasks using a plurality of computing units respectively according to the plurality of first execution results, so as to obtain a plurality of distributed protocol communication results according to a second parallel strategy. A second execution submodule, for executing the full collection communication subtasks using a plurality of computing units respectively according to the plurality of distributed protocol communication results, so as to obtain a second execution result according to a second parallel strategy.

[0134] In some embodiments, the plurality of first subtasks to be executed further include a first linear processing subtask, and the first execution submodule includes: a first execution unit, configured to respectively execute the first linear processing subtask using the plurality of computing units according to the plurality of first execution results to obtain the plurality of first linear processing results; and a second execution unit, configured to respectively execute the distributed protocol communication subtask using the plurality of computing units according to the plurality of first linear processing results to obtain the plurality of distributed protocol communication results according to the second parallel strategy.

[0135] In some embodiments, the multiple first subtasks to be executed also include a first fusion subtask and a first normalization subtask. The second execution submodule includes: a third execution unit, which is used to use multiple computing units to respectively execute the first fusion subtask according to the first data to be fused and the multiple distributed protocol communication results to obtain multiple first fusion results. A fourth execution unit, which is used to use multiple computing units to respectively execute the first normalization subtask according to the multiple first fusion results to obtain multiple first normalization results. A fifth execution unit, which is used to use multiple computing units to respectively execute the full collection communication subtask according to the multiple first normalization results to obtain the second execution result according to the second parallel strategy.

[0136] In some embodiments, the above-mentioned device also includes: a switching module, which is used to switch multiple first sub-tasks to be executed of the fusion task to multiple second sub-tasks to be executed in response to determining that the scale of the input data of the fusion task is less than or equal to a preset scale threshold, wherein the input data of the fusion task is multiple first execution results or data obtained according to the second execution results.

[0137] In some embodiments, at least one second to-be-executed subtask includes a full protocol communication subtask. The third execution module includes: a third execution submodule, configured to execute the full protocol communication subtask in at least one second to-be-executed subtask using multiple computing units according to the second to-be-processed data, and obtain a task execution result.

[0138] In some embodiments, the third execution submodule includes: a sixth execution unit, which is used to use multiple computing units to respectively execute the tasks to be processed according to the second data to be processed and the first parallel strategy to obtain multiple third execution results. A seventh execution unit is used to use multiple computing units to respectively execute at least one second subtask to be executed of the fusion task according to the multiple third execution results to obtain a fourth execution result. At least one second subtask to be executed is executed according to the first parallel strategy, and at least one second subtask to be executed includes a full protocol communication subtask. An eighth execution unit is used to obtain a task execution result according to the fourth execution result.

[0139] In some embodiments, at least one second subtask to be executed also includes a second linear processing subtask, a second fusion subtask, and a second normalization subtask. The seventh execution unit includes: a first execution subunit, which is used to execute the second linear processing subtask using multiple computing units respectively according to multiple third execution results to obtain multiple third linear processing results. A second execution subunit is used to execute the full-protocol communication subtask using multiple computing units respectively according to multiple third linear processing results to obtain multiple full-protocol communication results according to the first parallel strategy. The third execution subunit is used to execute the second fusion subtask using multiple computing units respectively according to the second data to be fused and multiple full-protocol communication results to obtain multiple second fusion results. The fourth execution subunit is used to execute the second normalization subtask using multiple computing units respectively according to the multiple second fusion results to obtain a fourth execution result.

[0140] In some embodiments, the fusion task is N-level, where N is an integer greater than 1. The first data to be fused of the first-level fusion task in the N-level fusion task is obtained based on the first input data. The first data to be fused of the n-th-level fusion task in the N-level fusion task is obtained based on the first fusion result of the n-1-th level, where n is an integer greater than 1 and less than or equal to N.

[0141] In some embodiments, there are N levels of tasks to be processed, and the N levels of tasks to be processed are at least one of attention tasks, activation tasks, and linear processing tasks. The first execution module includes: a fourth execution submodule, which is used to use multiple computing units to respectively execute at least one i-th level to be processed task according to the i-th level first to be processed data and the first parallel strategy, and obtain multiple i-th level first execution results. i is an integer greater than or equal to 1 and less than or equal to N, and N is an integer greater than 1.

[0142] In some embodiments, it also includes: a fourth execution module, which is used to perform a normalization task using a computing unit according to the first input data to obtain the first level of first data to be processed among the N levels of first data to be processed.

[0143] In some embodiments, when i is greater than 1, at least one i-th level to-be-processed task among the N-level to-be-processed tasks includes an i-th level linear processing task and an i-th level activation task, and the i-th level first to-be-processed data is obtained based on the i-th level second execution result. The fourth execution submodule includes: a ninth execution unit, for executing the i-th level linear processing task respectively using a plurality of computing units according to the i-th level first to-be-processed data and the first parallel strategy, to obtain a plurality of i-th level third linear processing results. A tenth execution unit, for executing the i-th level activation task using a plurality of computing units according to the plurality of i-th level third linear processing results, to obtain a plurality of i-th level first execution results.

[0144] In some embodiments, the fusion task is of N levels, where N is an integer greater than 1. The second data to be fused of the first level fusion task in the N-level fusion task is obtained based on the second input data, and the second input data is obtained based on the second execution result. The second data to be fused of the n-th level fusion task in the N-level fusion task is obtained based on the second fusion result of the n-1th level, where n is an integer greater than 1 and less than or equal to N.

[0145] In some embodiments, the tasks to be processed are of level N, and the tasks to be processed of level N are at least one of attention tasks, activation tasks, and linear processing tasks, and N is an integer greater than 1. The sixth execution unit includes: a fifth execution subunit, configured to execute at least one task to be processed of level i using multiple computing units according to the second data to be processed of level i and the first parallel strategy, respectively, to obtain multiple third execution results of level i. i is an integer greater than or equal to 1 and less than or equal to N.

[0146] In some embodiments, the apparatus further comprises: a fifth execution module, configured to perform a normalization task using a computing unit according to the second input data to obtain first-level second data to be processed among the N-level second data to be processed. The second input data is obtained according to the second execution result.

[0147] In some embodiments, when i is greater than 1, at least one of the N-level tasks to be processed includes an i-level linear processing task and an i-level activation task, and the i-level second data to be processed is obtained according to the i-1-level fourth execution result. The fifth execution subunit is further used to: according to the i-level second data to be processed and the first parallel strategy, use multiple computing units to respectively execute the i-level linear processing tasks to obtain multiple i-level fourth linear processing results. According to the multiple i-level fourth linear processing results, use multiple computing units to execute the i-level activation task to obtain multiple i-level third execution results.

[0148] In some embodiments, the first parallel strategy includes a tensor parallel strategy, and the second parallel strategy includes a sequence parallel strategy.

[0149] It can be understood that the apparatus of the present disclosure is described above, and the device of the present disclosure including the apparatus will be described below.

[0150] Figure 7 is a block diagram of a task execution device according to an embodiment of the present disclosure.

[0151] like Figure 7 As shown, the device 70 may include a task execution device 700. The task execution device 700 may be the device 600 described above.

[0152] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0153] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0154] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0155] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 to a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0156] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0157] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSP), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as a text processing method and / or a deployment method of a deep learning framework. For example, in some embodiments, the text processing method and / or the deployment method of a deep learning framework may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the text processing method and / or the method for deploying the deep learning framework described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the text processing method and / or the method for deploying the deep learning framework in any other appropriate manner (e.g., by means of firmware).

[0158] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system on chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0159] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0160] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories, read-only memories, erasable programmable read-only memories (EPROM) or flash memories, optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0161] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) display or a liquid crystal display (LCD)) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0162] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a Local Area Network (LAN), a Wide Area Network (WAN), and the Internet.

[0163] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0164] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0165] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A task execution method, comprising: According to the first data to be processed and the first parallel strategy, using multiple computing units to respectively execute the tasks to be processed, to obtain multiple first execution results; According to the plurality of the first execution results, using the plurality of the computing units to respectively execute the plurality of first to-be-executed subtasks of the fusion task to obtain a second execution result, wherein at least one of the first to-be-executed subtasks is executed according to the second parallel strategy; According to the second data to be processed, the plurality of computing units are used to respectively execute at least one second subtask to be executed of the fusion task to obtain a task execution result, wherein the second data to be processed is obtained according to the second execution result.

2. The method according to claim 1, wherein: The plurality of first to-be-performed subtasks include a distributed protocol communication subtask and a full collection communication subtask, The obtaining of the second execution result by using the plurality of computing units to respectively execute the plurality of first to-be-executed subtasks of the fusion task according to the plurality of the first execution results comprises: According to the plurality of the first execution results, using the plurality of the computing units to respectively execute the distributed protocol communication subtasks, so as to obtain a plurality of distributed protocol communication results according to the second parallel strategy; According to the plurality of the scatter-reduce communication results, the plurality of the computing units are used to respectively execute the full-gather communication subtasks, so as to obtain the second execution result according to the second parallel strategy.

3. The method according to claim 2, wherein: The plurality of first to-be-performed subtasks further include a first linear processing subtask, The step of using a plurality of the computing units to respectively execute the distributed protocol communication subtasks according to the plurality of the first execution results, so as to obtain a plurality of distributed protocol communication results according to the second parallel strategy, comprises: According to the plurality of first execution results, using the plurality of computing units to respectively execute the first linear processing subtasks to obtain a plurality of first linear processing results; According to the plurality of the first linear processing results, the plurality of the computing units are used to respectively execute the distributed protocol communication subtasks, so as to obtain the plurality of the distributed protocol communication results according to the second parallel strategy.

4. The method according to claim 2, wherein: The plurality of first subtasks to be performed further include a first fusion subtask and a first normalization subtask. The using a plurality of the computing units to respectively execute the full-collection communication subtasks according to the plurality of the distributed protocol communication results, so as to obtain the second execution result according to the second parallel strategy, comprises: According to the first data to be fused and the plurality of the decentralized protocol communication results, using the plurality of the computing units to respectively execute the first fusion subtasks to obtain a plurality of first fusion results; According to the plurality of the first fusion results, using the plurality of the computing units to respectively perform the first normalization subtasks to obtain a plurality of first normalization results; According to the plurality of the first normalization results, the full collection communication subtasks are respectively executed using the plurality of the computing units to obtain the second execution result according to the second parallel strategy.

5. The method according to claim 1, further comprising: In response to determining that the scale of the input data of the fusion task is less than or equal to a preset scale threshold, the multiple first subtasks to be executed of the fusion task are switched to the multiple second subtasks to be executed, wherein the input data of the fusion task is the multiple first execution results or data obtained according to the second execution results.

6. The method according to claim 1, wherein: At least one of the second to-be-performed subtasks includes a full protocol communication subtask, The step of using the plurality of computing units to respectively execute at least one second to-be-executed subtask of the fusion task according to the second to-be-processed data to obtain a task execution result includes: According to the second data to be processed, a plurality of the computing units are used to respectively execute at least one full protocol communication subtask in the second subtask to be executed to obtain the task execution result.

7. The method according to claim 6, wherein: The step of using the plurality of computing units to respectively execute at least one full protocol communication subtask in the second subtask to be executed according to the second data to be processed, and obtaining the task execution result includes: According to the second data to be processed and the first parallel strategy, using the plurality of computing units to respectively execute the tasks to be processed, to obtain a plurality of third execution results; According to the plurality of the third execution results, using the plurality of the computing units to respectively execute at least one of the second to-be-executed subtasks of the fusion task, to obtain a fourth execution result, wherein at least one of the second to-be-executed subtasks is executed according to the first parallel strategy, and at least one of the second to-be-executed subtasks includes the full protocol communication subtask; The task execution result is obtained according to the fourth execution result.

8. The method according to claim 7, wherein: At least one of the second subtasks to be performed further includes a second linear processing subtask, a second fusion subtask, and a second normalization subtask. The using the plurality of the computing units to respectively execute at least one of the second to-be-executed subtasks of the fusion task according to the plurality of the third execution results to obtain a fourth execution result includes: According to the plurality of the third execution results, using the plurality of the computing units to respectively execute the second linear processing subtasks to obtain a plurality of third linear processing results; According to the plurality of the third linear processing results, using the plurality of the computing units to respectively execute the full protocol communication subtasks, so as to obtain a plurality of full protocol communication results according to the first parallel strategy; According to the second data to be fused and the plurality of full protocol communication results, using the plurality of computing units to respectively execute the second fusion subtasks to obtain a plurality of second fusion results; According to the multiple second fusion results, the multiple computing units are used to respectively execute the second normalization subtask to obtain the fourth execution result.

9. The method according to claim 4, wherein: The fusion task is of level N, where N is an integer greater than 1. The first to-be-fused data of the first-level fusion task in the N-level fusion task is obtained according to the first input data; The first to-be-fused data of the nth level fusion task in the N-level fusion tasks is obtained according to the first fusion result of the n-1th level, where n is an integer greater than 1 and less than or equal to N.

10. The method according to claim 1, wherein: The tasks to be processed are of N levels, and the N-level tasks to be processed are at least one of attention tasks, activation tasks, and linear processing tasks. The method of using a plurality of computing units to respectively execute the tasks to be processed according to the first data to be processed and the first parallel strategy to obtain a plurality of first execution results includes: According to the i-th level first data to be processed and the first parallel strategy, multiple computing units are used to respectively execute at least one i-th level task to be processed, and multiple i-th level first execution results are obtained, where i is an integer greater than or equal to 1 and less than or equal to N, and N is an integer greater than 1.

11. The method according to claim 10, further comprising: According to the first input data, the computing unit is used to perform a normalization task to obtain first-level first data to be processed among N levels of the first data to be processed.

12. The method according to claim 10, wherein: In the case where i is greater than 1, at least one i-th level to-be-processed task among the N-th level to-be-processed tasks includes an i-th level linear processing task and an i-th level activation task, and the i-th level first to-be-processed data is obtained according to the i-1-th level second execution result, The method of using the plurality of computing units to respectively execute at least one i-th level to-be-processed task according to the i-th level first to-be-processed data and the first parallel strategy to obtain a plurality of i-th level first execution results includes: According to the i-th level first to-be-processed data and the first parallel strategy, using a plurality of the computing units to respectively perform the i-th level linear processing tasks, to obtain a plurality of i-th level third linear processing results; According to the multiple i-th level third linear processing results, the multiple computing units are used to execute the i-th level activation task to obtain multiple i-th level first execution results.

13. The method according to claim 12, wherein: The fusion task is of level N, where N is an integer greater than 1. The second to-be-fused data of the first-level fusion task in the N-level fusion task is obtained according to the second input data, and the second input data is obtained according to the second execution result; The second to-be-fused data of the nth-level fusion task in the N-level fusion task is obtained according to the second fusion result of the n-1th level, where n is an integer greater than 1 and less than or equal to N.

14. The method according to claim 10, wherein: The tasks to be processed are of N levels, and the N-level tasks to be processed are at least one of attention tasks, activation tasks, and linear processing tasks, and N is an integer greater than 1. The using the plurality of computing units to respectively execute the tasks to be processed according to the second data to be processed and the first parallel strategy to obtain a plurality of third execution results comprises: According to the i-th level second data to be processed and the first parallel strategy, multiple computing units are used to respectively execute at least one i-th level task to be processed to obtain multiple i-th level third execution results, where i is an integer greater than or equal to 1 and less than or equal to N.

15. The method according to claim 14, further comprising: According to the second input data, the normalization task is performed using the computing unit to obtain the first level of second data to be processed among the N levels of the second data to be processed, wherein the second input data is obtained according to the second execution result.

16. The method according to claim 14, wherein: In the case where i is greater than 1, at least one i-th level to-be-processed task among the N-th level to-be-processed tasks includes an i-th level linear processing task and an i-th level activation task, and the i-th level second to-be-processed data is obtained according to the fourth execution result of the i-1th level, The method of using a plurality of computing units to respectively execute at least one task to be processed at the i-th level according to the second data to be processed at the i-th level and the first parallel strategy to obtain a plurality of third execution results at the i-th level includes: According to the i-th level second to-be-processed data and the first parallel strategy, using a plurality of the computing units to respectively perform the i-th level linear processing tasks, to obtain a plurality of i-th level fourth linear processing results; According to the multiple i-th level fourth linear processing results, the multiple computing units are used to execute the i-th level activation task to obtain the multiple i-th level third execution results.

17. The method according to claim 1, wherein: The first parallel strategy includes a tensor parallel strategy, and the second parallel strategy includes a sequence parallel strategy.

18. A task execution device, comprising: A first execution module, configured to respectively execute the tasks to be processed using a plurality of computing units according to the first data to be processed and the first parallel strategy, and obtain a plurality of first execution results; a second execution module, configured to respectively execute, according to the plurality of the first execution results, the plurality of first to-be-executed subtasks of the fusion task using the plurality of the computing units, to obtain a second execution result, wherein at least one of the first to-be-executed subtasks is executed according to a second parallel strategy; The third execution module is used to use multiple computing units to respectively execute at least one second subtask to be executed of the fusion task according to the second data to be processed, so as to obtain a task execution result, wherein the second data to be processed is obtained according to the second execution result.

19. A task execution device, comprising the apparatus as claimed in claim 18.

20. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 17.

21. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 17.

22. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 17.

Images