Task execution method, device, computer equipment, storage medium and program product

Abstract

The present application relates to a task execution method, apparatus, computer device, computer-readable storage medium, and computer program product, and relates to the field of artificial intelligence chip technology. The present application can improve the processing efficiency of executing computing tasks and communication tasks on an artificial intelligence chip. The method includes: obtaining a task stream; in the same task stream, causing a first processing unit of the artificial intelligence chip to execute a computing task, and causing a second processing unit of the artificial intelligence chip to execute a communication task; wherein the computing task is independent of the communication task; the communication task is dependent on a corresponding computing task, and the communication task is independent of a non-corresponding computing task.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence chip technology, and in particular to a task execution method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the development of artificial intelligence technology, large language models have emerged. These models can be used to handle various natural language processing tasks, including but not limited to text generation, question-answering systems, and machine translation. For example, DeepSeek, a large language model, is currently widely used in scenarios such as intelligent dialogue, text generation, and code writing, helping users improve processing efficiency in various business operations.

[0003] Training and inference of large language models require extensive computing resources and are typically performed on AI chips. These chips are hardware specifically designed and optimized for AI tasks, including but not limited to GPUs (Graphics Processing Units), NPUs (Neural Network Processing Units), and GPGPUs (General-Purpose Computing on Graphics Processing Units).

[0004] When performing computations on an AI chip, both computational and communication tasks are involved. Computational tasks can be used to perform operations such as MMA (Matrix Multiply Accumulate), while communication tasks can be used to perform operations such as Reduce, which involves data transmission and information exchange.

[0005] For the execution of computing tasks and communication tasks on artificial intelligence chips, current technology can schedule the execution of computing tasks and communication tasks through the coordination of multiple different task streams. However, this requires interaction between different task streams, and its processing efficiency for the execution of computing tasks and communication tasks on artificial intelligence chips still needs to be improved. Summary of the Invention

[0006] Based on this, it is necessary to provide a task execution method, apparatus, computer equipment, computer-readable storage medium and computer program product to address the above technical issues.

[0007] In a first aspect, the present application provides a task execution method, comprising:

[0008] Get the task flow;

[0009] In the same task flow, causing the first processing unit of the artificial intelligence chip to perform a computing task, and causing the second processing unit of the artificial intelligence chip to perform a communication task;

[0010] The computing task is independent of the communication task; the communication task is independent of the corresponding computing task; and the communication task is independent of the non-corresponding computing task.

[0011] In a second aspect, the present application further provides a task execution device, comprising:

[0012] Acquisition module, used to obtain task flow;

[0013] an execution module, configured to, in the same task flow, cause the first processing unit of the artificial intelligence chip to perform a computing task, and cause the second processing unit of the artificial intelligence chip to perform a communication task;

[0014] The computing task is independent of the communication task; the communication task is independent of the corresponding computing task; and the communication task is independent of the non-corresponding computing task.

[0015] In a third aspect, the present application also provides an artificial intelligence device, comprising: one or more artificial intelligence chips; the artificial intelligence chips are used to execute the steps of the above method.

[0016] In a fourth aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0017] In a fifth aspect, the present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0018] In a sixth aspect, the present application also provides a computer program product, comprising a computer program, which implements the steps of the above method when executed by a processor.

[0019] The above-mentioned task execution method, apparatus, computer device, computer-readable storage medium, and computer program product enable the first processing unit of the artificial intelligence chip to perform a computing task and the second processing unit of the artificial intelligence chip to perform a communication task in the same task flow, thereby enabling the computing task and the communication task to be executed within a single task flow through scheduling between different hardware units. The computing task is independent of the communication task, and the communication task can be independent of non-corresponding computing tasks. In this case, the first processing unit and the second processing unit can execute their respective tasks simultaneously without blocking each other. When the communication task depends on the corresponding computing task, the first processing unit can complete the computing task without being blocked, and the communication task can process the communication task after the computing task is completed. Compared with the interaction between different task flows, the resource scheduling and switching efficiency within a single task flow is higher, which improves the processing efficiency of the execution of computing tasks and communication tasks on the artificial intelligence chip, and is shorter in time and has better performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A schematic diagram of the structure of an artificial intelligence device in one embodiment;

[0022] Figure 2 A flowchart of a task execution method in one embodiment;

[0023] Figure 3 A schematic diagram of a data processing flow in one embodiment;

[0024] Figure 4 A schematic diagram of time consumption in one embodiment;

[0025] Figure 5 is a structural block diagram of a task execution device in one embodiment;

[0026] Figure 6 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0028] The task execution method provided in the embodiment of the present application can be applied to Figure 1 The AI ​​device shown in the figure is a hardware device or system capable of performing AI tasks. The AI ​​device may include one or more AI chips (e.g., AI chip 0, AI chip 1, etc.). These multiple AI chips may be identical, partially identical, or completely different. AI chips include, but are not limited to, GPUs (Graphics Processing Units), NPUs (Neural Network Processing Units), and GPGPUs (General-Purpose Computing on Graphics Processing Units).

[0029] The task execution method provided in the embodiments of the present application can be executed by the control unit of an artificial intelligence chip in an artificial intelligence device. The artificial intelligence chip can include a first processing unit, a second processing unit, a buffer unit, and global memory. The first processing unit can be a processing unit for performing computational tasks, such as processing tasks related to matrix multiplication (such as matrix multiplication and accumulation, general matrix multiplication, convolution, etc.); the second processing unit can be a general-purpose processing unit in the artificial intelligence chip, which can be used to process operations other than matrix multiplication, such as addition and softmax operations. The second processing unit can be used to perform communication tasks and implement reduction operators such as allreduce and reduce-scatter. Inter-card communication can be achieved through the second processing units between different cards (between different artificial intelligence chips).

[0030] For the execution of computing tasks and communication tasks on artificial intelligence chips, current technology can schedule the execution of computing tasks and communication tasks through the coordination of multiple different task flows (such as the task flow of computing tasks and the task flow of communication tasks), so as to achieve, for example, parallel execution of the task flow of computing tasks and the task flow of communication tasks. Different processing logic or different kernels of the artificial intelligence chip are running on each task flow. This requires interaction between the task flow of computing tasks and the task flow of communication tasks. However, the scheduling granularity at the task flow level is relatively large, and its processing efficiency for the execution of computing tasks and communication tasks on artificial intelligence chips still needs to be improved.

[0031] In this regard, the task execution method provided by the embodiment of the present application enables the first processing unit of the artificial intelligence chip to execute the computing task and the second processing unit of the artificial intelligence chip to execute the communication task in the same task flow, so that the computing task and the communication task can be executed within a single task flow through scheduling between different hardware units, wherein the computing task does not depend on the communication task, and the communication task can be independent of the non-corresponding computing task. At this time, the first processing unit and the second processing unit can execute their respective tasks at the same time without blocking each other. When the communication task depends on the corresponding computing task, the first processing unit can complete the computing task without being blocked, and the communication task can process the communication task after the computing task is completed. Compared with the interaction between different task flows, the resource scheduling and switching efficiency within a single task flow is higher, which improves the processing efficiency of the execution of computing tasks and communication tasks on the artificial intelligence chip, and takes less time and has better performance.

[0032] In some embodiments, the computer device may include Figure 1 The artificial intelligence device shown. The computer device may be a server. The server may include a host and a device connected to each other. The host of the server may include a CPU (Central Processing Unit), and the device of the server may include Figure 1 The artificial intelligence device shown may include one or more artificial intelligence chips, including but not limited to GPU and GPGPU.

[0033] In an exemplary embodiment, Figure 2 As shown, a task execution method is provided, which can be applied to Figure 1 In the artificial intelligence chip of the artificial intelligence device shown, the method may include:

[0034] Step S201: Obtain the task flow.

[0035] Step S202: In the same task flow, the first processing unit of the artificial intelligence chip is caused to perform a computing task, and the second processing unit of the artificial intelligence chip is caused to perform a communication task. The computing task is independent of the communication task, the communication task is independent of the corresponding computing task, and the communication task is independent of the non-corresponding computing task.

[0036] In this embodiment, combined with Figure 1The control unit of the artificial intelligence chip can obtain a task flow, in which the control unit of the artificial intelligence chip causes the first processing unit to perform a computation task on related data, and causes the second processing unit to perform a communication task on related data. The task flow can be an ordered sequence of tasks that can be used to indicate the execution order of the computation tasks of the first processing unit and the communication tasks of the second processing unit. A computation task is independent of a communication task; the communication task can be dependent on a corresponding computation task or independent of a non-corresponding computation task. A communication task is dependent on a corresponding computation task if it must be performed based on the computation results obtained from the corresponding computation task; a communication task is independent of a non-corresponding computation task if it does not need to be performed based on the computation results obtained from the non-corresponding computation task. Thus, task switching and scheduling between the first and second processing units of the artificial intelligence chip can be implemented within a single task flow. If the communication task is independent of a non-corresponding computation task, the first and second processing units can execute their respective tasks simultaneously. That is, the computation task of the first processing unit and the communication task of the second processing unit can be executed simultaneously without blocking each other. When a communication task depends on a corresponding computing task, the first processing unit can complete the computing task without being blocked, and the second processing unit can execute the corresponding communication task after the first processing unit's computing task is completed. Compared to the interaction between different task flows in current technologies, this embodiment achieves higher efficiency in resource scheduling and switching within a single task flow, shortens time consumption, and improves performance.

[0037] The task execution method of this embodiment enables the first processing unit of the artificial intelligence chip to perform a computing task and the second processing unit of the artificial intelligence chip to perform a communication task within the same task flow. This allows the computing task and the communication task to be executed within a single task flow through scheduling between different hardware units. The computing task is independent of the communication task, and the communication task can be independent of non-corresponding computing tasks. In this case, the first processing unit and the second processing unit can execute their respective tasks simultaneously without blocking each other. When the communication task depends on the corresponding computing task, the first processing unit can complete the computing task without being blocked, and the communication task can process the communication task after the computing task is completed. Compared to the interaction between different task flows, resource scheduling and switching within a single task flow are more efficient, improving the processing efficiency of the execution of computing and communication tasks on the artificial intelligence chip, and are less time-consuming and have better performance.

[0038] In an exemplary embodiment, in the same task flow of step S202, causing the first processing unit of the artificial intelligence chip to perform a computing task and causing the second processing unit of the artificial intelligence chip to perform a communication task may include:

[0039] In the same task flow, the first processing unit is made to execute a data calculation task to obtain a data calculation result, and the second processing unit is made to execute a communication task of the calculation result.

[0040] In this embodiment, combined with Figure 1 The control unit of the artificial intelligence chip can obtain input data, which can be tensor data or data blocks after tensor data is blocked. The data can be transmitted by the control unit to the first processing unit for processing. In the same task flow, the control unit can enable the first processing unit to perform a calculation task on the data to obtain a calculation result of the data. For example, matrix multiplication and accumulation, general matrix multiplication, convolution and other operations can be performed on the data to obtain the calculation result of the data. The control unit of the artificial intelligence chip can also enable the second processing unit to obtain the calculation result of the data obtained by the first processing unit and perform a communication task of the calculation result of the data. For example, full reduction, reduce-and-scatter and other operations can be performed on the calculation result of the data.

[0041] The solution of this embodiment can be based on the first processing unit and the second processing unit of the artificial intelligence chip to implement the calculation and communication operations of the first processing unit and the second processing unit on data in a single task flow, thereby improving the efficiency of data processing compared to the interaction between different task flows.

[0042] In an exemplary embodiment, further, in the same task flow, the aforementioned step of causing the first processing unit to execute a data computing task to obtain a data computing result, and causing the second processing unit to execute a communication task of the computing result, may include:

[0043] In the same task flow, the first processing unit is enabled to sequentially execute the calculation task of each data block of the data to obtain the calculation result of each data block, and the second processing unit is enabled to sequentially execute the communication task of the calculation result of each data block obtained.

[0044] In this embodiment, within the same task flow, the control unit of the artificial intelligence chip can cause the first processing unit to sequentially perform the computation task for each data block of (tensor) data, thereby enabling the first processing unit to sequentially obtain the computation result for each data block. For each computation result of each data block obtained sequentially by the first processing unit, within the same task flow, the control unit can cause the second processing unit to sequentially perform the communication task for each computation result of the data block obtained by the first processing unit. As an example, the first processing unit can sequentially perform the computation task on data block 0 and data block 1, obtaining computation results for data block 0 and data block 1, respectively. After the first processing unit obtains the computation result for data block 0, the first processing unit can continue to perform the computation task for data block 1. Simultaneously, after the first processing unit obtains the computation result for data block 0, the second processing unit can obtain the computation result for data block 0 and then perform the communication task for the computation result of data block 0. That is, within a single task flow, while the first processing unit is performing the computation task on one data block, the second processing unit can perform the communication task for the computation result of another data block. It can be understood that the calculation and communication tasks performed on the data will have corresponding time consumption. The time consumption of the calculation task performed on the data can be recorded as the calculation task time consumption, and the time consumption of the communication task performed on the data can be recorded as the communication task time consumption. The solution of this embodiment can realize the partial parallelization of the calculation and communication tasks of data execution at the level of a single task flow, so that the time consumption of part of the calculation task and the time consumption of part of the communication task are mutually covered, thereby improving its data processing efficiency. As in the above example, when the first processing unit continues to execute the calculation task of data block 1, the second processing unit can execute the communication task of the calculation result of data block 0 in parallel, so that the time consumption of the calculation task of data block 1 and the time consumption of the communication task of the calculation result of data block 0 are mutually covered, thereby improving data processing efficiency. From this, it can also be explained that the computing tasks of data block 0 and data block 1 do not depend on the communication task, the communication task of the computing result of data block 0 depends on the computing task of data block 0 (i.e. the corresponding computing task), the communication task of the computing result of data block 1 depends on the computing task of data block 1 (i.e. the corresponding computing task), and the communication task of the computing result of data block 0 does not depend on the computing task of data block 1 (i.e. the non-corresponding computing task), and the communication task of the computing result of data block 1 does not depend on the computing task of data block 0 (i.e. the non-corresponding computing task).

[0045] In an exemplary embodiment, further, in the above method, in the same task flow, causing the first processing unit to perform a data computing task to obtain a data computing result, and causing the second processing unit to perform a communication task of the computing result, may further include the following steps:

[0046] In the same task flow, the first processing unit writes the calculation result of each data block into the buffer unit of the artificial intelligence chip, and the second processing unit reads the calculation result of the data block from the buffer unit.

[0047] In this embodiment, combined with Figure 1 , the artificial intelligence chip may include a cache / buffer unit, which may be a tensor buffer unit. Figure 3 In the same task flow, the control unit of the artificial intelligence chip can cause the first processing unit to sequentially write the computation results of each data block of (tensor) data into the buffer unit of the artificial intelligence chip. That is, each time a computation task is performed on data block i, the computation result of data block i is written to the buffer unit until all data blocks are processed, i.e., i is greater than the total number of data blocks, where the total number of data blocks is the total number of data blocks. Because the communication task for the computation result of data block i in the second processing unit depends on the computation task for data block i in the first processing unit, each time the first processing unit writes the computation result of data block i into the buffer unit, the second processing unit reads the computation result of data block i from the buffer unit and performs the communication task on the computation result of data block i, until the computation results of all data blocks are processed, i.e., i is greater than the total number of data blocks.

[0048] In an exemplary embodiment, further, in the above method, in the same task flow, causing the first processing unit to perform a data computing task to obtain a data computing result, and causing the second processing unit to perform a communication task of the computing result, may further include the following steps:

[0049] In the same task flow, the first processing unit is enabled to divide the data into blocks to obtain each data block.

[0050] In this embodiment, in the same task flow, the control unit of the artificial intelligence chip can cause the first processing unit to sequentially execute the calculation task of each data block of the data to obtain the calculation result of each data block, and the control unit can cause the second processing unit to sequentially execute the communication task of the calculation result of the data block obtained by the first processing unit each time. Moreover, before the first processing unit sequentially executes the calculation task of each data block of the data, the control unit can also cause the first processing unit to block the data to obtain each data block. The first processing unit can block the data according to a preset block method to obtain each data block. The preset block method can instruct the first processing unit to block the data according to a preset granularity to obtain each data block. The solution of this embodiment can execute the data block within a single task flow, without the need to split the (tensor) data in the upper-level software, and can save the change of the framework layer.

[0051] In an exemplary embodiment, the above method may further include the following steps:

[0052] Obtain the correspondence between the total time consumption and various data block methods; wherein, the total time consumption includes the calculation task unit time consumption, the communication task unit time consumption and the task overlapping time consumption; according to the correspondence, determine the target block method; wherein, the target block method is a data block method that makes the total time consumption meet the target conditions; the target block method is used for the first processing unit to block the data.

[0053] The selection of different block partitioning methods (different granularities) will result in different time consumption for computing and communication tasks. This embodiment can determine the target block partitioning method used by the first processing unit to partition the data based on the correspondence between the total time consumption and various data block partitioning methods. As a result, the control unit of the artificial intelligence chip can, in the same task flow, cause the first processing unit to partition the (tensor) data according to the target block partitioning method to obtain each data block. The target block partitioning method refers to a data block partitioning method that ensures that the total time consumption meets the target condition. The target condition can be a condition related to the size of the total time consumption, such as the total time consumption being less than a certain total time consumption threshold, the total time consumption being minimized, etc., thereby reducing the time consumption of data processing through analysis and application of the target block partitioning method.

[0054] Specifically, the total time consumption may include the unit time consumption of the computing task, the unit time consumption of the communication task, and the task overlapping time consumption. Among them, the unit time consumption of the computing task refers to the time consumption of executing the computing task of a data block, and the unit time consumption of the communication task refers to the time consumption of executing the communication task of the calculation result of a data block. As for the task overlapping time consumption, as described in the previous embodiment, there will be corresponding time consumption for the computing and communication tasks executed on the data. At the level of a single task flow, the computing and communication tasks of the data execution can be partially parallelized, so that the time consumption of part of the computing task and the time consumption of part of the communication task are mutually covered. For example, when the first processing unit continues to execute the computing task of data block 1, the second processing unit can execute the communication task of the calculation result of data block 0 in parallel, so that the time consumption of the computing task of data block 1 and the time consumption of the communication task of the calculation result of data block 0 are mutually covered. It can be understood that the task overlapping time consumption refers to the time consumption of the overlapping part of the computing task of part of the data block and the communication task of the calculation result of part of the data block.

[0055] For the total time, as an example, Figure 4As shown, the computational tasks for four data blocks are used as an example. The computational tasks for the four data blocks are denoted as computations 1 to 4, and the communication tasks for the computational results of the four data blocks are denoted as communications 1 to 4. The total time consumption can be divided into case (a) and case (b). In case (a), the computational time consumption for a single data block (computational task unit time consumption) can be greater than or equal to the communication time consumption for a single data block (communication task unit time consumption), that is, computation 1 can be greater than or equal to communication 1, and so on. In case (b), the computational time consumption for a single data block is less than the communication time consumption for a single data block, that is, computation 1 is less than communication 1, and so on. In this regard, in cases (a) and (b), the task overlap time is the time consumed during the parallel period of calculations 2 to 4 and communications 1 to 3. The task overlap time is determined by the larger of the time consumed during the parallel period of calculations 2 to 4 and communications 1 to 3. In case (a), the task overlap time is the time consumed during calculations 2 to 4, and in case (b), the task overlap time is the time consumed during communications 1 to 3.

[0056] For this purpose, let's assume that the size of input matrix A (data) is [M, K], and the size of input matrix B (data) is [K, N]. The granularity of the computation task for each block of dimensions M, N, and K in the input (tensor) data can be denoted as TM, TN, and TK. The size of the computation result for each data block is [TM, TN], and this computation result is used by the communication task.

[0057] Therefore, the corresponding relationship between the total time consumption and various data block methods can be expressed as:

[0058] ;

[0059] Where T represents the total time, t1 represents the unit time of the computing task, t2 represents the unit time of the communication task, and ceiling represents rounding up.

[0060] Specifically, as an example, the unit time consumption of the calculation task t1 can be further expressed as:

[0061] ,

[0062] ;

[0063] Among them, load can be expressed as the time required for the first processing unit to read input data from the global memory, Calc can be expressed as the time required for the first processing unit to perform related operations of the calculation task, such as the time required for matrix multiplication, store can be expressed as the time required for the first processing unit to write the calculation results of the data blocks into the buffer unit, load_bandwidth can be expressed as the bandwidth for reading data, HW_POWER can be expressed as the computing power of the first processing unit, store_bandwidth can be expressed as the bandwidth for writing data, and dsize can be expressed as the data size.

[0064] Specifically, as an example, the unit time consumption t2 of the communication task can be further expressed as:

[0065] ;

[0066] Bandwidth can be expressed as the bandwidth of inter-card communication.

[0067] It should be noted that the above formulas for calculating the unit time consumption of the task and the unit time consumption of the communication task are only used as an example. In specific applications, corresponding formulas can be used for calculation according to different implementation methods and algorithms of the actual kernel.

[0068] For the same hardware, the computing power of the processing unit, the read / write bandwidth within the card, and the bandwidth for inter-card communication are generally determined. The solution of this embodiment can select a target segmentation method that meets the target conditions based on the above correspondence. The appropriate target feedback method is used for the first processing unit to segment data, improving data processing efficiency and optimizing the performance of the fusion of computing and communication tasks.

[0069] In an exemplary embodiment, the first processing unit includes a tensor processing unit; the second processing unit includes a vector processing unit; and the task flow indicates an execution order of computing tasks of the tensor processing unit and communication tasks of the vector processing unit.

[0070] In this embodiment, the artificial intelligence device may include one or more artificial intelligence chips, such as GPUs. The artificial intelligence chips may include tensor processing units (Tensor Cores) and vector processing units (Vector Cores). The tensor processing unit may be a computing core with dedicated hardware acceleration, which can be used to process computationally intensive matrix multiplication operations (such as matrix multiplication and accumulation (MMA), general matrix multiplication (GEMM), and convolution operations). The vector processing unit may be a general-purpose processing unit of an artificial intelligence chip, such as a GPU, which can be used to process operations other than matrix multiplication, such as Add and Softmax.

[0071] The solution of this embodiment can be based on the tensor processing unit and vector processing unit included in the artificial intelligence chip, and realize the switching and scheduling of the tensor processing unit and the vector processing unit in a single task flow. When there is no data dependency, the tensor processing unit and the vector processing unit can execute their respective tasks at the same time without blocking each other. When there is data dependency between the two, the tensor processing unit can complete the computing task in sequence without being blocked, and the vector processing unit can perform the corresponding communication task after the computing task of the tensor processing unit is completed. In this way, computing and communication can be realized in parallel through scheduling switching between different hardware units, and the execution of all the above tasks can be within a single task flow, without the need to split the tensor in the upper-level software, and can save the modification of the framework layer. Moreover, compared with the interaction between different task flows, the resource scheduling and switching efficiency within a single task flow is higher, which improves the processing efficiency of the execution of computing tasks and communication tasks on the artificial intelligence chip, and takes less time and has better performance.

[0072] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0073] Based on the same inventive concept, the present application also provides a task execution device for implementing the aforementioned task execution method. The implementation solution provided by this device is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations of one or more task execution device embodiments provided below can be found in the above-mentioned limitations of the task execution method and will not be repeated here.

[0074] In an exemplary embodiment, Figure 5 As shown, a task execution device is provided, and the device 500 may include:

[0075] Acquisition module 501, used to acquire task flow;

[0076] An execution module 502 is configured to, in the same task flow, cause the first processing unit of the artificial intelligence chip to perform a computing task, and cause the second processing unit of the artificial intelligence chip to perform a communication task;

[0077] The computing task is independent of the communication task; the communication task is independent of the corresponding computing task; and the communication task is independent of the non-corresponding computing task.

[0078] In an exemplary embodiment, the execution module 502 is configured to enable the first processing unit to execute a data computing task to obtain a computing result of the data, and enable the second processing unit to execute a communication task of the computing result in the same task flow.

[0079] In an exemplary embodiment, the execution module 502 is used to enable the first processing unit to sequentially execute the calculation task of each data block of the data to obtain the calculation result of each data block in the same task flow, and to enable the second processing unit to sequentially execute the communication task of the calculation result of the data block obtained each time.

[0080] In an exemplary embodiment, the execution module 502 is also used to enable the first processing unit to write the calculation results of each data block into the buffer unit of the artificial intelligence chip, and enable the second processing unit to read the calculation results of the data block from the buffer unit in the same task flow.

[0081] In an exemplary embodiment, the execution module 502 is further configured to enable the first processing unit to divide the data into blocks in the same task flow to obtain each data block.

[0082] In an exemplary embodiment, the execution module 502 is also used to obtain the correspondence between the total time consumption and various data block methods; the total time consumption includes the calculation task unit time consumption, the communication task unit time consumption and the task overlapping time consumption; according to the correspondence, the target block method is determined; wherein, the target block method is the data block method that makes the total time consumption meet the target condition; the target block method is used by the first processing unit to block the data.

[0083] In an exemplary embodiment, the first processing unit includes a tensor processing unit; the second processing unit includes a vector processing unit; and the task flow indicates an execution order of the computing task of the tensor processing unit and the communication task of the vector processing unit.

[0084] Each module in the task execution device described above may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0085] In an exemplary embodiment, an artificial intelligence device is provided, which may include one or more artificial intelligence chips; the artificial intelligence chip can be used to execute the steps in the embodiments of the above-mentioned task execution method.

[0086] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 6 As shown. The computer device may include a processor, a memory, an input / output interface and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external device in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a task execution method is implemented.

[0087] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0088] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0089] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0090] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0091] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0092] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0093] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0094] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A task execution method, characterized in that: The method comprises: Get the task flow; In the same task flow, the tensor processing unit in the artificial intelligence chip is caused to sequentially perform matrix multiplication-related operations on each data block to obtain a calculation result for each data block, and the vector processing unit in the artificial intelligence chip is caused to sequentially perform a reduction operation on the calculation result of each data block obtained; the tensor processing unit is used to process matrix multiplication-related operations; and the vector processing unit is used to process operations other than matrix multiplication-related operations; Wherein, the matrix multiplication related operation does not depend on the reduction operation; the reduction operation depends on the corresponding matrix multiplication related operation, and the reduction operation does not depend on the non-corresponding matrix multiplication related operation; wherein, the reduction operation depends on the corresponding matrix multiplication related operation, and the reduction operation needs to be performed based on the calculation result obtained by the corresponding matrix multiplication related operation; the reduction operation does not depend on the non-corresponding matrix multiplication related operation, and the reduction operation does not need to be performed based on the calculation result obtained by the non-corresponding matrix multiplication related operation; when the reduction operation does not depend on the non-corresponding matrix multiplication related operation, the tensor processing unit and the vector processing unit perform their respective tasks simultaneously; when the reduction operation depends on the corresponding matrix multiplication related operation, the vector processing unit performs the corresponding reduction operation after the matrix multiplication related operation of the tensor processing unit is completed; The method further comprises: In the same task flow, the tensor processing unit is caused to block the data according to a target block mode to obtain each data block; the target block mode is determined according to the correspondence between the total time consumption and various data block modes; the total time consumption includes the computing task unit time consumption, the communication task unit time consumption and the task overlap time consumption; wherein the task overlap time consumption is the time consumption of the overlapping part of the matrix multiplication related operation of the part of the data block and the reduction operation of the calculation result of the part of the data block; The corresponding relationship is expressed as: Where T represents the total time, t1 represents the unit time of the computing task, t2 represents the unit time of the communication task, ceiling represents rounding up, M and N represent the size of the data, and TM and TN represent the granularity of the data block.

2. The method according to claim 1, characterized in that Also includes: In the same task flow, the tensor processing unit writes the calculation results of each data block into the buffer unit of the artificial intelligence chip, and the vector processing unit reads the calculation results of the data block from the buffer unit.

3. The method according to claim 1, characterized in that Also includes: Obtain the corresponding relationship between the total time consumption and various data block methods; The total time consumption includes the calculation task unit time consumption, the communication task unit time consumption and the task overlap time consumption; Determining a target block division method according to the corresponding relationship; The target block partitioning method is the data block partitioning method that makes the total time consumption meet the target condition.

4. The method according to any one of claims 1 to 3, characterized in that The task flow indicates an execution order of the matrix multiplication-related operations of the tensor processing unit and the reduction operations of the vector processing unit.

5. An artificial intelligence device, characterized in that: The device includes: one or more artificial intelligence chips; the artificial intelligence chip is used to execute the steps of any one of the methods of claims 1 to 4.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images