Operator task execution method, artificial intelligence chip and electronic device

Abstract

The embodiment of the present invention provides an operator task execution method, an artificial intelligence chip, and an electronic device for reducing the load of the computing core in the artificial intelligence chip and improving the execution efficiency of the operator. The method includes: the computing core executes a first task in the operator, and the first task includes at least one of the following: a computing task, a first data handling task; and a DMA controller executes a second data handling task in the operator. The DMA controller in the artificial intelligence chip shares part or all of the data handling tasks in the operator, thereby reducing the load of the computing core in the artificial intelligence chip and improving the execution efficiency of the operator.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to an operator task execution method, an artificial intelligence chip, and an electronic device. Background Art

[0002] The operations of artificial intelligence models can be implemented by operators in the computation graph. A computation graph is a multi-graph structure used to represent the computational tasks and data flow processes of artificial intelligence models. Operators refer to various operations performed on tensors at each layer in the artificial intelligence model. For example, the convolution operation performed by the convolution layer of the artificial intelligence model on the input data of the artificial intelligence model is a convolution operator.

[0003] In practical applications, artificial intelligence chips can be used to implement the calculations of artificial intelligence models. Currently, the computing cores in the artificial intelligence chips perform all the tasks of each operator. The load of the computing cores is high and the execution efficiency of the operators is low. Summary of the invention

[0004] Embodiments of the present invention provide an operator task execution method, an artificial intelligence chip, and an electronic device, which are used to reduce the load of a computing core in an artificial intelligence chip and improve operator execution efficiency.

[0005] In a first aspect, the present application provides an operator task execution method, which is applied to an artificial intelligence chip, wherein the artificial intelligence chip includes a computing core and a direct memory access DMA controller, and the method includes:

[0006] The computing core executes a first task in the operator, where the first task includes at least one of the following: a computing task and a first data handling task;

[0007] The DMA controller executes the second data transfer task in the operator.

[0008] In the above technical solution, the operator includes a first task and a second data transport task. The computing core in the artificial intelligence chip executes the first task in the operator, and the DMA controller in the artificial intelligence chip executes the second data transport task. That is to say, the DMA controller in the artificial intelligence chip shares part or all of the data transport tasks in the operator, thereby reducing the load of the computing core in the artificial intelligence chip and improving the execution efficiency of the operator.

[0009] In a possible implementation, the first task is a computing task, the input data of the operator includes first data, the first data is associated with the computing task, and the artificial intelligence chip further includes a memory;

[0010] The computing core executes the first task in the operator, including:

[0011] The computing core reads first data from a first storage location of the memory;

[0012] Calculate and check the first data to obtain a first calculation result;

[0013] The computing core writes the first computing result to the second storage location of the memory.

[0014] In the above technical solution, when the operator includes a computing task and a second data transporting task, all the data transporting tasks are executed by the DMA controller, which can reduce the load of the computing core to the greatest extent.

[0015] In a possible implementation, the first task is a computing task and a first data handling task, the input data of the operator includes first data and second data, the first data is associated with the computing task, the second data is associated with the first data handling task, and the artificial intelligence chip further includes a memory;

[0016] The computing core executes the first task in the operator, including:

[0017] The computing core reads first data from a first storage location of the memory, computes the first data to obtain a first computing result, and writes the first computing result to a second storage location of the memory;

[0018] The computing core reads the second data from the third storage location of the memory, and writes the second data to the fourth storage location of the memory.

[0019] In the above technical solution, when the operator includes a computing task, a first data transport task and a second data transport task, the computing core can execute all the computing tasks and a part of the data transport tasks, and the other part of the data transport tasks is executed by the DMA controller, which can reduce the load of the computing core and improve the execution efficiency of the operator.

[0020] In one possible implementation, the data volume of the second data is determined based on the first data volume, the first time taken by the computing core to execute the computing task, the data transfer speed of the computing core, and the data transfer speed of the DMA controller, and the first data volume is the data volume corresponding to the data in the input data other than the first data.

[0021] In the above technical solution, when the operator includes a computing task, a first data transport task and a second data transport task, when determining the amount of the second data that needs to be transported, factors such as the first data amount corresponding to all the data that needs to be transported in the input data of the operator, the first time taken by the computing core to execute the computing task, the data transport speed of the computing core, and the data transport speed of the DMA controller are considered, which helps to further achieve load balancing between the computing core and the DMA controller in the artificial intelligence chip.

[0022] In a possible implementation, the first task is a first data handling task, the input data of the operator includes second data, the second data is associated with the first data handling task, and the artificial intelligence chip further includes a memory;

[0023] The computing core executes the first task in the operator, including:

[0024] The computing core reads the second data from the third storage location of the memory, and writes the second data to the fourth storage location of the memory.

[0025] In the above technical solution, when the operator includes a first data transport task and a second data transport task, the computing core executes a part of the data transport task and the DMA controller executes the other part of the data transport task, which can reduce the load of the computing core and improve the execution efficiency of the operator.

[0026] In a possible implementation, the data volume of the second data is determined according to the data volume of the input data of the operator, the data transfer speed of the computing core, and the data transfer speed of the DMA controller.

[0027] In the above technical solution, when the operator includes a first data transport task and a second data transport task, when determining the amount of the second data that needs to be transported, factors such as the amount of input data of the operator, the data transport speed of the computing core, and the data transport speed of the DMA controller are taken into consideration, which helps to further achieve load balancing between the computing core and the DMA controller in the artificial intelligence chip.

[0028] In a possible implementation, the input data of the operator includes third data, and the third data is associated with the second data handling task; the artificial intelligence chip also includes a memory;

[0029] The DMA controller executes the second data transfer task in the operator, including:

[0030] The DMA controller reads the third data from the fifth storage location of the memory and writes the third data to the sixth storage location of the memory.

[0031] In a possible implementation, a difference between a second time duration taken by the computing core to execute the first task and a third time duration taken by the DMA controller to execute the second data transporting task is within a preset range.

[0032] In the above technical solution, the operator includes a first task and a second data transfer task. The difference between the second time duration taken by the computing core to execute the first task and the third time duration taken by the DMA controller to execute the second data transfer task is within a preset range, which can achieve load balancing of the computing core and the DMA controller in the artificial intelligence chip.

[0033] Furthermore, the second time duration taken by the computing core to execute the first task is equal to the third time duration taken by the DMA controller to execute the second data transfer task, which can maximize the load balancing of the computing core and the DMA controller in the artificial intelligence chip.

[0034] In a possible implementation, the artificial intelligence chip further includes a control unit, and before the computing core executes the first task in the operator and the DMA controller executes the second data transfer task in the operator, further includes:

[0035] The control unit divides the operator into a first task and a second data handling task;

[0036] The control unit allocates the first task to the computing core for processing, and allocates the second data transport task to the DMA controller for processing.

[0037] In the above technical solution, the artificial intelligence chip can divide the tasks of the operators and distribute the tasks within the artificial intelligence chip, thereby realizing the self-allocation of hardware resources for executing operator tasks.

[0038] In a possible implementation, before the computing core executes the first task in the operator and the DMA controller executes the second data transfer task in the operator, the method further includes:

[0039] The computing core receives a task allocation result sent by the central processing unit, the task allocation result includes a first task and a second data handling task obtained after the operator is divided, the processing resource allocated to the first task is the computing core, and the processing resource allocated to the second data handling task is the DMA controller;

[0040] The DMA controller receives the task allocation result sent by the central processing unit.

[0041] In the above technical solution, the central processing unit divides the operator tasks and allocates hardware resources to each divided task. The artificial intelligence chip executes each task of the operator according to the task allocation result of the central processing unit, thereby reducing the performance overhead of the artificial intelligence chip and improving the performance of the operator.

[0042] In a second aspect, the present application provides an artificial intelligence chip, including a computing core and a DMA controller;

[0043] A computing core, configured to: execute a first task in the operator, where the first task includes at least one of the following: a computing task and a first data handling task;

[0044] The DMA controller is used to: execute the second data transfer task in the operator.

[0045] In a third aspect, the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores program instructions; the processor executes the program instructions in the memory to implement the method in the above-mentioned first aspect or any possible implementation manner of the first aspect.

[0046] The technical effects that can be achieved in any of the second to third aspects mentioned above can refer to the description of the beneficial effects in the first aspect mentioned above or any possible implementation method of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic diagram of a system architecture applicable to an embodiment of the present application;

[0048] Figure 2 A schematic diagram of a computing kernel executing an image superposition operator in the related art;

[0049] Figure 3 A schematic diagram of the structure of an artificial intelligence chip applicable to the embodiments of the present application;

[0050] Figure 4 A flowchart of an operator task execution method provided by an embodiment of the present invention;

[0051] Figure 5 A schematic diagram of an artificial intelligence chip provided in an embodiment of the present application executing an image overlay operator;

[0052] Figure 6 A schematic diagram of an artificial intelligence chip providing an embodiment of the present application executing an image overlay operator. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and beneficial effects of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0054] Figure 1 This is a system architecture diagram applicable to the embodiments of the present application. The system architecture includes a central processing unit (CPU) and an artificial intelligence chip. The number of artificial intelligence chips can be one or more, and the present application does not specifically limit the number of artificial intelligence chips. It should be understood that the system architecture may also include other modules or components, which are not specifically limited in the present application.

[0055] The central processing unit is used to handle non-computing tasks in artificial intelligence applications. The central processing unit can include multiple processing cores, which can perform multiple tasks simultaneously, greatly improving the computing power and performance of computing devices.

[0056] Artificial intelligence chips are also called AI accelerators or computing cards, which are used to handle large amounts of computing tasks in artificial intelligence applications. Artificial intelligence chips can be graphics processing units (GPUs) or general-purpose computing on graphics processing units (GPGPUs).

[0057] The artificial intelligence chip includes a computing core and a memory. In practical applications, the artificial intelligence chip is used to realize the operation of the artificial intelligence model. The operation of the artificial intelligence model can be realized by the operators in the calculation graph. Each operator includes various operator tasks. For example, some operators only include various computing tasks, some operators only include data moving tasks, and some operators include both computing tasks and data moving tasks.

[0058] In the related art, for each operator, a computing core executes all tasks of the operator, such as all computing tasks and all data transfer tasks in the operator.

[0059] Take the operator as an example: image superposition operator. Figure 2 A schematic diagram showing a computing core executing an image overlay operator in the related art is shown.

[0060] like Figure 2 As shown, the image superposition operator is used to superimpose image 1 and image 2 to obtain image 3. The input data of the image superposition operator includes image 1 and image 2, the input data is stored in storage location 1 in the memory in the artificial intelligence chip, and the execution result of the image superposition operator is stored in storage location 2 in the memory in the artificial intelligence chip.

[0061] Superimposing image 1 and image 2 includes two types of operations, namely: superimposing region 1 of image 1 and image 3, and storing the superimposed result in storage location 2; and moving region 2 of image 1 from storage location 1 to storage location 2. In the related art, both types of operations are performed by the computing core. In this way, the load of the computing core is high and the execution efficiency of the operator is low.

[0062] In view of this, the present application provides an operator task execution method to reduce the load of the computing core and improve the efficiency of operator execution. The operator execution method provided in the present application can be implemented by an operator task execution device. The operator task execution device can be an artificial intelligence chip. The following method embodiments are introduced by taking the artificial intelligence chip as the execution subject as an example.

[0063] Combine the following Figure 3, providing a structural schematic diagram of an artificial intelligence chip applicable to an embodiment of the present application.

[0064] like Figure 3 As shown, the artificial intelligence chip includes a computing core, a direct memory access (DMA) controller and a memory.

[0065] The computing core is used to execute computing tasks in the operator, and the computing core may be, for example, a stream processor cluster (SPC for short).

[0066] The DMA controller is used to execute part or all of the data transfer tasks in the operator.

[0067] The memory can be used to store the input data of the operator and the execution results of each task of the operator.

[0068] It should be understood that the artificial intelligence chip may also include other modules or components, for example, the artificial intelligence chip may also include a control unit, which is not limited in this application. This application also does not limit the number of computing cores and DMA controllers included in the artificial intelligence chip.

[0069] The following is an introduction to an operator task execution method provided by the present application in conjunction with the accompanying drawings.

[0070] Figure 4 The flowchart of an operator task execution method provided by the present application is exemplarily shown. The operator task execution method can be applied to an artificial intelligence chip, which includes a computing core, a DMA controller and a memory, such as Figure 4 As shown, the operator task execution method includes:

[0071] Step 401: The computing core executes a first task in the operator, where the first task includes at least one of the following: a computing task and a first data transport task.

[0072] In an embodiment of the present application, the operator includes a first task and a second data handling task, wherein the first task can be implemented in multiple ways, for example, the first task is a computing task, or the first task is a computing task and a first data handling task, or the first task is a first data handling task.

[0073] Step 402: The DMA controller executes the second data transfer task in the operator.

[0074] Through the above method, the computing core in the artificial intelligence chip executes the first task in the operator, and the DMA controller in the artificial intelligence chip executes the second data transport task. That is to say, the DMA controller in the artificial intelligence chip shares part or all of the data transport tasks in the operator, thereby reducing the load of the computing core in the artificial intelligence chip and improving the execution efficiency of the operator.

[0075] Furthermore, the above step 401 and step 402 may be executed in parallel, thereby speeding up the execution of the entire operator.

[0076] Based on the above-mentioned different first tasks, there are multiple possible implementation methods for the above-mentioned artificial intelligence chip to execute the operator method.

[0077] In the first implementation mode, the first task is a computing task, the operator includes a computing task and a second data handling task, the input data of the operator includes first data and third data, the first data is associated with the computing task, and the third data is associated with the second data handling task. The above step 401 can be understood as: the computing core performs the computing task on the first data. The above step 402 can be understood as: the DMA controller performs the second data handling task on the third data.

[0078] In a possible manner, the above step 401 may include the following process:

[0079] S11, the computing core reads first data from a first storage location of the memory.

[0080] The first storage location is a set of storage addresses occupied by the first data in the memory. The set of storage addresses occupied by the first data may be continuous storage addresses or discontinuous storage addresses, and this application does not limit this. Taking the first data occupying multiple continuous storage addresses in the memory as an example, the first storage location can be represented by the following example 1 or example 2:

[0081] Example 1: the starting address and address length occupied by the first data in the memory, wherein the address length is related to the data size of the first data.

[0082] Example 2: the starting address and the ending address occupied by the first data in the memory.

[0083] S12, calculating and checking the first data to obtain a first calculation result.

[0084] S13, the computing core writes the first computing result to the second storage location of the memory.

[0085] The second storage location is a set of storage addresses occupied by the first calculation result in the memory. The set of storage addresses occupied by the first calculation result may be continuous storage addresses or discontinuous storage addresses, and this application does not limit this. Taking the first calculation result occupying multiple continuous storage addresses in the memory as an example, the second storage location can be represented by the following Example 3 or Example 4:

[0086] Example 3: the starting address and address length occupied by the first calculation result in the memory, wherein the address length is related to the data size of the first calculation result.

[0087] Example 4: the starting address and the ending address occupied by the first calculation result in the memory.

[0088] The above step 402 may include the following process:

[0089] S21, the DMA controller reads third data from a fifth storage location of the memory;

[0090] The fifth storage location is a set of storage addresses occupied by the third data in the memory before the data is moved. The set of storage addresses occupied by the third data in the memory before the data is moved may be continuous storage addresses or discontinuous storage addresses, and this application does not limit this. Taking the third data occupying a plurality of continuous storage addresses in the memory before the data is moved as an example, the fifth storage location may be represented by the following example 5 or example 6:

[0091] Example 5: the starting address and address length occupied by the third data in the memory before being moved, wherein the address length is related to the data size of the third data.

[0092] Example 6: the starting address and the ending address occupied by the third data in the memory before being moved.

[0093] S22, the DMA controller writes third data to a sixth storage location of the memory.

[0094] The sixth storage location is a set of storage addresses occupied by the third data after being moved in the memory. The set of storage addresses occupied by the third data after being moved in the memory may be continuous storage addresses or discontinuous storage addresses, which is not limited in this application.

[0095] Taking the case where the third data occupies a plurality of consecutive storage addresses in the memory after being moved as an example, the sixth storage location may be represented by the following Example 7 or Example 8:

[0096] Example 7: the starting address and address length occupied by the third data in the memory after being moved, wherein the address length is related to the data size of the third data.

[0097] Example 8: the starting address and the ending address occupied by the third data in the memory after being moved.

[0098] Take the operator as an image superposition operator, which is used to superimpose two images to obtain a superimposed image as an example. Figure 5 A schematic diagram showing an artificial intelligence chip provided in an embodiment of the present application executing an image overlay operator.

[0099] like Figure 5 As shown, the input data of the image overlay operator includes image 1 and image 2. Image 1 is stored in storage location 1 in the HBM in the artificial intelligence chip, and image 2 is stored in storage location 2 in the HBM. The image overlay operator includes computing tasks and data transfer tasks. The computing core in the artificial intelligence chip performs the computing tasks on area 1 and image 2 of image 1, and the DMA controller performs the data transfer task on area 2 of image 1.

[0100] The computing core performs computing tasks on area 1 of image 1 and image 2, which may include: the computing core reads area 1 of image 1 from storage location 1 in the HBM, reads image 2 from storage location 2, and then the computing core performs superposition calculation on the read area 1 of image 1 and image 2 to obtain a superposition result, and the computing core writes the superposition result into storage location 3 in the HBM.

[0101] The DMA controller performs a data transfer task on area 2 of image 1, which may include: the DMA controller reads area 2 of image 1 from storage location 1 in the HBM, and then writes area 2 of image 1 into storage location 3 in the HBM.

[0102] The result of the artificial intelligence chip executing the image superposition operator is image 3, which is stored in storage location 3 in HBM.

[0103] Through this first implementation, when the operator includes computing tasks and data transfer tasks, the computing core in the artificial intelligence chip performs computing tasks on all data that need to be calculated, and the DMA controller performs data transfer tasks on all data that need to be transferred, thereby reducing the load on the computing core to the greatest extent.

[0104] In the second implementation mode, the first task is a computing task and a first data handling task, the operator includes a computing task, a first data handling task and a second data handling task, the input data of the operator includes first data, second data and third data, the first data is associated with the computing task, the second data is associated with the first data handling task, and the third data is associated with the second data handling task. The above step 401 can be understood as: the computing core performs the computing task on the first data, and performs the first data handling task on the second data. The above step 402 can be understood as: the DMA controller performs the second data handling task on the third data.

[0105] In a possible manner, the above step 401 may include the following process:

[0106] S31, the computing core reads first data from a first storage location of the memory;

[0107] Here, the specific implementation of the first storage location can refer to the relevant description in the aforementioned S11, which will not be repeated here.

[0108] S32, calculating and checking the first data to obtain a first calculation result,

[0109] S33, the computing core writes the first computing result to the second storage location of the memory;

[0110] Here, the specific implementation of the second storage location can refer to the relevant description in the aforementioned S13, which will not be repeated here.

[0111] S34, the computing core reads the second data from the third storage location of the memory, and writes the second data to the fourth storage location of the memory.

[0112] The above step 402 may include the following process:

[0113] S21, the DMA controller reads third data from a fifth storage location of the memory;

[0114] Here, the specific implementation of the fifth storage location can refer to the relevant description in the aforementioned S21, which will not be repeated here.

[0115] S22, the DMA controller writes third data to a sixth storage location of the memory.

[0116] Here, the specific implementation of the sixth storage location can refer to the relevant description in the aforementioned S22, which will not be repeated here.

[0117] In one possible implementation, the data volume of the second data is determined based on the first data volume, the first time taken by the computing core to execute the computing task, the data transfer speed of the computing core, and the data transfer speed of the DMA controller. The first data volume is the data volume corresponding to the data in the input data except the first data, that is, the first data volume is the data volume corresponding to all the data that needs to be transferred in the first data.

[0118] The total amount of input data of the operator is S0+S, where the sum of the amount of the second data and the third data is S, that is, the first amount of data is S, the first time length taken by the computing core to perform the computing task is represented as t0, the data transfer speed of the computing core is represented as v1, the data transfer speed of the DMA controller is represented as v2, the second data is represented as S1, and the amount of the third data is represented as S2. The second time length taken by the computing core to perform the first task includes the sum of the first time length t0 taken by the computing core to perform the computing task and the time length t1 taken by the computing core to perform the first data transfer task, and the third time length t2 taken by the DMA controller to perform the second data transfer task, wherein t1=S1 / v1, t2=S2 / v2.

[0119] In a possible implementation, the difference between the second duration taken by the computing core to execute the first task and the third duration taken by the DMA controller to execute the second data transfer task is within a preset range. Therefore, S1 and S2 meet the following formula (1):

[0120] |S2 / v2- S1 / v1 - t0|≤x Formula (1)

[0121] In formula (1), x is a constant and can be set to a default value. The specific value of x can be set according to actual needs.

[0122] In a possible implementation, the second duration taken by the computing core to execute the first task is greater than or equal to the third duration taken by the DMA controller to execute the second data transfer task. The second duration is the sum of t0 and t1, and the third duration is t2, wherein t1=S1 / v1, and t2=S2 / v2.

[0123] Therefore, S1 and S2 meet the following formula (2)

[0124] t0+ S1 / v1≥S2 / v2 Formula (2)

[0125] In the above formula (1) and formula (2), t0, v1 and v2 can be obtained by calculation.

[0126] For example, the first time duration t0 taken by the computing core to execute the computing task includes the sum of the following three parts: <1> The time consumption of reading the first data from the memory is obtained, for example, by dividing the amount of the first data read by the bandwidth of the computing core reading data from the memory; <2> The time consumed in performing superposition calculation on the first data; <3> The time consumption of writing the first calculation result obtained by the superposition calculation into the memory is obtained, for example, by dividing the amount of data of the first calculation result to be stored by the bandwidth of the computing core for storing data in the memory.

[0127] For example, the data transfer speed v1 of the computing core can be obtained by dividing the amount of second data read by the computing core from the memory by the bandwidth of the computing core reading data from the memory, and the amount of second data divided by the bandwidth of the computing core storing data to the memory.

[0128] For example, the data transfer speed v2 of the DMA controller can be obtained by dividing the amount of second data read by the computing core from the memory by the bandwidth of the computing core reading data from the memory, and the amount of second data divided by the bandwidth of the computing core storing data to the memory.

[0129] After calculating the values ​​of t0, v1, and v2, the relationship between S1 and S2 can be obtained by formula (1) or formula (2), so that the minimum value of S1 and the maximum value of S2 can be calculated. The data volume of the second data is the minimum value of S1, and the data volume of the third data is the maximum value of S2. In this way, the data with the minimum data volume of S1 among all the data to be transported can be allocated to the computing core for transport, and the data with the maximum data volume of S2 among all the data to be transported can be allocated to the DMA controller for transport.

[0130] Take the operator as an image superposition operator, which is used to superimpose two images to obtain a superimposed image as an example. Figure 6 A schematic diagram showing an artificial intelligence chip provided in an embodiment of the present application executing an image overlay operator.

[0131] like Figure 6 As shown, the input data of the image overlay operator includes image 1 and image 2, image 1 is stored in storage location 1 in the HBM in the artificial intelligence chip, and image 2 is stored in storage location 2 in the HBM. The image overlay operator includes a computing task, a data transfer task 1, and a data transfer task 2. The computing task is performed by the computing core in the artificial intelligence chip on areas 1 and 2 of image 1, the data transfer task 1 is performed by the computing core on area 21 of image 1, and the data transfer task 2 is performed by the DMA controller on area 22 of image 1.

[0132] The computing core performs computing tasks on region 1 of image 1 and image 2, which may include: the computing core reads region 1 of image 1 from storage location 1 in the HBM, and reads image 2 from storage location 2 in the HBM, and then the computing core performs superposition calculation on the read region 1 of image 1 and image 2 to obtain a superposition result, and the computing core writes the superposition result into storage location 3 in the HBM.

[0133] The computing core performs the data moving task 1 on the region 21 of the image 1, which may include: the computing core reads the region 21 of the image 1 from the storage location 1 in the HBM, and then writes the region 21 of the image 1 into the storage location 3 in the HBM.

[0134] The DMA controller performs a data transfer task on the area 22 of the image 1, which may include: the DMA controller reads the area 22 of the image 1 from the storage location 1 in the HBM, and then writes the area 22 of the image 1 to the storage location 3 in the HBM.

[0135] The result of the artificial intelligence chip executing the image superposition operator is image 3, which is stored in storage location 3 in HBM.

[0136] Through this second implementation, when the operator includes computing tasks and data transfer tasks, the computing core in the artificial intelligence chip performs computing tasks on all data that need to be calculated, and performs transfer tasks on part of the data that needs to be transferred, and the DMA controller performs the data transfer task on the other part of the data that needs to be transferred, thereby reducing the load on the computing core and improving the execution efficiency of the operator.

[0137] In implementation mode three, the first task is a first data handling task, the input data of the operator includes second data and third data, the second data is associated with the first data handling task, and the third data is associated with the second data handling task. The above step 401 can be understood as: calculating and checking the second data to perform the first data handling task. The above step 402 can be understood as: the DMA controller performs the second data handling task on the third data.

[0138] In a possible manner, the above step 401 may include the following process:

[0139] S41, the computing core reads second data from a third storage location of the memory;

[0140] The third storage location is a set of storage addresses occupied by the second data in the memory before the data is moved. The set of storage addresses occupied by the second data in the memory before the data is moved may be continuous storage addresses or discontinuous storage addresses, and this application does not limit this. Taking the example of multiple continuous storage addresses occupied by the second data in the memory before the data is moved, the third storage location can be represented by the following Example 9 or Example 10:

[0141] Example 9: the starting address and address length occupied by the second data in the memory before being moved, wherein the address length is related to the data size of the second data.

[0142] Example 10: the starting address and the ending address occupied by the second data in the memory before being moved.

[0143] S42, the computing core writes the second data to the fourth storage location of the memory.

[0144] The fourth storage location is a set of storage addresses occupied by the second data in the memory after the second data is moved. The set of storage addresses occupied by the second data in the memory after the second data is moved may be continuous storage addresses or discontinuous storage addresses, and this application does not limit this. Taking the example of multiple continuous storage addresses occupied by the second data in the memory after the second data is moved, the fourth storage location can be represented by the following example 11 or example 12:

[0145] Example 11: the starting address and address length occupied by the second data in the memory after being moved, wherein the address length is related to the data size of the second data.

[0146] Example 12: the starting address and ending address occupied by the second data in the memory after being moved.

[0147] The above step 402 may include the following process:

[0148] S21, the DMA controller reads third data from a fifth storage location of the memory;

[0149] Here, the specific implementation of the fifth storage location can refer to the relevant description in the aforementioned S21, which will not be repeated here.

[0150] S22, the DMA controller writes third data to a sixth storage location of the memory.

[0151] Here, the specific implementation of the sixth storage location can refer to the relevant description in the aforementioned S22, which will not be repeated here.

[0152] In one possible implementation, the amount of the second data is determined according to the amount of the input data of the operator, the data transfer speed of the computing core, and the data transfer speed of the DMA controller. In one embodiment, the difference between the second time duration occupied by the computing core to execute the first data transfer task and the third time duration occupied by the DMA controller to execute the second data transfer task is less than a certain threshold. For example, when the second time duration occupied by the computing core to execute the first data transfer task is equal to the third time duration occupied by the DMA controller to execute the second data transfer task, the performance of the artificial intelligence chip is optimal.

[0153] Through this third implementation, when the operator only includes the task of moving data, the computing core in the artificial intelligence chip performs the task of moving data for a part of the data that needs to be moved, and the DMA controller performs the task of moving data for another part of the data that needs to be moved, thereby reducing the load of the computing core and improving the execution efficiency of the operator.

[0154] In the embodiment of the present application, there are multiple implementation methods for the task division of operators and the allocation of hardware resources to the tasks of operators. In one possible implementation method, the task division of operators and the allocation of hardware resources to the tasks of operators are implemented by an artificial intelligence chip. For example, the artificial intelligence chip also includes a control unit.

[0155] Before the computing core executes the first task in the operator and the DMA controller executes the second data handling task in the operator, the control unit divides the operator into the first task and the second data handling task, and allocates the first task to the computing core for processing and the second data handling task to the DMA controller for processing, thereby realizing the self-allocation of hardware resources for executing operator tasks.

[0156] In a possible implementation, a central processing unit implements task division of operators and allocates hardware resources to the tasks of operators.

[0157] Before the computing core executes the first task in the operator, the computing core receives a task allocation result sent by the central processor, the task allocation result includes the first task and the second data handling task obtained after the operator is divided, the processing resource allocated to the first task is the computing core, and the processing resource allocated to the second data handling task is the DMA controller;

[0158] Before the DMA controller executes the second data transfer task in the operator, the DMA controller receives the task allocation result sent by the central processor. In this way, the artificial intelligence chip executes each task of the operator according to the task allocation result of the central processor, thereby reducing the performance overhead of the artificial intelligence chip and improving the performance of the operator.

[0159] Based on the same technical concept, an embodiment of the present application provides an artificial intelligence chip, including a computing core and a DMA controller; wherein:

[0160] The computing core is used to execute a first task in the operator, where the first task includes at least one of the following: a computing task and a first data handling task.

[0161] The DMA controller is used to: execute the second data transfer task in the operator.

[0162] In one possible implementation, the first task is a computing task, the input data of the operator includes first data, the first data is associated with the computing task, and the artificial intelligence chip also includes a memory; a computing core is specifically used to: read the first data from a first storage location of the memory; calculate the first data to obtain a first calculation result; and write the first calculation result to a second storage location of the memory.

[0163] In one possible implementation, the first task is a computing task and a first data handling task, the input data of the operator includes first data and second data, the first data is associated with the computing task, and the second data is associated with the first data handling task, and the artificial intelligence chip also includes a memory; a computing core, specifically used to: read the first data from a first storage position of the memory, calculate the first data to obtain a first calculation result, and write the first calculation result to a second storage position of the memory; read the second data from a third storage position of the memory, and write the second data to a fourth storage position of the memory.

[0164] In one possible implementation, the data volume of the second data is determined based on the first data volume, the first time taken by the computing core to execute the computing task, the data transfer speed of the computing core, and the data transfer speed of the DMA controller, and the first data volume is the data volume corresponding to the data in the input data other than the first data.

[0165] In one possible implementation, the first task is a first data handling task, the input data of the operator includes second data, the second data is associated with the first data handling task, and the artificial intelligence chip also includes a memory; a computing core, specifically used to: read the second data from a third storage location of the memory, and write the second data to a fourth storage location of the memory.

[0166] In a possible implementation, the data volume of the second data is determined according to the data volume of the input data of the operator, the data transfer speed of the computing core, and the data transfer speed of the DMA controller.

[0167] In one possible implementation, the input data of the operator includes third data, and the third data is associated with the second data handling task; the DMA controller is specifically used for: the DMA controller reads the third data from the fifth storage location of the memory, and writes the third data to the sixth storage location of the memory.

[0168] In a possible implementation, a difference between a second time duration taken by the computing core to execute the first task and a third time duration taken by the DMA controller to execute the second data transporting task is within a preset range.

[0169] In one possible implementation, the artificial intelligence chip also includes a control unit, which is used to: divide the operator into a first task and a second data handling task; assign the first task to the computing core for processing, and assign the second data handling task to the DMA controller for processing.

[0170] In one possible implementation, the computing core is also used to: receive the task allocation result sent by the central processing unit, the task allocation result includes the first task and the second data handling task obtained after the operator is divided, the processing resources allocated to the first task are the computing core, and the processing resources allocated to the second data handling task are the DMA controller; the DMA controller is also used to: receive the task allocation result sent by the central processing unit.

[0171] Based on the same inventive concept, the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores program instructions; the processor executes the program instructions in the memory to implement the steps of the above-mentioned operator execution method.

[0172] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, devices (equipment), systems, chips, computer-readable storage media, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware, which are collectively referred to as "modules" or "systems" herein.

[0173] The present application is described with reference to at least one of the following diagrams of the method, apparatus (device) or system of the present application: a flowchart, a block diagram. It should be understood that at least one of the following can be implemented by computer program instructions: each process in the flowchart, each box in the block diagram, and a combination of a process in the flowchart and a box in the block diagram.

[0174] These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce instructions for implementing at least one of the following: Figure 1 Devices or blocks that specify functions in a process or multiple processes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements at least one of the following: Figure 1 A function specified in a process or multiple processes, in the box Figure 1 A function specified in one or more boxes.

[0175] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing at least one of the following: steps for specifying a function in one or more processes in a flowchart, steps for specifying a function in one or more boxes in a block diagram.

[0176] Although the present invention has been described in conjunction with specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present invention. Accordingly, this specification and the accompanying drawings are merely exemplary illustrations of the present invention as defined by the appended claims and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present invention. Obviously, those skilled in the art may make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, the present invention is intended to include such modifications and variations if they fall within the scope of the claims of the present invention and their equivalents.

Claims

1. An operator task execution method, characterized in that: Applied to an artificial intelligence chip, the artificial intelligence chip is a GPU or a GPGPU, the artificial intelligence chip includes a computing core, a control unit and a direct memory access DMA controller, the method includes: The control unit divides the operator into a first task and a second data handling task; The control unit allocates the first task to the computing core for processing, and allocates the second data transport task to the DMA controller for processing; The computing core executes a first task in the operator, where the first task includes a first data handling task, or the first task includes a computing task and a first data handling task; The DMA controller executes the second data transfer task in the operator; Among them, the amount of data corresponding to the data allocated to the computing core in the input data of the operator and the amount of data corresponding to the data allocated to the DMA controller satisfy: the difference between the second time taken by the computing core to execute the first task on the data allocated to the computing core and the third time taken by the DMA controller to execute the second data moving task on the data allocated to the DMA controller is within a preset range.

2. The method according to claim 1, characterized in that The first task is a computing task and a first data handling task, the input data of the operator includes first data and second data, the first data is associated with the computing task, the second data is associated with the first data handling task, and the artificial intelligence chip further includes a memory; The computing core executes a first task in the operator, including: The computing core reads the first data from a first storage location of the memory, calculates the first data to obtain a first calculation result, and writes the first calculation result to a second storage location of the memory; The computing core reads the second data from a third storage location of the memory, and writes the second data to a fourth storage location of the memory.

3. The method according to claim 2, characterized in that The data volume of the second data is determined based on the first data volume, the first time taken by the computing core to execute the computing task, the data transfer speed of the computing core, and the data transfer speed of the DMA controller. The first data volume is the data volume corresponding to the data in the input data other than the first data.

4. The method according to claim 1, characterized in that The first task is a first data handling task, the input data of the operator includes second data, the second data is associated with the first data handling task, and the artificial intelligence chip further includes a memory; The computing core executes a first task in the operator, including: The computing core reads the second data from a third storage location of the memory, and writes the second data to a fourth storage location of the memory.

5. The method according to claim 4, characterized in that The data volume of the second data is determined according to the data volume of the input data of the operator, the data transfer speed of the computing core, and the data transfer speed of the DMA controller.

6. The method according to claim 1, characterized in that The input data of the operator includes third data, and the third data is associated with the second data handling task; the artificial intelligence chip also includes a memory; The DMA controller executes the second data transfer task in the operator, including: The DMA controller reads the third data from a fifth storage location of the memory and writes the third data to a sixth storage location of the memory.

7. An artificial intelligence chip, characterized in that: It includes a computing core, a control unit and a direct memory access DMA controller; the artificial intelligence chip is a GPU or a GPGPU; The control unit is used to divide the operator into a first task and a second data handling task; allocate the first task to the computing core for processing, and allocate the second data handling task to the DMA controller for processing; The computing core is used to execute a first task in an operator, the operator includes the first task and a second data handling task, the first task includes a first data handling task, or the first task includes a computing task and a first data handling task; The DMA controller is used to execute the second data transfer task in the operator; Among them, the amount of data corresponding to the data allocated to the computing core in the input data of the operator and the amount of data corresponding to the data allocated to the DMA controller satisfy: the difference between the second time taken by the computing core to execute the first task on the data allocated to the computing core and the third time taken by the DMA controller to execute the second data moving task on the data allocated to the DMA controller is within a preset range.

8. The artificial intelligence chip according to claim 7, characterized in that: The first task is a computing task and a first data handling task, the input data of the operator includes first data and second data, the first data is associated with the computing task, the second data is associated with the first data handling task, and the artificial intelligence chip further includes a memory; The computing core is specifically used for: Reading the first data from a first storage location of the memory, calculating the first data to obtain a first calculation result, and writing the first calculation result to a second storage location of the memory; The second data is read from a third storage location of the memory, and the second data is written to a fourth storage location of the memory.

9. The artificial intelligence chip according to claim 8, characterized in that: The first task is a first data handling task, the input data of the operator includes second data, the second data is associated with the first data handling task, and the artificial intelligence chip further includes a memory; The computing core is specifically used for: The second data is read from a third storage location of the memory, and the second data is written to a fourth storage location of the memory.

10. The artificial intelligence chip according to any one of claims 7 to 9, characterized in that: The input data of the operator includes third data, and the third data is associated with the second data handling task; The DMA controller is specifically used to: read the third data from the fifth storage location of the memory, and write the third data to the sixth storage location of the memory.

11. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the memory stores program instructions; the processor executes the program instructions in the memory to implement the steps of the method as claimed in any one of claims 1 to 6.

Images