Computing device and execution core termination method

Abstract

The present invention provides a method for terminating a computing device and an execution core. The computing device includes an execution core and a global memory. The execution core includes a task scheduler, a plurality of computing cores, a bus interface device and a bus. The bus interface device couples the plurality of computing cores. The bus couples the bus interface device and the global memory. When terminating a task, the task scheduler issues a data refresh instruction to the computing core to store the cache data of the computing core in the global memory respectively. After storing the cache data in the global memory, the global memory responds to corresponding confirmation information based on the data refresh instruction respectively. After determining that all confirmation information has been obtained, the bus interface device sets the end flag corresponding to the end task to enable to complete the end task. The method for terminating the computing device and the execution core of the present invention can reduce the time overhead spent by the execution core in terminating the task by having the bus interface device be responsible for setting the end flag.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence chips, and in particular to a computing device and a method for terminating an execution core. Background Art

[0002] The computing device needs to have huge computing power when performing artificial intelligence (AI) applications. The huge computing power of the computing device comes from a large number of internal hardware execution units (EU). The execution unit can be called an execution core (kernel). Each execution core usually contains multiple calculation cores (CU), such as integer (INT) calculation core, floating point (FP) calculation core, tensor core (tensorcore) and / or vector core (vector core). The execution core can organize various types of calculation cores through programming and scheduling programs (scheduler), so that the computing device can support applications such as general computing, scientific computing and neural network computing.

[0003] When the computing device terminates a task for an execution core, the data in the cache used by the computing core corresponding to the execution core will be stored in the global memory. Then, when the computing core obtains the confirmation (ACK) information, the computing core will set the end flag in the global memory through instructions. However, since the confirmation information needs to be returned to each computing core that is executing the end task before these computing cores can issue instructions to set the corresponding flag, the time overhead on the communication path is increased. Summary of the invention

[0004] The present invention is directed to a computing device and a method for terminating an execution core, which can reduce the time overhead consumed by the execution core in terminating a task.

[0005] According to an embodiment of the present invention, a computing device includes an execution core and a global memory. The execution core includes a task scheduler, a plurality of computing cores, a bus interface device, and a bus. The task scheduler generates a plurality of thread requests. The plurality of thread requests are executed by the plurality of computing cores. The bus interface device couples the plurality of computing cores. The bus couples the bus interface device and the global memory. When the computing device performs a task termination, the task scheduler issues a corresponding data refresh instruction to each computing core to store the cache data corresponding to each computing core to the global memory. After the plurality of cache data are stored to the global memory, the global memory responds to the corresponding confirmation information based on each of the data refresh instructions. In addition, the bus interface device determines whether the confirmation information issued by the global memory for each computing core has been obtained, and after determining that all the confirmation information has been obtained, sets the termination flag corresponding to the termination task to enable to complete the termination task.

[0006] According to an embodiment of the present invention, the execution core in the termination method of the execution core is set in a computing device. The bus interface device in the execution core is coupled to the global memory in the computing device through a bus. The termination method includes: when performing the termination task, issuing a corresponding data refresh instruction to each computing core in the execution core to store the cache data corresponding to each computing core to the global memory respectively; after the multiple cache data are respectively stored in the global memory, responding to corresponding confirmation information through the global memory and based on each of the data refresh instructions respectively; and, through the bus interface device, judging whether the confirmation information issued by the global memory for each computing core has been obtained, and after judging that all the confirmation information has been obtained, setting the termination flag corresponding to the termination task to enable to complete the termination task.

[0007] Based on the above, the computing device and the execution core termination method described in the embodiment of the present invention, by having the bus interface device be responsible for setting the termination flag, can reduce the time overhead spent by the execution core in terminating the task. In the case of a large number of computing devices or a large number of execution cores in a single computing device, the termination task can be completed more quickly, the computing resources in the execution core can be released more effectively, and the computing device can execute the next task more quickly. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a block diagram of a computing device according to an embodiment of the present invention.

[0009] Figure 2 FIG. 4 is a block diagram of a bus interface device in a computing device according to an embodiment of the present invention.

[0010] Figure 3 It is a schematic diagram of an execution core in a computing device according to an embodiment of the present invention performing a task termination.

[0011] Figure 4 It is a schematic diagram of information transmission between a bus interface device and a global memory to send confirmation information to multiple computing cores in one embodiment of the present invention.

[0012] Figure 5 FIG. 1 is a schematic diagram of an example of a task scheduler and a task queue of each computing core according to an embodiment of the present invention.

[0013] Figure 6 is a flowchart of a method for terminating an execution core according to an embodiment of the present invention.

[0014] Description of Reference Numerals

[0015] 10: Computing device

[0016] 105: Execution Core

[0017] 110: Task Scheduler

[0018] 120, 120-1 to 120-N: Calculation core

[0019] 130: Cache

[0020] 140: Bus interface device

[0021] 150: Global memory

[0022] 210~230、310~340:mark

[0023] S410-S430: Steps of the execution core termination method

[0024] ENDF: End flag

[0025] ACK1: Acknowledgement

[0026] ENDFLAG1: End flag setting

[0027] T1, T2: time point

[0028] Task0~Task4: Tasks DETAILED DESCRIPTION

[0029] Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.

[0030] The term "coupled (or connected)" used in the entire specification of this case (including the claims) may refer to any direct or indirect means of connection. For example, if the text describes a first device coupled (or connected) to a second device, it should be interpreted that the first device can be directly connected to the second device, or the first device can be indirectly connected to the second device through other devices or some connection means. The terms "first", "second", etc. mentioned in the entire specification of this case (including the claims) are used to name the components (element) or to distinguish different embodiments or scopes, and are not used to limit the upper or lower limit of the number of components, nor to limit the order of components. In addition, wherever possible, components / components / steps with the same number in the drawings and embodiments represent the same or similar parts. Components / components / steps using the same number or the same terminology in different embodiments can refer to the relevant descriptions of each other.

[0031] Figure 1 1 is a block diagram of a computing device 10 according to an embodiment of the present invention. The computing device 10 of this embodiment may be an artificial intelligence processor, an artificial intelligence processing card, an artificial intelligence processing card cluster, a computer, or a computer cluster. The artificial intelligence processor may be any one of a graphics processing unit (GPU), a tensor processing unit (TPU), a neural network processing unit (NPU), a deep learning processing unit (DPU), an accelerated processing unit (APU), and a general-purpose graphics processing unit (GPGPU). The computing device 10 mainly includes an execution core 105 and a global memory 150. The computing device 10 may include multiple execution cores 105.

[0032] The execution core 105 includes a task scheduler 110 , a plurality of computing cores 120 , a bus interface device 140 and a bus BUS. The task scheduler 110 can cooperate with the computing cores 120 to generate a plurality of thread requests correspondingly according to external commands. The thread requests are executed by the computing cores 120 .

[0033] The computing core 120 includes one or a combination of an integer computing core, a floating point computing core, a tensor core, and a vector core. In other words, the computing core 120 can be an integer computing core, a floating point computing core, a tensor core, or a vector core. The execution core 105 of this embodiment may include computing cores 120 of different numbers and types.

[0034] The bus interface device 140 couples these computing cores 120. The bus BUS couples the bus interface device 140 and the global memory 150. The computing core 120 communicates with the global memory 150 through the bus interface device 140 and the bus BUS. When the computing device 10 ends a task, the task scheduler 110 issues a corresponding data refresh instruction to each computing core 120 to store the cache data corresponding to each computing core in the global memory 150 respectively. The execution core 105 of this embodiment also includes a plurality of caches 130. Each computing core 120 is respectively provided with a corresponding cache 130. The cache 130 of this embodiment includes an L1 high-speed cache. Before executing the aforementioned data refresh instruction, the cache data corresponding to each computing core 120 is stored in each corresponding cache 130. After the aforementioned cache data is respectively stored in the global memory 150, the global memory 150 responds to the corresponding confirmation information based on each of the data refresh instructions.

[0035] The bus interface device 140 determines whether the confirmation information for each computing core 120 issued by the global memory 150 has been obtained. And, after determining that all the confirmation information has been obtained, the bus interface device 140 sets the end flag ENDF corresponding to this end task to enable to complete the end task. The end flag ENDF of this embodiment is set in the global memory 150. When the end flag ENDF is disabled, the execution core 105 is executing an external command. When the end flag ENDF is enabled, the execution core 105 is not used. The computing device 10 can know the current computing resources according to the end flag ENDF corresponding to the execution core 105, and configure these computing resources. The corresponding embodiments and corresponding drawings are presented below for detailed description.

[0036] Figure 2 1 is a block diagram of a bus interface device 140 in a computing device according to an embodiment of the present invention. The bus interface device 140 may include a refresh atomic instruction table 141, a comparison logic 142, and an end flag storage component 144. The address of the final result flag will be carried on the refresh request issued from the computing core 120. The same execution core (kernel) may include multiple thread clusters executed on different computing cores 120. Each thread cluster will increase the value of the "estimated count" in the refresh atomic instruction table 141 when refreshing. In other words, if the address of the end flag is the same, the refresh requests located in the same execution core will be accumulated to the same "estimated count" field in the refresh atomic instruction table 141.

[0037] Each refresh request returns a response, indicating that the data has been written to the global memory 150. In addition, this response is accumulated to the corresponding "arrival count" field in the refresh atomic instruction table 141. The comparison logic 142 checks the "estimated count" field and the "arrival count" field in the refresh atomic instruction table 141. When the values ​​in the "estimated count" field and the "arrival count" field are the same, the comparison logic 142 issues a storage end flag request. The end flag storage component 144 writes the end flag to the end flag ENDF in the global memory 150 through the storage end flag request issued by the comparison logic 142.

[0038] Figure 3 1 is a schematic diagram of the execution core 105 in the computing device 10 in one embodiment of the present invention performing a task termination. Figure 1 and Figure 3 First, the task scheduler 110 in the execution core 105 obtains the corresponding end command from the outside or from the computing device 10, and then issues a corresponding data refresh instruction (such as Figure 3 After receiving the corresponding data refresh instruction, each computing core 120 completes its own operation (eg Figure 3 Label 220), and then continue to execute these data refresh instructions (such as Figure 3 The data refresh instruction is to store the cache data corresponding to each computing core 120 in the global memory 150, thereby releasing the data and computing resources in the computing core 120 and the cache 130. Figure 3 After label 220), its own computing resources will be released.

[0039] Each computing core 120 executes the corresponding data refresh instruction by an atomic instruction, for example, a fence atomic flush. The data refresh instruction of this embodiment is issued by the corresponding computing core 120 and reaches the global memory 150 through the bus interface device 140, i.e., the bus BUS. When the data refresh instruction is completed, that is, after the cache data corresponding to each computing core 120 is stored in the global memory 150, the global memory 150 responds to the corresponding confirmation information (such as Figure 3 Confirmation information ACK1).

[0040] The bus interface device 140 determines whether the confirmation information ACK1 for each computing core 120 issued by the global memory 150 has been obtained. After determining that all the confirmation information ACK1 has been obtained, the bus interface device 140 sets the end flag ENDF corresponding to the aforementioned end task to enable (such as Figure 3 The end flag is set ENDFLAG1) to complete this end task.

[0041] In other words, in the embodiment of the present invention, the bus interface device 140 is responsible for setting the end flag ENDF. Compared with the task scheduler 110 or the computing core 120 being mainly responsible for setting the end flag ENDF, it can be clearly seen that the information transmission in the embodiment of the present invention is shorter, and the time overhead consumed by the execution core 105 in terminating the task can be further reduced. In particular, in the case where there are a large number of execution cores 105 in the computing device 10, or a large number of computing devices 10, by reducing the time overhead consumed by the execution core 105 in terminating the task, the computing resources can be released more quickly, thereby increasing the computing speed.

[0042] Figure 4 It is a schematic diagram of information transmission in which the bus interface device 140 and the global memory 150 send confirmation information to multiple computing cores 120-1 to 120-N in one embodiment of the present invention. Figure 4 The horizontal axis represents time. Figure 4 The vertical axis lists the time points at which the global memory 150 sends confirmation information to the multiple computing cores 120-1 to 120-N. For example, the global memory 150 sends confirmation information to the computing core 120-1 at time point T1 (eg, Figure 4 Indicated as 310), the global memory 150 sends a confirmation message to the computing core 120-2. Figure 4 At 320, the global memory 150 sends a confirmation message to the computing core 120-N at time point T2 (eg, Figure 4 The bus interface device 140 counts these confirmation information to determine whether the confirmation information issued by the global memory 150 for each computing core 120 - 1 ˜ 120 -N has been obtained.

[0043] It is assumed here that the bus interface device 140 obtains confirmation information from each computing core 120-1 to 120-N at time point T2. Figure 4 At 340, the bus interface device 140 issues an atomic instruction Figure 3 The end flag sets ENDFLAG1, thereby setting the bit in Figure 3 The end flag ENDF in the global memory 150 .

[0044] Figure 5 1 is a schematic diagram of an example of a task scheduler 110 and a task queue of each computing core 120 - 1 to 120 - 8 according to an embodiment of the present invention. Figure 5The computing cores 120-1 to 120-8 are used as examples of multiple computing cores. Task Task0 represents the task being run by computing cores 120-1 to 120-8. Due to the distance between each computing core and the global resources, the time to complete task Task0 in different computing cores will not be exactly the same, but will be different. When ending a task, it used to be necessary to wait until all cache data corresponding to computing core 120 was refreshed to the global memory, and then the task scheduler 110 would uniformly release the computing resources of all computing cores. This results in the computing core with the longest time determining the release time of computing resources of other computing cores, resulting in low resource utilization of other computing cores. For example, Figure 5 Task 1 is unrelated to Task 0, so it needs to wait until Task 0 releases its resources before it can be scheduled to computing cores 120-1 and 120-2. After computing cores 120-1 and 120-2 complete their tasks, they need to wait until all other computing cores complete their tasks before they can schedule Task 1 to computing cores 120-1 and 120-2. This results in low resource utilization of computing cores 120-1 and 120-2.

[0045] In contrast, in the embodiment of the present invention, after the computing core 120-1 and the computing core 120-2 complete the corresponding task Task0, they directly issue a fence atomic flush to the bus interface device 140 to perform a data refresh instruction, and the data refresh instruction will be taken over by the bus interface device 140. Therefore, the computing core 120-1 and the computing core 120-2 can directly release the computing resources inside the computing core. At the same time, the task Task1 can be directly scheduled to the computing core 120-1 and the computing core 120-2 and executed. Therefore, the embodiment of the present invention improves task concurrency and the effective use of computing resources.

[0046] Figure 6 FIG. 1 is a flowchart of a method for terminating an execution core according to an embodiment of the present invention. The method for terminating an execution core is applicable to Figure 1 Computing device 10. Also refer to Figure 1 and Figure 6 In step S510, when the execution core 105 terminates the task, Figure 1The task scheduler 110 issues a corresponding data refresh instruction to each computing core 120 in the execution core 105, so as to store the cache data corresponding to each computing core 120 in the global memory 150 respectively. In step S520, after the aforementioned cache data are stored in the global memory 150 respectively, the corresponding confirmation information is responded to respectively through the global memory 150 and based on each of the aforementioned data refresh instructions. In step S530, through the bus interface device 140, it is determined whether the confirmation information issued by the global memory 150 for each computing core 120 has been obtained, and after it is determined that all the confirmation information has been obtained, the end flag ENDF corresponding to the aforementioned end task is set to enable through the bus interface device 140 to complete this end task. For the detailed implementation content of the aforementioned steps S510 to S530, please refer to the aforementioned embodiment.

[0047] In summary, the computing device and the execution core termination method described in the embodiment of the present invention reduce the time overhead of the execution core in terminating the task by having the bus interface device be responsible for setting the termination flag. In the case of a large number of computing devices or a large number of execution cores in a single computing device, the termination task can be completed more quickly, the computing resources in the execution core can be released more effectively, and the computing device can execute the next task more quickly.

[0048] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A computing device, characterized in that: include: Execution core; as well as Global memory, The execution core includes: Task scheduler, generates multiple thread requests; a plurality of computing cores, wherein the plurality of thread requests are executed by the plurality of computing cores; A bus interface device, coupled to the plurality of computing cores; and A bus, wherein the bus couples the bus interface device and the global memory, in, When the computing device performs a termination task for storing data in a cache used by a computing core corresponding to the execution core into a global memory, the task scheduler issues a corresponding data refresh instruction to each computing core to store the cache data corresponding to each computing core into the global memory respectively. Furthermore, after each computing core completes its own operation, it releases its own computing resources. At the same time, the task scheduler schedules the next task of each computing core to each computing core for execution. Furthermore, after storing the cache data in the global memory respectively, the global memory responds to corresponding confirmation information based on each of the data refresh instructions respectively. Furthermore, the bus interface device determines whether the confirmation information for each computing core issued by the global memory has been obtained, and after determining that all the confirmation information has been obtained, sets the end flag corresponding to the end task to enable to complete the end task.

2. The computing device according to claim 1, wherein: The end flag is set in the global memory, When the end flag is disabled, the execution core is executing an external command. When the end flag is enabled, the execution core is not used.

3. The computing device according to claim 1, wherein: The computing device further comprises: Multiple caches, each computing core corresponds to each cache, Before executing the data refresh instruction, the cache data corresponding to each computing core is stored in each corresponding cache.

4. The computing device according to claim 3, characterized in that: Each of the plurality of caches includes an L1 cache.

5. The computing device according to claim 1, wherein: The task scheduler cooperates with the plurality of computing cores to correspondingly generate the plurality of thread requests according to external commands.

6. The computing device according to claim 1, wherein: The plurality of computing cores include one or a combination of integer computing cores, floating point computing cores, tensor cores, and vector cores.

7. The computing device according to claim 1, wherein: The computing device is an artificial intelligence processor, an artificial intelligence processing card, an artificial intelligence processing card cluster, a computer or a computer cluster.

8. A method for terminating an execution core, wherein the execution core is disposed in a computing device, characterized in that: The bus interface device in the execution core is coupled to the global memory in the computing device via a bus, and the termination method includes: When performing an end task for storing data in a cache being used by a computing core corresponding to the execution core into a global memory, a corresponding data refresh instruction is issued to each computing core in the execution core to store the cache data corresponding to each computing core into the global memory respectively; Furthermore, after each computing core completes its own operation, it releases its own computing resources, and at the same time, the task scheduler schedules the next task of each computing core to each computing core for execution; after storing the cache data in the global memory, each of the data refresh instructions responds to corresponding confirmation information through the global memory; and Through the bus interface device, it is determined whether the confirmation information issued by the global memory for each computing core has been obtained, and after it is determined that all the confirmation information has been obtained, the end flag corresponding to the end task is set to enable to complete the end task.

9. The termination method according to claim 8, characterized in that: The end flag is set in the global memory, When the end flag is disabled, the execution core is executing an external command. When the end flag is enabled, the execution core is not used.

10. The termination method according to claim 8, characterized in that: The computing device further includes a plurality of buffers, wherein each computing core corresponds to each buffer. Before executing the data refresh instruction, the cache data corresponding to each computing core is stored in each corresponding cache. And, each of the plurality of caches includes an L1 cache.

Images