Method, computing device, medium and program product for executing kernel function

Abstract

The present invention relates to a method, computing device, medium and program product for executing a kernel function. The method includes: establishing a common use connection relationship for common resources among multiple thread bundles related to the same task; controlling the exit time of the current thread bundle of the current task among the multiple tasks based at least on the established common use connection relationship, so that before the current thread bundle exits, all other thread bundles of the current task that have a common use connection relationship with the current thread bundle have completed the use of the corresponding common resources; and in response to determining that the current thread bundle has exited, starting the corresponding thread bundle of the next task, the current task and the next task being executed by the same computing unit. The present invention can not only significantly improve the utilization rate of artificial intelligence chips, but also effectively reduce the execution cost and the difficulty of dynamic task allocation.

Description

Technical Field

[0001] Embodiments of the present invention generally relate to the field of artificial intelligence, and more particularly to a method, a computing device, a computer-readable storage medium, and a computer program product for executing a kernel function. Background Art

[0002] When executing a kernel function, the original task is typically split into multiple tasks and assigned to multiple compute units (CUs) for computation. Each CU is responsible for executing several tasks. It should be understood that a warp is the most basic execution unit, and each task on an AI chip typically consists of several warps. Examples of such AI chips include, but are not limited to, graphics processing units (GPUs), general-purpose graphics processing units (GPGPUs), and tensor processing units (TPUs). When executing a kernel function, the current task can generally only be switched to the next task after each of its multiple warps has completed. It should be understood that different warps have different responsibilities; some warps may terminate early, while others may start later. When a CU executes multiple tasks, idle CUs (also known as "execution bubbles," or gaps) may exist between the preceding and succeeding tasks.

[0003] There are two main traditional solutions for executing kernel functions. In the first solution, no processing is performed on the existing idle computing units. In the second solution, the original task is rewritten so that the number of tasks is the same as the number of computing units. For example, only one task is executed on a CU, and several subtasks are processed within the task, so that there is no task switching, and there is no idle computing unit between the previous and next task switches. For the first solution, since the idle computing units between the previous and next tasks are not processed, it will lead to a waste of resources of the artificial intelligence chip. In particular, for the case where the execution time of the task itself is short, the idle computing units caused by different warps may reach 10-20% of the warp execution time. For the second solution, on the one hand, rewriting the original task will result in a large amount of code being rewritten, which will lead to higher migration costs; on the other hand, since the number of dynamically available execution units is uncertain, it makes dynamic task allocation more difficult.

[0004] In summary, the shortcomings of traditional methods for executing kernel functions are that it is difficult to reduce execution costs and the difficulty of dynamic task allocation while avoiding waste of artificial intelligence chip resources. Summary of the Invention

[0005] The present invention provides a method, computing device, computer-readable storage medium, and computer program product for executing kernel functions, which can not only significantly improve the utilization rate of artificial intelligence chip resources, but also effectively reduce the execution cost and the difficulty of dynamic task allocation.

[0006] According to a first aspect of the present invention, a method for executing a kernel function is provided. The method comprises: establishing a common use connection relationship for a plurality of thread bundles associated with the same task for a common resource, wherein the same task is any one of the plurality of tasks for executing the kernel function; controlling the exit time of the current thread bundle of the current task among the plurality of tasks based at least on the established common use connection relationship, so that before the current thread bundle exits, all other thread bundles of the current task that have a common use connection relationship with the current thread bundle have completed the use of the corresponding common resource; and in response to determining that the current thread bundle exits, starting the corresponding thread bundle of the next task, wherein the current task and the next task are executed by the same computing unit, which is any one of the plurality of computing units for executing the kernel function.

[0007] In some embodiments, starting the corresponding warp of the next task includes: ensuring that the corresponding warp of the next task does not use common resources that other warps of the current task need to access.

[0008] In some embodiments, establishing a common usage connection relationship for a common resource among multiple thread warps related to the same task includes: numbering the multiple thread warps according to the exit order of the multiple thread warps related to the same task based on the original code of the same task; and marking the usage status of each thread warp in the multiple thread warps for the common resource, so as to establish a common usage connection relationship with the multiple thread warps for the common resource.

[0009] In some embodiments, controlling the exit time of the current thread warp of the current task among the multiple tasks based at least on the established common use connection relationship includes: in response to determining that each thread warp of the current task completes use of the corresponding common resource, notifying other thread warps that have a common use connection relationship with respect to the corresponding common resource.

[0010] In some embodiments, based at least on the established common usage connection relationship, controlling the exit time of the current warp of the current task among the plurality of tasks includes: in response to determining that each warp of the current task completes usage of the corresponding common resource, notifying other warps of the current task.

[0011] In some embodiments, notifying other warps having a common usage connection relationship with respect to the corresponding common resource includes: in response to determining that each warp of the current task completes use of the corresponding common resource, sending indication information indicating completion of use of the corresponding common resource to other warps having a common usage connection relationship with respect to the corresponding common resource.

[0012] In some embodiments, controlling the exit time of the current thread warp of the current task among the multiple tasks based at least on the established common use connection relationship further includes: in response to determining that the current thread warp of the current task has completed the current task and received indication information of completion of use of corresponding common resources sent by all other thread warps with a common use connection relationship, allowing the current thread warp of the current task to exit; and in response to determining that the current thread warp of the current task has not completed the current task or has not received indication information of completion of use of corresponding common resources sent by all other thread warps with a common use connection relationship, not allowing the current thread warp of the current task to exit.

[0013] In some embodiments, establishing a common use connection relationship for common resources among multiple thread bundles associated with the same task includes: confirming whether a signal indicating that a concurrent mode switch has been turned on is detected; in response to confirming that a signal indicating that a concurrent mode switch has been turned on is detected, establishing a common use connection relationship for common resources among multiple thread bundles associated with the same task; in response to confirming that a signal indicating that a concurrent mode switch has been turned on is not detected, sending an instruction for turning on the concurrent mode switch so that each computing unit in concurrent mode can execute two thread bundle groups concurrently.

[0014] In some embodiments, starting the corresponding thread bundle of the next task includes: in response to confirming that the current computing unit in concurrent mode needs to receive the second thread bundle group when the first thread bundle group has not yet completed the task, confirming whether the head thread bundle in the first thread bundle group has exited; in response to confirming that the head thread bundle in the first thread bundle group has exited, allowing the head thread bundle in the second thread bundle group to be sent down to the current computing unit for execution; confirming whether the non-head thread bundle in the first thread bundle group has exited; and in response to confirming that the non-head thread bundle in the first thread bundle group has exited, allowing the non-head thread bundle in the second thread bundle group to be sent down to the current computing unit for execution.

[0015] According to a second aspect of the present invention, a computing device is further provided. The computing device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the computing device to perform the method of the first aspect of the present invention.

[0016] According to a third aspect of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a machine, the method according to the first aspect of the present invention is executed.

[0017] According to a fourth aspect of the present invention, there is further provided a computer program product, comprising a computer program, wherein when the computer program is executed by a machine, the method of the first aspect of the present invention is performed.

[0018] The present invention can make the corresponding thread bundle of the latter task start to execute without waiting for all the thread bundles of the previous task to be completely completed, thereby making the previous and next tasks used for kernel function execution can be continuously overlapped and executed on one computing unit, and there is no idle computing unit between the previous and next tasks, thereby significantly improving the utilization rate of the artificial intelligence chip. At the same time, the present invention still follows the classic artificial intelligence chip task segmentation logic, so that the task dynamic scheduling capability is strong, and there is no need to modify the kernel function code on a large scale, which is conducive to improving the reusability of the original code. Therefore, the present invention can not only significantly improve the utilization rate of artificial intelligence chip resources, but also reduce the execution cost and the difficulty of dynamic task allocation.

[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other features, advantages and aspects of the embodiments of the present invention will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements.

[0021] Figure 1 The figure schematically shows a diagram of a computing device implementing a method for executing a kernel function according to an embodiment of the present invention.

[0022] Figure 2 A flowchart of a method for executing a kernel function according to an embodiment of the present invention is shown.

[0023] Figure 3 The diagram shows a common usage connection relationship of multiple warps related to the same task for common resources according to an embodiment of the present invention.

[0024] Figure 4A A schematic diagram of a method for executing a kernel function according to an embodiment of the present invention is shown.

[0025] Figure 4B Another schematic diagram of a method for executing a kernel function according to an embodiment of the present invention is shown.

[0026] Figure 5 A flowchart of a method for controlling the exit of a current warp of a current task according to an embodiment of the present invention is shown.

[0027] Figure 6 A flowchart of a method for starting a corresponding warp of a next task according to an embodiment of the present invention is shown.

[0028] Figure 7 A schematic diagram of a method for starting a corresponding warp of a next task according to an embodiment of the present invention is shown.

[0029] Figure 8 A schematic diagram is shown of a conventional method of not processing idle computing units when executing kernel functions.

[0030] Figure 9 A schematic diagram showing a traditional method of controlling the number of tasks to be the same as the number of computing units when executing a kernel function.

[0031] In the various drawings, the same or corresponding reference numerals denote the same or corresponding parts. DETAILED DESCRIPTION

[0032] The preferred embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although preferred embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to make the present invention more thorough and complete and to fully convey the scope of the present invention to those skilled in the art.

[0033] As used herein, the term "including" and its variations represent open inclusion, i.e., "including but not limited to." The term "based on" means "based at least in part on." The terms "one example embodiment" and "an embodiment" mean "at least one example embodiment." The term "another embodiment" means "at least one additional embodiment." The terms "first," "second," and so on may refer to different or identical objects.

[0034] As described above, there are two conventional solutions for executing kernel functions. In the first solution, idle computing units are not processed. Figure 8FIG. 8 is a schematic diagram showing a conventional method 800 for not processing idle computing units when executing kernel functions. Figure 8 As shown, when executing a kernel function, the next task (e.g., the first task, or "Task 1") can only be started after the zeroth, first, and second warps of the previous task (e.g., the zeroth task, or "Task 0") have all completed and exited. When executing Task 0, the zeroth warp of Task 0 completes first, followed by the first and second warps. The zeroth warp of Task 1 is not started until the second warp of Task 0 completes and exits. Therefore, there is a computing unit idle period 810 (or "execution bubble," or gap) between Task 0 and Task 1. Similarly, there is a computing unit idle period 810 between Task 1 and Task 2. In the first solution mentioned above, since the idle computing units between the previous and next tasks are not processed, it will lead to a waste of artificial intelligence chip resources. Especially for the case where the execution time of the task itself is short, the idle computing units caused by different warps may even reach 10-20% of the warp execution time.

[0035] In the second solution, the original tasks need to be rewritten so that the number of tasks is controlled to be the same as the number of computing units. Figure 9 A schematic diagram of a conventional method 900 for controlling the number of tasks to be the same as the number of computing units when executing a kernel function is shown. For example, only one task (e.g., Figure 9 The zeroth task 910 ("Task 0") shown in the figure processes several subtasks (e.g., subtask 0, subtask 1, ..., subtask M, where M is a natural number). This prevents task switching on the CU and eliminates idle compute units between task switches. Regarding the second solution, rewriting the original task requires extensive code rewriting, resulting in high migration costs. Furthermore, the number of dynamically available execution units is uncertain, making dynamic task allocation difficult.

[0036] In summary, the shortcomings of traditional methods for executing kernel functions are that it is difficult to reduce execution costs and the difficulty of dynamic task allocation while avoiding waste of artificial intelligence chip resources.

[0037] To at least partially address one or more of the above-mentioned problems and other potential problems, exemplary embodiments of the present invention provide a solution for executing kernel functions. In this solution, a common use connection relationship for common resources is established between multiple thread warps associated with the same task among multiple tasks used for kernel function execution; and based at least on the established common use connection relationship, the exit time of the current thread warp among multiple thread warps associated with the current task among the multiple tasks is controlled so that before the current thread warp exits, all other thread warps associated with the current task that have a common use connection relationship with the current thread warp have completed their use of the corresponding common resources; and when the current thread warp associated with the current task is determined to have exited, the corresponding thread warp of the next task is started. This solution allows the corresponding thread warp of the subsequent task to begin execution without waiting for all thread warps of the previous task to complete. This allows the previous and next tasks used for kernel function execution to be executed continuously and overlappingly on a single computing unit, without idle computing units between the previous and next tasks, thereby significantly improving the utilization of the artificial intelligence chip. Furthermore, the present invention still adheres to the classic task segmentation logic of artificial intelligence chips, thereby enabling strong dynamic task scheduling capabilities without requiring large-scale modification of kernel function code, thereby facilitating improved reusability of the original code. Therefore, the present invention can not only significantly improve the utilization rate of artificial intelligence chip resources, but also effectively reduce the execution cost and the difficulty of dynamic task allocation.

[0038] Figure 1 Schematically shows a schematic diagram of a computing device 100 for implementing a method for executing a kernel function according to an embodiment of the present invention. Figure 1As shown, the computing device 100 may have one or more processing units, including specialized processing units such as graphics processing units (GPUs), field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), general-purpose graphics processing units (GPGPUs), or tensor processing units (GPGPUs), as well as general-purpose processing units such as CPUs. The computing device 100 also includes at least: a shared connection relationship establishment unit 102, a current warp exit time control unit 104, and a next-task corresponding warp launch unit 106. It should be understood that the shared connection relationship establishment unit 102, the current warp exit time control unit 104, and the next-task corresponding warp launch unit 106 may be software modules that, for example, run on one or more processing units configured in the computing device 100. The one or more processing units may be integrated with the multiple computing units used to execute the kernel function, or they may be separately configured on different computing devices.

[0039] The common use connection relationship establishing unit 102 is used to establish a common use connection relationship for common resources among multiple warps related to the same task, where the same task is any one of multiple tasks for executing a kernel function.

[0040] Regarding the exit time control unit 104 of the current thread warp, it is used to control the exit time of the current thread warp with the current task among the multiple tasks based on at least the established common use connection relationship, so that before the current thread warp exits, all other thread warps of the current task that have a common use connection relationship with the current thread warp have completed the use of corresponding common resources.

[0041] Regarding the corresponding thread warp starting unit 106 for the next task, it is used to respond to determining whether the current thread warp has exited; and in response to determining that the current thread warp has exited, start the corresponding thread warp of the next task, the current task and the next task are executed by the same computing unit, and the same computing unit is any computing unit among multiple computing units used to execute kernel functions.

[0042] The following will be combined Figure 2 、 Figure 3 、 Figure 4A and Figure 4B A method 200 for executing a kernel function according to an embodiment of the present invention is described. Figure 2 A flowchart of a method 200 for executing a kernel function according to an embodiment of the present invention is shown. Figure 3 A schematic diagram illustrating a common usage connection relationship 300 of multiple warps related to the same task with respect to common resources according to an embodiment of the present invention is shown. Figure 4A A schematic diagram of a method for executing a kernel function according to an embodiment of the present invention is shown. Figure 4B FIG2 shows another schematic diagram of a method for executing a kernel function according to an embodiment of the present invention. It should be understood that the method 200 may be used, for example, in Figure 1 The method 200 is executed at the described computing device 100. The method 200 may also include additional actions not shown and / or may omit actions shown, and the scope of the present invention is not limited in this respect.

[0043] At step 202, the computing device 100 establishes a common use connection relationship for a plurality of thread warps related to the same task for a common resource. With respect to a plurality of thread warps related to the same task, it means that a plurality of thread warps serve the same task. For example, Figure 4A In FIG, the zeroth warp 310 - 1 , the first warp 320 - 1 , and the second warp 330 - 1 of the current task 410 for the same current task 410 are multiple warps related to the same task.

[0044] Regarding the method of establishing a common use connection relationship, in some embodiments, it includes, for example: the computing device 100 confirms whether a signal indicating that the concurrent mode switch has been turned on is detected; in response to confirming that the signal indicating that the concurrent mode switch has been turned on is detected, establishing a common use connection relationship for multiple thread bundles related to the same task for common resources; in response to confirming that the signal indicating that the concurrent mode switch has not been turned on is detected, sending an instruction for turning on the concurrent mode switch so that each computing unit in the concurrent mode can execute two thread bundle groups concurrently.

[0045] It should be understood that the concurrent mode switch is open to the upper-level software of the multiple computing units used to execute the kernel function to enable or turn on. When the concurrent mode switch is enabled, each of the multiple computing units can execute two thread warp groups concurrently; when the concurrent mode switch is not enabled, each computing unit can only execute one thread warp group.

[0046] It should be understood that the execution of a kernel function generally involves multiple tasks. For example, the kernel function executed is, for example, matrix multiplication. That is, C=A*B. Wherein, A and B are two input matrices to be multiplied. C is, for example, an output matrix, which is, for example, a 4086x4096 matrix. In some embodiments, in order to complete matrix multiplication, the original kernel function task of matrix multiplication is generally divided into multiple tasks (the number of tasks is, for example, 1024). Each task, for example, calculates an output block, the size of which is, for example, 128x128. For example, 32 CUs can be used to complete 1024 tasks. Each CU needs to run 32 tasks.

[0047] It should be understood that the previous and next tasks in the multiple tasks (kernel tasks) used for kernel function execution usually access common resources, so conflicts in access to common resources need to be prevented.

[0048] In some embodiments, a method for establishing a shared connection relationship includes, for example, the following: the computing device 100, based on source code of a single task, numbers multiple warps associated with the single task according to their exit order; and annotates the usage of a common resource by each of the multiple warps, thereby establishing a shared connection relationship with the multiple warps regarding the common resource. For example, for a current task (e.g., but not limited to, "Task 0"), the warps are numbered according to the exit order of multiple (e.g., N) warps associated with the current task. For example, the warps associated with the current task are numbered: Warp 0 ... Warp N (i.e., Warp 0 ... Warp N). In some embodiments, the shared resource is, for example, but not limited to, a shared memory address. For example, the computing device 100 divides the shared memory address into address segments to determine the warp that accesses each address segment. For example, the divided address segments are: a shared A segment (i.e., S_A segment, whose corresponding address segment is, for example, [0, 128KB]), a shared B segment (i.e., S_B segment, whose corresponding address segment is, for example, [128KB, 160KB]), and a shared C segment (i.e., S_C segment, whose corresponding address segment is, for example, [160KB, 192KB]). The warps accessing the shared A segment include, for example, the zeroth warp, the first warp, and the second warp (i.e., corresponding to Warp 0, Warp 1, and Warp 2, respectively). The warps accessing the shared B segment include the zeroth warp and the first warp (i.e., corresponding to Warp 0 and Warp 1, respectively). The warps accessing the shared C segment include the zeroth warp and the second warp (i.e., corresponding to Warp 0 and Warp 2, respectively), as shown in Table 1 below.

[0049] Table 1

[0050]

[0051] like Figure 3 As shown, in the established shared-use connection relationship 300, the zeroth warp 310 and the first warp 320 share a shared B segment 342. The zeroth warp 310, the first warp 320, and the second warp 330 share a shared A segment 340. The zeroth warp 310 and the second warp 330 share a shared C segment 344.

[0052] At step 204, the computing device 100 controls the exit time of the current thread warp of the current task among the multiple tasks based at least on the established common use connection relationship, so that before the current thread warp exits, all other thread warps of the current task that have a common use connection relationship with the current thread warp have completed the use of the corresponding common resources.

[0053] like Figure 4A As shown, when the zeroth warp 310-1 of the current task 410 completes the current task at time 420, the computing device 100 does not allow the zeroth warp 310-1 of the current task 410 to exit at time 420 because it has not yet received corresponding shared resource usage completion indication information sent by the first warp 320-1 and the second warp 330-1 of the current task 410, which have a common usage connection relationship with the shared A segment, shared B segment, and shared C segment. Instead, the computing device 100 controls the exit time of the zeroth warp 310-1 of the current task 410, delaying its exit to time 430. At time 430, the zeroth warp 310-1 of the current task 410 has received corresponding shared resource usage completion indication information sent by the first warp and the second warp, which have a common usage connection relationship with the shared A segment, shared B segment, and shared C segment. Therefore, the computing device 100 allows the zeroth warp 310-1 of the current task 410 to exit.

[0054] Regarding the method for controlling the exit time of the current thread bundle of the current task among the multiple tasks, it includes, for example, a variety of ways. In some embodiments, the method for controlling the exit time of the current thread bundle includes, for example: the computing device 100 allows the current thread bundle of the current task to exit in response to determining that the current thread bundle of the current task has completed the current task and received the indication information for the completion of the use of the corresponding common resources sent by all other thread bundles with a common use connection relationship; and does not allow the current thread bundle of the current task to exit in response to determining that the current thread bundle of the current task has not completed the current task or has not received the indication information for the completion of the use of the corresponding common resources sent by all other thread bundles with a common use connection relationship. The following will be combined with Figure 5 The method 500 for controlling the exit time of the current warp of the current task among the multiple tasks is described in detail, which will not be repeated here.

[0055] At step 206 , the computing device 100 determines whether the current warp has exited.

[0056] If the computing device 100 determines that the current thread warp has not exited, it jumps to step 210 and does not start the thread warp corresponding to the next task. Figure 4A As shown, if the computing device 100 determines that the zeroth warp 310-1 (e.g., Warp 0) of the current task 410 (e.g., Task0) has not exited, for example, between the time point 420 and the time point 430, the zeroth warp 310-2 (e.g., Warp 0) of the next task 460 (e.g., Task1) is not started.

[0057] At step 208 , if the computing device 100 determines that the current warp exits, the warp corresponding to the next task is started, and the current task and the next task are executed by the same computing unit, which is any computing unit among the multiple computing units for executing kernel functions.

[0058] In some embodiments, starting the corresponding warp of the next task includes: ensuring that the corresponding warp of the next task does not use common resources that other warps of the current task need to access.

[0059] Regarding the method for starting a warp corresponding to the next task, for example, the method includes: in response to determining that a current compute unit in concurrent mode needs to receive a second warp group while a first warp group has not yet completed its task, determining whether the head warp in the first warp group has exited; in response to determining that the head warp in the first warp group has exited, allowing the head warp in the second warp group to be dispatched to the current compute unit for execution; determining whether non-head warps in the first warp group have exited; and in response to determining that non-head warps in the first warp group have exited, allowing non-head warps in the second warp group to be dispatched to the current compute unit for execution. It should be understood that the head warp in a warp group is primarily used for data transfer operations, while the non-head warps in a warp group are primarily used for executing operations in tensor core enable kernels (TCore enable kernels), such as matrix multiplication and convolution (e.g., MMA / Conv). Therefore, a head warp will terminate its task earlier than non-head warps.

[0060] For example, if the computing device 100 determines that the zeroth warp 310-1 (eg, Warp 0 of Task 0) of the current task 410 (eg, Task 0) exits, the zeroth warp 310-2 (eg, Warp 0 of Task 1) of the next task 460 (eg, Task 1) is started.

[0061] like Figure 4B As shown, the zeroth warp 310 - 2 of the next task 460 (eg, Task 1) has overlapping execution sections 470 with the first warp 320 - 1 (eg, Warp 1 of Task 0) of the current task 410 (eg, Task 0) and the second warp 330 - 1 (eg, Warp 2 of Task 0) of the current task 410 (eg, Task 0).

[0062] Similarly, if the computing device 100 determines that the first warp 320-1 (e.g., Warp 1 of Task 0) of the current task 410 (e.g., Task 0) has exited, the first warp 320-2 (e.g., Warp 1 of Task 1) of the next task 460 (e.g., Task 1) is started. If the computing device 100 determines that the second warp 330-1 (e.g., Warp 2 of Task 0) of the current task 410 (e.g., Task 0) has exited, the second warp 330-2 (e.g., Warp 1 of Task 1) of the next task 460 (e.g., Task 1) is started.

[0063] In the above scheme, the present invention can make the corresponding thread bundle of the latter task start executing without waiting for all the thread bundles of the previous task to be completely completed, thereby making it possible for the previous and next tasks used for kernel function execution to be continuously overlapped and executed on one computing unit, and there is no idle computing unit between the previous and next tasks, thereby significantly improving the utilization rate of the artificial intelligence chip. At the same time, the present invention still follows the classic artificial intelligence chip task segmentation logic, so that the task dynamic scheduling capability is strong, and there is no need to modify the kernel function code on a large scale, which is conducive to improving the reusability of the original code. Therefore, the present invention can not only significantly improve the utilization rate of the artificial intelligence chip, but also effectively reduce the execution cost and the difficulty of dynamic task allocation.

[0064] The following will be combined Figure 5 A method 500 for allowing a current warp of a current task to exit according to an embodiment of the present invention is described. Figure 5 FIG. 5 is a flow chart of a method 500 for controlling the exit of a current warp of a current task according to an embodiment of the present invention. It should be understood that the method 500 may be implemented in, for example, Figure 1 The method 500 is executed at the described computing device 100. The method 500 may also include additional actions not shown and / or may omit actions shown, and the scope of the present invention is not limited in this respect.

[0065] At step 502 , the computing device 100 numbers a plurality of warps related to the same task according to the exit order of the warps based on the original code of the same task.

[0066] At step 504 , the computing device 100 marks the usage status of each warp in the plurality of warps for the common resource, so as to establish a common usage connection relationship with the plurality of warps for the common resource.

[0067] For example, Figure 4A As shown, for the shared A segment, the zeroth warp 310-1 of the current task 410 and the first warp 320-1 of the current task 410 have a common use connection relationship, and the zeroth warp 310-1 of the current task 410 and the second warp 330-1 of the current task 410 have a common use connection relationship. For the shared B segment, the zeroth warp 310-1 of the current task 410 and the first warp 320-1 of the current task 410 have a common use connection relationship. For the shared C segment, the zeroth warp 310-1 of the current task 410 and the second warp 330-1 of the current task 410 have a common use connection relationship.

[0068] At step 506 , if the computing device 100 determines that each warp of the current task has completed use of the corresponding common resource, the computing device 100 notifies other warps having a common use connection relationship with the corresponding common resource.

[0069] For example, if it is determined that each warp of the current task has completed the use of the corresponding public resource, indication information indicating the completion of the use of the corresponding public resource is sent to other warps having a common use connection relationship with the corresponding public resource. Figure 4A As shown, after the zeroth warp 310-1 of the current task 410 completes use of the shared A segment, it sends first indication information of completion of use of the shared A segment (indicated by marker 412) to the first warp 320-1 of the current task 410 and the second warp 330-1 of the current task 410, which have a common use connection relationship with the shared A segment. After the zeroth warp 310 completes use of the shared B segment, it sends first indication information of completion of use of the shared B segment (indicated by marker 414) to the first warp 320-1 of the current task 410, which has a common use connection relationship with the shared B segment. And after the zeroth warp 310-1 of the current task 410 completes use of the shared C segment, it sends second indication information of completion of use of the shared C segment (indicated by marker 416) to the second warp 330-1 of the current task 410, which has a common use connection relationship with the shared C segment.

[0070] It should be understood that in other embodiments, if the computing device 100 determines that each warp of the current task completes use of the corresponding common resource, it notifies all other warps of the current task, not limited to other warps with a common use connection relationship.

[0071] At step 508 , if the computing device 100 determines that the current warp of the current task has completed the current task and received indication information of completion of use of corresponding common resources sent by all other warps having a common use connection relationship, the current warp of the current task is allowed to exit.

[0072] If the computing device 100 determines that the current warp of the current task has not completed the current task or has not received indication information of completion of use of corresponding common resources sent by all other warps having a common use connection relationship, the current warp of the current task is not allowed to exit.

[0073] like Figure 4AAs shown, the zeroth warp 310-1 of the current task 410 completes the current task at time 420. However, at time 420, the zeroth warp 310-1 of the current task 410 has not yet received indications of completion of use of corresponding common resources from the first warp 320-1 of the current task 410 and the second warp 330-1 of the current task 410, with which the two warps have a common use connection. Therefore, the zeroth warp 310-1 of the current task 410 is not allowed to exit. For example, after the first warp 320-1 of the current task 410 completes use of the shared A segment, it sends a second indication of completion of use of the shared A segment (indicated by marker 422) to the zeroth warp 310-1 of the current task 410. After the second warp 330-1 of the current task 410 completes use of the shared A segment, it sends a third indication of completion of use of the shared A segment (indicated by marker 424) to the zeroth warp 310-1 of the current task 410. Furthermore, after the first warp 320-1 of the current task 410 completes the use of the shared B segment, it sends a second indication of completion of use of the shared B segment (as indicated by mark 426) to the zeroth warp 310-1 of the current task 410. After the second warp 330-1 of the current task 410 completes the use of the shared C segment, it sends a second indication of completion of use of the shared C segment (as indicated by mark 428) to the zeroth warp 310-1 of the current task 410. At time point 430, the zeroth warp 310-1 of the current task 410 has received indications of completion of use of corresponding common resources sent by the first warp and the second warp that have a common use connection relationship with respect to the shared A segment, the shared B segment, and the shared C segment. Therefore, the zeroth warp 310-1 of the current task 410 is allowed to exit. Figure 4A It can be seen that the zeroth warp 310-1 of the current task 410 completes the current task at time point 420. However, at time point 420, the zeroth warp 310-1 of the current task 410 is not allowed to exit. Instead, the exit time of the zeroth warp 310-1 of the current task 410 is controlled to be delayed to time point 430, that is, delayed until the indication information of completion of use of corresponding common resources sent by all other warps having a common use connection relationship is received.

[0074] Similar, such as Figure 4A As shown, at time point 440, the first thread warp 320-1 of the current task 410 completes the task and has received the indication information of completion of use of corresponding public resources sent by the zeroth thread warp 310-1 of the current task 410 having a common use connection relationship for the shared A segment and the shared B segment. Therefore, the first thread warp 320-1 of the current task 410 is allowed to exit.

[0075] Similarly, at time point 450, the second thread warp 330-1 of the current task 410 completes the task and has received the corresponding public resource usage completion indication information sent by the zeroth thread warp 310-1 of the current task 410 that has a common usage connection relationship with the shared A segment and the shared C segment. Therefore, the second thread warp 330-1 of the current task 410 is allowed to exit.

[0076] By adopting the above-mentioned means, the present invention can bind the exit time of each thread warp to the completion status of the use of the corresponding common resources, thereby ensuring that when each thread warp exits, the other thread warps with a common use connection relationship have completed the use of the corresponding common resources, thereby avoiding resource conflicts with the corresponding thread warp of the next task.

[0077] The following will be combined Figure 6 and Figure 7 A method 600 for starting a corresponding warp of a next task according to an embodiment of the present invention is described. Figure 6 A flowchart of a method 600 for starting a corresponding warp of a next task according to an embodiment of the present invention is shown. Figure 7 FIG. 6 is a schematic diagram showing a method for starting a corresponding warp of a next task according to an embodiment of the present invention. It should be understood that the method 600 may be used, for example, in Figure 1 The method 600 is executed at the described computing device 100. The method 600 may also include additional actions not shown and / or may omit actions shown, and the scope of the present invention is not limited in this respect.

[0078] At step 602 , in response to confirming that the current computing unit in concurrent mode needs to receive the second warp group when the first warp group has not finished its task, it is confirmed whether the head warp in the first warp group has exited.

[0079] Regarding the thread warp group, it is a thread warp group (or workgroup) used for tasks related to the kernel function. The thread warp group is abbreviated as "tg".

[0080] Regarding the first warp group, it is, for example, several warps (Warp0 . . . WarpN) of the current task associated with the kernel function. In some embodiments, the first warp group is, for example but not limited to, Warp0 . . . Warp3 of Task0 .

[0081] The second warp group may be, for example, several warps (Warp0 . . . WarpN) of the next task associated with the kernel function. In some embodiments, the second warp group may be, for example but not limited to, Warp0 . . . Warp3 of Task1 .

[0082] It should be understood that the first warp group and the second warp group are two adjacent warp groups that can be executed concurrently among the multiple warp groups, for example, tg0 and tg1 among the three warp groups (tg0, tg1, and tg2), or tg1 and tg2.

[0083] like Figure 7 As shown, the first thread bundle group 710 starts to be executed at the first time point 712. When the first thread bundle group 710 has not completed the task, it needs to receive the second thread bundle group 720. The first thread bundle group 710 includes a head thread bundle and a non-head thread bundle. The second thread bundle group 720 also includes a head thread bundle and a non-head thread bundle. The head thread bundle is used for the data transfer operation related to the kernel function execution; the non-head thread bundle is used for the kernel function execution operation. Figure 7 As shown, at a first time point 712, the head warp of the first warp group 710 is started to execute a task. For example, if the computing device 100 determines that the second warp group 720 needs to be received before the first warp group 710 completes its task, the computing device 100 further determines whether the head warp in the first warp group 710 has exited.

[0084] At step 604 , in response to confirming that the head warp in the first warp group has not exited, the head warp in the second warp group is not allowed to be issued to the current computing unit.

[0085] For example, between the first time point 712 and the second time point 714, the head warp of the first warp group 710 has not yet completed its task and exited. At this time, the head warp of the second warp group 720 is not allowed to be sent to the current computing unit.

[0086] At step 606 , in response to confirming that the head warp in the first warp group has exited, the head warp in the second warp group is allowed to be sent to the current computing unit for execution.

[0087] For example, if the computing device 100 confirms that the head warp in the first warp group 710 has exited (for example, at the second time point 714, the head warp in the first warp group 710 completes the task and exits), the head warp in the second warp group 720 is allowed to be sent to the current computing unit for execution.

[0088] At step 608 , it is determined whether the non-head warp in the first warp group has exited.

[0089] For example, the computing device 100 confirms whether the non-head warp in the first warp group 710 has exited (eg, at the third time point 716 , the non-head warp in the first warp group 710 completes its task and exits).

[0090] At step 610 , in response to confirming that the non-head warp in the first warp group has not exited, the non-head warp in the second warp group is not allowed to be issued to the current computing unit.

[0091] For example, if it is confirmed that the non-head warp of the first warp group 710 has not yet completed its task and exited (for example, Figure 7 At this time, it is not allowed to send the non-head warp of the second warp group 720 to the current computing unit.

[0092] At step 612 , in response to confirming that the non-head warp in the first warp group has exited, the non-head warp in the second warp group is allowed to be issued to the current computing unit for execution.

[0093] like Figure 7 As shown, if the computing device 100 confirms that the non-head thread warp in the first thread warp group has exited (for example, at the third time point 716, the non-head thread warp in the first thread warp group 710 ends the task and exits), the non-head thread warp in the second thread warp group is allowed to be sent to the current computing unit for execution.

[0094] Similarly, in some embodiments, if the computing device 100 further confirms that the head warp in the second warp group 720 has exited (for example, at the fourth time point 722, the head warp in the second warp group 720 completes its task and exits), the head warp in the third warp group 730 is allowed to be delivered to the current computing unit for execution. If the computing device 100 further confirms that the non-head warps in the second warp group have exited (for example, at the fifth time point 724, the non-head warps in the second warp group 720 complete their task and exit), the non-head warps in the third warp group 730 are allowed to be delivered to the current computing unit for execution.

[0095] In the above scheme, the present invention enables concurrent execution of two thread warp groups within a tensor core function within the same compute unit (CU), significantly improving the execution efficiency of the thread warp groups within the CU and reducing the kernel function's runtime. Furthermore, the present invention enables execution of kernel functions where the number of thread warp groups exceeds the number of CUs.

[0096] The various processes and processing described above, such as methods 200 through 500, may be performed on a computing device. The computing device may include, for example, at least one processor (at least one graphics processor and at least one central processing unit); and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor. In some embodiments, methods 200 through 500 may be implemented as a computer software program or program product tangibly embodied on a machine-readable medium. In some embodiments, part or all of the computer program may be loaded and / or installed onto the computing device via read-only memory (ROM) and / or a communication unit. When the computer program is loaded into random-access memory (RAM) and executed by the GPU and CPU, one or more actions of methods 200 through 500 described above may be performed.

[0097] The present invention may be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions for performing various aspects of the present invention. The computer-readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof.

[0098] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. Various aspects of the present invention are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each block in the flowcharts and / or block diagrams, as well as combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0099] These computer-readable program instructions can be provided to a central processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine such that when these instructions are executed by the central processing unit of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0100] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented using a dedicated hardware-based system that performs the prescribed function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0101] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved. This is not a limitation herein.

[0102] The above specific embodiments do not constitute a limitation on the scope of protection of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors.

Claims

1. A method for executing a kernel function, characterized in that include: Establishing a common use connection relationship for common resources among multiple thread warps related to the same task, wherein the multiple thread warps related to the same task refer to multiple thread warps serving the same task; Controlling an exit time of a current warp of a current task among the plurality of tasks based at least on the established common use connection relationship, so that before the current warp exits, all other warps of the current task that have a common use connection relationship with the current warp complete use of corresponding common resources; and In response to determining that the current warp exits, a warp corresponding to a next task is started, wherein the current task and the next task are executed by a same computing unit, which is any computing unit among a plurality of computing units for executing kernel functions.

2. The method according to claim 1, characterized in that The corresponding thread warp that starts the next task includes: This ensures that the corresponding warp of the next task does not use the common resources that other warps of the current task need to access.

3. The method according to claim 1, wherein Establishing a connection relationship for multiple thread warps related to the same task to use common resources includes: Numbering the plurality of thread warps related to the same task according to the exit order of the plurality of thread warps based on the original code of the same task; and A usage status of each of the plurality of warps for a common resource is marked, so as to establish a common usage connection relationship with the plurality of warps for the common resource.

4. The method according to claim 3, characterized in that Controlling an exit time of a current warp of a current task among the plurality of tasks based at least on the established common use connection relationship includes: In response to determining that each warp of the current task completes use of the corresponding common resource, other warps having a common use connection relationship with the corresponding common resource are notified.

5. The method according to claim 3, characterized in that Controlling an exit time of a current warp of a current task among the plurality of tasks based at least on the established common use connection relationship includes: In response to determining that each warp of the current task completes usage of the corresponding common resource, other warps of the current task are notified.

6. The method according to claim 4, characterized in that Notifications to other thread bundles that have a common usage connection relationship with the corresponding public resource include: In response to determining that each warp of the current task completes use of the corresponding common resource, indication information indicating completion of use of the corresponding common resource is sent to other warps having a common use connection relationship with the corresponding common resource.

7. The method according to claim 6, wherein At least based on the established common use connection relationship, controlling the exit time of the current warp of the current task among the plurality of tasks further includes: In response to determining that the current warp of the current task has completed the current task and received indication information of completion of use of corresponding common resources sent by all other warps having a common use connection relationship, allowing the current warp of the current task to exit; and In response to determining that the current warp of the current task has not completed the current task or has not received indication information of completion of use of corresponding common resources sent by all other warps having a common use connection relationship, the current warp of the current task is not allowed to exit.

8. The method according to claim 1, characterized in that Establishing a connection relationship for multiple thread warps related to the same task to use common resources includes: confirming whether a signal indicating that the concurrent mode switch has been turned on is detected; In response to confirming that a signal indicating that the concurrent mode switch has been turned on is detected, establishing a common use connection relationship for a plurality of thread warps associated with the same task with respect to a common resource; In response to confirming that the signal indicating that the concurrent mode switch has been turned on is not detected, an instruction for turning on the concurrent mode switch is sent to enable each computing unit in the concurrent mode to concurrently execute two warp groups.

9. The method according to claim 1, characterized in that The corresponding thread warp that starts the next task includes: In response to confirming that the current computing unit in the concurrent mode needs to receive the second warp group when the first warp group has not yet finished its task, confirming whether the head warp in the first warp group has exited; In response to confirming that the head warp in the first warp group has exited, allowing the head warp in the second warp group to be sent to the current computing unit for execution; confirming whether a non-head warp in the first warp group has exited; and In response to confirming that the non-head warp in the first warp group has exited, the non-head warp in the second warp group is allowed to be issued to the current computing unit for execution.

10. A computing device, characterized in that: include: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a machine, performs the method according to any one of claims 1 to 9.

12. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a machine, performs the method according to any one of claims 1 to 9.

Images