Methods, chips, devices, media, and program products for issuing kernel functions

Abstract

This invention relates to a method, chip, apparatus, medium, and program product for issuing tasks to kernel functions. The method includes: configuring configuration information about computing units, and causing the configuration information to be issued to an allocation unit in the computing core along with task data of the kernel function. The configuration information includes computing unit description information for the kernel function, and the kernel function is at least one of a plurality of kernel functions to be executed by the computing core. In response to the allocation unit receiving the configuration information about the computing units, the method performs any of the following based on the configuration information: determining the number of computing units used by the kernel function; and determining the computing units used to execute the relevant tasks of the kernel function. This invention can significantly improve the resource utilization of hardware computing units in artificial intelligence chips.

Description

Technical Field

[0001] The embodiments of the present invention generally relate to the field of artificial intelligence technology, and more specifically to a method for issuing kernel functions for tasks, an artificial intelligence chip, a computing device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] In traditional methods for issuing kernel tasks, the computation kernel typically issues the received kernel task requests sequentially. It then uses a polling mechanism to select a computation unit (CU) that meets the resource requirements of a thread group (tg) and assigns the thread group to that CU for execution. If the current kernel task requires a large number of thread groups—for example, exceeding the total number of computation units in the kernel—or if the current kernel task will fill all the computation units in the kernel, the thread groups required by the next kernel task will be in a waiting state because there are no available computation units to execute them.

[0003] In summary, the traditional methods for issuing tasks related to kernel functions have the following shortcomings: it is difficult to allocate computing units reasonably, which leads to excessively long waiting times for kernel functions in the queue to wait for available computing units, thereby reducing the resource utilization rate of the hardware computing units of artificial intelligence chips. Summary of the Invention

[0004] This invention provides a method, chip, device, medium, and program product for issuing kernel functions, which can significantly improve the resource utilization of the hardware computing unit of an artificial intelligence chip.

[0005] According to a first aspect of the present invention, a method for distributing tasks of kernel functions is provided, the method comprising: configuring configuration information about computing units, and causing the configuration information to be distributed to an allocation unit in a computing core along with task data of a kernel function, the configuration information including computing unit description information for a kernel function, the kernel function being at least one of a plurality of kernel functions to be executed by the computing core; in response to the allocation unit receiving the configuration information about computing units, performing any of the following based on the configuration information: determining the number of computing units used by the kernel function; and determining the computing units for executing the relevant tasks of the kernel function.

[0006] In some embodiments, the configuration information regarding the computing units includes a first descriptor or a second descriptor, wherein the first descriptor indicates the maximum number of computing units used by the kernel function, and the second descriptor indicates the corresponding computing units used to perform the relevant tasks of the kernel function.

[0007] In some embodiments, in response to determining that a first descriptor included in the configuration information indicates the maximum number of computing units used by the current kernel function, the allocation unit selects computing units for performing the relevant tasks of the current kernel function based on the first descriptor, such that the number of selected computing units is less than or equal to the maximum number of computing units indicated by the first descriptor.

[0008] In some embodiments, in response to determining that the second descriptor included in the configuration information indicates a computing unit for executing the relevant task of the current kernel function, the allocation unit selects the corresponding computing unit for executing the current kernel function based on the second descriptor; and sends the relevant thread group of the current kernel function to the selected corresponding computing unit.

[0009] In some embodiments, sending the relevant thread group of the current kernel function to the selected corresponding computing unit includes: the allocation unit confirming whether all the selected corresponding computing units have met the predetermined detection conditions; and in response to confirming that all the selected corresponding computing units have met the predetermined detection conditions, the allocation unit sends the relevant thread group of the current kernel function to the computing units that meet the predetermined detection conditions.

[0010] In some embodiments, sending the relevant thread group of the current kernel function to the selected corresponding computing unit further includes: in response to confirming that there are some computing units in the selected corresponding computing units that have not yet met the predetermined detection conditions, the allocation unit sends the relevant thread group of the current kernel function to the computing units that have already met the predetermined detection conditions, while waiting for the some computing units that have not yet met the predetermined detection conditions to meet the predetermined detection conditions again; and the allocation unit sends the corresponding thread group in the relevant thread group of the current kernel function to the some computing units that have met the predetermined detection conditions again.

[0011] In some embodiments, the method further includes: the allocation unit selecting at least a portion of the available computing units to execute the next kernel function based on a polling mechanism; or the allocation unit selecting at least a portion of the available computing units to execute the next kernel function based on the computing unit description information for the next kernel function included in the configuration information.

[0012] According to a second aspect of the present invention, an artificial intelligence chip is also provided, the artificial intelligence chip comprising: a command processor configured to configure configuration information about computing units, the configuration information being sent to an allocation unit in the computing core along with task data of a kernel function, the configuration information including computing unit description information for the kernel function, the kernel function being at least one of a plurality of kernel functions to be executed by the computing core; and a computing core comprising: an allocation unit configured to determine, based on the received configuration information about computing units, the number of computing units used by the kernel function, or to determine computing units for executing related tasks of the kernel function; and a plurality of computing units, at least a portion of the plurality of computing units being configured to execute related tasks of the kernel function.

[0013] In some embodiments, the allocation unit is further configured to: in response to determining that the configuration information indicates a maximum number of computing units used by the kernel function, select computing units for executing the relevant tasks of the kernel function based on the configuration information, such that the number of selected computing units is less than or equal to the maximum number; or in response to determining that the configuration information indicates computing units for executing the relevant tasks of the kernel function, select corresponding computing units for executing the kernel function based on the configuration information.

[0014] According to a third aspect of the present invention, a computing device is also provided. The computing device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the computing device to perform the method of the first aspect of the present invention.

[0015] According to a fourth aspect of the present invention, a computer-readable storage medium is also provided. The computer-readable storage medium stores a computer program that, when executed by a machine, performs the method of the first aspect of the present invention.

[0016] According to a fifth aspect of the present invention, a computer program product is also provided, comprising a computer program that, when executed by a machine, performs the method of the first aspect of the present invention.

[0017] This invention configures the computing units by sending configuration information along with the task data of the kernel function to the allocation unit in the computing core. This allows the allocation unit to determine the number of computing units used by the kernel function or the computing units used to execute the related tasks of the kernel function based on the configuration information. As a result, this invention allows the computing units occupied by the kernel function to be flexibly configured on demand. Therefore, when multiple kernel functions are started simultaneously on the same device, the problem of insufficient computing resources or excessive waiting time for later kernel functions will not occur. Thus, this invention can significantly improve the resource utilization of the hardware computing units of artificial intelligence chips.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0019] The above and other features, advantages, and aspects of the various embodiments of the present invention will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements.

[0020] Figure 1 The diagram illustrates a computing apparatus for implementing a method of issuing tasks related to kernel functions according to an embodiment of the present invention.

[0021] Figure 2 A flowchart illustrating a method for issuing kernel functions according to an embodiment of the present invention is shown.

[0022] Figure 3 A flowchart of a method for selecting a computing unit according to an embodiment of the present invention is shown.

[0023] Figure 4 A schematic diagram of the structure of an artificial intelligence chip according to some embodiments of the present invention is shown.

[0024] Figure 5 A flowchart of a method for selecting a computing unit according to some embodiments of the present invention is shown.

[0025] In the various figures, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0026] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0027] The term "comprising" and its variations as used herein signify open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc., may refer to different or the same objects.

[0028] The following example uses a computing core with 16 CUs to illustrate the traditional method of assigning tasks to kernel functions. It should be understood that AI chips (e.g., but not limited to GPUs or GPGPUs) may contain more than 16 CUs; ​​the 16 CUs used here are for illustrative purposes only. For example, the allocation unit of the computing core receives 32 thread groups from kernel function k0 and 4 thread groups from kernel function k1 (e.g., each thread group of kernel function k1 requires more resources). The allocation unit will first allocate the 32 thread groups of kernel function k0 to the 16 CUs, so that each CU is allocated 2 thread groups. If the resources in the CUs are sufficient to run the 4 thread groups of kernel function k0 simultaneously, all 16 CUs are occupied by kernel function k0, and each occupied CU is not fully utilized. However, the remaining free resources in each CU are insufficient to run the subsequent kernel function k1. This will make it difficult for the thread group of kernel function k1 to be allocated to the corresponding CU. It can only wait until all the tasks on the CU occupied by kernel function k0 are completed before the thread group of kernel function k1 can be allocated to an idle CU for execution.

[0029] Therefore, the traditional method of issuing kernel functions has the following shortcomings: unreasonable allocation of computing units, which leads to excessively long waiting times for kernel functions in the queue to wait for available computing units, thereby reducing the resource utilization rate of the hardware computing units of artificial intelligence chips.

[0030] To at least partially address one or more of the aforementioned problems and other potential issues, exemplary embodiments of the present invention propose a method for issuing tasks to kernel functions. In this method, configuration information regarding computing units is configured in the command processor, and this configuration information is issued to the allocation unit within the computing core along with the kernel function's task data. Upon receiving the configuration information, the allocation unit determines the number of computing units used by the kernel function, or determines the computing units used to execute the related tasks of the kernel function, based on the configuration information. Since the allocation unit can select the corresponding computing units or the number of computing units based on the configuration information when issuing kernel function tasks, the present invention allows for flexible and on-demand configuration of the computing units occupied by the kernel function. Therefore, when multiple kernel functions are started simultaneously on the same device, the problem of insufficient computing resources or excessively long waiting times for later kernel functions is avoided. Thus, the present invention can significantly improve the resource utilization rate of the hardware computing units in artificial intelligence chips.

[0031] Figure 1 A schematic diagram of a computing device 100 for implementing a method of issuing tasks related to kernel functions according to an embodiment of the present invention is shown. Figure 1 As shown, the computing device 100 may have one or more processing units and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor. The processing unit includes dedicated processing units such as graphics processing units (GPUs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), general-purpose computing on graphics processing units (GPGPUs), and general-purpose processing units such as CPUs. The computing device 100 also includes at least: an information configuration unit 102 for computing units and a computing unit determination unit 104.

[0032] The information configuration unit 102 for the computing unit is used to configure configuration information for the computing unit and to send the configuration information to the allocation unit in the computing core along with the task data of the kernel function. The configuration information includes computing unit description information for the kernel function, and the kernel function is at least one of a plurality of kernel functions to be executed by the computing core.

[0033] Regarding the computing unit determination unit 104, it is configured to, in response to the allocation unit receiving the configuration information regarding the computing units, perform any of the following based on the configuration information: determine the number of computing units used by the kernel function; and determine the computing units used to perform the relevant tasks of the kernel function.

[0034] The following will combine Figure 2 and Figure 4 A method 200 for issuing kernel functions according to an embodiment of the present invention is described. It should be understood that method 200 can, for example, be implemented in... Figure 1 The described computing device 100 performs the operation. Method 200 may also include additional actions not shown and / or the actions shown may be omitted; the scope of the invention is not limited in this respect.

[0035] At step 202, the computing device 100 configures configuration information about the computing unit, and causes the configuration information to be sent to the allocation unit in the computing core along with the task data of the kernel function. The configuration information includes computing unit description information for the kernel function, and the kernel function is at least one of a plurality of kernel functions to be executed by the computing core.

[0036] Regarding the task data of the kernel function, it is, for example, the command processor (cp) in the computing device 100 (such as... Figure 4 (Indicated by marker 402) The data received regarding the computation task of the kernel function.

[0037] Regarding the computing device 100, which includes at least an artificial intelligence chip 400, it should be understood that the computing device 100 may also include other components. For example... Figure 4 As shown, the artificial intelligence chip 400 includes at least a command processor 402 and a computing core 406. The computing core 406 includes at least a receiving unit 408, an allocation unit 412, and a computing unit 414. In some embodiments, the computing core 406 further includes a processing unit 410.

[0038] The configuration information regarding the computing units may include, for example, a first descriptor or a second descriptor. The first descriptor indicates the maximum number of computing units used by the kernel function, and the second descriptor indicates the corresponding computing units used to execute the related tasks of the kernel function. In some embodiments, the first descriptor may include, for example, the parameter "maxCuNum" and its value. The second descriptor may include, for example, the parameter "cu_mask" and multiple flag bits, which indicate the identifiers of the computing units used to execute the kernel function. For example, the second descriptor may be "cu_mask=0xff," which includes four flag bits. The first flag bit "f" from low to high indicates computing units CU0, CU1, CU2, and CU3, and the second flag bit "f" from low to high indicates computing units CU4, CU5, CU6, and CU7. For example, if the configuration information issued along with the task data of kernel function k0 is "maxCuNum=8," then this configuration information indicates that the maximum number of computing units used by kernel function k0 is 8.

[0039] Regarding the computing core, it is used to execute kernel functions. In some embodiments, the computing core is, for example, a stream processor cluster (SPC). It should be noted that the present invention does not limit the number of artificial intelligence chips and computing cores.

[0040] For example, when a kernel function is launched, the computing device 100 configures configuration information about the computing unit based on that kernel function, such that the configuration information at least indicates the launched kernel function and either a first descriptor or a second descriptor. Then, the task data of the launched kernel function and the configuration information about the computing unit are sent via the task dispatching unit 404 included in the command processor 402 to the receiving unit 408 in the computing core 406 (e.g., ...). Figure 4 (As indicated by marker 416). The task data of the kernel function and the configuration information about the computing unit received by the receiving unit 408 are further provided to the allocation unit 412.

[0041] In some embodiments, a method for configuring configuration information regarding computing units includes, for example, determining configuration information regarding computing units based on the number of thread groups of at least one of a plurality of kernel functions to be executed, the computing resources occupied by each thread group, and the computing units included in the computing core. In some embodiments, the computing device 100 configures a first descriptor or a second descriptor for a first kernel function, wherein the first descriptor is used to indicate the maximum number of computing units used by the kernel function.

[0042] At step 204, in response to the allocation unit receiving the configuration information about the computing units, the computing device 100 performs any of the following based on the configuration information: determining the number of computing units used by the kernel function; and determining the computing units used to perform the relevant tasks of the kernel function.

[0043] For example, if the computing device 100 determines that the first descriptor included in the configuration information indicates the maximum number of computing units used by the current kernel function, the allocation unit selects computing units for performing the relevant tasks of the current kernel function based on the first descriptor, such that the number of selected computing units is less than or equal to the maximum number of computing units indicated by the first descriptor.

[0044] In some embodiments, if the allocation unit 412 of the computing device 100 determines that the first descriptor included in the configuration information (configuration information issued following the task data of kernel function k0, for example including: maxCuNum=8) indicates the maximum number of computing units used by the current kernel function. For example, if the first descriptor in the configuration information "maxCuNum=8" issued following the task data of kernel function k0 indicates that the maximum number of computing units used by kernel function k0 is "8", the allocation unit 412 selects a number of computing units less than or equal to "8" to execute the relevant tasks of kernel function k0 based on this configuration information. Specifically, for example, the allocation unit 412 allocates the 32 thread groups of kernel function k0 to the selected 8 computing elements. It should be understood that the above configuration information does not include configuration information for kernel function k1. In this case, the allocation unit 412 may, for example, select 4 computing units from the computing units based on a polling rule to execute the 4 thread groups of kernel function k1 respectively.

[0045] In some embodiments, the configuration information regarding the computing units following the task data of kernel function k0 includes maxCuNum=8, and the configuration information regarding the computing units following the task data of kernel function k1 includes maxCuNum=4. Based on this configuration information, allocation unit 412 selects "8" computing units to execute the relevant tasks of kernel function k0. Allocation unit 412 also selects "4" computing units to execute the relevant tasks of kernel function k1. For example, allocation unit 412 allocates the 32 thread groups of kernel function k0 to the selected 8 computing elements based on a polling rule; and allocates the 4 thread groups of kernel function k1 to the selected 4 computing elements based on a polling rule.

[0046] In some embodiments, if the computing device 100 determines that the second descriptor included in the configuration information indicates a computing unit for executing a relevant task of the current kernel function, the allocation unit selects a corresponding computing unit for executing the current kernel function based on the second descriptor; and sends the relevant thread group of the current kernel function to the selected corresponding computing unit.

[0047] For example, if the allocation unit 412 of the computing device 100 determines that the second descriptor included in the configuration information (configuration information issued along with the task data of the kernel function k0, for example including: cu_mask=0xff) indicates the computing unit for executing the relevant task of the current kernel function. For example, the computing units for executing the relevant task of the kernel function k0 indicated by the second descriptor "mask=0xff" in the configuration information "cu_mask=0xff" are CU0, CU1, CU2, CU3, CU4, CU5, CU6, and CU7. The allocation unit 412 allocates the 32 thread groups of the kernel function k0 to the computing units CU0 to CU7.

[0048] In some embodiments, the allocation unit 412 confirms whether the selected corresponding computing units CU0 to CU7 have met the predetermined detection conditions; and if it is confirmed that at least some of the selected corresponding computing units CU0 to CU7 have met the predetermined detection conditions, the allocation unit 412 allocates the 32 thread groups of the kernel function k0 to the computing units CU0 to CU7.

[0049] In the above scheme, by configuring the configuration information of the computing units and sending the configuration information along with the task data of the kernel function to the allocation unit in the computing core, the allocation unit of the computing core can determine the number of computing units used by the kernel function or the computing units used to execute the related tasks of the kernel function based on the configuration information. Thus, the present invention can configure the computing units occupied by the task data of the kernel function to be executed as needed, unlike the traditional method which allocates the thread group of the current kernel function to all computing units of the computing core, resulting in excessively long waiting times for kernel functions in the queue. The present invention allows the computing units occupied by the kernel function to be flexibly configured as needed, thus preventing the problem of insufficient computing resources or excessively long waiting times for later kernel functions when multiple kernel functions are started simultaneously on the same device. Therefore, the present invention can significantly improve the resource utilization rate of the hardware computing units of artificial intelligence chips.

[0050] As mentioned above, the computing device 100 can also determine the selected computing unit through configuration information, and then send the thread group to be executed to the computing unit to execute the corresponding kernel function task. Therefore, the method 200 also includes, for example, a method 300 for selecting a computing unit. The following will be combined with... Figure 3 and Figure 4 A method 300 for selecting a computing unit according to an embodiment of the present invention is described. It should be understood that method 300 can, for example, be implemented in... Figure 1 The described computing device 100 performs the procedure. Method 300 may also include additional actions not shown and / or the actions shown may be omitted; the scope of the invention is not limited in this respect.

[0051] At step 302, the allocation unit of the computing device 100 confirms whether the selected corresponding computing units have all met the predetermined detection conditions.

[0052] The predetermined detection conditions are, for example, that the computing unit is in an idle state; or that although the computing unit is in an occupied state, the remaining computing resources of the computing unit are greater than the computing resources required by the predetermined number of thread groups of the kernel function.

[0053] In some embodiments, the allocation unit 412 detects whether the selected computing unit is in an idle state; if the selected corresponding computing unit is in an idle state, it determines that the selected corresponding computing unit meets the predetermined detection conditions; if the selected corresponding computing unit is in an occupied state and the remaining computing resources of the computing unit are less than the computing resources required by a predetermined number of thread groups of the kernel function, it determines that the selected corresponding computing unit does not meet the predetermined detection conditions.

[0054] In step 304, in response to confirming that the selected corresponding computing units have met the predetermined detection conditions, the allocation unit sends the relevant thread group of the current kernel function to the computing units that meet the predetermined detection conditions.

[0055] In some embodiments, if the allocation unit 412 determines that all the selected corresponding computing units have met the predetermined detection conditions (e.g., all the selected computing units CU0 to CU7 have met the predetermined detection conditions), the allocation unit 412 sends the relevant thread group of the current kernel function to the computing units that meet the predetermined detection conditions (e.g., allocate the relevant thread group of the current kernel function to the selected computing units CU0 to CU7).

[0056] In some embodiments, if the allocation unit 412 determines that at least some of the selected corresponding computing units have met the predetermined detection conditions (for example, among the selected computing units CU0 to CU7, CU0 to CU6 have met the predetermined detection conditions, while CU7 has not yet met the predetermined detection conditions), the allocation unit 412 distributes the relevant thread group of the current kernel function (for example, distributes the 28 thread groups of kernel function k0) to the at least some computing units (for example, the selected computing units CU0 to CU6) that meet the predetermined detection conditions.

[0057] In step 306, in response to confirming that there are some computing units among the selected corresponding computing units that have not yet met the predetermined detection conditions, the allocation unit sends the relevant thread group of the current kernel function to the computing units that have already met the predetermined detection conditions, while waiting for the portion of computing units that have not yet met the predetermined detection conditions to meet the predetermined detection conditions again.

[0058] For example, if the allocation unit 412 determines that there are some computing units among the selected corresponding computing units that have not yet met the predetermined detection conditions (e.g., CU7 among the selected computing units CU0 to CU7 has not yet met the predetermined detection conditions), the allocation unit 412 sends the relevant thread group of the current kernel function (e.g., sends the 28 thread groups of kernel function k0) to the at least some computing units (e.g., the selected computing units CU0 to CU6) that meet the predetermined detection conditions, while waiting for the some computing units (e.g., computing unit CU7) that have not yet met the predetermined detection conditions to meet the predetermined detection conditions again.

[0059] At step 308, the allocation unit distributes the corresponding thread group from the relevant thread group of the current kernel function to the partial calculation unit that has re-met the predetermined detection conditions.

[0060] For example, if the allocation unit 412 determines that a portion of the computing units (e.g., computing unit CU7) that did not meet the predetermined detection conditions at step 306 have met the predetermined detection conditions again, the allocation unit 412 sends the corresponding thread group in the relevant thread group of the current kernel function (e.g., sends the 4 thread groups of kernel function k0) to the portion of the computing units (e.g., computing unit CU7) that have met the predetermined detection conditions again.

[0061] In step 310, after the allocation unit distributes the relevant thread group of the current kernel function to at least some of the computing units that meet the predetermined detection conditions, the allocation unit selects at least some of the available computing units to execute the next kernel function based on a polling mechanism.

[0062] For example, while the allocation unit 412 distributes the relevant thread group of the current kernel function (e.g., distributes 28 thread groups of kernel function k0) to at least some of the computing units (e.g., the selected computing units CU0 to CU6) that meet the predetermined detection conditions, the allocation unit 412 selects at least some of the available computing units that meet the predetermined detection conditions based on a polling mechanism to continue executing the next kernel function.

[0063] The method for the allocation unit to select available computing units that meet predetermined detection conditions to continue executing the next kernel function includes, for example, the allocation unit selecting at least a portion of available computing units that meet predetermined detection conditions to continue executing the next kernel function based on a polling mechanism. It should be understood that the allocation unit may also select the corresponding computing units identified in the configuration information that meet the predetermined detection conditions to continue executing the next kernel function based on the computing unit description information about the next kernel function in the configuration information (e.g., k1 cu_mask=0xf00).

[0064] For example, if the configuration information only includes the computation unit description information for the current kernel function (e.g., kernel function k0) (e.g., cu_mask=0xff), the computation units selected by allocation unit 412 for executing kernel function k0 are CU0 to CU7. After allocation unit 412 distributes the 32 thread groups of kernel function k0 to the computation units CU0 to CU7 that meet the predetermined detection conditions, allocation unit 412 selects at least a portion of the available computation units CU8 to CU15 that meet the predetermined detection conditions (e.g., computation units CU8 to CU11) based on a polling mechanism to continue executing the next kernel function (e.g., kernel function k1). For example, allocation unit 412 allocates the 4 thread groups of kernel function k1 to computation units CU8 to CU11. It should be understood that when selecting a computation unit for executing kernel function k1, if computation units CU0 to CU7 for executing kernel function k0 have not yet been released, allocation unit 412 may select computation units CU8 to CU11 to execute kernel function k1; if at least some of the computation units CU0 to CU7 for executing kernel function k0 have been released, allocation unit 412 may also select at least some of the released computation units to execute kernel function k1, rather than being limited to selecting at least some computation units that meet the predetermined detection conditions from the remaining computation units CU8 to CU15.

[0065] In the above scheme, the allocation unit determines whether the selected computing unit has met the predetermined detection conditions, and distributes the relevant thread group of the current kernel function to at least some of the computing units that meet the predetermined detection conditions. It waits for the computing units that have not yet met the predetermined detection conditions to meet the predetermined detection conditions again before distributing the relevant thread group of the current kernel function. It also selects at least some of the available computing units that meet the predetermined detection conditions to continue executing the next kernel function. This invention further shortens the allocation time of the relevant thread group of multiple kernel functions to be executed, thereby optimizing the resource utilization of multiple kernel functions using the same device.

[0066] like Figure 4As shown, the artificial intelligence chip 400 includes, for example, a command processor 402 and a computing core 406, wherein the command processor 402 includes, for example, a task issuing unit 404, and the computing core 406 includes, for example, a receiving unit 408, a processing unit 410, an allocation unit 412, and a computing unit 414.

[0067] Regarding the command processor 402, it is configured to configure configuration information regarding computing units, which is distributed to allocation units within the computing core along with task data for kernel functions. It should be understood that in some embodiments, the configuration information regarding computing units may also be user-configurable. The command processor 402 includes at least a task distribution unit 404, configured to distribute tasks related to kernel functions to computing core 406. Specifically, as... Figure 4 As shown, the command processor 402 is configured, for example, to send the task data of the kernel function and the configuration information about the computing unit to the receiving unit 408 of the computing core 406 via the task sending unit 404, such as... Figure 4 As shown in the middle label 416.

[0068] Regarding computational core 406, it includes at least an allocation unit 412 and a computational unit 414. The allocation unit 412 is configured to determine the number of computational units used by the kernel function based on received configuration information regarding the computational units, or to determine the computational units used to execute related tasks of the kernel function. The computational unit 414 includes a plurality of computational units (CUs), at least a portion of which are configured to execute related tasks of the kernel function.

[0069] Regarding the receiving unit 408, it is configured to receive tasks related to kernel functions issued by the task issuing unit 404 of the command processor 402.

[0070] Regarding the processing unit 410, it is at least configured to receive task data and configuration information (such as kernel function data) sent from the receiving unit 408. Figure 4 (as shown in label 418), and provides the relevant thread group and configuration information of the current kernel function to the allocation unit 412 (such as...). Figure 4 (As shown in the middle mark 420).

[0071] In some embodiments, the allocation unit 412 is further configured to: in response to determining that the configuration information indicates the maximum number of computing units used by the kernel function, select computing units for executing the relevant tasks of the kernel function based on the configuration information, such that the number of selected computing units is less than or equal to the maximum number, and allocate the relevant thread group of the current kernel function to the selected computing units, as indicated by label 422.

[0072] In some embodiments, the allocation unit 412 is further configured to: in response to determining that configuration information indicates a computing unit for executing the kernel function, select a corresponding computing unit based on the configuration information to execute the kernel function; and allocate the relevant thread group of the current kernel function to the selected computing unit, as indicated by label 422.

[0073] Regarding computing unit 414, it includes at least a plurality of compute units (CUs). In some embodiments, each CU has a unique identifier so that allocation unit 412 can determine the computing unit that performs the relevant task of the kernel function based on the identifier of the CU indicated by the status bit included in the second descriptor in the configuration information; and allocate the relevant thread group of the current kernel function to the determined computing unit.

[0074] In the above scheme, by configuring the command processor of the artificial intelligence chip with configuration information about the computing units, and by sending the configuration information to the allocation unit in the computing core along with the task data of the kernel function; and by configuring the allocation unit of the computing core to determine the number of computing units used by the kernel function based on the received configuration information about the computing units, or to determine the computing units used to execute the related tasks of the kernel function; the present invention allows the computing units occupied by the kernel function to be flexibly configured on demand, and thus, when multiple kernel functions are started simultaneously on the same device, the problem of subsequent kernel functions having no computing resources available or waiting time being too long will not occur. Therefore, the present invention can significantly improve the resource utilization rate of the hardware computing units of the artificial intelligence chip.

[0075] The following will combine Figure 4 and Figure 5 The present invention describes a method 500 for selecting a computing unit, according to some embodiments thereof. It should be understood that method 500 may, for example, be implemented in... Figure 1 The described computing device 100 performs the operation. Method 500 may also include additional actions not shown and / or the actions shown may be omitted; the scope of the invention is not limited in this respect.

[0076] At step 502, the computing device 100 responds to the allocation unit receiving the configuration information about the computing unit.

[0077] The configuration information for the computing units includes, for example, descriptions of the computing units for multiple kernel functions to be executed by the computing core. Each computing unit description includes, for example, a first descriptor or a second descriptor for the corresponding kernel function. The first descriptor indicates the maximum number of computing units used by the corresponding kernel function, and the second descriptor indicates the corresponding computing units used to execute the relevant tasks of the corresponding kernel function. For example, the computing unit configuration information issued following the task data of kernel function k0 includes: cu_mask=0xff, and the configuration information issued following the task data of kernel function k1 includes: format cu_mask=0xf00. It should be understood that by configuring the computing unit description information (first descriptor or second descriptor) for multiple kernel functions simultaneously in the configuration information, the present invention allows multiple kernel functions to use the computing core in parallel and flexibly configures the computing units occupied by the computing core.

[0078] In some embodiments, the proportion of computing units used by the first kernel function and the second kernel function can be precisely controlled by the maximum number of computing units used by the first kernel function as indicated by the first descriptor of the first kernel function, and the maximum number of computing units used by the second kernel function as indicated by the first descriptor of the second kernel function. For example, the maximum number of computing units used by the first kernel function is 80% of the total number of computing units, and the maximum number of computing units used by the second kernel function is 20% of the total number of computing units.

[0079] It should be understood that in some embodiments, a computational core may be a collection of multiple computational cores that can be used to execute multiple kernel functions, and the present invention does not limit the number of computational cores.

[0080] In some embodiments, the configuration information related to the computing unit is a user-configurable interface.

[0081] At step 504, in response to determining that the first descriptor included in the configuration information indicates the maximum number of computing units used by the current kernel function, the allocation unit selects computing units for performing the relevant tasks of the current kernel function based on the first descriptor, such that the number of selected computing units is less than or equal to the maximum number of computing units indicated by the first descriptor.

[0082] For example, the computing device 100 configures the first descriptor maxCuNum in the configuration information of kernel function k0 to 8 and configures the first descriptor maxCuNum in the configuration information of kernel function k1 to 4. As a result, the allocation unit 412 determines that the maximum number of computing units used to execute kernel function k0 is 8 and the maximum number of computing units used to execute kernel function k1 is 4.

[0083] The method by which the allocation unit determines the computational units used to execute the kernel function is, for example, based on a round-robin (rr) scheduling algorithm. For instance, allocation unit 412, based on the rr algorithm, sequentially allocates the corresponding thread group of the kernel function to each selected computational unit according to the order of the polling queue, wherein the number of selected computational units is equal to or less than the maximum number of computational units used by the kernel function indicated in the first descriptor.

[0084] For example, in response to determining that the configuration information includes a first descriptor indicating the maximum number of computing units used by the current kernel function, if the maximum number of computing units used indicated by the first descriptor is 4, the allocation unit 412 selects computing units (e.g., marking at least some of the computing units among CU0, CU1, CU2, and CU3) for performing the relevant tasks of the current kernel function based on the first descriptor, such that the number of selected computing units is less than or equal to the maximum number of computing units indicated by the first descriptor.

[0085] At step 506, in response to determining that the second descriptor included in the configuration information indicates a computing unit for executing the relevant task of the current kernel function, the allocation unit selects the corresponding computing unit for executing the current kernel function based on the second descriptor.

[0086] Regarding the second descriptor, it indicates, for example, the corresponding computational unit (e.g., configured by the parameter cu_mask) for performing the relevant task of the kernel function. In some embodiments, the second descriptor indicates the corresponding computational unit for performing the relevant task of the kernel function in hexadecimal form. The generation process of the second descriptor includes, for example, converting the binary number indicating the selection state into a corresponding hexadecimal number in order from least significant bit to most significant bit to indicate the corresponding computational unit for performing the relevant task of the kernel function.

[0087] For example, the configuration information of kernel function k0 contains the second descriptor cu_mask=0xff, and the configuration information of kernel function k1 contains cu_mask2=0xf00. This configuration information indicates that the computing unit 414 for executing kernel function k0 consists of 8 CUs (CU0 to CU7), and the computing unit 414 for executing kernel function k1 consists of 4 CUs (CU8 to CU11). In some embodiments, the computing resources in each computing unit can run four thread groups of kernel function k0 or one thread group of kernel function k1. The allocation unit then selects 8 CUs (CU0 to CU7) for executing kernel function k0 and 4 CUs (CU8 to CU11) for executing kernel function k1 based on the second descriptor.

[0088] As mentioned above, each bit of the binary number corresponding to the hexadecimal status bit of the second descriptor corresponds one-to-one with a computational unit. For example, the least significant bit in the binary number corresponds to the least significant bit identified in the computational unit, and these correspondences increment sequentially. For instance, when using the binary number 101101 to indicate the selection status of a computational unit, CU0 corresponds to the least significant bit (1), CU1 to the second bit (0), CU2 to the third bit (1), CU3 to the fourth bit (1), CU4 to the fifth bit (0), and CU5 to the sixth bit (1). Thus, the selection status of the computational unit is indicated through the binary number. In the corresponding binary number, the value of each bit represents the selection status of each computational unit. For example, "1" indicates that the current computational unit is selected to execute the kernel function, and "0" indicates that the current computational unit is not selected to execute the kernel function.

[0089] In the above scheme, the allocation unit in the kernel function is based on the number of computing units used by the kernel function indicated by the first descriptor included in the configuration information or the computing units used to execute the related tasks of the kernel function indicated by the second descriptor. The allocation unit determines the computing units assigned to the corresponding thread group of the kernel function. Therefore, the present invention can achieve precise allocation and control of computing units in a computing kernel that runs multiple kernel functions simultaneously.

[0090] The various processes and procedures described above, such as methods 200, 300, and 500, can be executed at a computing device. This computing device may include, for example, at least one processor (at least one graphics processor and at least one central processing unit); and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor. In some embodiments, methods 200, 300, and 500 may be implemented as computer software programs or program products tangibly contained in a machine-readable medium. In some embodiments, part or all of the computer program may be loaded and / or installed on the computing device via read-only memory (ROM) and / or a communication unit. When the computer program is loaded into random-access memory (RAM) and executed by the GPU and CPU, one or more actions of methods 200, 300, and 500 described above can be performed.

[0091] This invention can be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention. The computer-readable storage medium may be a tangible device capable of holding and storing instructions used by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof.

[0092] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network, to an external computer or external storage device. Various aspects of the invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0093] These computer-readable program instructions can be provided to the central processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the central processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0094] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0095] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps loaded in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0096] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors.

Claims

1. A method for issuing kernel functions, characterized in that, The method includes: Configure configuration information about computing units, and cause the configuration information to be sent to the allocation units in the computing core along with the task data of the kernel function. The configuration information includes computing unit description information for the kernel function, and the kernel function is at least one of a plurality of kernel functions to be executed by the computing core. The configuration information about the computing units includes a first descriptor or a second descriptor, the first descriptor indicating the maximum number of computing units used by the kernel function, and the second descriptor indicating the corresponding computing units used to execute the relevant tasks of the kernel function. In response to the allocation unit receiving the configuration information about the computing unit, it performs any of the following based on the configuration information: Determine the number of computational units used by the kernel function; and Determine the computational units used to perform the relevant tasks of the kernel function.

2. The method according to claim 1, characterized in that, Also includes: In response to determining that the first descriptor included in the configuration information indicates the maximum number of computing units used by the current kernel function, the allocation unit selects computing units for performing the relevant tasks of the current kernel function based on the first descriptor, such that the number of selected computing units is less than or equal to the maximum number of computing units indicated by the first descriptor.

3. The method according to claim 1, characterized in that, Also includes: In response to the determination that the second descriptor included in the configuration information indicates a computing unit for executing the relevant task of the current kernel function, the allocation unit selects the corresponding computing unit for executing the current kernel function based on the second descriptor; as well as Distribute the relevant thread groups of the current kernel function to the selected corresponding computing unit.

4. The method according to claim 3, characterized in that, Sending the relevant thread group of the current kernel function to the selected corresponding computation unit includes: The allocation unit confirms whether all selected corresponding computing units have met the predetermined detection conditions; as well as In response to the confirmation that the selected corresponding computing units have met the predetermined detection conditions, the allocation unit sends the relevant thread groups of the current kernel function to the computing units that meet the predetermined detection conditions.

5. The method according to claim 4, characterized in that, Sending the relevant thread group of the current kernel function to the selected corresponding computation unit also includes: In response to the confirmation that some of the selected computing units have not yet met the predetermined detection conditions, the allocation unit sends the relevant thread group of the current kernel function to the computing units that have met the predetermined detection conditions, while waiting for the portion of computing units that have not yet met the predetermined detection conditions to meet the predetermined detection conditions again. as well as The allocation unit distributes the corresponding thread group from the relevant thread group of the current kernel function to the partial computing unit that has re-met the predetermined detection conditions.

6. The method according to claim 1, characterized in that, Also includes: The allocation unit selects at least a portion of the available computing units to execute the next kernel function based on a polling mechanism. or Based on the computational unit description information for the next kernel function included in the configuration information, the allocation unit selects at least a portion of the available computational units to execute the next kernel function.

7. An artificial intelligence chip, characterized in that, The artificial intelligence chip includes: A command processor is configured to configure information about computing units. This configuration information is sent to allocation units within the computing core along with task data for kernel functions. The configuration information includes computing unit description information for the kernel functions, where the kernel function is at least one of a plurality of kernel functions to be executed by the computing core. The configuration information about the computing units includes a first descriptor or a second descriptor, where the first descriptor indicates the maximum number of computing units used by the kernel function, and the second descriptor indicates the corresponding computing units used to execute the relevant tasks of the kernel function. The computational kernel includes: An allocation unit, configured to determine, based on received configuration information about computing units, the number of computing units used by a kernel function, or to determine the computing units used to perform related tasks of the kernel function; and Multiple computing units, at least some of which are configured to perform related tasks of the kernel function.

8. The artificial intelligence chip according to claim 7, characterized in that, The allocation unit is further configured to: In response to the determination that the configuration information indicates the maximum number of computational units used by the kernel function, computational units for executing the relevant tasks of the kernel function are selected based on the configuration information, such that the number of selected computational units is less than or equal to the maximum number. or In response to determining that the configuration information indicates the computing unit for executing the kernel function, the corresponding computing unit is selected based on the configuration information to execute the kernel function.

9. A computing device, characterized in that, include: At least one processor; as well as A memory that is communicatively connected to the at least one processor; in The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a machine, performs the method according to any one of claims 1-6.

11. A computer program product, characterized in that, Includes a computer program, which, when executed by a machine, performs the method according to any one of claims 1-6.

Images