A task computing method and device

Abstract

The present application relates to chip technical field, provide a kind of task computing method and device.Method includes: resource scheduling module itself executes first sub-computing task, and second sub-computing task is issued to computing module, so that computing module executes the second sub-computing task;Wherein, the parallelism of first sub-computing task is lower than the parallelism of second sub-computing task.The present application is calculated by the sub-computing task of high parallelism, and the sub-computing task of low parallelism is calculated by PRA itself, so that the sub-computing task of low parallelism does not need to occupy UCU, so as to make full use of the computing capacity of UCU and PRA, improve the computing performance of overall computing task group.

Description

TECHNICAL FIELD

[0001] The present application relates to the chip technical field, in particular to a task computing method and device. BACKGROUND

[0002] When the graphics processor (Graphics Processing Unit, GPU for short) executes a computing task, a computer task allocation (CTA for short) module reads the computing task issued by a host CPU, the CTA parses the corresponding task to obtain a task group (workgroup for short) to be processed, and issues the task item (workitem for short) to be processed by each workgroup.

[0003] A program resource allocation (PRA for short) module receives the workitem group composed of the CTA to calculate a unified computer unit (UCU for short). The PRA forms different numbers of warps according to the parallelism of the UCU, for example, the parallelism of the UCU is 32, the CTA issues 34 workitems, and the PRA forms two warps. warp = ceil(number / 32).

[0004] Meanwhile, the PRA configures registers and other resources for the UCU, and pre-fetches some data for subsequent calculation of the UCU. Note that in order to make some unified scheduling for the UCU, the PRA has some calculation logic (such as addition, subtraction, multiplication, division, bit calculation, load / store, etc.). After the corresponding register resources and data are ready, the PRA schedules the UCU to start calculation, and after the UCU finishes calculation, the PRA recycles the corresponding register resources. Some tasks with low parallelism cannot reasonably use the computing power of the GPU.

[0005] Therefore, it is urgent to overcome the defects of the prior art in the technical field. SUMMARY

[0006] The technical problem solved by the present application is to provide a task computing method and device to improve computing performance.

[0007] The present application adopts the following technical solutions:

[0008] In a first aspect, the present application provides a task computing method, which comprises:

[0009] The resource scheduling module itself performs the first sub-computing task, and the second sub-computing task is issued to the computing module, so that the computing module performs the second sub-computing task.

[0010] The parallelism of the first sub-computing task is lower than the parallelism of the second sub-computing task.

[0011] Preferably, the resource scheduling module itself performs the first sub-computing task, and specifically includes:

[0012] A data transmission path is established between the resource scheduling module and the cache area, and the resource scheduling module uses the data transmission path to obtain data required for performing the first sub-computing task from the cache area, so as to perform the first sub-computing task by using the data and the instruction of the first sub-computing task.

[0013] Preferably, the cache area includes a three-level cache connected with a display memory, a one-level cache connected with an external storage, and a two-level cache connected between the one-level cache and the three-level cache; and the data transmission path is established between the resource scheduling module and the cache area, and specifically includes:

[0014] A data transmission path is established between the resource scheduling module and the two-level cache, and / or a data transmission path is established between the resource scheduling module and the three-level cache; and a data transmission path is established between the one-level cache and the computing module.

[0015] Preferably, the first sub-computing task and the second sub-computing task are distinguished by a task issuing module and issued to the resource scheduling module, and specifically include:

[0016] The task issuing module splits the computing task group into a plurality of sub-computing tasks, generates a task identifier according to the parallelism of each sub-computing task, and issues each sub-computing task and the corresponding task identifier to the resource scheduling module, so that the resource scheduling module identifies the first sub-computing task and the second sub-computing task according to the task identifier;

[0017] The task identifier includes a first task identifier for representing the first sub-computing task and a second task identifier for representing the second sub-computing task.

[0018] Preferably, the method further includes:

[0019] The task issuing module further issues a dependency relationship between the sub-computing tasks to the resource scheduling module.

[0020] The resource scheduling module performs scheduling of the sub-computing tasks according to the dependency relationship.

[0021] Preferably, the resource scheduling module performs scheduling of the sub-computing tasks according to the dependency relationship, and specifically includes:

[0022] According to the dependency relationship, a dependency table item is generated; wherein the dependency table item comprises task items on which each sub-computing task depends and execution conditions of each task item;

[0023] After each sub-computing task is executed, the dependency table item is updated;

[0024] Every interval of a preset period, a third sub-computing task and a fourth sub-computing task are found from the dependency table item, the third sub-computing task is executed, and the fourth sub-computing task is issued to a computing module;

[0025] The third sub-computing task is a first sub-computing task whose dependent task items have all been completed, and the fourth sub-computing task is a second sub-computing task whose dependent task items have all been completed.

[0026] Preferably, the execution condition is represented by a state identifier of a task item; the dependency table item further comprises a waiting identifier of each sub-computing task; and after each sub-computing task is executed, the dependency table item is updated, specifically comprising:

[0027] After the corresponding sub-computing task is executed, the state identifier of the task item representing the sub-computing task is updated to completed;

[0028] The waiting identifier of the sub-computing task whose dependent task items have all been completed is updated to no need to wait, so that the resource scheduling module identifies the sub-computing task as the third sub-computing task or the fourth sub-computing task when the waiting identifier of the corresponding sub-computing task is identified as no need to wait.

[0029] Preferably, the execution condition of the second sub-computing task is returned by the computing module to the resource scheduling module, specifically comprising:

[0030] After the second sub-computing task is executed, the computing module sends the second sub-computing task and a corresponding execution completion identifier to the resource scheduling module together, so that the resource scheduling module identifies the execution condition of the second sub-computing task as execution completion.

[0031] In a second aspect, the application provides a task computing device, comprising a task issuing module, a resource scheduling module and a computing module;

[0032] The task issuing module is configured to issue a first sub-computing task and a second sub-computing task to the resource scheduling module;

[0033] The resource scheduling module is configured to execute the first sub-computing task by itself, and issue the second sub-computing task to the computing module;

[0034] The computing module is configured to execute the second sub-computing task; wherein the parallelism of the first sub-computing task is lower than the parallelism of the second sub-computing task.

[0035] Preferably, the device further comprises a memory.

[0036] The memory is provided with a cache area, and a data transmission path is established between the resource scheduling module and the cache area.

[0037] The resource scheduling module uses the data transmission path to obtain data required for executing the first sub-computing task from the cache area, so as to execute the first sub-computing task using the data and the instructions of the first sub-computing task.

[0038] In a third aspect, the present application further provides a non-volatile computer storage medium, which stores computer executable instructions, and the computer executable instructions are executed by one or more processors to complete the method in the first aspect.

[0039] In a fourth aspect, a chip is provided, which comprises a processor and an interface, and is configured to call and run a computer program stored in a memory, and execute the method in the first aspect.

[0040] In a fifth aspect, a computer program product is provided, which comprises instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes the method in the first aspect.

[0041] The present application can make the sub-computing task with low parallelism not occupy the UCU, so as to fully utilize the computing capacity of the UCU and the PRA, and improve the computing performance of the whole computing task group. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments of the present application will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0043] Figure 1 is a schematic diagram of a task computing device in the prior art provided by the embodiments of the present application;

[0044] Figure 2 is a schematic diagram of a task computing method in the prior art provided by the embodiments of the present application;

[0045] Figure 3is a schematic diagram of an architecture of a task computing device in the prior art provided by an embodiment of the present application;

[0046] Figure 4 is a schematic diagram of a first task computing method provided by an embodiment of the present application;

[0047] Figure 5 is a schematic diagram of an architecture of a first task computing device provided by an embodiment of the present application;

[0048] Figure 6 is a flowchart of a first task computing method provided by an embodiment of the present application;

[0049] Figure 7 is a flowchart of a second task computing method provided by an embodiment of the present application;

[0050] Figure 8 is a flowchart of a third task computing method provided by an embodiment of the present application;

[0051] Figure 9 is a flowchart of a fourth task computing method provided by an embodiment of the present application;

[0052] Figure 10 is a schematic diagram of a second task computing method provided by an embodiment of the present application;

[0053] Figure 11 is a schematic diagram of a third task computing method provided by an embodiment of the present application;

[0054] Figure 12 is a schematic diagram of a second task computing method provided by an embodiment of the present application;

[0055] Figure 13 is a schematic diagram of a third task computing device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0057] Unless otherwise required by context, as used herein the term "comprising" is to be interpreted as meaning "including but not limited to". In describing the embodiments of the disclosure, the terms "one embodiment", "some embodiments", "an exemplary embodiment", "an example", "a certain example" or "some examples" are intended to mean that a certain feature, structure, material, or characteristic is included in at least one embodiment or example of the present disclosure. These terms are not necessarily intended to refer to the same embodiment or example. Furthermore, these terms are not necessarily intended to refer to an exclusive "or" relationship between features, structures, materials, or characteristics that such terms describe, although they can for example be present in one embodiment or example in a "one of" relationship or in an "exclusive or" relationship. It is also noted that the terms "comprise", "comprising", "comprises", "include", "including", "includes", "contain", "containing", "has", "having", "may have", "may have", "may have" and "may have" are not necessarily intended to be construed as security features, structures, materials, or characteristics of the present disclosure.

[0058] In the description of the present disclosure, the terms "first", "second", etc. are used only for the purpose of description, and are not to be interpreted or implied as indicating or implying relative importance or implying the number of technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise stated, the meaning of "a plurality of" is two or more. In addition, for example, in the description, the same type of nouns can also be described as two independent individuals by adding "A", "B" at the end, in which case the features defined with "A", "B" are only used for the purpose of distinguishing the same type of individual description, and cannot be interpreted or implied as indicating or implying relative importance or implying the number of technical features indicated.

[0059] In describing some embodiments, "coupled", "coupling" and "connected" and their derivatives can be used. For example, the term "connected" can be used in describing some embodiments to indicate that two or more components are in direct physical or electrical contact with each other. For another example, the term "coupled" can be used in describing some embodiments to indicate that two or more components are in direct physical or electrical contact with each other. However, the term "connected" or "coupled" can also mean that two or more components are not in direct contact with each other, but still cooperate or interact with each other, such as "optical coupling", "wireless connection", etc. The embodiments disclosed herein are not necessarily limited to the content of the present disclosure.

[0060] In the description of the present disclosure, the expression "A and / or B" (where A and B are used to represent specific feature content) includes the following three combinations: only A, only B, and a combination of A and B.

[0061] As used herein, "about," "approximately," or "around" in reference to a value includes the recited value and the average value within an acceptable range of deviation from the recited value, as determined by one of ordinary skill in the art taking into account the measurement being discussed and the error associated with measuring a particular quantity (i.e., the limitations of the measurement system).

[0062] Furthermore, the technical features involved in each of the embodiments of the application described below can be combined with each other as long as there is no conflict.

[0063] In the prior art, due to the multi-core characteristics, GPUs are often suitable for performing high-parallelism computing tasks, but due to the limited resources of each operation core, the execution performance of GPUs is not superior for low-parallelism computing tasks, and when a small number of operation cores are used to execute low-parallelism computing tasks, the remaining idle operation cores still need to wait until a computing task is completed before the subsequent task can be performed. With the development of artificial intelligence technology, some models also often use GPUs for computing. The internal structure of a GPU is shown in the task computing device in Figure 1 , and the computing task is mainly issued by the CTA to the PRA, and then dispatched by the PRA to each UCU for parallel computing (i.e., executing the task). In addition to high-parallelism sub-computing tasks, large language models also have some low-parallelism sub-computing tasks, such as calculating the mean and variance of 1024 numbers, taking the maximum value, etc. In the prior art, these low-parallelism operations are also operated using UCU, i.e., as shown in Figure 2 , the PRA issues a computing task group (i.e., kick task, represented as program in Figure 2 ) and related information to the UCU, and the UCU performs the operation, and then waits for the UCU to feed back a done signal to the PRA, and then the PRA feeds back a done signal to the CTA. The PRA only performs resource allocation and address calculation of the UCU program, which results in a long time-consuming for task issuing and low overall UCU utilization. In order to solve this problem, the embodiments of the present application provide a task computing method, as shown in Figure 3 and Figure 4 , the method comprises:

[0064] In step 201, the resource scheduling module itself executes a first sub-computing task. The resource scheduling module is also called PRA or resource scheduler.

[0065] In step 202, a second sub-computing task is issued to a computing module, so that the computing module executes the second sub-computing task; the computing module is also called UCU or unified computing unit.

[0066] The parallelism of the first sub-computing task is lower than the parallelism of the second sub-computing task, that is, the first sub-computing task is a sub-computing task with low parallelism, and the second sub-computing task is a sub-computing task with high parallelism. The sub-computing task can be understood as part of the operation of a large computing task group that needs to be operated, that is, a computing task group is divided into multiple sub-computing tasks.

[0067] It should be noted that the low parallelism and the high parallelism can be obtained by first calculating the parallelism and then comparing the parallelism with a preset value, for example, the parallelism lower than the preset value is regarded as low parallelism, and the parallelism not lower than the preset value is regarded as high parallelism. The preset value is obtained by experience analysis by a person skilled in the art. In actual use, a person skilled in the art can also pre-classify various operations, for example, define basic mathematical operations such as addition, subtraction, multiplication and division between two numerical values as low parallelism, and define matrix operations with a dimension higher than a preset dimension as high parallelism, so as to classify the sub-computing tasks according to the operations required by the sub-computing tasks into the first sub-computing task or the second sub-computing task.

[0068] Since the PRA itself has computing resources and computing capability, in the embodiment, the sub-computing task with high parallelism is calculated by the UCU, and the sub-computing task with low parallelism is operated by the PRA itself, so that the sub-computing task with low parallelism does not need to occupy the UCU, thereby fully utilizing the computing capability of the UCU and the PRA and improving the computing performance of the whole computing task group.

[0069] In actual use, the sub-computing task execution needs to be implemented in cooperation with instructions and data. The instructions are often issued in the sub-computing task, and the data needs to be obtained by the module itself from the cache, that is, the resource scheduling module itself executes the first sub-computing task, specifically including:

[0070] A data transmission path is established between the resource scheduling module and the cache area. The resource scheduling module uses the data transmission path to obtain the data required for executing the first sub-computing task from the cache area, so as to execute the first sub-computing task by using the data and the instructions of the first sub-computing task. The data transmission path is pre-established. An optional implementation manner is to connect the resource scheduling module and the memory where the cache area is located, so as to form the data transmission path.

[0071] In a specific application scenario, as shown in Figure 5 The cache area includes a three-level cache connected with a video memory, a one-level cache connected with an external storage, and a two-level cache connected between the one-level cache and the three-level cache. The data transmission path is established between the resource scheduling module and the cache area, specifically including that the one-level cache is represented as a cache L0 in Figure 5 , and the two-level cache is represented as a cache L1 inFigure 5 The third level cache is represented as cache L3 in the figure. Figure 5 The second level cache is represented as cache L2 in the figure.

[0072] A data transmission path is established between the resource scheduling module and the second level cache, and / or a data transmission path is established between the resource scheduling module and the third level cache; wherein a data transmission path is established between the first level cache and the computing module. It can be understood that the first level cache is a cache for direct use by the UCU, and the storage location of the data therein can change following the computing process of the UCU, such as reordering, and the second level cache and the third level cache can be understood as caches not directly used by the UCU, and therefore the location of the data stored therein is relatively fixed, thereby facilitating the PRA to obtain and use.

[0073] In actual use, the first sub-computing task and the second sub-computing task are distinguished by the task issuing module and issued to the resource scheduling module, as shown in Figure 6 The method comprises the following steps:

[0074] In step 301, the task issuing module splits the computing task group into multiple sub-computing tasks, and generates a task identifier according to the parallelism of each sub-computing task. The task issuing module is also called CTA or computing task issuing module.

[0075] In step 302, each sub-computing task and the corresponding task identifier are jointly issued to the resource scheduling module, so that the resource scheduling module can identify the first sub-computing task and the second sub-computing task according to the task identifier.

[0076] The task identifier includes a first task identifier for representing the first sub-computing task and a second task identifier for representing the second sub-computing task, the first task identifier is generated for a sub-computing task with low parallelism, and the second task identifier is generated for a sub-computing task with high parallelism.

[0077] In actual use, the task identifier can be represented as one bit of data, using b'00 to represent the first task identifier and using b'01 to represent the second task identifier.

[0078] In actual use, each sub-computing task is not independent, but has a certain dependency relationship, such as This computing process is actually implemented by splitting into three sub-computing tasks, which are: task 1: multiplying matrix Q and matrix K to store as matrix X; task 2: performing a softmax operation on matrix X to store the result as vector Y; and task 3: completing the multiplication of vector Y and matrix V to store the result as Z. Task 3 depends on task 2, and task 2 depends on task 1. In order to cope with this scenario, the embodiment also provides a preferred implementation, as shown inFigure 7 As shown in the method of the embodiment, the method further comprises:

[0079] In step 401, the task issuing module also issues the dependency relationship between the sub-computing tasks to the resource scheduling module.

[0080] In step 402, the resource scheduling module schedules the sub-computing tasks according to the dependency relationship. The scheduling of the sub-computing tasks can be determining the execution order of the sub-computing tasks. For the second sub-computing task, the order of being issued to the computing module can be regarded as the execution order, that is, the second sub-computing task can be considered to be executed immediately after being issued to the computing module. The sub-computing tasks that do not have a dependency relationship with each other can be executed in parallel.

[0081] The embodiment enables the resource scheduling module to autonomously control the scheduling of the sub-computing tasks by issuing the dependency relationship to the resource scheduling module, so that the resource scheduling module and the computing module can execute the corresponding sub-computing tasks in parallel, thereby further improving the computing efficiency of the overall computing task group.

[0082] In a specific application scenario, the resource scheduling module schedules the sub-computing tasks according to the dependency relationship, such as Figure 8 As shown in the method of the embodiment, the method further comprises:

[0083] In step 501, a dependency table item is generated according to the dependency relationship. The dependency table item includes the task item on which each sub-computing task depends and the execution status of each task item.

[0084] In step 502, the dependency table item is updated after each sub-computing task is executed, that is, the execution status of the task item corresponding to each sub-computing task is updated. The task item is only used to distinguish the sub-computing task of the dependent party from the sub-computing task of the depended party, and its essence is still a sub-computing task. For example, if there are multiple sub-computing tasks, which are sub-computing task A, sub-computing task B and sub-computing task C, and the sub-computing task A depends on the sub-computing task C, then in this dependency relationship, the sub-computing task C is called task item C, but its essence still represents the sub-computing task C, and the execution status of the task item C is the execution status of the sub-computing task C. Similarly, if the sub-computing task B depends on the sub-computing task A, then in this dependency relationship, the sub-computing task A is called task item A, but its essence still represents the sub-computing task A, and the execution status of the task item A is the execution status of the sub-computing task A.

[0085] In step 503, every interval of a preset period, find the third sub-computing task and the fourth sub-computing task from the dependency table item, execute the third sub-computing task, and issue the fourth sub-computing task to the computing module; the preset period is obtained by the person skilled in the art according to experience analysis, such as a cycle of a timer as a preset period.

[0086] Wherein, the third sub-computing task is the first sub-computing task whose dependent task items have all been completed, and the fourth sub-computing task is the second sub-computing task whose dependent task items have all been completed. In actual use, there is also a corresponding order for each sub-computing task. When there are multiple third sub-computing tasks, the third sub-computing task at the front of the order is executed first, and when the third sub-computing task is completed and the next preset period position is reached, the third sub-computing task at the front is found again for execution. Similarly, when there are multiple fourth sub-computing tasks, the fourth sub-computing task at the front of the order is issued first, and when the fourth sub-computing task is completed and the next preset period position is reached, the fourth sub-computing task at the front is found again for issuance.

[0087] In actual application scenarios, the execution of each sub-computing task is also stored in the dependency table item, and when the corresponding sub-computing task is executed, it is not involved in the subsequent search for the third sub-computing task and the fourth sub-computing task.

[0088] In a specific application scenario, the execution is represented by the state identifier of the task item; the dependency table item also includes the waiting identifier of each sub-computing task; and the dependency table item is updated after the execution of each sub-computing task, such as Figure 9 As shown, specifically including:

[0089] In step 601, after the execution of the corresponding sub-computing task, the state identifier of the task item representing the sub-computing task is updated to completed.

[0090] In step 602, the waiting identifier of the sub-computing task whose dependent task items have all been completed is updated to no need to wait, so that the resource scheduling module identifies the sub-computing task as the third sub-computing task or the fourth sub-computing task when identifying that the waiting identifier of the corresponding sub-computing task is no need to wait.

[0091] When the state identifiers of each task item dependent on the corresponding sub-computing task are all completed, the sub-computing task is regarded as the sub-computing task whose dependent task items have all been completed.

[0092] Among them, the first sub-computing task is executed by the resource scheduling module itself, so its execution condition can be directly obtained, and the execution condition of the second sub-computing task is returned to the resource scheduling module by the computing module, specifically including: after the second sub-computing task is executed, the computing module sends the second sub-computing task and the corresponding execution completion identifier to the resource scheduling module, so that the resource scheduling module identifies the execution condition of the second sub-computing task as execution completion.

[0093] It should be noted that the parallelism of each sub-computing task and the dependency relationship between each sub-computing task in the embodiment can be considered as known, or can be obtained by analyzing the task issuing module, and the specific implementation is known to those skilled in the art, which will not be described here.

[0094] Based on the method described in the above embodiment, the specific application scene is combined, and the implementation process in the specific scene of the application is described by the technical description in the specific scene.

[0095] In the prior art, the CTA reads the computing task issued by the host CPU, the CTA analyzes the corresponding task to obtain the workgroup to be processed, and simultaneously issues the workitem to be processed by each workgroup.

[0096] The PRA receives the workitem group composed of the warp to perform calculation by the UCU unified computing unit. Among them, the PRA will form different numbers of warps according to the parallelism of the unified computing unit, for example, if the parallelism of the unified computing unit is 32, and the CTA issues 34 workitems, the PRA will form two warps. warp = ceil(number / 32).

[0097] At the same time, the PRA configures registers and other resources for the unified computing unit, and pre-fetches some data for subsequent calculation of the unified computing unit. It should be noted that in order to make some unified scheduling for the unified computing unit, the PRA has some computing logic (addition, subtraction, multiplication, division, bit calculation, load / store). After the corresponding register resources and data are prepared, the PRA will schedule the unified computing unit to start calculation, and after the unified computing unit finishes calculation, the PRA will recycle the corresponding register resources.

[0098] As can be seen, in the prior art, the PRA only serves as a task issuing and resource allocation module, but the computing logic is not fully applied, the PRA only issues a kick task and related information to the UCU, and then waits for the UCU to feed back a done signal to the PRA, and then the PRA feeds back a done signal to the CTA. The PRA at most does resource allocation and address calculation of the UCU program.

[0099] Considering that PRA has its own computing capability, and there are some small kicks in large language models, such as mean and variance of 1024 numbers, maximum value operation, etc., using UCU to run will cause the task to be issued for a long time, and the overall UCU utilization is not high, that is, the task with low parallel degree cannot reasonably use the computing power of the GPU, and if it is completed using CPU, it will consume communication between CPU and GPU, thereby prolonging the computing time, which is unacceptable for the delay caused by the edge AI scene. In order to solve this problem and enhance the overall performance of the GPU, the embodiment proposes the above-mentioned task computing method, which specifically includes:

[0100] As shown in Figure 5 , first, a data connection path is established between PRA and cache L1; wherein PRA has its own code interval, and can also obtain its own data from cache L1, and can also obtain data from cache L1 and transmit it to UCU.

[0101] CTA splits one kick into multiple subkicks, some of which will be issued to UCU for operation, and some of which only need to complete the corresponding operation at the PRA level.

[0102] Multiple subkicks can be used as multiple instructions of PRA, so that the previous subkick is calculated in UCU, and the next subkick is calculated in PRA, so that PRA does not need to start a new task, and only needs to receive a done signal from UCU to PRA, and can call the thread in PRA for the following calculation, while the resources of UCU can be effectively shared to other computing tasks. Through this method, the computing resources of PRA and CacheL1 can be effectively utilized, and the resources of UCU can be effectively utilized in tasks with high parallel degree, thereby improving the running performance of the computing task and shortening the running time.

[0103] The following is an example of two practical application scenarios:

[0104] The first application scenario is to perform FastRCNN model calculation. In FastRCNN, AI models A and B before and after are suitable for GPU and AI chip execution (i.e. high parallel degree), but the middle is connected with an NLmeans layer (i.e. low parallel degree) which is suitable for execution on CPU. However, the process of transferring the task from one GPU to CPU and then transferring the calculation result to GPU will greatly affect the performance. In this scheme, the task is divided into three subKick groups to form a Kick, which is processed by UCU and PRA respectively, i.e. Figure 10As shown, model A and B are executed in UCU, and the NLmeans layer is executed in PRA, so as to realize the whole process on the GPU.

[0105] The second application scenario is a process of a large model For example, the calculation process is divided into three steps:

[0106] The first step is to complete the multiplication of matrix Q and matrix K, and the result is stored as matrix X, which is suitable for GPU completion.

[0107] The second step is to complete the softmax operation on matrix X, and the result is stored as vector Y, which is particularly inefficient in GPU implementation.

[0108] The third step is to complete the multiplication of vector Y and matrix V, and the result is stored as Z, which is suitable for GPU completion.

[0109] As Figure 11 shown, the first step and the third step in this embodiment use the UCU operation of the GPU; the second step uses the PRA calculation of the GPU.

[0110] This embodiment reduces system latency by deploying tasks with low parallelism on PRA for execution.

[0111] Among them, CTA will send a complete calculation task group (abbreviated as: tick) to PRA, and tick uses a buffer structure to represent it, which includes multiple sub-planning tasks (abbreviated as: subtick), and a field composed of 32 bits is used to indicate a subkick. Bit[31:30]=b’00 represents that the subkick is executed in PRA (i.e. the first sub-computing task); bit[31:30]=b’01 represents that the subkick is executed in UCU (i.e. the second sub-computing task); bit[31:30]=b’11 represents that the kick is executed and can send a done signal to CTA. PRA and UCU set the dependency relationship through bit[29-27]. Bit

[29] =1, set the dependency relationship; Bit

[28] =1, clear the dependency relationship; Bit

[27] =1, whether to wait for the dependency.

[0112] All dependencies are handled by PRA, and the dependencies are stored in the dependency table item. Whether there is a dependency is realized by looking up the table. Every interval cycle (i.e. the preset period), PRA will execute a dependency relationship clearing operation. Specifically: PRA will detect whether the computing unit of PRA or UCU returns the information that the dependency relationship is cleared (a bit signal, free_sync, i.e. execution completion identifier), so as to update the dependency relationship table item.

[0113] Furthermore, at each cycle interval, the PRA executes a task delivery operation. Specifically: First, the PRA checks whether the next task to be delivered requires waiting for dependencies. If there are no dependent tasks, and assuming the PRA's computing unit or UCU can receive the data, the task will be delivered. If the task has dependencies, the PRA queries the dependency table to see if the dependencies have been resolved. If the dependency has been resolved (i.e., the wait flag is "no waiting required"), the task is delivered; if the dependency has not been resolved (i.e., the wait flag is "wait required"), the task will be stuck in the PRA and will not be delivered.

[0114] Based on the above-described task calculation methods, this embodiment also provides a task calculation device, such as... Figure 12 As shown, it includes a task distribution module, a resource scheduling module, and a computing module. The task distribution module distributes a first sub-computation task and a second sub-computation task to the resource scheduling module. The resource scheduling module executes the first sub-computation task itself and distributes the second sub-computation task to the computing module. The computing module executes the second sub-computation task. The first sub-computation task is a low-parallelism sub-computation task, and the second sub-computation task is a high-parallelism sub-computation task.

[0115] The device also includes a memory; the memory is provided with a cache area, and a data transmission path is established between the resource scheduling module and the cache area; the resource scheduling module uses the data transmission path to obtain the data required to execute the first sub-computation task from the cache area, so as to use the data and the instructions of the first sub-computation task to execute the first sub-computation task.

[0116] like Figure 13 The diagram shown is a schematic representation of another architecture of the task computing device according to an embodiment of the present invention. The task computing device of this embodiment includes one or more processors 21 and a memory 22. Figure 13 Take a processor 21 as an example.

[0117] Processor 21 and memory 22 can be connected via a bus or other means. Figure 13 Taking the example of a connection between China and Israel via a bus.

[0118] The memory 22, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs and non-volatile computer-executable programs, such as the task calculation method in the above embodiment. The processor 21 executes the task calculation method by running the non-volatile software program and instructions stored in the memory 22.

[0119] The memory 22 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state memory device. In some embodiments, the memory 22 can optionally include a memory that is remotely located relative to the processor 21, and these remotely located memories can be connected to the processor 21 through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0120] The program instructions / modules are stored in the memory 22, and when executed by the one or more processors 21, perform the task calculation method in the above-mentioned embodiments.

[0121] It is worth noting that the information interaction, execution process, and the like between the modules and units in the above-mentioned apparatus and system are based on the same concept as the processing method embodiments of the present application, and the specific content can be referred to the description in the method embodiments of the present application, which will not be described here.

[0122] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the embodiments can be completed by a program instructing relevant hardware, and the program can be stored in a computer-readable storage medium, and the storage medium can include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and the like.

[0123] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A task calculation method, characterized in that, include: The resource scheduling module executes the first sub-computation task itself and sends the second sub-computation task to the computing module so that the computing module can execute the second sub-computation task. Among them, the parallelism of the first sub-computation task is lower than that of the second sub-computation task; A data transmission path is established between the resource scheduling module and the cache. The resource scheduling module uses the data transmission path to obtain the data required to execute the first sub-computation task from the cache, and then uses the data and the instructions of the first sub-computation task to execute the first sub-computation task. The cache includes a Level 3 cache connected to the video memory, a Level 1 cache connected to external storage, and a Level 2 cache connected between the Level 1 cache and the Level 3 cache. Establishing a data transmission path between the resource scheduling module and the cache includes: establishing a data transmission path between the resource scheduling module and the Level 2 cache, and / or establishing a data transmission path between the resource scheduling module and the Level 3 cache. A data transmission path is established between the Level 1 cache and the computation module. It also includes: the task distribution module distributes the dependencies between each sub-computation task to the resource scheduling module; the resource scheduling module schedules each sub-computation task according to the dependencies; the resource scheduling module handles all dependencies, the dependencies are stored in the dependency table, and the existence of dependencies is determined by looking up the table; every preset period, the resource scheduling module will perform a dependency clearing operation.

2. The task calculation method according to claim 1, characterized in that, The first and second sub-computation tasks are distinguished by the task distribution module and distributed to the resource scheduling module, including: The task distribution module splits the computing task group into multiple sub-computing tasks, generates task identifiers based on the parallelism of each sub-computing task, and distributes each sub-computing task and its corresponding task identifier to the resource scheduling module so that the resource scheduling module can identify the first sub-computing task and the second sub-computing task based on the task identifier. The task identifier includes a first task identifier for representing a first sub-computation task and a second task identifier for representing a second sub-computation task.

3. The task calculation method according to claim 1, characterized in that, The resource scheduling module schedules each sub-computation task according to the dependency relationship, including: Based on the dependencies, dependency entries are generated; wherein, the dependency entries include the task items that each sub-computation task depends on and the execution status of each task item; After each sub-computation task is completed, update the dependency table entries; At each preset interval, the third sub-computation task and the fourth sub-computation task are found from the dependency table entries, the third sub-computation task is executed, and the fourth sub-computation task is sent to the computing module. The third sub-computation task is the first sub-computation task whose dependent tasks have all been completed, and the fourth sub-computation task is the second sub-computation task whose dependent tasks have all been completed.

4. The task calculation method according to claim 3, characterized in that, The execution status is represented by the status indicator of the task item; the dependency table entry also includes the waiting indicator of each sub-computation task; updating the dependency table entry after each sub-computation task has been executed includes: After the corresponding sub-computation task is completed, the status flag of the task item representing that sub-computation task will be updated to "completed". The waiting flags of sub-computation tasks whose dependent tasks have all been completed are updated to "no waiting required". This allows the resource scheduling module to identify the corresponding sub-computation task as the third or fourth sub-computation task when it recognizes that the waiting flag of the corresponding sub-computation task is "no waiting required".

5. The task calculation method according to claim 3, characterized in that, The execution status of the second sub-computation task is returned by the computation module to the resource scheduling module, including: After the second sub-computation task is completed, the computing module sends the second sub-computation task and its corresponding execution completion identifier to the resource scheduling module so that the resource scheduling module can identify the execution status of the second sub-computation task as completed.

6. A task computing apparatus for implementing the task computing method as described in any one of claims 1-5, characterized in that, It includes a task distribution module, a resource scheduling module, and a computing module; The task distribution module is used to distribute the first sub-computation task and the second sub-computation task to the resource scheduling module; The resource scheduling module is used to execute the first sub-computation task itself and to send the second sub-computation task to the computing module; The computing module is used to execute the second sub-computation task; wherein the parallelism of the first sub-computation task is lower than that of the second sub-computation task. It also includes a memory, which has a cache area, and a data transmission path is established between the resource scheduling module and the cache area; The resource scheduling module uses the data transmission path to obtain the data required to execute the first sub-computation task from the cache, and then uses the data and the instructions of the first sub-computation task to execute the first sub-computation task.

Images