Multi-task adaptive allocation method and equipment suitable for streaming processor

Abstract

The invention relates to the technical field of computing power optimization, in particular to a multi-task self-adaptive allocation method and device suitable for a streaming processor, and the method comprises the following steps: carrying out fine-grained decomposition on a multi-task application, splitting the multi-task application into a plurality of operators needing to be called, and recording the operator calling dependency relationship of the operators; the method comprises the following steps: dividing execution of a multi-task application into a plurality of stages, and recording a plurality of operators needing to be called in each stage; computing resources are dynamically allocated to operators needing to be called in each stage, and a computing resource allocation plan is generated; and according to the computing resource allocation plan, sending the computing instruction to the computing chip and starting execution until all stages are executed. According to the method, by dynamically adjusting the number of the physical cores allocated to each subtask, multi-task maximized parallel concealment is achieved, and the parallelism and the hardware resource utilization rate of chip hardware in the whole calculation process can be effectively improved; operation efficiency is improved, and idle core resources are reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computing power optimization, in particular to a multi-task adaptive allocation method and device suitable for a stream processor. BACKGROUND

[0002] Artificial intelligence is undergoing a process of evolution from the generative artificial intelligence (Generative AI) stage to the agent artificial intelligence (Agentic AI) stage, and the large model field is about to experience explosive growth in computing power demand. Among a series of artificial intelligence applications, GEMM (General Matrix Multiplication) type calculation is the most basic, most extensive, and most obvious computing power occupation. As a powerful tool, GEMM widely supports and accelerates the core algorithms in many scientific computing, engineering applications, and machine learning fields. The efficiency improvement of GEMM operation and the overall efficiency improvement of application have a strong correlation, so currently there are various types of hardware architecture chips used to accelerate GEMM calculation and thus improve application performance.

[0003] From the earliest attempt to use GPGPU (General-Purpose Graphics Processing Unit) to accelerate GEMM calculation, after more than ten years of trial and error in various industries and fields, the current chip field has formed a heterogeneous architecture system with matrix-like cores as the core of computing power. Among them, FPGA (Field Programmable Gate Array) and ASIC (Application-Specific Integrated Circuit) are the representatives of customized computing power carriers, and are becoming an important supplement to GPGPU. FPGA occupies the flexible adaptation track with its semi-customized characteristics. Its special DSP (Digital Signal Processing) unit can realize dynamic switching of multi-precision matrix operation through hardware reconfiguration, but the peak computing power of FPGA is limited, and the programming difficulty is significantly higher than that of general-purpose chips. It is not as widely used as ASIC in current artificial intelligence training. In contrast, ASIC realizes the ultimate energy efficiency ratio with a fully customized architecture, giving birth to different types of chips such as NPU (Neural Processing Unit), TPU (Tensor Processing Unit), etc. These chips cancel the non-core modules with low operation intensity, introduce matrix-like cores in combination with the needs of GEMM calculation, and at the same time retain the stream processor structure of GPGPU, so that more than 95% of transistor resources are concentrated in matrix-like cores and vector processing units, improving the computing efficiency of matrix-matrix multiplication, matrix-vector multiplication, and large data throughput.

[0004] Generally speaking, the current mainstream heterogeneous architecture generally adopts a collaborative design of "stream processor + matrix-like core", but the addition of FPGA and ASIC leads to the increase of ecology and scheduling complexity, and the power field presents the dual contradictions of supply and demand imbalance and efficiency waste: in the cloud scenario, the ten thousand card level cluster becomes the standard configuration for large model training, but the cross device communication and scheduling fault tolerance difficulty increases dramatically, and the bandwidth limitation of the traditional interconnection protocol makes the data transmission between CPU and acceleration chip become a common bottleneck; the end side scenario faces the challenge of resource fragmentation, although the low delay characteristics of ASIC are more suitable for ultra-low latency expert activation under MoE model training, but the existing scheduling scheme generally lacks adaptive adjustment ability and cannot dynamically match the high throughput computing characteristics of the matrix-like core. In addition, performance benchmark evaluation programs such as HPL, HPL-MxP, etc. are used to comprehensively evaluate the capabilities of the power cluster, and the evaluation content includes multiple types of computing tasks. In the mixed precision computing and burst task request scenario, even if an advanced hardware scheduler is used, the power utilization rate of the ASIC dedicated integrated circuit is often less than 50%, and the memory wall and resource idling problem is prominent. Therefore, a method is needed that can realize the precise matching of multiple, multiple computing tasks and matrix-like core resources, general stream processors, and release the stream processor power potential. SUMMARY

[0005] To solve the technical problems existing in the prior art, the present application provides a multi-task adaptive allocation method and device suitable for a stream processor, which fully considers the multi-thread concurrency of the stream processor and the high operation throughput characteristics of the matrix-like core, can effectively improve the parallelism and hardware resource utilization rate of the chip hardware in the whole computing process, dynamically adjusts the number of physical cores allocated to each sub-task, realizes maximum parallel masking of multi-task under the premise of ensuring result correctness, improves operation efficiency, reduces core resource idling, and releases the stream processor power potential.

[0006] The first object of the present application is to provide a multi-task adaptive allocation method suitable for a stream processor.

[0007] The second object of the present application is to provide a computer device.

[0008] The third object of the present application is to provide a storage medium.

[0009] The first object of the present application can be achieved by adopting the following technical solutions:

[0010] A multi-task adaptive allocation method suitable for a stream processor, the method comprising:

[0011] S1, finely decompose the multi-task application, split the multi-task application into multiple operators that need to be called, and record the calling and dependence relationship of the operators;

[0012] S2, according to the obtained operator call sequence dependence, the execution of the multi-task application is divided into several stages, and the several operators to be called in each stage are recorded;

[0013] S3, according to the change of the calculation task quantity of each operator in each stage, the calculation resource of each operator to be called in each stage is dynamically allocated, the number of matrix core resources and the number of stream processors to which each called operator in each stage is allocated are determined, and a calculation resource allocation plan is generated;

[0014] S4, according to the calculation resource allocation plan, the calculation instruction is sent to the calculation chip and the execution is started until all stages are executed.

[0015] Specifically, the step S1 comprises:

[0016] According to the multi-task application, the multi-task application is split into called operators including matrix multiplication GEMM, triangular matrix multiplication TRSM, LU decomposition GETRF, broadcast communication Bcast, and the calling sequence constraint of the called operators is recorded.

[0017] 3. The multi-task adaptive allocation method suitable for stream processors according to claim 2, wherein the step S2 comprises:

[0018] According to the obtained calling sequence constraint, the execution of the multi-task application is divided into stages, and the operator call requirements in each stage are stacked, so that the operator calls in each stage can be executed simultaneously, and the operator calls in the stage are not subject to the calling sequence constraint. The operator calls between stages follow the calling sequence constraint of the operators.

[0019] Specifically, the step S3 comprises:

[0020] The number of tasks required by the GEMM type operator in the current stage is calculated, the GEMM type calculation task is preferentially allocated to the matrix core, the number of threads required by each non-GEMM type operator in the current stage is calculated, and the non-GEMM type calculation task is preferentially allocated to the general stream processor unit, so that the actual execution time of all operators in a stage is close.

[0021] When the current stage is the last stage of the multi-task application, the next step is performed; otherwise, the calculation resource of the operator call of the next stage is allocated, the number of matrix core resources and the number of stream processors to which each called operator in each stage is allocated are determined, and a calculation resource allocation plan is generated.

[0022] Specifically, the non-GEMM type calculation task preferentially allocated to the general stream processor unit comprises:

[0023] The number of threads required by each non-GEMM type operator in the calculation stage is calculated, the proportion of the number of threads required by each operator is obtained, the threads of the current stream processor hardware are grouped based on the proportion of the number of threads required by each operator and the number of threads of the current stream processor hardware, so that the proportion of the number of physical threads of each group is consistent with the proportion of the number of threads required by each operator, and the non-GEMM type computing task is allocated to the threads of the general stream processor unit.

[0024] The second object of the present application can be achieved by adopting the following technical solution:

[0025] A computer device comprises a processor and a memory for storing a program executable by the processor, and when the processor executes the program stored in the memory, the above-mentioned multi-task adaptive allocation method suitable for a stream processor is implemented.

[0026] The third object of the present application can be achieved by adopting the following technical solution:

[0027] A storage medium stores a program, and when the program is executed by a processor, the above-mentioned multi-task adaptive allocation method suitable for a stream processor is implemented.

[0028] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0029] The present application provides a multi-task adaptive allocation method and device suitable for a stream processor containing a matrix-like core, which divides a multi-task application into a plurality of operators to be called by performing fine-grained decomposition on the multi-task application, divides the execution of the multi-task application into a plurality of stages, dynamically allocates computing resources to the operators to be called in each stage according to the computing task amount of each operator in each stage, so that the actual execution time of all operators in a stage is close, obtains a computing resource allocation plan, fully considers the multi-thread concurrency of the stream processor and the high operation throughput characteristics of the matrix-like core, effectively improves the parallelism and hardware resource utilization rate of the chip hardware in the whole computing process, dynamically adjusts the number of physical cores allocated to each sub-task, realizes maximum parallel masking of the multi-task under the premise of ensuring the correctness of the result, improves the operation efficiency, reduces the idle of core resources, and releases the computing power potential of the stream processor. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can also obtain other drawings from the structures shown in the drawings without creating any creative labor.

[0031] Figure 1 This is a flowchart of a multi-task adaptive allocation method applicable to a streaming processor containing a matrix-like core, as described in an embodiment of the present invention.

[0032] Figure 2 This is an architecture diagram of the Ascend A2 series NPU computing unit in an embodiment of the present invention;

[0033] Figure 3 This is a schematic diagram of the task scheduling plan at each stage in an embodiment of the present invention;

[0034] Figure 4 This is a schematic diagram of a task scheduling plan specifying the number of steps in an embodiment of the present invention. Detailed Implementation

[0035] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments, and the implementation of the present invention is not limited thereto. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] Example 1:

[0037] This invention provides a multi-task adaptive allocation method suitable for streaming processors. It fully considers the multi-threaded concurrency and high computational throughput characteristics of streaming processors, such as matrix-like cores, and effectively improves the parallelism and hardware resource utilization of chip hardware throughout the entire computing process. At the same time, it dynamically adjusts the number of physical cores allocated to each subtask, and maximizes the parallel masking of multiple tasks while ensuring the correctness of the results. This improves computational efficiency, reduces idle core resources, and releases the computing potential of streaming processors.

[0038] like Figure 1 As shown, the multi-task adaptive allocation method for streaming processors according to the present invention includes the following steps:

[0039] S1. Perform fine-grained decomposition on the multi-tasking application, breaking it down into multiple operators that need to be called, and recording the calling dependencies of multiple operators.

[0040] Specifically, the multi-task application is broken down into called operators based on the synchronization instructions in the multi-task application. The called operators include: matrix multiplication GEMM, triangular matrix multiplication TRSM, LU decomposition GETRF, and broadcast communication Bcast. The order of calling the called operators is recorded and constrained.

[0041] For any application, as long as the means of application performance acceleration through a specific chip is selected, it inevitably involves the calling of various operators in the Basic Linear Algebra Subprograms (BLAS). Specifically, the called operators include four kinds of operators, namely, matrix multiplication GEMM, triangular matrix multiplication TRSM, LU decomposition GETRF, and broadcast communication Bcast. For example, the matrix multiplication GEMM is used for a matrix multiplication calculation task, the triangular matrix multiplication TRSM is used for a triangular solution calculation task, the GETRF is used for an LU decomposition calculation task. The broadcast communication Bcast is used for a result broadcast task. The calling operation of various operators is often a non-blocking operation, and the calling sequence depends on the definition of an explicit synchronization instruction. According to the design of the synchronization instruction in the multi-task application, the application calling operator is arranged in this step, and the called operator and the necessary sequence constraint are combed. This necessary sequence constraint lays a solid foundation for the subsequent stage division and ensures the correctness of the task execution logic.

[0042] In this embodiment, the HPL-MxP benchmark program is taken as an example for decomposition, and the called operators are obtained, including matrix multiplication GEMM, triangular matrix multiplication TRSM, LU decomposition GETRF, and broadcast communication Bcast. The matrix multiplication GEMM is the main calculation core of the HPL-MxP benchmark program, which is used for high-precision matrix multiplication operation in mixed-precision iteration and accounts for the main part of the overall calculation amount. TRSM (Triangular Solve with Multiple Right-Hand Sides) solves the upper / lower triangular matrix equation and is used to solve the high-precision part of the linear system, for example, to process the triangular matrix in the iterative refinement process. GETRF (LU Factorization with Partial Pivoting) is the LU decomposition with partial pivot selection, which is used in the initial decomposition stage at low precision to provide a basis for subsequent mixed-precision iteration. Bcast (Broadcast) is a data broadcast communication, which is usually used to synchronize matrix blocks or vectors in an MPI parallel environment. It is used to synchronize the calculation data (such as the face element obtained after updating the LU decomposition) between different processes in a distributed memory system. There is an explicit definition of the calling sequence in the HPL-MxP benchmark program: the triangular matrix multiplication TRSM and the broadcast communication Bcast calling must wait for the LU decomposition GETRF to be executed; after the X-time LU decomposition GETRF→triangular matrix multiplication TRSM→broadcast communication Bcast loop is executed, the matrix multiplication GEMM is called, and the matrix multiplication GEMM calling must wait for the X-time triangular matrix multiplication TRSM in the loop to be executed. In order to make the embodiment more specific, X=4 is defined in this embodiment.

[0043] As Figure 2 shown, is the architecture diagram of the Ascend A2 series NPU computing unit. Taking the Ascend A2 series NPU computing unit as an example, the matrix-like core AIC is the matrix-like core described in the present application. The Cube computing unit in the matrix-like core AIC can complete the matrix multiplication calculation of 16x16x16 at a time within one clock cycle, the number of calculations is 4096 times, and supports the input and output of data types such as INT8, FP16, and FP32. In order to match the high data throughput characteristics of the AIC, a multi-level cache including L1, L0A, L0B, and L0C is designed; the AIV is the general stream processor described in the present application, the minimum data operation granularity is 32 bytes or 256 bits, and the 256-bit bitwise data calculation operation can be completed at a time within one clock cycle.

[0044] S2, according to the obtained operator calling and depending relationship, the execution of the multi-task application is divided into several stages, and several operators in each stage need to be called.

[0045] Specifically, according to the obtained calling and depending sequence constraint, the execution of the multi-task application is divided into different stages, and the operator calling requirement of each stage is stacked, so that the operator calling in each stage can be executed at the same time, and the operator calling in the stage is not subject to the sequence constraint; the inter-stage operator calling follows the operator calling sequence constraint.

[0046] In this embodiment, the operator to be called and the sequence constraint are obtained in step S1, and in step S2, the benchmark test execution is divided into Y stages. In the benchmark test, Y is often much larger than X, in order to make this embodiment more specific, Y=100 is defined in this embodiment. As Figure 3 shown, is a part of the stage task scheduling plan in the embodiment of the present application. From the calling and depending sequence constraint of the operator obtained in step S1, it can be concluded that in the X-time calling cycle, 2X stages are accumulated. The end of the 2X stage completes a complete X-time calling cycle, at this time, the next cycle is entered, that is, the 2X+1 stage. The 2X+1 stage accumulates 2X stages again, that is, the second complete X-time calling cycle is completed, at this time, the 2X+2X=4X stage is reached. The subsequent cycles are carried out in this way.

[0047] As Figure 4As shown, it is a task scheduling plan schematic diagram of step number, in the benchmark test running process, the first stage executes the operator GETRF, LU decomposition is carried out to the matrix, and upper triangular matrix U and lower triangular matrix L are obtained; the second stage executes the operator TRSM and Bcast, and surface element updating is realized through triangular matrix multiplication; the third stage executes the operator GETRF, and a new round of matrix LU decomposition is carried out to obtain upper triangular matrix U and lower triangular matrix L; the fourth stage executes the operator TRSM and Bcast, and surface element updating is realized through triangular matrix multiplication; the eighth stage executes the operator TRSM and Bcast, and surface element updating is realized through triangular matrix multiplication; the ninth stage executes the operator GETRF and GEMM, the operator GETRF is called to carry out a new round of matrix LU decomposition, and the operator GEMM is called to combine the surface element decomposition results of the previous eight stages, that is, tail matrix updating operation is carried out. In this way, the operator execution requirement of each stage can be obtained, the dependence between stages is met, and a plurality of operators in the stage are called simultaneously.

[0048] S3, according to the calculation task quantity of each operator in each stage, dynamically allocate calculation resources to each operator to be called in each stage, determine the number of matrix core resources and stream processors allocated to each called operator in each stage, and finally generate a calculation resource allocation plan. In this way, the actual execution time of all operators in a stage is close, the idle of calculation resources can be avoided to the maximum extent, the optimal parallelism in the stage is achieved, and the mapping of operator calling and hardware resources is realized.

[0049] S31, according to the calculation scale of each operator in each stage, perform mapping and allocation of hardware resources, calculate the number of tasks required by the GEMM class operator in the stage, preferentially allocate the GEMM class computing task to the matrix core, calculate the number of threads required by each non-GEMM class operator in the stage, and preferentially allocate the non-GEMM class computing task to the general stream processor unit, so that the actual execution time of all operators in a stage is close, and the bucket effect is avoided.

[0050] S32, if the current stage is the last stage of the multi-task application, then jump to step S4 for the next step; otherwise, continue to allocate calculation resources to the operator calling of the next stage, determine the number of matrix core resources and stream processor resources allocated to each called operator in each stage, and generate a calculation resource allocation plan.

[0051] In this embodiment, taking the second stage as an example, step S3 specifically includes the following steps:

[0052] A, calculate the number of tasks required by the GEMM class operator in the stage.

[0053] In this embodiment, only the TRSM operator in the second stage belongs to the GEMM type, and therefore the TRSM calculation task is distributed to the 25 available matrix cores of the GEMM type, and the TRSM operator is allocated a total of 25 matrix cores of the GEMM type.

[0054] B, the number of threads required by each non-GEMM type operator in the current stage is calculated to obtain the proportion of the number of threads required by each operator, and the threads of the current stream processor hardware are grouped based on the proportion of the number of threads required by each operator and the number of threads of the current stream processor hardware, so that the proportion of the number of physical threads in each group is consistent with the proportion of the number of threads required by each operator.

[0055] In this embodiment, the only non-GEMM type operator in the second stage is the Bcast operator, and therefore the Bcast operator can exclusively occupy all stream processor resources for a total of 50; the Bcast operator can exclusively occupy all stream processor resources, and therefore the Bcast task cycle is distributed to all 50 available threads, and finally the Bcast operator is allocated a total of 50 general stream processor resources.

[0056] C, return to step A, and allocate resources to the operator calls in the third stage to finally determine the number of matrix core resources and the number of stream processor resources allocated to each called operator in each stage, and generate a calculation resource allocation plan.

[0057] According to the change of the calculation task amount of each operator in different stages, the number of physical cores allocated to each subtask is dynamically adjusted, the maximum parallel masking of multiple tasks is realized under the premise of ensuring the correctness of the results, the operation efficiency is improved, and the core resource idling is reduced.

[0058] S4, according to the calculation resource allocation plan obtained in step S3, send the calculation instructions to the calculation chip and start execution until all stages are executed.

[0059] In this embodiment, in the second stage, the operators to be issued are TRSM and Bcast, and the step S3 has obtained the calculation resource allocation plan (TRSM is allocated 25 matrix cores and Bcast is allocated 50 stream processor threads). Therefore: the TRSM calling instruction is issued, the calculation core type is matrix core, and the core number is 25; the Bcast calling instruction is issued, the calculation core type is general stream processor, and the thread number is 50; and then waiting until the TRSM and Bcast operators are executed and the hardware resources are released. Since there is no data dependency between the TRSM task and the Bcast task, simultaneous execution of the two does not affect the correctness of the result, the calling instructions can be issued at the same time and run at the same time, and parallel masking is achieved. In addition, by applying the calculation resource allocation plan obtained in step S3, all available calculation resources of the matrix core and the stream processor are occupied by the calculation task, thereby achieving the purpose of improving the operation efficiency and reducing the idle of the calculation core resources.

[0060] In this embodiment, a multi-task adaptive allocation method suitable for a stream processor is provided, which fully considers the multi-thread concurrency of the stream processor and the high operation throughput characteristics of the matrix core, preferentially allocates matrix calculation to the matrix core, and allocates non-matrix calculation to the general thread unit, thereby effectively improving the parallelism of the chip hardware in the whole calculation process and the hardware resource utilization rate. According to the calculation task amount of each operator in different stages, the number of physical cores allocated to each sub-task is dynamically adjusted, the maximum parallel masking of multi-tasks is realized under the premise of ensuring the correctness of the result, the operation efficiency is improved, and the idle of the core resources is reduced.

[0061] Embodiment 2:

[0062] The computer device provided in this embodiment can be a server, a computer or the like, and includes a processor, a memory, an input device, a display and a network interface connected through a system bus, the processor is used to provide calculation and control capability, the memory includes a non-volatile storage medium and an internal memory, the non-volatile storage medium stores an operating system, a computer program and a database, the internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run, and when the processor executes the computer program stored in the memory, the multi-task adaptive allocation method suitable for a stream processor in the above embodiment 1 is implemented, as follows:

[0063] S1, finely decompose a multi-task application, split the multi-task application into a plurality of operators to be called, and record the calling and dependency relationship of the operators;

[0064] S2, according to the obtained calling and dependency relationship of the operators, divide the execution of the multi-task application into a plurality of stages, and record a plurality of operators to be called in each stage;

[0065] S3, dynamically allocating computing resources to each operator to be invoked in each stage according to the computing task amount of each operator in each stage, determining the number of matrix core resources and the number of stream processors to which each invoked operator in each stage is allocated, and generating a computing resource allocation plan;

[0066] S4, sending computing instructions to a computing chip and starting execution according to the computing resource allocation plan, until all stages are executed.

[0067] Embodiment 3:

[0068] The embodiment provides a storage medium, which is a computer-readable storage medium, and stores a computer program. When the program is executed by a processor, the processor executes the computer program stored in the memory, and the method for adaptively allocating multiple tasks to a stream processor in the above embodiment 1 is implemented, as follows:

[0069] S1, performing fine-grained decomposition on a multi-task application, splitting the multi-task application into multiple operators to be invoked, and recording the invocation and dependency relationship of the operators;

[0070] S2, dividing the execution of the multi-task application into several stages according to the obtained operator invocation and dependency relationship, and recording the several operators to be invoked in each stage;

[0071] S3, dynamically allocating computing resources to each operator to be invoked in each stage according to the computing task amount of each operator in each stage, determining the number of matrix core resources and the number of stream processors to which each invoked operator in each stage is allocated, and generating a computing resource allocation plan;

[0072] S4, sending computing instructions to a computing chip and starting execution according to the computing resource allocation plan, until all stages are executed.

[0073] The above embodiment is a preferred embodiment of the present application, but the embodiment of the present application is not limited by the above embodiment, and any change, modification, replacement, combination, simplification made without departing from the spirit and principle of the present application should be an equivalent replacement mode, and all are included in the protection scope of the present application.

Claims

1. A multi-task adaptive allocation method suitable for streaming processors, characterized in that, Includes the following steps: S1. Perform fine-grained decomposition on the multi-tasking application, breaking it down into multiple operators that need to be called, and recording the order and dependencies of operator calls. S2. Based on the obtained operator call sequence dependency, divide the execution of the multi-task application into several stages and record several operators that need to be called in each stage. S3. Based on the changes in the computational workload of each operator in each stage, dynamically allocate computational resources to the operators that need to be called in each stage, determine the number of matrix core resources and the number of stream processor resources allocated to each called operator in each stage, and generate a computational resource allocation plan. S4. According to the computing resource allocation plan, the computing instructions are sent to the computing chip and execution begins until all stages are completed.

2. The multi-task adaptive allocation method for streaming processors according to claim 1, characterized in that, Step S1 includes: Based on the synchronous instructions in the multi-tasking application, the multi-tasking application is broken down into called operators, including matrix multiplication (GEMM), triangular matrix multiplication (TRSM), LU decomposition (GETRF), and broadcast communication (Bcast). The order of calling the called operators is recorded and constrained.

3. The multi-task adaptive allocation method suitable for streaming processors according to claim 2, characterized in that, Step S2 includes: The execution of the multi-task application is based on the obtained call order constraints. The operator call requirements of each stage are pushed onto the stack, so that the operator calls within each stage can be executed simultaneously, and the operator calls within a stage are not subject to the order constraints; the operator calls between different stages, however, follow the operator call order constraints.

4. The multi-task adaptive allocation method for streaming processors according to claim 1, characterized in that, Step S3 includes: Calculate the number of tasks required for GEMM-class operators in the current stage, and prioritize the allocation of GEMM-class computing tasks to the matrix core. Calculate the number of threads required for each non-GEMM-class operator in the current stage, and prioritize the allocation of non-GEMM-class computing tasks to the general-purpose streaming processor unit, so that the actual execution time of all operators in a stage is close. If the current stage is the last stage of a multi-tasking application, proceed to the next step; otherwise, continue to allocate computing resources for the operator calls in the next stage, determine the number of matrix core resources and the number of stream processor resources allocated to each called operator in each stage, and generate a computing resource allocation plan.

5. The multi-task adaptive allocation method suitable for streaming processors according to claim 4, characterized in that, The calculation of the number of threads required for each non-GEMM operator in the current stage, and the priority allocation of non-GEMM computing tasks to general-purpose streaming processor units, includes: The number of threads required by each non-GEMM operator during the computation phase is calculated, and the proportion of threads required by each operator is obtained. Based on the proportion of threads required by each operator and the number of threads in the current streaming processor hardware, the threads of the current streaming processor hardware are grouped so that the proportion of physical threads in each group is consistent with the proportion of threads required by each operator. Non-GEMM computation tasks are then allocated to the threads of the general-purpose streaming processor unit.

6. A computer device comprising a processor and a memory for storing a processor-executable program, characterized in that, When the processor executes the program stored in the memory, it implements the multi-task adaptive allocation method for streaming processors as described in any one of claims 1-5.

7. A storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the multi-task adaptive allocation method for streaming processors as described in any one of claims 1-5.

Images