Effective SM division strategy based on kernel on-chip storage resource request

Abstract

The invention discloses an effective SM division strategy based on a kernel on-chip storage resource request. The effective SM division strategy comprises a storage resource demand evaluator, an SM division optimizer and a thread block scheduler, the method comprises the following steps: acquiring an on-chip storage resource demand of a kernel through a storage resource demand evaluator; obtaining the allocation number of SM corresponding to the kernel through the SM division optimizer according to the on-chip storage resource demand, and obtaining the mapping relation between the kernel and SM allocation according to the on-chip storage resource demand of the kernel and the allocation number of the SM corresponding to the kernel; and the thread block scheduler is used for scheduling the thread blocks of the kernel to the corresponding SM for execution according to the mapping relationship. According to the technical scheme, effective allocation of SM resources among different kernels can be realized.

Description

Technical Field

[0001] The invention belongs to the technical field of GPGPU micro-architecture, and in particular relates to an effective SM partitioning strategy based on kernel on-chip storage resource request. Background Art

[0002] In data centers and cloud computing, GPGPU is often used to process high-performance concurrent computing tasks submitted by multiple users. In the scenario of multi-task concurrent execution, multiple kernels are executed concurrently, which makes the GPGPU resource competition problem more prominent. It exists not only within the kernel of the same task, but also between kernels of different tasks. In order to improve the system throughput, it is necessary to optimize the resource allocation and concurrent access in the multi-task and multi-kernel scenario.

[0003] In order to improve the overall resource utilization of GPGPU in multi-kernel concurrent execution scenarios, there is a current type of research work that optimizes resource allocation in multi-kernel scenarios through streaming multiprocessor (SM) partitioning strategies to improve the overall operating efficiency of GPGPU systems. SM partitioning strategies can usually be considered from two aspects: spatial partitioning and temporal partitioning. Since the temporal partitioning SM strategy requires context switching between different kernels, it is easy to cause a large time overhead. At present, researchers basically study GPGPU performance optimization in multi-kernel scenarios from the perspective of spatial multitasking to improve the throughput of GPGPU systems. Adriaens et al. first proposed using the spatial multitasking average (Even) partitioning strategy to evenly allocate SM resources to multiple concurrently executed kernels. However, the Even partitioning strategy does not take into account the differences in resource requirements of different kernels, and simply performing an average partitioning will result in low utilization of some SMs. In response to this situation, Zhao et al. proposed an efficient SM partitioning scheme based on classification-driven search. Before performing SM partitioning, the scheme needs to first determine the types of different tasks, and then search for the best SM partitioning result based on the type of task. However, accurate task classification is a challenging task, and it will add extra time when searching for SM partitioning schemes. In response to this situation, Thomas et al. proposed a dynamic optimization strategy for concurrent execution of multiple tasks. This strategy uses the Roofline model to classify tasks, and adopts three static SM partitioning schemes according to the type of task. Compared with the Even partitioning strategy, the Dynamic optimizations strategy implements a more efficient SM partitioning scheme and improves the overall resource utilization of GPGPU. However, once the SM division between concurrent tasks is determined, the above scheme will not change. Even when a task is completed, other concurrent tasks cannot use the SM resources occupied by it, which easily causes a decrease in the overall resource utilization of GPGPU. In addition, existing work assumes that concurrent tasks start to execute at the same time, and does not effectively deal with the situation where concurrent tasks start to execute one after another. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes an effective SM partitioning strategy based on kernel on-chip storage resource request to solve the above problems in the prior art.

[0005] To achieve the above object, the present invention provides an effective SM partitioning strategy based on kernel on-chip storage resource request, including: a storage resource demand evaluator, an SM partitioning optimizer and a thread block scheduler;

[0006] Wherein, the on-chip storage resource requirement of the kernel is obtained through the storage resource requirement evaluator;

[0007] The SM partition optimizer obtains the number of SMs allocated to the kernel according to the on-chip storage resource requirements, and obtains a mapping relationship between the kernel and the SM allocation according to the on-chip storage resource requirements of the kernel and the number of SMs allocated to the kernel;

[0008] The thread block scheduler is used to schedule the kernel's thread blocks to the corresponding SM for execution according to the mapping relationship.

[0009] Optionally, the storage resource requirement evaluator obtains kernel related information, and calculates the kernel's on-chip storage resource requirement based on the kernel related information, wherein the kernel related information includes the demand data for on-chip storage resources when the kernel is executed.

[0010] Optionally, the on-chip storage resource requirement calculation process of the kernel is:

[0011] SRR=Num reg +Num lm +Num sm +Num cm +Num gm

[0012] Among them, Num reg 、Num lm 、Num sm 、Num cm 、Num gm They represent the requirements of a kernel for registers, local data cache, shared cache, constant cache, and global cache respectively. SRR represents the on-chip storage resource requirements of the kernel.

[0013] Optionally, the SM partition optimizer includes an SM partitioner and an SM partition table, wherein the SM partition optimizer allocates the number of SMs corresponding to the kernel according to on-chip storage resource requirements, and the SM partition table is a mapping table structure for mapping the kernel and SM allocation.

[0014] Optionally, in the SM divider, the process of obtaining the number of allocated SMs corresponding to the kernel includes:

[0015] According to the on-chip storage resource requirements, the proportion of on-chip storage resource requirements for each kernel is calculated:

[0016]

[0017] Where n represents the number of kernels, SRR i Indicates the storage resource requirement of the i-th kernel, SRRR i Indicates the storage resource demand ratio of the i-th kernel;

[0018] According to the proportion of on-chip storage resource requirements of each kernel, the number of SMs allocated to the kernel is calculated:

[0019] SMNum ki =SRRR i *SMNum total ;

[0020] Among them, SMNum total Indicates the total number of SMs in the current GPGPU architecture, SMNum ki Indicates the number of SMs allocated to the i-th kernel.

[0021] Optionally, the SM partition table includes several table entries, wherein the table entries are respectively used to store valid bits, kernel numbers, SM allocation quantities, SM scheduling allocation information, and on-chip storage resource requirements.

[0022] Optionally, in the SM partition table, the SM partition table is written by a storage resource requirement evaluator according to the on-chip storage resource requirement of the kernel and an SM partitioner according to the allocated number of SMs corresponding to the kernel to obtain a mapping relationship between the kernel and the SM allocation;

[0023] Among them, after obtaining the on-chip storage resource requirements of the kernel, the valid bit of the SM partition table is adjusted to valid, and the storage resource requirement evaluator writes the kernel number and the on-chip storage resource requirements according to the on-chip storage resource requirements of the kernel and the kernel number. After obtaining the allocated number of SMs corresponding to the kernel, the SM partitioner adjusts the SM allocation number and SM scheduling allocation information in the SM partition table according to the allocated number of SMs corresponding to the kernel.

[0024] Optionally, in the thread block scheduler, a valid bit in the SM partition table is obtained, and when the valid bit is valid, the kernel number and SM scheduling allocation information are obtained, the assigned SM number is obtained according to the SM scheduling allocation information, and the kernel's thread block is scheduled to the corresponding SM for execution according to the kernel number and the assigned SM number.

[0025] Compared with the prior art, the present invention has the following advantages and technical effects:

[0026] 1. Aiming at the problem of fierce resource competition and low resource utilization in multi-kernel application scenarios, the present invention analyzes PTX instructions, calculates the requirements of different kernels for on-chip storage resources, and determines the division ratio of SM between different kernels.

[0027] 2. In order to improve the utilization of SM resources, on the one hand, the kernels that start to execute later can orderly seize the SM resources allocated to them, reducing the context switching overhead during concurrent kernel execution; on the other hand, when a concurrent kernel has no thread block scheduling, it indicates that the kernel is in the final stage. At this time, the SM division result is recalculated according to the on-chip storage resource utilization of other kernels, so that other kernels can non-mandatorily seize the SM resources occupied by the kernel in the final stage. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0029] Figure 1 A logical structure diagram of the SM partitioning strategy according to an embodiment of the present invention;

[0030] Figure 2 is a comparison result of normalized system throughput of two comparison methods in an embodiment of the present invention;

[0031] Figure 3 The comparison results of the average normalized turnaround time of three different strategies in the embodiment of the present invention;

[0032] Figure 4 This is a comparison result of the system throughput of the embodiment of the present invention, which adopts the non-mandatory preemption strategy and does not adopt the non-mandatory preemption strategy. DETAILED DESCRIPTION

[0033] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0034] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0035] The present invention discloses an effective SM partitioning strategy based on kernel on-chip storage resource request, which includes three functional components: an on-chip storage resource requirement evaluator (SRE), an SM partitioning optimizer (SMPO) and an improved thread block scheduler (TBS).

[0036] They have the following functions respectively: First, the storage resource requirement estimator is responsible for evaluating the on-chip storage resource requirement (Storage Resource Requirement, SRR) of the kernel. The on-chip storage resource requirement is used to calculate the on-chip storage resource requirement ratio (Storage Resource Requirement Ratio, SRRR) of each kernel. Based on the on-chip storage resource requirement ratio, the number of SM allocations corresponding to the kernel can be calculated. Then, the SM partition optimizer consists of the SM partitioner (SMPartitioner, SMP) and the SM partition table (SM Partition Table, SMPT). The SM partitioner calculates the number of SM allocations for each kernel. The SM partition table is used to store the mapping relationship between the SM allocation information of each concurrent kernel and the size of the on-chip storage resource requirement.

[0037] Finally, the thread block scheduler schedules the kernel's thread blocks to the corresponding SM for execution according to the mapping relationship between kerenl and SM in the mapping table of SMPT. In addition, when the kernel has no thread block scheduling, TBS will re-modify the SMPT.

[0038] The present invention proposes a new SM partitioning strategy, which can realize the effective allocation of SM resources among different kernels. In addition, by means of orderly preemption and non-mandatory preemption of SM resources, the context switching overhead during concurrent kernel execution is reduced, so that the SM resources can be fully utilized. Compared with the Even partitioning strategy and the Dynamic optimizations strategy, the present invention can more effectively improve the overall resource utilization and system throughput of GPGPU.

[0039] The above technical solution is explained as follows:

[0040] The present invention discloses an effective SM partitioning strategy based on kernel on-chip storage resource request, comprising a storage resource demand evaluator, an SM partitioning optimizer and an improved thread block scheduler;

[0041] First, the storage resource requirement estimator evaluates the kernel's on-chip storage resource requirement (Storage Resource Requirement, SRR):

[0042] SRR=Num reg +Num lm +Num sm +Num cm +Num gm

[0043] Among them, Num reg 、Num lm 、Num sm 、Num cm 、Num gm The sub-table shows the demand of a kernel for registers, local data cache, shared cache, constant cache and global cache.

[0044] Then, the SM partition optimizer consists of the SM partitioner and the SM partition table.

[0045] The SM partition table includes isValid, kNum, SRR, which correspond to whether it has been written, kernel number and SRR corresponding to the kernel number in sequence. According to the above calculated SRR, it is written to the corresponding position in the corresponding row of the corresponding kernel number, and isValid is set to 1.

[0046] The SM allocator calculates the number of SMs to allocate for each kernel based on the above:

[0047] The SM partitioner uses SRR to calculate the on-chip storage resource requirement ratio (Storage Resource Requirement Ratio, SRRR) of each kernel:

[0048]

[0049] Where n represents the number of kernels, SRR i Indicates the storage resource requirement of the i-th kernel.

[0050] Based on SRRR, the number of SMs allocated to the kernel can be calculated:

[0051] SMNum ki =SRRR i *SMNum total

[0052] Among them, SRRR iIndicates the proportion of on-chip storage resources required by the i-th kernel, SMNum total Indicates the total number of SMs in the current GPGPU architecture, SMNum ki Indicates the number of SMs allocated to the i-th kernel.

[0053] The SM partition table is used to store the SM allocation information and on-chip storage resource requirements of each concurrent kernel. It contains multiple table entries, which mainly store the SM allocation status and on-chip storage resource requirements of each kernel. Each table entry consists of fields such as isValid, kNum, SMNum, bitMask, and SRR.

[0054] Among them, isValid is a valid bit, indicating whether the table entry is valid. Its value is 1, indicating that the table entry is valid, and 0, indicating that the table entry is invalid. If it is valid, scheduling is performed according to the table entry.

[0055] kNum stores the number of the corresponding kernel. The elements in the table item correspond to the number of a kernel, indicating that the kernel number is associated with the kernel under that number.

[0056] SMNum indicates the number of SMs allocated to the kernel.

[0057] bitMask is a bit array whose length is equal to the number of SMs in the GPGPU architecture. Each bit of bitMask indicates whether a certain SM is assigned to the corresponding kernel. If a bit is set to 1, it means that the corresponding kernel can be scheduled to execute on the SM corresponding to the number. On the contrary, if a bit is set to 0, it cannot be scheduled to execute on the corresponding SM.

[0058] SRR is used to store the on-chip storage resource requirement size of the corresponding kernel.

[0059] The maximum number of rows in the SM partition table, i.e., the mapping table, is equal to the maximum number of kernels that can be executed concurrently by GPGPU, and each row corresponds to a kernel.

[0060] Finally, the thread block scheduler schedules the kernel's thread blocks to the corresponding SM for execution according to the mapping relationship between kerenl and SM in the SMPT mapping table. In addition, when the kernel has no thread block scheduling, the thread block scheduler will re-modify the SMPT.

[0061] The above technical solution of the present invention is described in detail with reference to relevant data and drawings:

[0062] The present invention provides an effective SM partitioning strategy based on kernel on-chip storage resource request, which is composed of three functional components, namely, an on-chip storage resource requirement evaluator (SRE), an SM partitioning optimizer (SMPO) and an improved thread block scheduler (TBS). The logical structure is as follows: Figure 1 The specific work of the three functional components is as follows:

[0063] First, the storage resource requirement estimator estimates the kernel's on-chip storage resource requirement (Storage Resource Requirement, SRR). And uses the on-chip storage resource requirement to calculate the on-chip storage resource requirement ratio (Storage Resource Requirement Ratio, SRRR) of each kernel. Based on the on-chip storage resource requirement ratio, the number of SMs allocated to the kernel can be calculated.

[0064] Then, the SM partition optimizer consists of the SM partitioner (SM Partitioner, SMP) and the SM partition table (SMPartition Table, SMPT).

[0065] ① The SM allocator calculates the number of SMs to be allocated for each kernel.

[0066] ②The SM partition table is used to store the SM allocation information and on-chip storage resource requirements of each concurrent kernel. It contains multiple table entries, which mainly store the SM allocation status and on-chip storage resource requirements of each kernel. Each table entry consists of fields such as isValid, kNum, SMNum, bitMask, and SRR. Among them, bitMask is a bit array whose length is equal to the number of SMs in the GPGPU architecture. Each bit of bitMask indicates whether a certain SM is assigned to the corresponding kernel. If a bit is set to 1, it means that the corresponding kernel can be scheduled to execute on the SM corresponding to the number, otherwise it cannot be scheduled to execute on the corresponding SM.

[0067] Finally, the thread block scheduler (TBS) schedules the kernel's thread blocks to the corresponding SMs for execution according to the mapping relationship between kerenl and SM in the mapping table of the SM partition optimizer. In addition, when the kernel has no thread block scheduling, the thread block scheduler will re-modify the SMPT.

[0068] The function of the storage resource requirement estimator is to calculate the number of SMs allocated to the kernel, including the following specific steps:

[0069] First, the storage resource requirement estimator can obtain kernel-related information from the PTX (ParallelThread eXecution instruction set) file generated by the CUDA program compilation process, including the demand data for on-chip storage resources during kernel execution, such as registers, local data cache, shared cache, constant cache, global cache and other on-chip storage resources, thereby evaluating each kernel's demand for on-chip storage resources.

[0070] After the storage resource requirement estimator collects all the storage resource requirement data of kerenl, use:

[0071] SRR=Num reg +Num lm +Num sm +Num cm +Num gm

[0072] Calculate the SRR of each kernel. reg 、Num lm 、Num sm 、Num cm 、Num gm They represent the demand of a kernel for registers, local data cache, shared cache, constant cache, and global cache respectively.

[0073] After the SRR is calculated, a table entry is allocated for the kernel in the SM partition table of the SM partition optimizer logic component, that is, an entry with an isValid field of 0 is found, the kernel number and SRR are written together, and the isValid of the entry is set to 1.

[0074] Then, after the storage resource requirement estimator has estimated the SRR of each kernel, the SM resource can be divided by the SM partition optimizer SMPO to allocate a suitable number of SMs to each kernel. The SMPO is composed of the SMP and the SMPT.

[0075] (1)SMP

[0076] The SM partitioner SMP is mainly used to calculate the number of SMs allocated to each kernel.

[0077] This process is divided into three steps. First, the demand ratio of on-chip storage resources of each kernel is calculated. Then, the number of SMs allocated to each kernel is calculated based on the ratio. Finally, the SMNum field of the corresponding kernel in the SMPT is modified according to the result of SM allocation.

[0078] SMP uses:

[0079]

[0080] Calculate the proportion of on-chip storage resources required by each kernel. Where n represents the number of kernels, SRR i Indicates the storage resource requirement of the i-th kernel.

[0081] After calculating the proportion of on-chip storage resources required by each kernel, use:

[0082] SMNum ki =SRRR i *SMNum total

[0083] The number of SMs allocated to each kernel can be calculated. i Indicates the proportion of on-chip storage resources required by the i-th kernel, SMNum total Indicates the total number of SMs in the current GPGPU architecture, SMNum ki Indicates the number of SMs allocated to the i-th kernel.

[0084] According to SMNum ki The above SMs are allocated through the SM divider, and a corresponding number of SMs are allocated to each kernel. After the allocation, the SM number is recorded, and the position used by the flag corresponding to the SM number is set to 1.

[0085] When a new kernel starts executing or a kernel finishes executing, the above process is repeated to recalculate the number of allocated SMs for the latest concurrent kernel.

[0086] (2)SMPT

[0087] In the SM partition table SMPT, bitMask is a bit array whose length is exactly equal to the number of SMs contained in GPGPU. The positions of each bit in the bit array from left to right are 0, 1, ..., and the bit at position 0 indicates whether the SM numbered 0 is assigned to the kernel, the bit at position 1 indicates whether the SM numbered 1 is assigned to the kernel, and so on. For example, the kNum field of a table entry in the SMPT is 1, and the bitMask field is 11110000, which means that the SMs numbered 0-3 are assigned to the kernel numbered 1.

[0088] After the SM partitioner SMP calculates the number of SMs allocated to each kernel, it needs to modify the bitMask field corresponding to each kernel in the SM partition table to provide a scheduling basis for the thread scheduler.

[0089] Algorithm 1 shows the process of modifying bitMask when each kernel starts executing.

[0090] Algorithm 1modify_bitMask / / bitMask modification algorithm:

[0091] Input: isValid and SMNum of each SMPT entry

[0092] Output: bitMask of each SMPT entry

[0093]

[0094] Finally, after the SM allocation to each kernel is determined by the SM partition optimizer, the thread block scheduling of each kernel can be performed according to the bitMask in the SM partition table. To this end, the original thread scheduler needs to be optimized. The improved thread block scheduler includes the following specific steps:

[0095] i. Each time a thread block is scheduled, the SM partition table is first accessed to determine that each kernel's thread block is scheduled to the corresponding SM for execution;

[0096] ii. The kernels that start executing later will seize the SM resources allocated to them in order. k Already assigned to kernel k that started executing first j If mandatory preemption is adopted at this time, it will lead to k i and k j The context switches between different thread blocks of k, thus generating additional time overhead. However, the method proposed in the present invention prohibits the continuation of scheduling k at this time. j Thread blocks to SM kWhen k j In SM k When a thread block on the top of the thread is finished executing, it starts scheduling k i Thread blocks to SM k The above scheduling process is called ordered preemptive scheduling.

[0097] iii. The kernel in the execution phase non-compulsory preemption of SM resources allocated by the kernel in the end of execution phase. i No thread block is scheduled for execution, which means k i Go to the end of execution. If k i In SM j When a thread block on the kernel is finished executing, a kernel k in the running stage is scheduled. j A thread block to SM k The above scheduling process is called non-mandatory preemptive scheduling.

[0098] The improved thread block scheduling algorithm consists of two for loops. The first for loop implements scheduling in the SM allocated by the kernel corresponding to the thread block. By querying the SMPT table, the assigned SM is determined. If one of the SMs has idle resources, the thread block is scheduled to execute on the corresponding SM. It should be noted that when there are multiple SMs with idle resources, the thread blocks are scheduled in a polling manner. This ensures that each thread block can only be scheduled in the SM allocated by the kernel to which it belongs. Even if a certain SM is occupied by the kernel that starts executing first, it can ensure that the SM resources allocated to it can be preempted in order in subsequent scheduling. When the first for loop does not return the allocated SM resources for the thread block, the second for loop is used to implement non-mandatory preemption of the SM resources allocated by the kernel at the end of execution.

[0099] In order to verify the effectiveness of the SM partitioning strategy proposed in the present invention, a corresponding experimental environment was constructed, and the performance of the present invention, the Even partitioning strategy and the Dynamic optimizations strategy were compared and analyzed through performance indicators such as system throughput and average normalized turnaround time. Among them, the Even partitioning strategy is used as a benchmark comparison method. Therefore, the experiments all adopted the cycle-level simulator GPGPU-Sim v4.0.0 that simulates the GPGPU architecture. The GPGPU architecture simulated in the experiment is the Turing architecture corresponding to the RTX 2060 product launched by NVIDIA, which includes 30 SMs, and each SM includes 64k of L1 Data cache, 64k of shared memory and 64k of Constant cache and other resource configurations. More GPGPU configuration information is shown in Table 1.

[0100] Table 1

[0101]

[0102] At the same time, the relevant codes in GPGPU-Sim were modified to implement the method proposed in the present invention. After constructing the experimental environment, 8 test programs were selected and combined in pairs to construct 28 test program pairs. The experimental process mainly conducted experimental analysis on these 28 test program pairs. These test programs are from test program sets such as ISPASS-2009, NVIDIA CUDA SDK, Rodinia, and PolyBench. Table 2 shows the list of selected test programs.

[0103] Table 2

[0104]

[0105] In order to better analyze the experimental results, each test program is classified into two categories: compute-intensive (COM) and memory-intensive (MEM).

[0106] Figure 2The comparison results of the normalized system throughput of the present invention and the two comparison methods are shown. Compared with the Even partitioning strategy, the average system throughput of the present invention is increased by 18%, and the maximum improvement is more than 30%; compared with the Dynamic optimizations strategy, the average system throughput of the present invention is increased by 8%, and the maximum improvement is 64%. It can be seen that the system throughput of the present invention is the highest. Moreover, for the concurrent execution of different types of tasks, the strategies of the present invention show different degrees of performance improvement. Even for the combined execution of memory-intensive applications, the present invention also achieves a certain degree of performance improvement. This is due to the fact that the method calculates the proportion of SM partitioning according to the actual on-chip storage resource requirements of different kernels, making the SM partitioning more accurate, so that more matching SM resources can be allocated to different concurrent tasks. In addition, the method of the present invention further improves the on-chip resource utilization in the multi-kernel concurrent execution scenario through two methods: orderly preemption and non-mandatory preemption. The Even partitioning strategy evenly distributes system resources in the multi-kernel scenario to ensure that each kernel can run in a relatively fair environment, but does not consider the actual demand for resources of each kernel. Therefore, among the three strategies, its average system throughput is the lowest. For the Dynamic optimizations strategy, it identifies the types of concurrent tasks and obtains the empirical value of SM division based on the combination of different types of tasks. This strategy only divides SM according to the type of task combination, but does not really divide according to the specific resource requirements of the application, which makes this resource division method less adaptable. For example, the system throughput of two groups of compute-intensive and memory-intensive combinations such as PDTD and SN, CP and SN is the same as that of the Even strategy. When two memory-intensive task combinations are executed, the system throughput performance is not very good in most test examples, especially when three groups of tasks such as 2MM and SN, 2MM and SYRK, and 2MM and HG are executed concurrently, their system throughput has decreased compared to the Even strategy.

[0107] The average normalized turnaround time is inversely proportional to the system throughput to a certain extent. Therefore, theoretically speaking, the average normalized turnaround time of the present invention should also be the shortest. Figure 3 The comparison results of the average normalized turnaround time of three different strategies are shown. It can be seen that Figure 3 The presented results are basically consistent with the theoretical analysis, and the average normalized turnover time of the present invention is the lowest. Compared with the Even partitioning strategy, the average normalized turnover of the present invention is reduced by 15% on average, and can be reduced by 39% at most; compared with Dynamic optimizations, the average normalized turnover of the present invention is reduced by 7% on average, and can be reduced by 25% at most.

[0108] In order to verify the effectiveness of the non-mandatory preemption strategy in the method of the present invention, that is, when a kernel is in the final stage of execution, other kernels in the non-final stage can schedule thread blocks to execute on the SM assigned to the kernel in the final stage, the system throughput of the present invention using the non-mandatory preemption strategy and not using the non-mandatory preemption strategy is also compared. The comparison results are shown in Figure 2. Figure 4 As shown in the figure, we can see that compared with not using the non-mandatory preemption strategy, the average system throughput increased by 7.4% and the maximum increased by 28.8% after using the non-mandatory preemption strategy. The main reason is that after adopting the non-mandatory preemption strategy, when it is detected that a kernel is in the end stage, that is, it has no thread blocks scheduled for execution, the thread blocks of other kernels can be scheduled to execute on their corresponding SM in a timely manner, which can improve the resource utilization of the SM corresponding to the kernel.

[0109] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An effective SM partitioning strategy based on kernel on-chip storage resource requests, characterized in that: include: Storage resource requirement estimator, SM partition optimizer, and thread block scheduler; Wherein, the on-chip storage resource requirement of the kernel is obtained through the storage resource requirement evaluator; The SM partition optimizer obtains the number of SMs allocated to the kernel according to the on-chip storage resource requirements, and obtains a mapping relationship between the kernel and the SM allocation according to the on-chip storage resource requirements of the kernel and the number of SMs allocated to the kernel; The thread block scheduler is used to schedule the kernel's thread blocks to the corresponding SM for execution according to the mapping relationship.

2. The strategy according to claim 1, characterized in that: The storage resource requirement evaluator obtains kernel related information, and calculates the kernel's on-chip storage resource requirement based on the kernel related information, wherein the kernel related information includes the on-chip storage resource requirement data when the kernel is executed.

3. The strategy according to claim 2, characterized in that: The calculation process of the kernel's on-chip storage resource requirements is as follows: SRR=Number reg +Number lm +Number sm +Number cm +Number gm Among them, Num reg 、Num lm 、Num sm 、Num cm 、Num gm They represent the requirements of a kernel for registers, local data cache, shared cache, constant cache, and global cache respectively. SRR represents the on-chip storage resource requirements of the kernel.

4. The strategy according to claim 1, characterized in that: The SM partition optimizer includes an SM partitioner and an SM partition table, wherein the SM partition optimizer allocates the number of SMs corresponding to the kernel according to the on-chip storage resource requirements, and the SM partition table is a mapping table structure for the mapping relationship between kernel and SM allocation.

5. The strategy according to claim 1, characterized in that: In the SM divider, the process of obtaining the number of SMs allocated to the kernel includes: According to the on-chip storage resource requirements, the proportion of on-chip storage resource requirements for each kernel is calculated: Where n represents the number of kernels, SRR i Indicates the storage resource requirement of the i-th kernel, SRRR i Indicates the storage resource demand ratio of the i-th kernel; According to the proportion of on-chip storage resource requirements of each kernel, the number of SMs allocated to the kernel is calculated: SMNum ki =SRRR i *SMNum total ; Among them, SMNum total Indicates the total number of SMs in the current GPGPU architecture, SMNum ki Indicates the number of SMs allocated to the i-th kernel.

6. The strategy according to claim 4, characterized in that: The SM partition table includes several table entries, wherein the table entries are respectively used to store valid bits, kernel numbers, SM allocation quantities, SM scheduling allocation information, and on-chip storage resource requirements.

7. The strategy according to claim 6, characterized in that: In the SM partition table, the storage resource requirement evaluator writes the SM partition table according to the on-chip storage resource requirement of the kernel and the SM partitioner according to the allocated number of SMs corresponding to the kernel to obtain a mapping relationship between the kernel and the SM allocation; Among them, after obtaining the on-chip storage resource requirements of the kernel, the valid bit of the SM partition table is adjusted to valid, and the storage resource requirement evaluator writes the kernel number and the on-chip storage resource requirements according to the on-chip storage resource requirements of the kernel and the kernel number. After obtaining the allocated number of SMs corresponding to the kernel, the SM partitioner adjusts the SM allocation number and SM scheduling allocation information in the SM partition table according to the allocated number of SMs corresponding to the kernel.

8. The strategy according to claim 7, characterized in that: In the thread block scheduler, a valid bit in the SM partition table is obtained. When the valid bit is valid, the kernel number and SM scheduling allocation information are obtained, the allocated SM number is obtained according to the SM scheduling allocation information, and the kernel's thread block is scheduled to the corresponding SM for execution according to the kernel number and the allocated SM number.

Images