GPGPU (General Purpose Graphics Processing Unit) stream multiprocessor node source scheduling method, equipment and medium

Abstract

The invention discloses a GPGPU (General Purpose Graphics Processing Unit) stream multiprocessor source-saving scheduling method, equipment and a medium, belongs to the technical field of GPGPU scheduling execution, and is used for solving the technical problems that in the existing GPGPU scheduling execution process, the energy efficiency utilization rate of an intensive load is low, memory contention is easy to form, waste of resources and energy consumption is caused, and power consumption is not favorably saved. The method comprises the following steps of: performing all activation processing on the stream multiprocessor unit connected with the thread block scheduling unit; performing category judgment on the task load related to the intensive load, and determining a memory access SM core corresponding to the memory access intensive load; performing suspension processing on the stream multiprocessor unit corresponding to the memory access SM core; computing control is carried out on a computing SM core in the residual stream multiprocessor unit to obtain a computing execution state of the computing SM core; and performing activation control on the stream multiprocessor unit corresponding to the memory access SM core to obtain a task execution state of the current task load.

Description

Technical Field

[0001] The present application relates to the field of GPGPU scheduling and execution, and in particular to a GPGPU stream multiprocessor resource saving scheduling method, device and medium. Background Art

[0002] Due to the significant computing power, accelerators like GPGPU are widely used for different workloads. These architectures allow thousands of threads to be executed in parallel through programming models such as CUDA (Compute Unified Device Architecture) or OpenCL (Open Computing Language). The programming model leads to a hierarchy of threads, a group of threads form a warp (basic scheduling unit consisting of 32 threads) or wavefront (a collection of threads executed in parallel), and a group of warps form a thread block called CTA (cooperative thread array) or workgroup. There are two levels of scheduling inside GPGPU, a thread block or CTA scheduler that assigns CTAs to each core, and a warp scheduler that determines which warp to execute within the core. In order to improve the performance of these parallel architectures, improved warp schedulers or CTA schedulers have been proposed to increase resource utilization and increase the parallelism of program execution.

[0003] However, for memory-intensive workloads, maximizing the use of all cores is not necessarily the most energy-efficient. At the same time, due to memory contention, insufficient resources of the memory access controller, etc., the pipeline may be paused during operation, resulting in a waste of resources and energy. Summary of the invention

[0004] The embodiments of the present application provide a GPGPU stream multiprocessor resource-saving scheduling method, device and medium, which are used to solve the following technical problems: During the existing GPGPU scheduling execution process, the energy efficiency utilization rate of intensive loads is low, which easily leads to memory contention, resulting in waste of resources and energy consumption, and is not conducive to saving power consumption.

[0005] The present application embodiment adopts the following technical solutions: On the one hand, an embodiment of the present application provides a GPGPU stream multiprocessor resource saving scheduling method, comprising: activating all stream multiprocessor units connected to a thread block scheduling unit; transmitting the task load assigned by the thread block scheduling unit to the stream multiprocessor unit, and judging the category of the task load as an intensive load through a monitoring cycle and an SM core in the stream multiprocessor unit, and determining a memory access SM core corresponding to a memory access intensive load; wherein the intensive load comprises: a computationally intensive load and the memory access intensive load; suspending the stream multiprocessor unit corresponding to the memory access SM core; and performing computational control on the computational SM cores in the remaining stream multiprocessor units to obtain a computational execution state of the computational SM core; wherein the computational execution state comprises: an activation state and a resource saving state; according to the computational execution state of the computational SM core, activating and controlling the stream multiprocessor unit corresponding to the memory access SM core to obtain a task execution state of the current task load.

[0006] The embodiment of the present application can identify and suspend memory access intensive loads on SM (streaming multiprocessor, which is an important computing unit in NVIDIA Volta and later processor architectures), give priority to computing compute-intensive loads, and when all compute-intensive loads are completed, activate the thread blocks of the suspended memory access intensive loads for computing, and at the same time, power gate or clock gate enable the SM core that completes the compute-intensive load, thereby reducing static power consumption and dynamic power consumption, thereby improving energy efficiency. In this way, compute-intensive tasks and memory-access intensive tasks are executed separately, reducing the impact of memory-access intensive tasks on compute-intensive tasks, reducing memory contention, and at the same time, the streaming multiprocessor enters a power-saving state after the calculation is completed, thereby saving power consumption.

[0007] In a feasible implementation, all stream multiprocessor units connected to the thread block scheduling unit are activated, specifically including: transmitting the received thread task load to the thread block scheduling unit through the Host server; wherein each thread block in the thread block scheduling unit can only be executed on one of the stream multiprocessor units, and one of the stream multiprocessors can execute multiple thread blocks simultaneously; before allocating the task load to the stream multiprocessor units through the thread block scheduling unit, controlling all SM cores in the stream multiprocessor units to remain activated.

[0008] In a feasible implementation, before determining the category of the task load as an intensive load through the monitoring cycle and the SM core in the stream multiprocessor unit and determining the memory access SM core corresponding to the memory access intensive load, the method further includes: configuring the monitoring cycle of the SM core in each of the stream multiprocessor units to obtain a clock monitoring cycle; wherein the unit number of the clock monitoring cycle is associated with the task attribute of the thread task load; through the clock monitoring cycle, monitoring the memory access intensive load for cycles related to memory access pauses to obtain a memory access pause cycle; wherein the memory access pause cycle is the total number of cycles for monitoring the SM core pauses due to memory access.

[0009] In a feasible implementation, the task load is judged as a type of intensive load through the monitoring cycle and SM core in the stream multiprocessor unit, and the memory access SM core corresponding to the memory access intensive load is determined, specifically including: when the memory access pause cycle is greater than one-third of the clock monitoring cycle, the task load currently being run by the SM core is determined as the memory access intensive load; the number of running SM cores corresponding to the memory access intensive load is identified and collected; when the number of running SM cores is greater than one-half of the number of SM cores in an activated state, the running SM core corresponding to the number of running SM cores is the memory access SM core; wherein the memory access SM core only includes the memory access intensive load; and the stream multiprocessor unit corresponding to the memory access SM core is determined as a memory access stream multiprocessor unit for processing the memory access intensive load.

[0010] In a feasible implementation, the stream multiprocessor unit corresponding to the memory access SM core is suspended, specifically including: the stream multiprocessor unit includes an improved Warp scheduler and a pipeline module; when the stream multiprocessor unit is determined to be a memory access stream multiprocessor unit for processing the memory access intensive load, the Warp scheduler in the memory access stream multiprocessor unit is subjected to execution identification processing of relevant memory access instructions; when the memory access instruction is identified, the Warp scheduler related to the memory access instruction is suspended; based on the Warp scheduler in the memory access stream multiprocessor unit being in a suspended state, the memory access stream multiprocessor unit is controlled to enter a suspended state.

[0011] In a feasible implementation, the computing SM cores in the remaining stream multi-processor units are computationally controlled to obtain the computing execution status of the computing SM cores, specifically including: determining the remaining stream multi-processor units as computing stream multi-processor units; wherein the computing stream multi-processor units only contain the computing intensive loads; performing computational control on the computing intensive loads through the computing stream multi-processor units; if the computing SM cores in the computing stream multi-processor units are in continuous computing of the computing intensive loads, determining the computing SM cores as being in an activated state; if the computing SM cores in the computing stream multi-processor units have completed computing of the computing intensive loads, determining the computing SM cores as being in a power-saving state; and continuously monitoring the computing execution status of the computing SM cores in all the computing stream multi-processor units.

[0012] In a feasible implementation manner, according to the computational execution state of the computational SM core, the stream multiprocessor unit corresponding to the memory access SM core is activated and controlled to obtain the task execution state of the current task load, specifically including: if the computational execution state of the computational SM core has the activation state, then the stream multiprocessor unit corresponding to the memory access SM core is not activated and controlled; if the computational execution states of the computational SM cores are all the resource-saving states, then the memory access stream multiprocessor unit corresponding to the memory access SM core is activated and controlled; when the memory access SM core in the memory access stream multiprocessor unit is in the activation state, the memory access intensive load in the memory access stream multiprocessor unit is task executed until all the memory access intensive loads in the memory access stream multiprocessor unit are executed to obtain the memory access completion state; wherein, the task execution state includes: the resource-saving state of the computational intensive load and the memory access completion state of the memory access intensive load.

[0013] In a feasible implementation manner, after activating and controlling the stream multi-processor unit corresponding to the memory access SM core to obtain the task execution status of the current task load, the method further includes: if the task execution status includes the resource saving status and the memory access completion status, then controlling all the stream multi-processor units to be readjusted to the activation state; based on the activation state, controlling the stream multi-processor unit to wait for receiving the next round of thread task loads assigned by the thread block scheduling unit.

[0014] In a second aspect, an embodiment of the present application also provides a GPGPU stream multi-processor resource saving scheduling device, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor so that the at least one processor can execute a GPGPU stream multi-processor resource saving scheduling method described in any of the above-mentioned embodiments.

[0015] In the third aspect, an embodiment of the present application also provides a non-volatile computer storage medium, characterized in that the storage medium is a non-volatile computer-readable storage medium, and the non-volatile computer-readable storage medium stores at least one program, each of which includes instructions, and when the instructions are executed by a terminal, the terminal executes a GPGPU stream multi-processor resource scheduling method described in any of the above embodiments.

[0016] The present application provides a GPGPU stream multiprocessor resource saving scheduling method, device and medium. Compared with the prior art, the embodiments of the present application have the following beneficial technical effects: 1. Resource optimization: By activating and suspending processing, this method can dynamically adjust the use of streaming multiprocessor units according to the task type, thereby optimizing the use efficiency of hardware resources.

[0017] 2. Reduce power consumption: Through power saving control (putting certain SM cores into a low-power state), this method helps reduce the overall power consumption of the GPU, which is important for improving energy efficiency and extending device life.

[0018] 3. Improve performance: By classifying the task load and allocating memory access-intensive loads to specific SM cores, the efficiency of task processing and the overall performance of the GPU can be improved.

[0019] 4. Load balancing: By monitoring the period and classifying the task loads of the SM cores, a balanced distribution of different types of loads can be achieved, thus avoiding overloading of some units while idling others.

[0020] 5. Dynamic response: This method can dynamically adjust the state of memory access to the SM core according to the current computational execution state of the SM core, allowing the GPU to flexibly respond to different computing requirements.

[0021] 6. Reduce delays: By optimizing the scheduling strategy, task execution delays caused by resource conflicts or insufficient resources can be reduced.

[0022] 7. Enhanced scalability: This method can adapt to GPGPU systems of different sizes, provides a general resource management method, and enhances the scalability of the system.

[0023] 8. Simplified programming model: This scheduling approach may provide developers with a more transparent and efficient programming model because the work of task scheduling and resource management is handled automatically.

[0024] 9. Improve stability: By precisely controlling the state of the SM core, system instability caused by resource competition can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings: Figure 1 A flow chart of a GPGPU stream multiprocessor resource saving scheduling method provided in an embodiment of the present application; Figure 2 A schematic diagram of a low power consumption architecture provided in an embodiment of the present application; Figure 3 A schematic diagram of a suspend source flow provided in an embodiment of the present application; Figure 4 A schematic diagram of the operation effect of a resource-saving scheduling method provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of a GPGPU stream multiprocessor resource-saving scheduling device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0027] The present application embodiment provides a GPGPU stream multiprocessor resource saving scheduling method, such as Figure 1 As shown, the GPGPU stream multiprocessor resource saving scheduling method specifically includes steps S101-S104: It should be noted that Figure 2 A schematic diagram of a low power consumption architecture provided in an embodiment of the present application is shown as follows: Figure 2As shown, a GPGPU streaming multiprocessor resource-saving scheduling system, wherein the low-power architecture includes a thread block scheduling unit, a streaming multiprocessor unit (SM), a dynamic monitoring unit, an operation reduction unit, and a gating enabling unit.

[0028] The thread block scheduling unit can distribute the threads received from the Host to the streaming multiprocessor units. Each thread block can only be executed on one streaming multiprocessor unit. A streaming multiprocessor can execute multiple thread blocks at the same time. For the suspended streaming multiprocessor, even if there are still remaining computing resources, thread blocks will not be allocated again. Only after a round of suspension-resource saving process is completed, the thread will be reallocated.

[0029] The stream multiprocessing unit contains an improved warp scheduler and pipeline. Once a stream multiprocessor enters the suspended state, the warp scheduler will continue to issue warps for execution until a warp needs to execute a memory access instruction. These warps will be paused to limit requests to the memory system. Once all warps are paused, the stream multiprocessor core is truly in a suspended state.

[0030] The dynamic monitoring unit can dynamically monitor the load situation in the current stream multiprocessor, determine whether it is a memory access intensive load or a computation intensive load, and send the determination result to the operation reduction unit.

[0031] When the reduction unit is running, the memory access intensive load will be suspended, and the computation intensive load will be run first. After the computation of the computation intensive load is completed, the streaming multiprocessor that executes the computation intensive load will be enabled to enter the power saving state, and the power gating or clock gating will be turned off as needed.

[0032] The gate enable unit has clock gating and power gating enable, which can dynamically start and shut down the clock and power of the streaming multiprocessor.

[0033] As a feasible implementation method, the thread block scheduling unit can distribute the threads received from the Host (server) to the stream multiprocessor unit. Each thread block can only be executed on one stream multiprocessor unit. One stream multiprocessor can execute multiple thread blocks at the same time. For the suspended stream multiprocessor, even if there are still remaining computing resources, thread blocks will not be allocated again. Only after a round of suspension-resource saving process is completed, the thread will be reallocated. Among them, the dynamic monitoring unit can dynamically monitor the load situation in the current stream multiprocessor, determine whether it is a memory access intensive load or a computing intensive load, and send the judgment result to the operation reduction unit. Among them, the operation reduction unit will suspend the memory access intensive load, give priority to running the computing intensive load, and then after the computing intensive load is calculated, the stream multiprocessor that executes the computing intensive load will be enabled to enter the resource saving state, and the power gating or clock gating will be turned off as required. Among them, the gating enable unit has clock gating and power gating enable, which can dynamically turn on and off the clock and power of the stream multiprocessor to put the stream multiprocessor into the resource saving state.

[0034] S101 , performing activation processing on all the streaming multiprocessor units connected to the thread block scheduling unit.

[0035] Specifically, the received thread task load needs to be transmitted to the thread block scheduling unit through the host server. Each thread block in the thread block scheduling unit can only be executed on one stream multiprocessor unit, and one stream multiprocessor can execute multiple thread blocks at the same time.

[0036] Furthermore, before distributing the task load to the streaming multiprocessor unit through the thread block scheduling unit, the SM cores in all the streaming multiprocessor units are controlled to remain in an activated state.

[0037] In one embodiment, Figure 3 A schematic diagram of a suspending resource flow provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, taking 6 stream multiprocessors (SMs) as an example, in step 0, all SM cores are in an activated state, so that task loads can be distributed to the stream multiprocessor units through the thread block scheduling unit in the subsequent process.

[0038] S102, the task load assigned by the thread block scheduling unit is transmitted to the stream multiprocessor unit, and the task load is classified into intensive loads through the monitoring cycle and SM core in the stream multiprocessor unit, and the memory access SM core corresponding to the memory access intensive load is determined. The intensive load includes: computing intensive load and memory access intensive load.

[0039] Specifically, the monitoring cycle of the SM core in each stream multiprocessor unit is configured to obtain a clock monitoring cycle, wherein the unit number of the clock monitoring cycle is associated with the task attribute of the thread task load.

[0040] Furthermore, the memory access intensive load is monitored for the period of memory access stalls through the clock monitoring cycle to obtain the memory access stall period, wherein the memory access stall period is the total number of periods during which the SM core is monitored to stall due to memory access.

[0041] Further, when the memory access pause period is greater than one third of the clock monitoring period, the task load currently being run by the SM core is determined as a memory access intensive load.

[0042] Furthermore, the number of running SM cores corresponding to the memory access intensive load is identified and collected.

[0043] Furthermore, when the number of running SM cores is greater than one-half of the number of activated SM cores, the running SM cores corresponding to the number of running SM cores are set as memory access SM cores, wherein the memory access SM cores only include memory access intensive loads.

[0044] Further, the streaming multiprocessor unit corresponding to the memory access SM core is determined as a memory access streaming multiprocessor unit for processing memory access intensive loads.

[0045] In one embodiment, Figure 2 As shown in the figure, in step 1, the loads running on three SM cores are judged as memory access intensive loads and enter the suspended state and suspend execution. The algorithm for determining whether an SM is a memory access intensive load is as follows: each SM core has a monitoring cycle, which is set to 15,000 clock cycles. During these 15,000 clock cycles, the total number of cycles in which the SM core is paused due to memory access is monitored. When the number of paused clock cycles (memory access pause cycle) is greater than one-third of the monitoring cycle (clock monitoring cycle), the load running on the core is judged as a memory access intensive load. When the number of SM cores judged as memory access intensive loads (the number of running SM cores) is greater than or equal to one-half of the activated SM cores, the SM cores judged as memory access intensive loads (memory access SM cores: SM4, SM5) are suspended.

[0046] In one embodiment, the following core code is included: FunctionMONITOR for each (Monitor cycle) do (here do means for loop) for (i=1; i<= Counter_active_state; i++) do (here do means for loop) if stall cycle(i)>mon cycle / 3 then Counter_mem++. stall cycle(i) = 0. end if end for if Counter_mem>= Counter_active_state / 2 then select a COREj that is in active_state. COREj switch from active_state to suspended state. Counter_ suspended _state ++. Counter_active_state --. Counter_mem -- . end if end for end function Among them, the key words in the code are explained: Monitor cycle: The monitoring cycle is set according to the current SM core's execution cycle of a thread block (the number of cycles from when the thread block is distributed to the SM core to when it is executed, which is an estimate).

[0047] Stall cycle: The total number of cycles in which the SM core is stalled due to memory access operations.

[0048] Counter_active_state: indicates the number of activated SM cores. The active state indicates that the computing core can run normally.

[0049] Counter_ suspended state: Indicates the number of SM cores that have entered the suspended state.

[0050] Counter_mem: Indicates the number of SM cores that are judged to be memory access intensive.

[0051] Through the code implementation of the above dynamic monitoring unit algorithm, the following operations can be looped in each detection cycle (one clock cycle in hardware implementation): the memory access pause cycle of each activated computing core is judged. When the total memory access pause cycle of a computing core is greater than 1 / 3 of the monitoring cycle, the current computing core is judged to be a memory access intensive load, and the memory access intensive load counter (Counter_mem) is increased by 1. When it is judged that the memory access intensive load counter is greater than or equal to 1 / 2 of the number of activated computing cores, an activated computing core is selected to enter the suspended state, and the corresponding suspended state counter is increased by 1, the activated state counter is reduced by 1, and the memory intensive load counter is reduced by 1.

[0052] S103, suspend the stream multiprocessor unit corresponding to the memory access SM core, and perform computation control on the computing SM cores in the remaining stream multiprocessor units to obtain the computation execution state of the computing SM core, wherein the computation execution state includes: activation state and resource saving state.

[0053] Specifically, the stream multiprocessor unit includes an improved Warp scheduler and a pipeline module. When the stream multiprocessor unit is determined to be a memory access stream multiprocessor unit for processing memory access intensive loads, the Warp scheduler in the memory access stream multiprocessor unit is subjected to execution identification processing related to the memory access instruction. When the memory access instruction is identified, the Warp scheduler related to the memory access instruction is suspended.

[0054] Furthermore, based on the Warp scheduler in the memory access stream multiprocessor unit being in a paused state, the memory access stream multiprocessor unit is controlled to enter a suspended state.

[0055] Further, the remaining stream multiprocessor units are determined as computation stream multiprocessor units, wherein the computation stream multiprocessor units only contain computationally intensive loads.

[0056] Furthermore, it is also necessary to perform computation control on the computation-intensive load through the computation stream multiprocessor unit. If the computation SM core in the computation stream multiprocessor unit is in the continuous computation of the computation-intensive load, the computation SM core is determined to be in the activated state. If the computation SM core in the computation stream multiprocessor unit has completed the computation of the computation-intensive load, the computation SM core is determined to be in the power saving state.

[0057] Furthermore, the computation execution status of the computation SM cores in all computation stream multiprocessor units is continuously monitored.

[0058] In one embodiment, if Figure 2As shown in the figure, in step 2, two computing SM cores (SM2, SM3) of computing intensive loads have completed the calculation and are enabled to enter the power saving state (the operation of computing intensive loads has been completed). The two computing SM cores (SM0, SM1) are still in the continuous operation of computing intensive loads, so the computing SM cores are determined to be in the activated state, and the remaining two memory access stream multiprocessor units (SM4, SM5) are still in the suspended state.

[0059] In one embodiment, the following core code is included: Function REDUCE while Counter_suspended_state>0 do if COREi finished then power gate COREi select a COREj that is in suspended state. COREj switch from suspended to active state. Counter_ suspended _state --. end if end while end Function Among them, the key words in the code are explained: COREi: indicates an activated SM core.

[0060] COREj: indicates a SM core in suspended state.

[0061] Through the above code implementation of the run reduction unit algorithm, when the suspended state counter value > 0, it means that there is a suspended memory access intensive load. Under this judgment condition, when any computation-intensive computing core is executed, its corresponding clock core power gating enable is turned off to enter the power saving mode, and a suspended access intensive computing core is selected to enter the active state to continue running. In this way, the access intensive load can be executed separately, which not only reduces the impact on the computation-intensive load, but also reduces the competition between the access intensive loads.

[0062] S104 . According to the computing execution state of the computing SM core, activation control is performed on the stream multiprocessor unit corresponding to the memory access SM core to obtain the task execution state of the current task load.

[0063] Specifically, if the computing execution state of the computing SM core is in an activated state, activation control is not performed on the streaming multiprocessor unit corresponding to the memory access SM core.

[0064] If the computing execution states of the computing SM cores are all in the resource-saving state, the memory access stream multiprocessor unit corresponding to the memory access SM core is activated and controlled.

[0065] When the memory access SM core in the memory access stream multiprocessor unit is in an activated state, task execution processing is performed on the memory access intensive load in the memory access stream multiprocessor unit until all memory access intensive loads in the memory access stream multiprocessor unit are completed and a memory access completion state is obtained.

[0066] The task execution status includes: the resource saving status of the computationally intensive load and the memory access completion status of the memory access intensive load.

[0067] In one embodiment, Figure 2 As shown, in step 3, the computing SM cores (SM0, SM1, SM2, SM3) of all computing-intensive task loads have completed the operation and become the power-saving state, and then the previously suspended memory access intensive loads are activated one by one for execution, that is, the stream multiprocessor units corresponding to the memory access SM cores (SM4, SM5) are activated and controlled, and all become activated. Then the memory access intensive loads perform task execution processing until all memory access intensive loads in the memory access stream multiprocessor units are completed and the memory access completion state is obtained, that is, this round of thread task load processing is completed.

[0068] Furthermore, if the task execution status includes both the resource saving status and the memory access completion status, all stream multiprocessor units are controlled to be readjusted to the activation status. Finally, based on the activation status, the stream multiprocessor units are controlled to wait for receiving the next round of thread task loads assigned by the thread block scheduling unit.

[0069] In one embodiment, Figure 4 A schematic diagram of the operation effect of a resource-saving scheduling method provided in an embodiment of the present application is shown in FIG. Figure 4As shown, during the execution process, after judging the number of square-inch cycles of the SM computing core, based on the preset operating cycle value, the proportional relationship between the operating cycles is used to determine the computing-intensive load or memory access-intensive load to which the different SM computing cores belong, and then corresponding activation operation or suspension waiting and other processing are performed. That is, the SM core that completes the computing-intensive load is power-gated or clock-gated to reduce static power consumption and dynamic power consumption, thereby improving energy efficiency. In this way, computing-intensive tasks and memory access-intensive tasks are executed separately, which reduces the impact of memory access-intensive tasks on computing-intensive tasks, reduces memory contention, and at the same time, after the calculation is completed, the stream multiprocessor enters a power-saving state, saving power.

[0070] In addition, the embodiment of the present application also provides a GPGPU stream multiprocessor resource saving scheduling device, such as Figure 5 As shown, the GPGPU stream multiprocessor resource saving scheduling device 500 specifically includes: At least one processor 501. And a memory 502 in communication with the at least one processor 501. The memory 502 stores instructions that can be executed by the at least one processor 501, so that the at least one processor 501 can execute: Activate all the stream multiprocessor units connected to the thread block scheduling unit; The task load assigned by the thread block scheduling unit is transmitted to the stream multiprocessor unit, and the task load is classified into intensive loads through the monitoring cycle and SM core in the stream multiprocessor unit to determine the memory access SM core corresponding to the memory access intensive load; wherein the intensive load includes: computing intensive load and memory access intensive load; Suspend the stream multiprocessor unit corresponding to the memory access SM core; and perform computation control on the computing SM cores in the remaining stream multiprocessor units to obtain the computation execution state of the computing SM cores; wherein the computation execution state includes: activation state and resource saving state; According to the computing execution state of the computing SM core, the stream multiprocessor unit corresponding to the memory access SM core is activated and controlled to obtain the task execution state of the current task load.

[0071] The embodiment of the present application can identify and suspend memory access intensive loads on SM (streaming multiprocessor, which is an important computing unit in NVIDIA Volta and later processor architectures), give priority to computing compute-intensive loads, and when all compute-intensive loads are completed, activate the thread blocks of the suspended memory access intensive loads for computing, and at the same time, power gate or clock gate enable the SM core that completes the compute-intensive load, thereby reducing static power consumption and dynamic power consumption, thereby improving energy efficiency. In this way, compute-intensive tasks and memory-access intensive tasks are executed separately, reducing the impact of memory-access intensive tasks on compute-intensive tasks, reducing memory contention, and at the same time, the streaming multiprocessor enters a power-saving state after the calculation is completed, thereby saving power consumption.

[0072] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0073] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0074] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0075] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0076] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0077] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0078] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0079] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0080] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0081] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0082] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the specification of the present application.

Claims

1. A GPGPU streaming multiprocessor resource saving scheduling method, characterized in that: The method comprises: Activate all the stream multiprocessor units connected to the thread block scheduling unit; The task load assigned by the thread block scheduling unit is transmitted to the stream multiprocessor unit, and the task load is subjected to a category judgment of a relevant intensive load through a monitoring cycle and an SM core in the stream multiprocessor unit, so as to determine a memory access SM core corresponding to a memory access intensive load; wherein the intensive load includes: a computationally intensive load and the memory access intensive load; Suspending the stream multiprocessor unit corresponding to the memory access SM core; and performing computation control on the computation SM cores in the remaining stream multiprocessor units to obtain the computation execution state of the computation SM core; wherein the computation execution state includes: an activation state and a resource saving state; According to the computing execution state of the computing SM core, activation control is performed on the stream multiprocessor unit corresponding to the memory access SM core to obtain the task execution state of the current task load.

2. A GPGPU streaming multiprocessor resource saving scheduling method according to claim 1, characterized in that: The streaming multiprocessor units connected to the thread block scheduling units are fully activated, including: The received thread task load is transmitted to the thread block scheduling unit through the host server; wherein each thread block in the thread block scheduling unit can only be executed on one stream multiprocessor unit, and one stream multiprocessor can execute multiple thread blocks at the same time; Before distributing the task load to the stream multiprocessor unit through the thread block scheduling unit, all SM cores in the stream multiprocessor unit are controlled to remain in an activated state.

3. A GPGPU streaming multiprocessor resource saving scheduling method according to claim 1, characterized in that: Before determining the category of the task load related to the intensive load through the monitoring cycle and the SM core in the stream multiprocessor unit and determining the memory access SM core corresponding to the memory access intensive load, the method further includes: Performing a monitoring cycle configuration on the SM core in each of the stream multiprocessor units to obtain a clock monitoring cycle; wherein the unit number of the clock monitoring cycle is associated with a task attribute of a thread task load; The memory access intensive load is monitored for cycles related to memory access pauses through the clock monitoring cycle to obtain a memory access pause cycle; wherein the memory access pause cycle is the total number of cycles during which the SM core is monitored to pause due to memory access.

4. A GPGPU streaming multiprocessor resource saving scheduling method according to claim 3, characterized in that: By using the monitoring cycle and the SM core in the stream multiprocessor unit, the task load is subjected to a category judgment related to the intensive load, and a memory access SM core corresponding to the memory access intensive load is determined, specifically including: When the memory access pause period is greater than one third of the clock monitoring period, determining the task load currently being run by the SM core as the memory access intensive load; Identify and collect the number of running SM cores corresponding to the memory access intensive load; When the number of the running SM cores is greater than one half of the number of the SM cores in the activated state, the running SM cores corresponding to the number of the running SM cores are used as the memory access SM cores; wherein the memory access SM cores only include the memory access intensive load; A streaming multiprocessor unit corresponding to the memory access SM core is determined as a memory access streaming multiprocessor unit for processing the memory access intensive load.

5. A GPGPU streaming multiprocessor resource saving scheduling method according to claim 4, characterized in that: The stream multiprocessor unit corresponding to the memory access SM core is suspended, specifically comprising: The stream multiprocessor unit includes an improved Warp scheduler and a pipeline module; When the stream multiprocessor unit is determined to be a memory access stream multiprocessor unit for processing the memory access intensive load, performing execution identification processing on the Warp scheduler in the memory access stream multiprocessor unit regarding the memory access instruction; When the memory access instruction is identified, pausing the Warp scheduler related to the memory access instruction; Based on the Warp scheduler in the memory access stream multiprocessor unit being in a paused state, the memory access stream multiprocessor unit is controlled to enter a suspended state.

6. A GPGPU streaming multiprocessor resource saving scheduling method according to claim 1, characterized in that: The calculation SM core in the remaining stream multiprocessor unit is operated and controlled to obtain the calculation execution state of the calculation SM core, specifically including: Determining the remaining stream multiprocessor units as computational stream multiprocessor units; wherein the computational stream multiprocessor units only contain the computationally intensive load; Performing computational control on the computationally intensive load by means of the computational stream multiprocessor unit; If the computing SM core in the computing stream multiprocessor unit is in the continuous operation of the computing intensive load, determining the computing SM core as an activated state; If the computing SM core in the computing stream multiprocessor unit has completed the operation of the computing intensive load, the computing SM core is determined to be in a power saving state; The computing execution status of the computing SM cores in all the computing stream multiprocessor units is continuously monitored.

7. A GPGPU streaming multiprocessor resource saving scheduling method according to claim 1, characterized in that: According to the computing execution state of the computing SM core, the stream multiprocessor unit corresponding to the memory access SM core is activated and controlled to obtain the task execution state of the current task load, specifically including: If the computing execution state of the computing SM core has the activation state, then activation control is not performed on the stream multiprocessor unit corresponding to the memory access SM core; If the computing execution states of the computing SM cores are all in the resource-saving state, activating and controlling the memory access stream multiprocessor unit corresponding to the memory access SM core; When the memory access SM core in the memory access stream multiprocessor unit is in an activated state, performing task execution processing on the memory access intensive load in the memory access stream multiprocessor unit until all the memory access intensive loads in the memory access stream multiprocessor unit are executed, and a memory access completion state is obtained; The task execution status includes: a resource saving status of the computationally intensive load and a memory access completion status of the memory access intensive load.

8. The GPGPU streaming multiprocessor resource saving scheduling method according to claim 1, characterized in that: After activating and controlling the stream multiprocessor unit corresponding to the memory access SM core to obtain the task execution status of the current task load, the method further includes: If the task execution status includes the resource saving status and the memory access completion status, controlling all the stream multiprocessor units to readjust to the activation status; Based on the activation state, the streaming multiprocessor unit is controlled to wait for receiving a next round of thread task loads allocated by the thread block scheduling unit.

9. A GPGPU streaming multiprocessor resource-saving scheduling device, characterized in that: The device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, so that the at least one processor can execute the GPGPU streaming multi-processor resource scheduling method according to any one of claims 1-8.

10. A non-volatile computer storage medium, characterized in that: The storage medium is a non-volatile computer-readable storage medium, and the non-volatile computer-readable storage medium stores at least one program, each of which includes instructions, and when the instructions are executed by a terminal, the terminal executes a GPGPU stream multi-processor resource scheduling method according to any one of claims 1-8.

Images