Stream processor dynamic allocation method and system for multi-task core grain architecture GPU (Graphics Processing Unit)

Abstract

The invention discloses a multi-task core grain architecture GPU-oriented stream processor dynamic allocation method and system. The method comprises the following steps: equally dividing the number of stream processors for all tasks; calculating a task bandwidth demand and an actual available bandwidth, obtaining a mismatching degree according to the task bandwidth demand and the actual available bandwidth, and then redistributing the number of stream processors of each task; calculating the available memory bandwidth of each task in different spatial arrangement modes, and selecting an optimal spatial arrangement mode according to the available memory bandwidth; the method comprises the following steps: determining an idle stream processor and an unfinished task, calculating the kernel residual execution time of the idle stream processor and the stealing overhead of the unfinished task, and stealing the idle stream processor according to the kernel residual execution time and the stealing overhead. According to the method and the device, the problem of allocation of stream processor resources among a plurality of tasks can be solved, extensible high throughput and high resource utilization rate in the multi-task core grain architecture GPU are realized, and the method and the device can be widely applied to the technical field of computers.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method and system for dynamic allocation of stream processors for multi-tasking chip architecture GPUs. Background Technology

[0002] Existing multi-task GPU stream processor allocation methods include static equal allocation and dynamic hill-climbing algorithm-based allocation. However, static equal allocation methods suffer from problems such as over-allocation or insufficient resources, lack of flexibility, and difficulty in quickly adapting to different tasks; while dynamic hill-climbing algorithm-based allocation methods suffer from problems such as affecting the performance of multi-task systems and difficulty in keeping up with changes in task search characteristics.

[0003] Emerging chip-level GPU architectures introduce cross-chip remote access, a non-uniform memory access (NUMA) architecture that requires consideration of not only the quantity dimension but also the spatial dimension when allocating stream processor resources. However, previous allocation techniques lacked consideration of the spatial arrangement of stream processors, making it difficult to achieve optimal allocation in multi-tasking chip-level GPU architectures. Summary of the Invention

[0004] The main objective of this application is to propose a method and system for dynamic allocation of stream processors for multi-task chip architecture GPUs, which can solve the problem of allocating stream processor resources among multiple tasks and achieve scalable high throughput and high resource utilization in multi-task chip architecture GPUs.

[0005] To achieve the above objectives, one aspect of this application proposes a method for dynamic allocation of stream processors for multi-task chip architecture GPUs, comprising the following steps: Distribute the number of stream processors equally among all tasks to complete the initial allocation; Calculate the task bandwidth requirement and the actual available bandwidth, obtain the mismatch degree based on the task bandwidth requirement and the actual available bandwidth, and then reallocate the number of stream processors for each task based on the mismatch degree; Calculate the available memory bandwidth for each task under different spatial arrangement modes, and select the optimal spatial arrangement mode based on the available memory bandwidth. The spatial arrangement modes include local priority mode and global equal distribution mode. Idle stream processors and unfinished tasks are identified, the remaining kernel execution time of the idle stream processor and the stealing overhead of the unfinished tasks are calculated, and then the idle stream processor is stolen based on the remaining kernel execution time and the stealing overhead.

[0006] In some embodiments, calculating the task bandwidth requirement and the actual available bandwidth, and obtaining the mismatch degree based on the task bandwidth requirement and the actual available bandwidth, specifically includes: Establish the task bandwidth requirement function and the actual available bandwidth function; Calculate the task bandwidth requirement corresponding to each task based on the task bandwidth requirement function; The actual available bandwidth corresponding to each task is calculated based on the actual available bandwidth function. The mismatch degree corresponding to each task is obtained by subtracting the bandwidth requirement of the task from the actual available bandwidth.

[0007] In some embodiments, the reallocation of the number of stream processors for each task based on the mismatch degree specifically includes: If the mismatch degree of the task is positive, the number of stream processors corresponding to the task is reduced to a preset number; if the mismatch degree of the task is negative, the number of stream processors corresponding to the task is increased to a preset number. When the mismatch degree of the task is negative and the current number of stream processors of the task is less than the initial number, the number of stream processors corresponding to the task is increased to the preset number, and the number of stream processors corresponding to the task with the largest mismatch degree is reduced. Determine the number of thread blocks for the task whose number of stream processors is increased, and allocate partial delay to the task based on the number of thread blocks.

[0008] In some embodiments, calculating the available memory bandwidth of each task under different spatial arrangement patterns specifically includes: The spatial arrangement mode of the stream processor is determined to be the local priority mode; Under the initial allocation, each of the tasks is divided into compute-intensive tasks or memory-intensive tasks; Calculate the first available memory bandwidth for the memory-intensive task in the local priority mode; Calculate the second available memory bandwidth for the memory-intensive task under the global equalization mode; Calculate the third available memory bandwidth for the computationally intensive task in the globally evenly distributed mode.

[0009] In some embodiments, selecting the optimal spatial arrangement pattern based on the available memory bandwidth specifically includes: When the second available memory bandwidth of the memory-intensive task is greater than the first available memory bandwidth, and the third available memory bandwidth of the compute-intensive task is greater than or equal to the task bandwidth requirement, the spatial arrangement mode is selected as the global even distribution mode. When the second available memory bandwidth of the memory-intensive task is less than or equal to the first available memory bandwidth, or the third available memory bandwidth of the compute-intensive task is less than the task bandwidth requirement, the spatial arrangement mode is selected as the local priority mode.

[0010] In some embodiments, determining the idle stream processor and the unfinished tasks, and calculating the remaining kernel execution time of the idle stream processor and the stealing overhead of the unfinished tasks, specifically includes: Determine whether the thread block launch queue of each task is empty, and determine the task kernel completion edge; Based on the completion edge of the task kernel, check whether there is an idle stream processor, and obtain the idle stream processor; Determine whether the thread block emission queue of each task contains unemitted thread blocks to obtain the unfinished tasks; Calculate the task to which the idle stream processor belongs, and the remaining kernel execution time for the current kernel; Calculate the theft cost of the unfinished task.

[0011] In some embodiments, the step of stealing the idle stream processor based on the remaining kernel execution time and the stealing overhead specifically includes: The idle stream processor is stolen when the stealing overhead is less than the remaining kernel execution time; After the theft is completed, the idle stream processor is preempted, and a new kernel thread block of the task to which the idle stream processor belonged before the theft is injected.

[0012] To achieve the above objectives, another aspect of this application proposes a dynamic allocation system for stream processors in a multi-tasking GPU architecture, comprising: The initial allocation module is used to evenly distribute the number of stream processors among all tasks, completing the initial allocation; The dynamic allocation module is used to calculate the task bandwidth requirement and the actual available bandwidth, obtain the mismatch degree based on the task bandwidth requirement and the actual available bandwidth, and then reallocate the number of stream processors for each task based on the mismatch degree. The mode selection module is used to calculate the available memory bandwidth of each task under different spatial arrangement modes, and select the optimal spatial arrangement mode according to the available memory bandwidth. The spatial arrangement modes include local priority mode and global equal distribution mode. A stream processor stealing module is used to determine idle stream processors and unfinished tasks, calculate the remaining kernel execution time of the idle stream processor and the stealing overhead of the unfinished tasks, and then steal the idle stream processor based on the remaining kernel execution time and the stealing overhead.

[0013] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0014] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method.

[0015] The embodiments of this application include at least the following beneficial effects: The method and system for dynamic allocation of stream processors for multi-task chip architecture GPUs of this application first evenly distributes the number of stream processors among all tasks to complete the initial allocation; then, it calculates the task bandwidth requirements and actual available bandwidth, obtains the mismatch degree based on the task bandwidth requirements and actual available bandwidth, and then reallocates the number of stream processors for each task according to the mismatch degree; then, it calculates the available memory bandwidth of each task under different spatial arrangement modes, selects the optimal spatial arrangement mode based on the available memory bandwidth, and the spatial arrangement modes include local priority mode and global equal distribution mode; finally, it determines the idle stream processors and unfinished tasks, calculates the remaining kernel execution time of the idle stream processors and the stealing overhead of the unfinished tasks, and then steals the idle stream processors according to the remaining kernel execution time and the stealing overhead. This application, on the one hand, further measures the bandwidth mismatch by real-time monitoring and online analysis of task bandwidth requirements and actual available bandwidth, thereby dynamically adjusting the allocation of stream processor resources, including changing the number and spatial arrangement of stream processors among different tasks, to bridge the gap between the bandwidth requirements and available bandwidth of all tasks; on the other hand, by identifying idle stream processors at the task kernel completion edge and implementing a stream processor stealing mechanism, it improves the utilization of stream processor resources, solves the problem of stream processor resource allocation among multiple tasks, and achieves scalable high throughput and high resource utilization in multi-task chip architecture GPUs. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments of this application are described below. It should be understood that the drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions in this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating the steps of a method for dynamically allocating stream processors for a multi-tasking GPU chip architecture according to an embodiment of this application; Figure 2 This is a schematic diagram illustrating the process of dynamically allocating the number of multi-task stream processors according to one embodiment of this application; Figure 3 A flowchart illustrating the selection of a multi-task stream processor spatial arrangement mode according to an embodiment of this application; Figure 4 This is a schematic diagram illustrating the process of stream processor theft provided in one embodiment of this application; Figure 5 This is a schematic diagram of the structure of a stream processor dynamic allocation system for a multi-tasking chip architecture GPU provided in one embodiment of this application; Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0020] Graphics processing units (GPUs) have become a compelling and crucial computing platform in cloud devices and data centers. GPU-based computing acceleration is a major driver of performance improvements in high-performance computing (HPC) systems, large-scale cloud infrastructures, and data analytics applications. In recent years, the number of streaming multiprocessors (SMs) in GPUs has been continuously increasing to provide powerful computing capabilities. As semiconductor process nodes continue to evolve, transistors are shrinking to near physical limits, and Moore's Law, which pursues economic efficiency, is slowing down, bringing challenges in yield, cost, and development cycles. This forces the search for more cost-effective technologies to meet the industry's growing demand for GPU performance, break through current manufacturing area limitations, and drive continuous improvement in GPU computing power.

[0021] Chiplet technology utilizes advanced packaging and signal transmission technologies to integrate multiple GPU chips into a larger, lower-cost computing system via silicon interposers or organic substrates. A key challenge for chiplet-based GPUs is fully utilizing abundant computing and storage resources. A single task often leads to underutilization of GPU resources. To improve utilization, GPUs are spatially partitioned to allow multiple tasks to be deployed simultaneously on a single GPU. Spatial multitasking offers a potential opportunity to efficiently utilize the resources of chiplet-based GPUs. However, traditional multitasking in chiplet-based GPUs does not intelligently maximize resource utilization and lacks attention to bandwidth matching for individual tasks. Furthermore, memory bandwidth in chiplet architecture depends not only on the storage system but also on the interconnect links between chiplets, introducing spatial arrangement challenges.

[0022] The allocation of stream processor resources in a multi-tasking chip architecture GPU has a significant impact on overall performance, involving the utilization of computing and storage resources. It is necessary to fully consider the architecture and task characteristics during GPU operation and dynamically adjust the stream processor allocation scheme to improve the overall execution efficiency of the GPU.

[0023] Existing multi-task GPU stream processor allocation methods include static equal allocation and dynamic hill-climbing algorithm for optimal allocation.

[0024] Static equal allocation typically divides stream processor resources into multiple groups of equal size, allocating a fixed number of stream processor resource groups to each task before task deployment. (1) However, the allocation granularity of static equal allocation is usually too large, resulting in over-allocation or slight shortage of resources for a particular task. (2) Static equal allocation also lacks flexibility, usually requiring a comprehensive overall assessment of the task's running phase, and cannot adapt to the task's running characteristics in real time, making it difficult to achieve optimal allocation throughout the entire running phase. (3) Static equal allocation usually relies on the programmer's subjective decision-making, requiring extensive program modifications and offline feature analysis, making it difficult to quickly adapt to different tasks.

[0025] Dynamic hill-climbing algorithms for optimal resource allocation typically iteratively adjust resource allocation at a certain granularity during runtime, searching for the optimal allocation by evaluating the benefits of these adjustments. However, iterative searches require lengthy analysis time to reach convergence, and the search process can impact the performance of multi-tasking systems. Furthermore, hill-climbing algorithms are generally suitable for tasks with fluctuating runtime characteristics; their search speed struggles to keep pace with these changes.

[0026] Emerging chip-level GPU architectures introduce cross-chip remote access, a non-uniform memory access (NUMA) architecture. This means that stream processor resource allocation needs to consider not only the quantity dimension but also the spatial dimension. Previous allocation techniques lacked consideration of the spatial arrangement of stream processors, making it difficult to achieve optimal allocation in multi-tasking chip-level GPU architectures.

[0027] In view of this, this application proposes a dynamic allocation method for stream processors in a multi-task chip architecture GPU. First, the number of stream processors is evenly distributed among all tasks to complete the initial allocation. Next, the bandwidth requirements and actual available bandwidth of each task are calculated. Based on these requirements and available bandwidth, a mismatch degree is obtained, and the number of stream processors for each task is reallocated according to the mismatch degree. Then, the available memory bandwidth for each task under different spatial arrangement modes is calculated, and the optimal spatial arrangement mode is selected based on the available memory bandwidth. Spatial arrangement modes include local priority mode and global equal distribution mode. Finally, idle stream processors and unfinished tasks are determined, and the remaining kernel execution time of the idle stream processors and the stealing overhead of the unfinished tasks are calculated. Then, based on the remaining kernel execution time and the stealing overhead, the idle stream processors are stolen. This application, on the one hand, further measures the bandwidth mismatch by real-time monitoring and online analysis of task bandwidth requirements and actual available bandwidth, thereby dynamically adjusting the allocation of stream processor resources, including changing the number and spatial arrangement of stream processors among different tasks, to bridge the gap between the bandwidth requirements and available bandwidth of all tasks; on the other hand, by identifying idle stream processors at the task kernel completion edge and implementing a stream processor stealing mechanism, it improves the utilization of stream processor resources, solves the problem of stream processor resource allocation among multiple tasks, and achieves scalable high throughput and high resource utilization in multi-task chip architecture GPUs.

[0028] Reference Figure 1 , Figure 1 This is a flowchart illustrating the steps of a method for dynamically allocating stream processors for a multi-tasking GPU architecture according to an embodiment of this application. This application proposes a method for dynamically allocating stream processors for a multi-tasking GPU architecture, which may include, but is not limited to, the following steps S101 to S104: Step S101: Distribute the number of stream processors equally among all tasks to complete the initial allocation; Specifically, during the initial allocation phase, stream processor resources are evenly distributed among all tasks.

[0029] Step S102: Calculate the task bandwidth requirement and the actual available bandwidth. Based on the task bandwidth requirement and the actual available bandwidth, obtain the mismatch degree, and then reallocate the number of stream processors for each task according to the mismatch degree. In some alternative embodiments, the number of stream processors allocated among tasks is dynamically adjusted by measuring the task bandwidth requirements against the actual available bandwidth.

[0030] As an optional implementation, the step of calculating the task bandwidth requirement and the actual available bandwidth, and obtaining the mismatch degree based on the task bandwidth requirement and the actual available bandwidth, can be further divided into the following steps S1021 to S1024: Step S1021: Establish the task bandwidth requirement function and the actual available bandwidth function; Step S1022: Calculate the task bandwidth requirement for each task according to the task bandwidth requirement function; Step S1023: Calculate the actual available bandwidth for each task based on the actual available bandwidth function; Specifically, a periodic dynamic bandwidth allocation program runs on the GPU, and the hardware monitors the task's execution characteristics. The established functions for task bandwidth requirements and actual available bandwidth are shown in the following equations: ; ; in, For task bandwidth requirements, This represents the actual available bandwidth. This refers to the number of instructions executed per cycle by a single stream processor under ideal conditions, operating without interruption. To monitor the number of last-level cache accesses for every thousand instructions, The size of the cached line in bytes. For the execution frequency of the stream processor, The number of stream processors allocated; To monitor the proportion of cross-core last-level cache accesses of a task out of all its last-level cache accesses, For local available bandwidth, This represents the available bandwidth at the remote end.

[0031] Furthermore, locally available bandwidth and available bandwidth at remote ends The calculation formula is as follows: ; ; in, To monitor the hit rate of the last-level cache access of the obtained task, The bandwidth that the last-level buffer can provide. To monitor the missing rate of the last-level cache access of the obtained task, The bandwidth that a memory block can provide. The proportion of task memory block accesses to the total number of memory block accesses is monitored. The size of the microchip transmitted in each cycle of the cross-chip transmission link. To monitor the proportion of data transmitted through this cross-core link to the total data transmitted through that link. This refers to the operating frequency of the cross-core link.

[0032] Step S1024: Subtract the task bandwidth requirement from the actual available bandwidth to obtain the mismatch degree for each task.

[0033] Specifically, the bandwidth requirements of the computation task With actual available bandwidth The gap is used to measure the degree of mismatch. : .

[0034] As an optional implementation, the step of reallocating the number of stream processors for each task based on the mismatch degree can be further divided into the following steps S1025 to S1027: Step S1025: If the mismatch degree of the task is positive, reduce the number of stream processors corresponding to the task to a preset number; if the mismatch degree of the task is negative, increase the number of stream processors corresponding to the task to a preset number. Specifically, the number of stream processors allocated is adjusted by reducing the number of stream processors allocated to tasks with a positive mismatch (MD>0) and increasing the number of stream processors allocated to tasks with a negative mismatch (MD<0) until the expected mismatch of all tasks is either positive or negative.

[0035] Step S1026: When the mismatch degree of a task is negative and the current number of stream processors for the task is less than the initial number, increase the number of stream processors corresponding to the task to the preset number, and reduce the number of stream processors from the task corresponding to the largest mismatch degree. Specifically, detect if there are tasks with a negative mismatch degree MD and the number of stream processors allocated is less than the initial number (the number of stream processors at the initial allocation), increase the number of stream processors allocated to that task slightly, and reduce the number of stream processors allocated to the task with the largest mismatch degree MD.

[0036] Step S1027: Determine the number of thread blocks for the task that increases the number of stream processors, and allocate partial delay to the task based on the number of thread blocks.

[0037] Specifically, it checks whether the task that increases the number of allocated stream processors has enough thread blocks to utilize all allocated stream processors. If the number of remaining thread blocks is insufficient, it immediately increases the number of stream processors just enough to match the number of thread blocks, and the remaining allocation is delayed until the task executes the new kernel.

[0038] It should be noted that, as Figure 2 The diagram illustrates the process of dynamically allocating the number of stream processors for multiple tasks. This embodiment employs a bandwidth-aware dynamic adjustment method. By periodically monitoring task bandwidth requirements and actual available bandwidth, and using task bandwidth mismatch as the adjustment target, the number of stream processors allocated to each task is dynamically adjusted. This method can bridge the mismatch between task bandwidth requirements and available bandwidth as much as possible, improving the utilization of GPU computing and storage resources, thereby further enhancing the performance of GPU multi-task parallel processing. Compared to traditional methods, this method fully considers task operation characteristics and chip GPU architecture characteristics, flexibly adjusting stream processor allocation, and is more suitable for task combinations with rich chip architecture and operation characteristics.

[0039] Step S103: Calculate the available memory bandwidth of each task under different spatial arrangement modes, and select the optimal spatial arrangement mode based on the available memory bandwidth. Spatial arrangement modes include local priority mode and global equal distribution mode. In some alternative embodiments, the optimal spatial arrangement mode is selected by evaluating the mismatch between the bandwidth requirements and available bandwidth of tasks under different multitasking stream processor spatial arrangement modes. The selection is mainly between two spatial arrangement modes: (1) Local priority mode, which prioritizes the arrangement of stream processors assigned to the same task into the same core; (2) Global equal distribution mode, which evenly distributes the stream processors assigned to the same task into all cores.

[0040] As an optional implementation, the step of calculating the available memory bandwidth for each task under different spatial arrangement patterns can be further divided into the following steps S1031 to S1035: Step S1031: Determine the spatial arrangement mode of the stream processor as local priority mode; Step S1032: Under the initial allocation, each task is divided into compute-intensive tasks or memory-intensive tasks; Specifically, a local priority spatial arrangement pattern is initially adopted. Then, based on monitoring data under the initial stream processor allocation, tasks are classified into compute-intensive tasks and memory-intensive tasks according to whether their bandwidth requirements are less than the average allocated memory bandwidth. Compute-intensive tasks primarily consume CPU or stream processor computing resources during execution; these tasks are characterized by performing a large number of mathematical or logical calculations, while their memory bandwidth requirements are relatively low. Memory-intensive tasks primarily consume memory bandwidth during execution; these tasks are characterized by frequently accessing memory to perform data read and write operations, while their computing resource requirements are relatively low.

[0041] Step S1033: Calculate the first available memory bandwidth for memory-intensive tasks in local priority mode; Step S1034: Calculate the second available memory bandwidth for memory-intensive tasks in the global equal-sharing mode; Step S1035: Calculate the available memory bandwidth for computationally intensive tasks in the globally distributed mode.

[0042] Specifically, a periodic spatial layout mode selection procedure is run on the GPU, and the hardware monitors the task's runtime characteristics. The available memory bandwidth for memory-intensive tasks is evaluated and compared under two spatial layout modes. The first available memory bandwidth under the local priority mode is... Second available memory bandwidth in global equal distribution mode The calculation formula is as follows: ; ; in, The bandwidth that a single in-core storage module can provide. The total bandwidth increased for inter-chip interconnect links. The number of chips occupied by the stream processors allocated to the task in local priority mode. This represents the total number of core particles. The proportion of task-wide last-level cache accesses out of all last-level cache accesses. This represents the proportion of the bandwidth requirement of a task to the total bandwidth requirement of all tasks.

[0043] As an optional implementation, the step of selecting the optimal spatial arrangement pattern based on the available memory bandwidth can be further divided into the following steps S1036 or S1037: Step S1036: When the second available memory bandwidth of a memory-intensive task is greater than the first available memory bandwidth, and the third available memory bandwidth of a computationally intensive task is greater than or equal to the task bandwidth requirement, select the global even distribution mode for the spatial arrangement mode. Step S1037: When the second available memory bandwidth of a memory-intensive task is less than or equal to the first available memory bandwidth, or the third available memory bandwidth of a computationally intensive task is less than the task bandwidth requirement, select the local priority mode for spatial arrangement.

[0044] Specifically, the system assesses whether the available bandwidth for computationally intensive tasks in the globally distributed mode can meet their bandwidth requirements. The globally distributed mode is selected only if memory-intensive tasks can obtain greater available memory bandwidth in this mode, and the bandwidth requirements of computationally intensive tasks are met; otherwise, the local priority mode is selected. The system then compares the selected spatial arrangement mode with the current spatial arrangement mode; if they are different, a stream processor spatial arrangement mode switch is performed.

[0045] It should be noted that, as Figure 3 The diagram illustrates the process of selecting a multi-task stream processor spatial layout mode. The multi-task stream processor spatial layout mode selection method of this application aims to maximize the performance of memory-intensive tasks without sacrificing the performance of computationally intensive tasks. It evaluates and compares the available bandwidth of memory-intensive tasks under different spatial layout modes and assesses whether the bandwidth requirements of computationally intensive tasks can be met under different spatial layout modes, thereby selecting the optimal multi-task stream processor spatial layout mode. Compared to traditional methods, this method incorporates consideration of the stream processor's spatial characteristics, taking into account the performance of different types of tasks, and is more suitable for multi-task stream processor allocation under a chip-level architecture.

[0046] Step S104: Determine the idle stream processor and unfinished tasks, calculate the remaining kernel execution time of the idle stream processor and the stealing overhead of the unfinished tasks, and then steal the idle stream processor based on the remaining kernel execution time and the stealing overhead.

[0047] In some alternative embodiments, the multitasking stream processor stealing method maximizes the utilization of stream processor resources at the task kernel completion edge by identifying idle stream processors at the task kernel completion edge and evaluating the benefits of having other tasks temporarily steal idle stream processors.

[0048] Understandably, each task is deployed to the GPU at the kernel level, and the next kernel injection is not allowed until the current kernel has finished executing. Each kernel contains a certain number of thread blocks, and each thread block is bound to a stream processor. When a kernel is nearing completion, the remaining thread blocks are usually insufficient to fully utilize the stream processor resources, leaving some stream processors idle. Due to the limitations of the GPU's operating mechanism, idle stream processors cannot obtain new thread blocks for execution according to the mapping relationship with tasks, but their idle resource nature gives them the potential to obtain thread blocks through additional mechanisms. The stream processor stealing mechanism temporarily runs thread blocks from another task on idle stream processors and restores them when the stealing ends. Stream processor stealing can be divided into a stealing phase and a return phase. The stealing phase requires changing the mapping relationship between tasks and stream processors to run thread blocks from other tasks on idle stream processors. The return phase requires preempting the stolen stream processor to clear the stream processor and restore the mapping relationship between the stream processor and tasks before the stealing.

[0049] As an optional implementation, the step of determining the idle stream processor and unfinished tasks, and calculating the remaining kernel execution time of the idle stream processor and the stealing overhead of the unfinished tasks, can be further divided into the following steps S1041 and S1045: Step S1041: Determine whether the thread block launch queue of each task is empty, and determine the task kernel completion edge; Step S1042: Based on the task kernel completion edge, check if there are any idle stream processors and obtain the idle stream processors; Step S1043: Determine whether the thread block emission queue of each task has unemitted thread blocks to obtain unfinished tasks; Specifically, the task kernel completion edge is identified by checking if the thread block emission queue of a certain task is empty. Then, the stream processor status is monitored at the task kernel completion edge to check if there are any idle stream processors, thus identifying idle stream processors. Next, it is checked whether the thread block emission queue of a computationally intensive task has unissued thread blocks, thus identifying unfinished tasks that need to temporarily acquire resources.

[0050] Step S1044: Calculate the task to which the idle stream processor belongs and the remaining kernel execution time for the current kernel; Step S1045: Calculate the theft cost of the unfinished task.

[0051] Specifically, assess the remaining time (i.e., remaining kernel execution time) for the task to which the stream processor to be stolen belongs to complete the current kernel and the overhead of implementing stream processor theft.

[0052] For the remaining kernel execution time ( Evaluation of ) : Detect the slowest-progressing thread block among the stream processors belonging to the same task as the stream processor to be stolen, and calculate the remaining number of instructions to be executed based on the number of instructions already executed and the total number of instructions. ), and then combine this with the average number of instructions per cycle executed by the task ( The estimated remaining time for the task to complete the current kernel execution is calculated using the following formula: ; For theft expenses ( Evaluation: Based on the context switching time of the task thread block to be executed after the theft ( ) and the number of thread blocks that a stream processor is expected to host ( The time overhead for the stream processor to complete context switching preemption during the return phase is calculated using the following formula: .

[0053] As an optional implementation, the step of stealing idle stream processors based on the remaining kernel execution time and theft overhead can be further divided into the following steps S1046 and S1047: Step S1046: When the stealing overhead is less than the remaining kernel execution time, steal from the idle stream processor; Step S1047: After the theft is completed, the idle stream processor is preempted, and a new kernel thread block of the task to which the idle stream processor belonged before the theft is injected.

[0054] Specifically, the remaining kernel execution time and theft overhead are compared. If the theft overhead is less than the remaining kernel execution time, the stream processor is stolen. When the task on the stolen stream processor finishes its current kernel execution and begins executing a new kernel, the stolen stream processor is returned. Then, the returned stream processor is preempted and injected with the new kernel thread block of the task that was previously the one stolen.

[0055] It should be noted that, as Figure 4 The diagram illustrates the process of stream processor theft. This application's multi-task stream processor theft method proposes a method for utilizing idle stream processors at the task kernel completion edge. Potential theft targets are obtained by identifying task kernel completion edges and detecting idle stream processors. Tasks that temporarily acquire stolen resources are identified by detecting computationally intensive tasks with redundant thread blocks. The remaining kernel execution time and theft overhead are evaluated and compared to determine whether to implement stream processor theft. Based on the hierarchical characteristics of GPU task execution, this method, through the theft and return of stream processors, ensures that idle stream processors at the kernel completion edge are fully utilized under appropriate conditions, thereby improving the utilization rate of stream processor resources.

[0056] In summary, the processing flow of the dynamic allocation method for stream processors in a multi-tasking GPU architecture according to the embodiments of this application is as follows: The first step is to measure the task bandwidth requirements and available bandwidth, and dynamically adjust the allocation of the number of stream processors among tasks. The second step is to evaluate the benefits of tasks under different stream processor spatial arrangement modes and select the better spatial arrangement mode. The third step is to monitor the stream processor status at the edge of the task kernel completion and temporarily steal idle stream processors to reallocate them to available tasks.

[0057] The above describes the method for dynamic allocation of stream processors for multi-tasking chip architecture GPUs according to embodiments of this application. It can be recognized that, compared with traditional stream processor allocation methods, the embodiments of this application have the following advantages: I. A bandwidth-aware dynamic stream processor allocation method for multi-tasking chip architecture GPUs is provided, adjusting stream processor allocation based on the mismatch between task bandwidth requirements and available bandwidth. This method can dynamically sense task execution characteristics, combined with the hardware characteristics of multi-tasking chip architecture GPUs, focusing on bridging the mismatch between task bandwidth requirements and available bandwidth, intelligently and dynamically adjusting the multi-tasking stream processor allocation, supporting various GPU architectures and various task types, supporting dynamic adjustment based on tracking real-time task execution characteristics, and possessing the ability to utilize GPU resources more efficiently.

[0058] Second, a method for selecting multi-task stream processor spatial arrangement modes applicable to multi-task chip architecture GPUs is provided. This method selects the stream processor spatial arrangement mode by comparing the available bandwidth of memory-intensive tasks under different spatial arrangement modes and evaluating whether the bandwidth requirements of compute-intensive tasks can be met under different spatial arrangement modes. This method can intelligently select the multi-task stream processor spatial arrangement mode that maximizes the execution efficiency of all tasks, improving task adaptation flexibility.

[0059] Thirdly, a method for stealing stream processors suitable for multi-tasking kernel architecture GPUs is provided, which steals idle stream processors for reuse at the task kernel completion edge. This method identifies available idle stream processors by detecting the thread block queue and stream processor status, compares the remaining kernel execution time with the stealing overhead to evaluate the stealing benefits, and thus makes a stealing decision. This method enables full utilization of idle stream processors at the task kernel completion edge, improving GPU computing resource utilization.

[0060] Reference Figure 5 This application also provides a dynamic allocation system for stream processors in a multi-tasking GPU architecture, comprising: The initial allocation module is used to evenly distribute the number of stream processors among all tasks, completing the initial allocation; The dynamic allocation module is used to calculate the task bandwidth requirements and the actual available bandwidth, obtain the mismatch degree based on the task bandwidth requirements and the actual available bandwidth, and then reallocate the number of stream processors for each task according to the mismatch degree. The mode selection module is used to calculate the available memory bandwidth of each task under different spatial arrangement modes, and select the optimal spatial arrangement mode based on the available memory bandwidth. The spatial arrangement modes include local priority mode and global equal distribution mode. The stream processor stealing module is used to identify idle stream processors and unfinished tasks, calculate the remaining kernel execution time of the idle stream processor and the stealing cost of the unfinished tasks, and then steal the idle stream processor based on the remaining kernel execution time and the stealing cost.

[0061] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0062] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0063] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0064] Please see Figure 6 , Figure 6 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 1001 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 1002 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1002 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1002 and is called and executed by the processor 1001 using the methods described in the embodiments of this application. Input / output interface 1003 is used to implement information input and output; The communication interface 1004 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 1005 transmits information between various components of the device (e.g., processor 1001, memory 1002, input / output interface 1003, and communication interface 1004); The processor 1001, memory 1002, input / output interface 1003 and communication interface 1004 are connected to each other within the device via bus 1005.

[0065] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0066] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0067] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0068] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0069] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0070] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0071] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0072] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0073] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0074] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0075] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0076] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0077] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0078] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0079] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0080] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for dynamically allocating stream processors for multi-tasking GPU chip architectures, characterized in that, The method comprises the following steps: allocating the number of stream processors to all tasks, and completing initial allocation; calculating task bandwidth demand and actual available bandwidth, obtaining a mismatch degree according to the task bandwidth demand and the actual available bandwidth, and then reallocating the number of stream processors of each task according to the mismatch degree; calculating available memory bandwidth of each task under different spatial arrangement modes, and selecting an optimal spatial arrangement mode according to the available memory bandwidth, wherein the spatial arrangement mode comprises a local priority mode and a global uniform distribution mode; determining an idle stream processor and an unfinished task, calculating core residual execution time of the idle stream processor and stealing overhead of the unfinished task, and then stealing the idle stream processor according to the core residual execution time and the stealing overhead.

2. The method of claim 1, wherein, The calculation of the task bandwidth demand and the actual available bandwidth, and the obtaining of the mismatch degree according to the task bandwidth demand and the actual available bandwidth specifically comprise: establishing a task bandwidth demand function and an actual available bandwidth function; calculating the task bandwidth demand corresponding to each task according to the task bandwidth demand function; calculating the actual available bandwidth corresponding to each task according to the actual available bandwidth function; performing subtraction operation on the task bandwidth demand and the actual available bandwidth to obtain the mismatch degree corresponding to each task.

3. The method of claim 1, wherein, The reallocation of the number of stream processors of each task according to the mismatch degree specifically comprises: if the mismatch degree of the task is a positive value, reducing the number of stream processors corresponding to the task to a preset number, and if the mismatch degree of the task is a negative value, increasing the number of stream processors corresponding to the task to a preset number; when the mismatch degree of the task is a negative value and the current number of stream processors of the task is less than the initial number, increasing the number of stream processors corresponding to the task to a preset number, and reducing the corresponding number of stream processors from the task corresponding to the maximum mismatch degree; determining the number of thread blocks of the task for which the number of stream processors is increased, and performing partial delay allocation on the task according to the number of thread blocks.

4. The method of claim 1, wherein, The calculation of the available memory bandwidth of each task under different spatial arrangement modes specifically comprises: determining that the spatial arrangement mode of the stream processor is the local priority mode; under the initial allocation, dividing each task into a compute-intensive task or a memory-intensive task; calculating a first available memory bandwidth of the memory-intensive task under the local priority mode; calculating a second available memory bandwidth of the memory-intensive task under the global uniform distribution mode; calculating a third available memory bandwidth of the compute-intensive task under the global uniform distribution mode.

5. The method of claim 4, wherein, The selection of the optimal spatial arrangement mode according to the available memory bandwidth specifically comprises: when the second available memory bandwidth of the memory-intensive task is greater than the first available memory bandwidth, and the third available memory bandwidth of the compute-intensive task is greater than or equal to the task bandwidth demand, selecting the spatial arrangement mode as the global uniform distribution mode; When the second available memory bandwidth of the memory-intensive task is less than or equal to the first available memory bandwidth, or the third available memory bandwidth of the computation-intensive task is less than the task bandwidth requirement, the spatial arrangement mode is selected as the local priority mode.

6. The method of claim 1, wherein, The determining idle stream processors and unfinished tasks, calculating the kernel remaining execution time of the idle stream processors and the stealing overhead of the unfinished tasks, specifically comprises: determining whether the thread block emission queue of each task is empty, and determining a task kernel completion edge; According to the task kernel completion edge, it is determined whether there is an idle stream processor, and the idle stream processor is obtained; determining whether the thread block emission queue of each task has an unemitted thread block, and obtaining the unfinished task; calculating the kernel remaining execution time of the task to which the idle stream processor belongs to complete the current kernel; calculating the stealing overhead of the unfinished task.

7. The method of claim 1, wherein, The idle stream processor is stolen according to the kernel remaining execution time and the stealing overhead, specifically comprising: When the stealing overhead is less than the kernel remaining execution time, the idle stream processor is stolen; After the stealing is completed, the idle stream processor is preempted, and a new kernel thread block of the task to which the idle stream processor belongs before being stolen is injected.

8. A stream processor dynamic allocation system for a multitask core architecture GPU, the system comprising: Comprise: An initial allocation module for equally dividing the number of stream processors for all tasks to complete initial allocation; A dynamic allocation module for calculating task bandwidth requirements and actual available bandwidth, obtaining a mismatch degree according to the task bandwidth requirements and the actual available bandwidth, and then reallocating the number of stream processors for each task according to the mismatch degree; A mode selection module for calculating the available memory bandwidth of each task under different spatial arrangement modes, selecting the optimal spatial arrangement mode according to the available memory bandwidth, and the spatial arrangement mode comprising a local priority mode and a global equal division mode; A stream processor stealing module for determining idle stream processors and unfinished tasks, calculating the kernel remaining execution time of the idle stream processors and the stealing overhead of the unfinished tasks, and then stealing the idle stream processors according to the kernel remaining execution time and the stealing overhead.

9. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method of any one of claims 1 to 7 when executing the computer program.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1 to 7.

Images