Workgroup scheduling method, graphics processor and electronic device

Abstract

The embodiment of the disclosure provides a workgroup scheduling method, a graphics processor and electronic equipment, the workgroup scheduling method is applied to an arbitration module of the graphics processor, the graphics processor further comprises at least two execution modules, and the scheduling method comprises the following steps: for each execution module, determining a first score of the execution module based on resource request of the workgroup and residual resource information of the execution module; in the case that the first score of each execution module is less than a first score threshold, determining a target execution module from the at least two execution modules based on a first waiting time length of each execution module, the first waiting time length of the execution module is a time length required for predicting that the execution module releases target resource information, and the target resource information is determined based on the resource request of the workgroup and the residual resource information of the execution module; and scheduling the workgroup to the target execution module.

Description

Technical Field

[0001] This disclosure relates to, but is not limited to, the field of computer technology, and in particular to a workgroup scheduling method, a graphics processor, and an electronic device. Background Technology

[0002] In a GPU (Graphics Processing Unit), the received working kernel is usually split into multiple workgroups, and multiple workgroups can be scheduled to be executed in different execution modules.

[0003] In related technologies, workgroups are basically scheduled through polling, feedback control, and resource awareness, which have problems such as poor scheduling effect, low throughput, and reduced GPU performance. Summary of the Invention

[0004] This disclosure provides a workgroup scheduling method, a graphics processor, and an electronic device.

[0005] This disclosure provides a scheduling method for a workgroup, applied in the arbitration module of a graphics processor. The graphics processor further includes at least two execution modules. The scheduling method includes: For each execution module, a first score is determined based on the resource requests of the workgroup and the remaining resource information of the execution module; If the first score of each execution module is less than the first score threshold, the target execution module is determined from at least two execution modules based on the first waiting time of each execution module. The first waiting time of the execution module is the predicted time required for the execution module to release the target resource information. The target resource information is determined based on the resource request of the workgroup and the remaining resource information of the execution module. Schedule the workgroup to the target execution module.

[0006] This disclosure provides a graphics processor, including an arbitration module and at least two execution modules, wherein: The arbitration module is used to determine a first score for each execution module based on the resource request of the workgroup and the remaining resource information of the execution module. If the first score of each execution module is less than a first score threshold, a target execution module is determined from at least two execution modules based on the first waiting time of each execution module. The first waiting time of the execution module is the predicted time required for the execution module to release the target resource information, which is determined based on the resource request of the workgroup and the remaining resource information of the execution module. The workgroup is then scheduled to the target execution module. The target execution module is used to execute each task in the workgroup.

[0007] This disclosure provides an electronic device including the graphics processor described above.

[0008] In this embodiment, firstly, each execution module is scored based on the resource requests of the workgroup and the remaining resource information of each execution module. Since this scheme considers both resource demand and remaining resource availability, it efficiently determines whether the resources of each execution module are saturated, thereby guiding the allocation of the workgroup and possessing advantages such as strong responsiveness and high resource awareness. Secondly, when the scores of each execution module are all below a set scoring threshold, i.e., the resources of each execution module cannot meet the needs of the workgroup, the target execution module can be further determined based on the waiting time of each execution module. Since this scheme considers the resource release of the execution module in the future period and predicts the time required for each execution module to release sufficient resources, it more dynamically measures scheduling latency, thereby achieving more accurate and timely scheduling selection. Finally, by combining the remaining resource information of each execution module and the time required for each execution module to release sufficient resources, the goal of scheduling the workgroup by combining resource availability and resource reclamation time is achieved. Even under resource constraints or uneven load conditions, workgroup scheduling can be performed efficiently, improving not only scheduling efficiency and fairness but also overall throughput, thereby enhancing load balancing and processor performance.

[0009] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0011] Figure 1 A schematic diagram illustrating the implementation flow of a workgroup scheduling method provided in this embodiment of the disclosure; Figure 2 A schematic diagram of the composition structure of a graphics processor provided in an embodiment of this disclosure; Figure 3 A schematic diagram of the composition structure of an arbitration module provided in this embodiment of the disclosure. Figure 1 ; Figure 4 A schematic diagram of the composition structure of a first computing unit provided in an embodiment of this disclosure; Figure 5 A schematic diagram of the composition structure of a resource computing module provided in an embodiment of this disclosure; Figure 6 A schematic diagram of the composition structure of an arbitration module provided in this embodiment of the disclosure. Figure 2 ; Figure 7 A schematic diagram illustrating the composition of a time prediction module provided in an embodiment of this disclosure; Figure 8 A schematic diagram of the composition structure of a second computing unit provided in an embodiment of this disclosure; Figure 9 This is a schematic diagram illustrating the connection relationship between an arbitration module and its upstream and downstream components, as provided in an embodiment of this disclosure. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this disclosure clearer, the disclosure will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this disclosure. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0013] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0014] In the following description, the terms “first, second, third” are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that “first, second, third” may be interchanged in a specific order or sequence where permitted, so that the embodiments of this disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0015] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used herein is for the purpose of describing embodiments of this disclosure only and is not intended to be limiting of this disclosure.

[0016] The method provided in this disclosure can be executed by an electronic device, which can be a laptop, tablet, desktop computer, set-top box, mobile device (e.g., mobile phone, portable music player, personal digital assistant, dedicated messaging device, portable gaming device), or a server. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0017] The technical solutions in the embodiments of this disclosure will now be clearly and completely described with reference to the accompanying drawings.

[0018] Figure 1 This is a schematic diagram illustrating the implementation flow of a workgroup scheduling method provided in an embodiment of this disclosure, as shown below. Figure 1 As shown, in the arbitration module applied to the graphics processor, the graphics processor also includes at least two execution modules. The scheduling method includes steps S11 to S13, wherein: Step S11: For each execution module, determine the first score of the execution module based on the resource requests of the workgroup and the remaining resource information of the execution module.

[0019] Here, the arbitration module can be any suitable hardware module capable of implementing this function. The GPU includes at least the arbitration module and at least two execution modules. It is understood that the execution modules in GPUs of different architectures can be different. For example, the execution module can be an SM (Streaming Multiprocessor), a CU (Compute Unit), etc.

[0020] A workgroup (or thread block) consists of a group of threads, for example, 1024 threads. All threads within a workgroup share the same memory space (e.g., shared memory) and can be synchronized via barriers. It is understood that different workgroups are independent of each other.

[0021] This resource request may include the demand for at least one resource. This resource may include, but is not limited to, shared memory, general-purpose registers (GPRs), and tasks within a workgroup. Tasks may be obtained by further dividing workgroups into thread groups (Warp or Wavefront), and a workgroup may include multiple tasks. A thread group may include 32 or 64 threads. All threads within a thread group execute the same instructions synchronously but operate on different data. A thread group is the smallest unit of execution unit scheduling. Therefore, this resource request may include, but is not limited to, the size of shared memory, the size of general-purpose registers, and the number of tasks in a workgroup (referred to as task demand).

[0022] This remaining resource information may include the remaining amount of at least one resource. For example, the remaining amount of shared memory, the remaining amount of general-purpose registers, the remaining amount of available tasks (referred to as task remaining amount), etc. It is understood that the number of resources included in this remaining resource information is the same as the resources included in the workgroup's resource request.

[0023] The first score is used to characterize whether the remaining resources in the execution module can meet the resource requirements of the workgroup. It can be understood that a larger first score indicates that the remaining resources are significantly greater than the required resources; a smaller first score indicates that the remaining resources are significantly less than the required resources. In implementation, when the workgroup's resource request includes requirements for multiple resources, if the remaining resources of each resource in the execution module are greater than the corresponding required resources, the first score can be larger; if the remaining resources of at least one resource in the execution module are less than the corresponding required resources, the first score can be smaller. Therefore, the first score of the execution module can be used to determine whether the execution module can execute the workgroup.

[0024] The method for determining the first score can be any suitable method.

[0025] In some implementations, a correspondence between each resource request, each remaining resource information and each first score can be established in advance. Therefore, based on this correspondence, a first score that matches both the resource request and the remaining resource information can be obtained.

[0026] In some implementations, the first score of the execution module can be determined based on the second scores of each resource. The second score of a resource is determined based on the resource's demand and remaining quantity. For example, if the second score of each resource is greater than a set second score threshold, the lowest second score is used as the first score of the execution module; if the second score of at least one resource is less than the second score threshold, a preset second target score is used as the first score of the execution module. The score thresholds (including the first and second score thresholds) can be any suitable small value, such as 0.05, 1, etc. It is understood that the first and second score thresholds can be the same or different. The target score (including the first and second target scores) can be any suitable small value, such as 0, 1, etc. It is understood that the first and second target scores can be the same or different.

[0027] Step S12: If the first score of each execution module is less than the first score threshold, determine the target execution module from at least two execution modules based on the first waiting time of each execution module. The first waiting time of the execution module is the predicted time required for the execution module to release the target resource information. The target resource information is determined based on the resource request of the workgroup and the remaining resource information of the execution module.

[0028] Here, the first score of an execution module can be less than or equal to the first score threshold. During implementation, if the first score of each execution module is less than the first score threshold, it indicates that none of the execution modules can meet the needs of the workgroup; that is, the resources of each execution module are saturated.

[0029] The target resource information may include the difference in quantity of at least one resource. It is understood that the resource included in the target resource information is one of the resources in the resource request. In some implementations, when the resource request includes at least two resources, if the remaining quantity of a certain resource in the execution module does not meet the demand for that resource, then the target resource information includes the difference in quantity of that resource; if the remaining quantity of at least two resources in the execution module does not meet the demand for that resource, then the target resource information includes the difference in quantity of those at least two resources. The difference in quantity of resources is determined based on the resource request (i.e., the demand for resources) and the remaining resource information (i.e., the remaining quantity of resources). For example, the difference in quantity may be the difference between the demand and the remaining quantity, or a weighted sum of those differences.

[0030] The first waiting period represents the time during which the execution module releases sufficient resources to meet the required needs; that is, the waiting period during which the remaining resources of the execution module can satisfy the needs of the workgroup. It can be understood that after the first waiting period, the remaining amount of each resource of the execution module is sufficient to meet the needs.

[0031] The determination of the first waiting duration can be done in any suitable way. In some implementations, the first waiting duration of the execution module can be determined based on the amount of resources released by the execution module in the current clock cycle. In some implementations, the first waiting duration can be determined based on the second waiting duration of each resource. The second waiting duration of a resource represents the waiting time for that resource to meet the required amount.

[0032] The target execution module can be determined in any suitable way. In some implementations, the execution module corresponding to the shortest first waiting time can be directly selected as the target execution module. In some implementations, the execution module corresponding to the first waiting time that matches a preset waiting time threshold can be selected as the target execution module, where the waiting time threshold can be any suitable duration. In some implementations, the target execution module is selected based on each first waiting time and the load of each execution module. For example, if the first waiting times of execution modules A1 and A2 are close, but the load of execution module A1 is greater than the load of execution module A2, then execution module A2 can be selected as the target execution module.

[0033] Step S13: Schedule the workgroup to the target execution module.

[0034] Here, the execution module, as the core hardware unit of the GPU, is responsible for scheduling and managing the parallel computing tasks of workgroups. After a workgroup (thread block) is assigned to the target execution module, it is further broken down into smaller hardware execution units, namely, Warp or Wavefront. The target execution module allocates shared memory, registers, etc., for the workgroup. Threads within the workgroup communicate through shared memory, while registers are used to store each thread's private data. Understandably, the execution module can concurrently process multiple workgroups, but this is limited by resources such as registers and shared memory.

[0035] In this embodiment, firstly, each execution module is scored based on the resource requests of the workgroup and the remaining resource information of each execution module. Since this scheme considers both resource demand and remaining resource availability, it efficiently determines whether the resources of each execution module are saturated, thereby guiding the allocation of the workgroup and possessing advantages such as strong responsiveness and high resource awareness. Secondly, when the scores of each execution module are all below a set scoring threshold, i.e., the resources of each execution module cannot meet the needs of the workgroup, the target execution module can be further determined based on the waiting time of each execution module. Since this scheme considers the resource release of the execution module in the future period, it predicts the time required for each execution module to release sufficient resources, thus more dynamically measuring scheduling latency and achieving more accurate and timely scheduling selection. Finally, by comprehensively considering the remaining resource information of each execution module and the time required for each execution module to release sufficient resources, it achieves workgroup scheduling by combining resource availability and resource reclamation time. Even under resource constraints or uneven load conditions, workgroup scheduling can be performed efficiently, improving not only scheduling efficiency and fairness but also overall throughput, thereby enhancing load balancing and processor performance.

[0036] In some implementations, the resource request of the working group includes the demand for at least one resource, and the remaining resource information of the execution module includes the remaining amount of each resource; the step S11, "determining the first score of the execution module based on the resource request of the working group and the remaining resource information of the execution module," includes steps S111 and S112, wherein: Step S111: For each resource, determine the second score of the resource based on the resource demand and the resource remaining quantity.

[0037] Here, the second score is used to characterize whether the remaining amount of resources can meet the demand. It can be understood that a larger second score indicates that the remaining amount of resources is much greater than the demand; conversely, a smaller second score indicates that the remaining amount of resources is much less than the demand.

[0038] The second score can be determined in any suitable way. In some implementations, for each resource, a correspondence between each demand quantity, each surplus quantity, and each second score can be pre-established. Based on this correspondence, a second score matching both the demand quantity and the surplus quantity can be obtained. In some implementations, for each resource, the second score can be determined based on the number of shift bits corresponding to the resource, the surplus quantity of the resource, and the coefficient corresponding to the resource.

[0039] Step S112: Determine the first score of the execution module based on the second score of each resource.

[0040] Here, the first rating can be the second rating of a certain resource, the average of all the second ratings, or a preset second target rating.

[0041] The first score can be determined in any suitable manner. In some embodiments, if the second scores of all resources are greater than the second score threshold, it indicates that the execution module can execute the workgroup. In this case, the minimum second score or the average of all second scores can be used as the first score of the execution module. In some embodiments, if the second score of at least one resource is not greater than the second score threshold, it indicates that the execution module cannot execute the workgroup. Therefore, the second target score can be used as the first score of the execution module. It is understood that the second target score should be less than the first score threshold.

[0042] In this embodiment, on the one hand, by scoring the relatively scarce resources such as general-purpose registers, shared memory, and remaining task resources in each execution module, the resource carrying capacity is determined to guide the allocation of workgroups. This achieves a dynamic scoring mechanism to quantify the supply and demand relationship of resources. Compared with static binding, this improves resource utilization, reduces fragmentation, and enhances the flexibility of workgroup allocation. On the other hand, the execution modules are scored based on the second score of each resource to assess the load of the execution modules in real time. This allows for dynamic adjustment of workgroup allocation, avoiding resource idleness or overload, thereby improving the resource utilization of the execution modules. At the same time, the scores of each execution module can guide the workgroup to run on the most energy-efficient execution modules, reducing overall power consumption and achieving a balance between performance and energy efficiency. This achieves the goal of multi-objective optimization of utilization, fairness, and energy efficiency.

[0043] In some implementations, the step S111 of "determining a second rating of the resource based on the resource demand and the remaining resource" includes steps S1111 and S1112, wherein: Step S1111: When the demand for resources is not greater than the remaining amount of resources, the remaining amount of resources is shifted based on the shift number corresponding to the resources to obtain the shift result. Based on the shift result of the resources, the second score of the resources is determined. The shift number corresponding to the resources is determined based on the total amount of resources in the execution module.

[0044] Here, if the remaining amount of the resource is not less than the remaining amount of the resource, it indicates that the remaining amount of the resource in the execution module can meet the needs of the resource in the workgroup.

[0045] Different resources can correspond to the same or different shift bits.

[0046] Since the number of bits shifted determines the input bit width of the multiplier and also affects the accuracy of the resource scoring calculation, the number of bits shifted for a resource can be determined based on the total amount of the resource. In some implementations, the total amount and the number of shifts can be proportional; that is, the larger the total amount, the larger the number of shifts; the smaller the total amount, the smaller the number of shifts. For example, the number of shifts for shared memory is greater than the number of shifts for a task.

[0047] In some implementations, the shift data of the resource can be determined using the following formula (1-1). ,Right now: (1-1); in, Indicates the total amount of resources. Indicates the first coefficient. This indicates rounding down to the nearest integer.

[0048] The shift result is obtained by right-shifting the remaining amount by the number of shift bits. For example, if the shift amount for shared memory is I, then the shift result for shared memory can be obtained by right-shifting the remaining amount of shared memory, shared_mem_avaiable, by I bits; if the shift amount for a general-purpose register is J, then the shift result for a general-purpose register can be obtained by right-shifting the remaining amount of the general-purpose register, gpr_avaiable, by J bits; if the shift amount for a task is K, then the shift result for a task can be obtained by right-shifting the remaining amount of the task, wave_avaiable, by K bits.

[0049] The second score can be determined in any suitable way. In some implementations, for each resource, a correspondence between each shift result and each second score can be established in advance, and a second score that matches the shift result can be obtained based on this correspondence.

[0050] In some implementations, the step S1111, "determining the second score of the resource based on the shift result corresponding to the resource," includes steps S141 and S142, wherein: Step S141: Obtain the coefficient corresponding to the resource; wherein, the coefficient corresponding to the resource is determined based on the demand of other resources.

[0051] Here, different resources can correspond to different coefficients. These coefficients can be preset or dynamically determined. In some implementations, the coefficient for a resource can be determined based on the demand for other resources. For example, the product of the demands for other resources can be used as the coefficient for that resource. Another example is using a weighted average of the products of the demands for other resources as the coefficient for that resource.

[0052] In some implementations, when the resources of a workgroup include shared memory, general-purpose registers, and tasks within the workgroup, the coefficient `shared_mem_coeff` corresponding to the shared memory can be the product of the general-purpose register requirement `gpr_req` and the task requirement `wave_req`; the coefficient `gpr_coeff` corresponding to the general-purpose register can be the product of the shared memory requirement `shared_mem_req` and `wave_req`; and the coefficient `wave_coeff` corresponding to the tasks within the workgroup can be the product of `shared_mem_req` and `gpr_req`. In some implementations, the coefficients corresponding to each resource in each workgroup can be pre-calculated. During implementation, the coefficients corresponding to the resource can be obtained by reading files, accessing memory, etc. Alternatively, the coefficients corresponding to each resource in the workgroup can be received from other devices.

[0053] Step S142: Determine the second score of the resource based on the first product between the shift result and the coefficient corresponding to the resource.

[0054] Here, the second rating can be the first product, a weighted sum of the first products, etc. For example, the first product can be used as the second rating for the resource.

[0055] In some implementations, if the resources of a workgroup include shared memory, general-purpose registers, and tasks contained within the workgroup, then: The second rating for this shared memory could be: (shared_mem_avaiable) I) * shared_mem_coeff; The second rating of this general-purpose register could be: (gpr_avaiable) J)* gpr_coeff; The second rating for this task could be: (wave_avaiable) K)* wave_coeff.

[0056] In this way, different coefficients are designed for different resources, improving the rationality and accuracy of the coefficients. Furthermore, since these coefficients are determined based on the demand for other resources, the hardware overhead and area of ​​the multiplier in the arbitration module can be significantly reduced. When resource supply exceeds demand, the remaining amount is first shifted and then multiplied with the coefficient to obtain the resource score. Since both shifting and multiplication are hardware-friendly operations, the introduced latency can be controlled at the microsecond level, improving computational efficiency and making it suitable for scenarios with high real-time requirements. Simultaneously, by mathematically transforming the resource supply and demand relationship into a comparable scalar value, the limitations of fixed resource dimension weights in traditional scoring are overcome.

[0057] Step S1112: When the demand for resources exceeds the remaining resources, the preset first target score is used as the second score for the resources.

[0058] Here, the first target score can be any suitable, relatively small value. It is understood that the first target score should be less than the second score threshold. During implementation, if the remaining amount of the resource is less than the required amount, indicating that the remaining amount of the resource in the execution module cannot meet the needs of the workgroup, the first target score can be used as the second score for the resource.

[0059] In this embodiment, different strategies are used to determine resource scores based on the supply and demand relationship of resources, thereby quantifying the supply and demand relationship and improving the accuracy of resource scoring. This allows the scoring to simultaneously consider performance and energy efficiency. Furthermore, different shift bits are selected for different resources to balance the area overhead and computational accuracy of the multiplier; shifting the remaining amount can save certain hardware costs.

[0060] In some embodiments, step S112 includes steps S1121 and S1122, wherein: Step S1121: If the second score of each resource is greater than the set second score threshold, the lowest second score is taken as the first score of the execution module.

[0061] Here, the second scoring threshold can be any suitable, small value.

[0062] If the second score of each resource is greater than the second score threshold, it indicates that the remaining amount of each resource in the execution module is greater than the corresponding demand. Therefore, the execution module can execute the workgroup, and the lowest second score can be used as the first score of the execution module. In some implementations, if the resources of the workgroup include shared memory, general-purpose registers, and tasks contained in the workgroup, when the second score of shared memory is less than the second score of general-purpose registers, and the second score of general-purpose registers is less than the second score of tasks, then the second score of shared memory is used as the first score of the execution module.

[0063] Step S1122: If the second score of at least one resource is not greater than the second score threshold, the preset second target score is used as the first score of the execution module.

[0064] Here, the second target score can be any suitable small value. In implementation, if the second score of at least one resource is not greater than the second score threshold, it indicates that the remaining quantity of at least one resource in the execution module is less than the corresponding demand. Therefore, the execution module cannot execute the workgroup. In this case, the second target score can be used as the first score of the execution module.

[0065] In this embodiment, based on the comparison results of the scores of each resource and the score threshold, the set score or the minimum second score is used as the score of the execution module. On the one hand, by setting the score threshold, resources below the baseline can be filtered out, ensuring that the workgroup can be scheduled to the execution module that meets the minimum performance requirements, reducing performance loss caused by resource mismatch. On the other hand, through dynamic screening and optimal allocation, the dual optimization of resource utilization and performance is achieved.

[0066] In some implementations, the scheduling method further includes step S14, wherein: Step S14: If the first score of at least one execution module is not less than the first score threshold, determine the target execution module based on the first score of at least one execution module.

[0067] Here, if the first score of an execution module is not less than a first score threshold, it indicates that the execution module can execute a workgroup. Therefore, a target execution module can be determined from at least one execution module capable of executing a workgroup. In implementation, if only one execution module has a first score not less than the first score threshold, that execution module is taken as the target execution module; if at least two execution modules have first scores not less than the first score threshold, then the target execution module can be determined based on the first scores of these at least two execution modules.

[0068] The target execution module can be determined in any suitable way. In some embodiments, the execution module corresponding to the highest first score is selected as the target execution module. In some embodiments, the execution module corresponding to the first score that matches a preset scoring threshold can be selected as the target execution module, where the scoring threshold can be any suitable value. In some embodiments, the target execution module is selected based on each first score and the load of each execution module. For example, if the first scores of execution modules B1 and B2 are close, but the load of execution module B1 is greater than the load of execution module B2, then execution module B2 can be selected as the target execution module.

[0069] In this embodiment, if there is an execution module that meets the requirements of the workgroup, the target execution module is selected based on the score of each execution module that meets the requirements of the workgroup, so as to ensure maximum resource utilization and accurate load balancing.

[0070] In some implementations, step S12, "determining the target execution module from at least two execution modules based on the first waiting time of each execution module," includes steps S121 and S122, wherein: Step S121: Determine the target first waiting time based on the first waiting time of each execution module; Step S122: From at least two execution modules, select the execution module that matches the target first waiting time as the target execution module.

[0071] Here, the target first waiting time can be determined in any suitable way. For example, a certain first waiting time can be used as the target first waiting time, such as the minimum first waiting time or the first waiting time that matches a preset waiting time threshold. Another example is to use the average of all first waiting times as the target first waiting time.

[0072] The first waiting time to be adapted to the target can be the same as or the closest to the target first waiting time. During implementation, the first waiting time of each execution module can be compared with the target first waiting time to determine the first waiting time to be adapted to the target first waiting time. The execution module corresponding to the adapted first waiting time is then taken as the target execution module.

[0073] In some implementations, the various first wait durations can be directly compared to obtain the minimum first wait duration. The execution module corresponding to the minimum first wait duration can then be used as the target execution module. It can be understood that the minimum first wait duration refers to the shortest duration within which the execution module can most quickly meet resource demands. In this way, directly using the execution module with the minimum wait duration as the target execution module minimizes latency and improves overall throughput.

[0074] In this embodiment, the target execution module is selected based on the first waiting time of each execution module. Since this scheme fully considers the rate at which each execution module releases resources, it improves the accuracy of the target execution module and enables efficient scheduling of the workgroup under resource constraints.

[0075] In some implementations, the scheduling method further includes step S120 before determining the target execution module, wherein: Step S120: For each execution module, determine the first waiting time based on the resource requests of the workgroup and the amount of tasks released by the execution module in the current clock cycle.

[0076] Here, the determination of the first waiting duration can be done in any suitable way. In some implementations, a correspondence between each resource request, each released task quantity, and each first waiting duration can be pre-established. Based on this correspondence, a first waiting duration that matches both the resource request and the released task quantity can be obtained. In some implementations, the first waiting duration of the execution module can be determined based on the second waiting duration of each resource. The second waiting duration of the resource is determined based on the task quantity released in the current clock cycle and the average release quantity of the resource in the previous clock cycle.

[0077] In this embodiment of the disclosure, the waiting time is determined based on resource demand and task release volume in order to accurately predict resource availability time and reduce idle time caused by resource fragmentation.

[0078] In some implementations, the resource request of the workgroup includes the demand for at least one resource; the step S120, "determining the first waiting time of the execution module based on the resource request of the workgroup and the amount of tasks released by the execution module in the current clock cycle," includes steps S1201 and S1202, wherein: Step S1201: For each resource, based on the amount of tasks released in the current clock cycle and the average release amount of the resource in the previous clock cycle, determine the average release amount of the resource in the current clock cycle, and based on the average release amount of the resource in the current clock cycle, determine the second waiting time of the resource.

[0079] Here, in each clock cycle (or beat), if a task terminates from the execution module in the current beat (i.e., the current clock cycle), the execution module can report information such as the workgroup identifier and the work core identifier to which the task belongs to to the arbitration module. The arbitration module can then obtain the corresponding resource release amount based on the task. It is understandable that if no task terminates from the execution module in the current beat, the corresponding resource release amount is 0.

[0080] The average release amount of resources in the current clock cycle can be determined in any suitable way. In some implementations, a correspondence can be pre-established between the task amount released in each current cycle, each first average release amount (i.e., the average release amount of the previous cycle), and each second average release amount (i.e., the average release amount of the current cycle). Then, based on this correspondence, a second average release amount that matches both the task amount released in the current cycle and the first average release amount can be obtained. In some implementations, the task amount released in the current cycle can be determined based on the sum of the first product and the second product. Here, the first product refers to the product between the first weight corresponding to the resource and the first average release amount, and the second product refers to the product between the task amount released in the current cycle and the second weight corresponding to the task amount.

[0081] The second waiting time can be determined in any suitable way. In some embodiments, a correspondence between each second average release amount and each second waiting time can be pre-established, and a second waiting time adapted to the second average release amount can be obtained based on this correspondence. In some embodiments, the second waiting time can be determined by the ratio between the first difference and the second average release amount. For example, the ratio, a weighted average of the ratio, etc., can be used as the second waiting time. The first difference refers to the difference between the demand for resources and the remaining amount of resources.

[0082] Step S1202: Determine the first waiting time of the execution module based on the second waiting time of each resource.

[0083] Here, the first waiting time can be determined in any suitable way. In some implementations, a certain second waiting time can be directly used as the first waiting time of the execution module. For example, the minimum second waiting time, the maximum second waiting time, etc., can be used as the first waiting time. In some implementations, the average of the various second waiting times, the weighted average of the average, etc., can be used as the first waiting time.

[0084] In this embodiment, on the one hand, the waiting time of resources is determined based on the amount of tasks released and the average release amount of the previous cycle, so as to accurately predict the resource release trend of the next cycle, realize intelligent load prediction and quantify the resource release pattern, and improve the dynamic resource prediction capability. On the other hand, the waiting time of execution modules is determined based on the waiting time of each resource, so as to ensure that all resources of the execution modules can meet the demand, thereby accurately quantifying the resource supply and demand relationship. At the same time, the waiting time of each execution module can guide the workgroup to run on the optimal execution module, so as to more dynamically measure the scheduling latency and realize multiple optimizations among performance, energy efficiency and latency.

[0085] In some implementations, step S1211, "determining the average release amount of resources in the current clock cycle based on the amount of tasks released in the current clock cycle and the average release amount of resources in the previous clock cycle," includes steps S151 to S154, wherein: Step S151: Determine the second weight corresponding to the task quantity based on the first weight corresponding to the resource; wherein, the first weight is different for different resources.

[0086] Here, the first weight can be any suitable value less than 1. Different resources can correspond to different first weights. It is understood that the sum of the first weight and the second weight is no greater than 1.

[0087] The second weight can be determined in any suitable way. In some embodiments, the second weight can be the difference between a preset value and a first weight, and the preset value can be any suitable value, such as 1. In some embodiments, a correspondence between each first weight and each second weight can be established in advance, and a second weight that matches the first weight can be obtained based on this correspondence.

[0088] Step S152: Determine the first product between the first weight and the average release amount of resources in the previous clock cycle.

[0089] Here, the first product refers to the product between the first weight and the average release amount of resources in the previous round.

[0090] Step S153: Determine the second product between the second weight and the amount of task released in the current clock cycle.

[0091] Here, the second product refers to the product between the second weight and the amount of task released in the current beat.

[0092] Step S154: Based on the sum of the first product and the second product, determine the average release amount of resources in the current clock cycle.

[0093] Here, the average release amount can be the sum, a weighted sum, or the sum of the sum and a preset constant. The preset constant can be a small constant used to protect the hardware from overflow during initialization or when no resources are released for an extended period. For example, the sum of the sum and the preset constant can be used as the average release amount of the resource in the current cycle.

[0094] In some implementations, the average release of resources in the current auction can be determined according to the following formulas (1-2). ,Right now: (1-2); in, This indicates the average amount of resources released in the previous round. Indicates the first weight. Indicates the second weight. This indicates the amount of tasks released in the current round. This represents a preset constant.

[0095] In this embodiment, on the one hand, different resources are assigned different weights, which improves the rationality of the weights and achieves the best optimization effect; on the other hand, the average release amount of the current round is determined based on the weight of the resources, the weight of the task, the average release amount of the resources in the previous round, and the release amount of the task, so as to quantify the relative importance of resources and tasks and improve the ability to accurately predict resources.

[0096] In some implementations, the step S1211, "determining the second waiting time of the resource based on the average release amount of the resource in the current clock cycle," includes steps S161 and S162, wherein: Step S161: Determine the first difference between the required amount of resources and the remaining amount of resources in the execution module; Step S162: Determine the second waiting time of the resource based on the ratio between the first difference and the average release amount of the resource in the current clock cycle.

[0097] Here, the first difference represents the supply-demand gap of the resource.

[0098] The second waiting time can be the ratio, a weighted average of the ratio, etc. In some implementations, the second waiting time of the resource can be determined according to the following formulas (1-3). ,Right now: (1-3); in, This indicates the demand for resources. Indicates the remaining amount of resources. This indicates the average amount of resources released during the current clock cycle.

[0099] In this embodiment of the disclosure, the waiting time for resources is determined based on the difference between resource supply and demand and the average release amount of the current batch. This is achieved by mathematically transforming the resource supply and demand gap into a comparable value, thereby accurately predicting the time required to release sufficient resources and facilitating more dynamic measurement of scheduling delays in the future.

[0100] Based on the above embodiments, this disclosure provides a graphics processor. Figure 2 This is a schematic diagram of the composition structure of a graphics processor provided in an embodiment of the present disclosure, such as... Figure 2 As shown, the graphics processor 20 includes an arbitration module 21 and at least two execution modules 22, wherein: Arbitration module 21 is used to determine a first score for each execution module based on the resource request of the workgroup and the remaining resource information of the execution module; if the first score of each execution module is less than the first score threshold, a target execution module is determined from at least two execution modules based on the first waiting time of each execution module, where the first waiting time of the execution module is the predicted time required for the execution module to release the target resource information, and the target resource information is determined based on the resource request of the workgroup and the remaining resource information of the execution module; and the workgroup is scheduled to the target execution module. Target execution module 22 is used to execute each task in the workgroup.

[0101] Here, the arbitration module 21 can be any suitable hardware module capable of implementing this function. The workgroup includes a group of threads, and the resource request may include a demand for at least one resource. This resource may include, but is not limited to, at least one of shared memory, general-purpose registers, tasks included in the workgroup, etc. The remaining resource information may include the remaining amount of at least one resource. It is understood that the number of resources included in the remaining resource information is the same as the number of resources included in the workgroup's resource request. In implementation, the arbitration module's determination of the execution module's first score can be found in the specific implementation of the aforementioned step S11.

[0102] If none of the execution modules can meet the needs of the workgroup, then the target execution module can be determined based on the first waiting time for each execution module to release sufficient resources. In implementation, the process by which this arbitration module determines the target execution module can be found in the specific implementation of step S12 described above.

[0103] The execution module, as the core hardware unit of the GPU, is responsible for scheduling and managing the parallel computing tasks of the workgroup. It is understood that the execution modules in GPUs with different architectures may differ. In implementation, the process of the arbitration module scheduling the target execution module and the execution module executing the task can be found in the specific implementation of step S13 above.

[0104] In this embodiment, firstly, each execution module is scored based on the resource requests of the workgroup and the remaining resource information of each execution module. Since this scheme considers both resource demand and remaining resource availability, it efficiently determines whether the resources of each execution module are saturated, thereby guiding the allocation of the workgroup and possessing advantages such as strong responsiveness and high resource awareness. Secondly, when the scores of each execution module are all below a set scoring threshold, i.e., the resources of each execution module cannot meet the needs of the workgroup, the target execution module can be further determined based on the waiting time of each execution module. Since this scheme considers the resource release of the execution module in the future period, it predicts the time required for each execution module to release sufficient resources, thus more dynamically measuring scheduling latency and achieving more accurate and timely scheduling selection. Finally, by comprehensively considering the remaining resource information of each execution module and the time required for each execution module to release sufficient resources, it achieves workgroup scheduling by combining resource availability and resource reclamation time. Even under resource constraints or uneven load conditions, workgroup scheduling can be performed efficiently, improving not only scheduling efficiency and fairness but also overall throughput, thereby enhancing load balancing and processor performance.

[0105] In some implementations, the arbitration module includes a resource calculation module, a first comparison module, a distribution module, and a resource recording module corresponding to each execution module; the resource recording module is used to record the remaining resource information of each execution module; each resource calculation module is used to determine the first score of the corresponding execution module based on the resource request of the working group and the remaining resource information of the corresponding execution module; the first comparison module is used to select the execution module with the highest first score as the target execution module if the first score of at least one execution module is not less than a first score threshold, and to pass the identification information of the working group and the target execution module to the distribution module; the distribution module is used to schedule the working group according to the identification information of the target execution module; and send the resource request of the working group and the identification information of the target execution module to the resource recording module; the resource recording module is also used to update the remaining resource information of the target execution module based on the resource request of the working group.

[0106] Here, the resource recording module can be any suitable hardware or software capable of implementing this function. This module records the resource quantity for each task, the remaining resource information for each execution module, and so on. The remaining resource information includes the remaining quantity of various resources.

[0107] The resource computing module can be any suitable hardware module capable of implementing this function. The number of resource computing modules is the same as the number of execution modules, and each resource computing module corresponds one-to-one with each execution module. In implementation, the first score of the corresponding execution module determined by the resource computing module can be found in the specific implementation of step S11 above.

[0108] The first comparison module can be any suitable hardware module capable of implementing this function, such as a comparator. This first comparison module is primarily used to compare the first score of each execution module with the magnitude of a first score threshold.

[0109] The identification information of the target execution module may include, but is not limited to, number, name, etc.

[0110] The distribution module can be any suitable hardware module capable of implementing this function. In implementation, when the distribution module receives the identification information of the execution module, it schedules the workgroup to the execution module corresponding to that identification information.

[0111] After the distribution module has scheduled the workgroup, it can also send the workgroup's resource requests to the resource recording module in a timely manner, so that the resource recording module can update the remaining resource information of the target execution module synchronously.

[0112] like Figure 3 As shown, the arbitration module 21 includes a resource calculation module 211, a first comparison module 212, a distribution module 213, and a resource recording module 214 corresponding to each execution module 22.

[0113] In this embodiment, firstly, each execution module is scored based on the resource requests of the workgroup and the remaining resource information of each execution module. Since this scheme considers both resource demand and resource availability, it can efficiently determine whether the resources of each execution module are saturated, thereby guiding the allocation of resources to the workgroup, and has the advantages of strong responsiveness and high resource awareness. Secondly, if there is an execution module that meets the workgroup's requirements, the execution module with the highest score that meets the workgroup's requirements is selected as the target execution module to ensure maximum resource utilization and accurate load balancing. Finally, the remaining resource information of the target execution module is updated in real time using a resource recording module to ensure the accuracy and reliability of the remaining resource information.

[0114] In some implementations, the resource request of the working group includes the demand for at least one resource, the remaining resource information of the execution module includes the remaining amount of each resource, and the resource calculation module includes a first calculation unit and a first comparison calculation unit corresponding to each resource; each first calculation unit is used to determine a second score of the resource based on the demand for the corresponding resource and the remaining amount of the resource in the execution module corresponding to the resource calculation module; the first comparison calculation unit is used to determine a first score of the execution module corresponding to the resource calculation module based on the second score of each resource.

[0115] Here, the first computing unit can be any suitable hardware module capable of implementing this function. The number of the first computing units is adapted to the number of resources. In implementation, the second score for determining the corresponding resource by the first computing unit can be found in the specific implementation of the aforementioned step S111.

[0116] The first comparison calculation unit can be any suitable hardware module capable of implementing this function. In implementation, the first comparison calculation unit determines the first score of the corresponding execution module, as detailed in the specific implementation of step S112 above.

[0117] In this embodiment, on the one hand, the first computing unit scores the relatively scarce resources such as general-purpose registers, shared memory, and remaining task resources in each execution module to determine its resource carrying capacity and guide the allocation of workgroups. This achieves a dynamic scoring mechanism to quantify the supply and demand relationship of resources. Compared with static binding, this improves resource utilization, reduces fragmentation, and enhances the flexibility of workgroup allocation. On the other hand, the first comparison computing unit scores the execution modules based on the second score of each resource to assess the load of the execution modules in real time. This allows for dynamic adjustment of workgroup allocation, avoiding resource idleness or overload, thereby improving the resource utilization of the execution modules. At the same time, the scores of each execution module can guide the workgroup to run on the most energy-efficient execution module, reducing overall power consumption and achieving a balance between performance and energy efficiency. This achieves the goal of multi-objective optimization of utilization, fairness, and energy efficiency.

[0118] In some implementations, the first calculation unit includes a comparison calculation unit, a shift calculation unit, and a first multiplication calculation unit; the comparison calculation unit is used to compare the required amount of resources with the remaining amount of resources and output a comparison result; the shift calculation unit is used to shift the remaining amount of resources according to the shift number corresponding to the resources when the comparison result is the first comparison result, to obtain a shift result; the first multiplication calculation unit is used to determine the first product between the shift result and the coefficient corresponding to the resources, and use the first product as the second score of the resources.

[0119] Here, the comparison calculation unit can be any suitable hardware module capable of implementing this function, such as a comparator. The comparison result can include, but is not limited to, a first comparison result and a second comparison result. The first comparison result indicates that the remaining amount of resources is not less than the required amount, and the second comparison result indicates that the remaining amount of resources is less than the required amount. The first comparison result can be represented by any suitable value, such as 1. The second comparison result can be represented by any suitable value, such as 0. It is understood that the values ​​of the first comparison result and the second comparison result are different.

[0120] In some implementations, the output may also include the remaining amount of resources. In practice, the comparison calculation unit can obtain the remaining amount of resources from the resource recording module.

[0121] The shift calculation unit can be any suitable hardware module capable of implementing this function, such as a shift register. In implementation, the shift calculation unit is connected to the comparison calculation unit. When the comparison result output by the comparison calculation unit is the first comparison result, the remaining amount of the resource is obtained from the comparison calculation unit, and the remaining amount of the resource is shifted according to the shift bit corresponding to the resource, and the shift result is output.

[0122] Since the number of shift bits determines the input bit width of the multiplier and also affects the accuracy of the resource scoring calculation, the shift bit number for each resource can be pre-determined by software based on the total amount of resources. In this way, the shift calculation unit can obtain the shift bit number for that resource, which is determined based on the total amount of resources in the execution module. In implementation, the process of shifting the remaining amount of resources by the shift calculation unit can be found in the specific implementation of step S1111 described above.

[0123] The first multiplication calculation unit can be any suitable hardware module capable of implementing this function, such as a multiplier. In implementation, the first multiplication calculation unit is connected to the shift calculation unit. After obtaining the coefficients corresponding to the resources, the first multiplication calculation unit calculates the product between the coefficients corresponding to the resources and the shift result, and outputs the product.

[0124] Different resources can correspond to different coefficients. During implementation, the coefficients for each resource in each workgroup can be pre-calculated using software.

[0125] like Figure 4 As shown, the first calculation unit 41 includes a comparison calculation unit 411, a shift calculation unit 412, and a first multiplication calculation unit 413.

[0126] In this embodiment, firstly, different shift amounts are selected for different resources to balance the area overhead and computational accuracy of the multiplier. Simultaneously, shifting the remaining amount saves hardware overhead. Secondly, different coefficients are designed for different resources, improving the rationality and accuracy of the coefficients. Furthermore, since these coefficients are determined based on the demand of other resources, the hardware overhead and area of ​​the multiplier in the arbitration module can be significantly reduced. Finally, when resource supply exceeds demand, the remaining amount is first shifted using the shift calculation unit, and then the first multiplication calculation unit multiplies the shifted result with the coefficient to obtain the resource score. Since both shifting and multiplication are hardware-friendly operations, the introduced latency can be controlled at the microsecond level, improving computational efficiency and making it suitable for scenarios with high real-time requirements. Additionally, mathematical transformation converts the resource supply and demand relationship into a comparable scalar value, overcoming the limitation of fixed resource dimension weights in traditional scoring methods.

[0127] In some implementations, the resource calculation module further includes a selection unit, which is used to determine a selection result based on the comparison result output by each comparison calculation unit; the first comparison calculation unit is further used to take a preset second target score as the first score of the execution module when the selection result is the first selection result; and to take the smallest second score as the first score of the execution module when the selection result is the second selection result.

[0128] Here, the selection unit can be any suitable hardware module capable of implementing this function, such as a multiplexer. The selection result can include the comparison result of the output of each comparison calculation unit. For example, if there are 3 comparison calculation units, the selection result can be 000, indicating that the comparison result output by all 3 comparison calculation units is 0; if the selection result is 001, it indicates that the comparison result output by the first two comparison calculation units is 0 and the comparison result output by the last comparison calculation unit is 1.

[0129] The selection result may include, but is not limited to, a first selection result, a second selection result, etc. A first selection result indicates that the comparison result output by at least one comparison calculation unit is the second comparison result. A second selection result indicates that the comparison result output by each calculation unit is the first comparison result.

[0130] The first score can be a preset second target score or the minimum second score. In implementation, the process by which the first comparison calculation unit determines the first score can be found in the specific implementation of the aforementioned steps S1121 and S1122.

[0131] like Figure 5 As shown, the resource calculation module 211 includes a first calculation unit 41, a first comparison calculation unit 51, and a selection unit 52 corresponding to each resource.

[0132] In this embodiment of the disclosure, the first comparison calculation unit uses the set score or the minimum second score as the score of the execution module based on the comparison results of the scores of each resource and the score threshold. On the one hand, by setting the score threshold, resources below the baseline can be filtered out, ensuring that the workgroup can be scheduled to the execution module that meets the minimum performance requirements, reducing performance loss caused by resource mismatch. On the other hand, through dynamic screening and optimal allocation, the dual optimization of resource utilization and performance is achieved.

[0133] In some implementations, the arbitration module further includes a time prediction module and a second comparison module corresponding to each execution module; the first comparison module is further configured to broadcast the resource request of the working group when the first score of each execution module is less than the first score threshold; the resource recording module is further configured to receive the task information corresponding to each clock cycle sent by each execution module; and update the remaining resource information of each execution module based on the task information corresponding to each clock cycle sent by each execution module; each time prediction module is configured to determine the first waiting time of the execution module based on the resource request of the working group and the amount of task released by the corresponding execution module in the current clock cycle; the second comparison module is configured to compare the first waiting time of each execution module and select the execution module with the smallest first waiting time as the target execution module.

[0134] Here, when none of the execution modules can meet the needs of the workgroup, the first comparison module can broadcast the workgroup's resource requests to each time prediction module.

[0135] Task information may include, but is not limited to, the identifier of the workgroup to which the task belongs, the identifier of the work core to which the task belongs, etc.

[0136] The resource recording module is also used to record resource information corresponding to each task. After a cycle ends, it determines the amount of resources released by the execution module in that cycle based on the task information sent by the execution module, and updates the remaining resource information of the execution module based on the amount of resources released.

[0137] The time prediction module can be any suitable hardware module capable of implementing this function. The number of time prediction modules is the same as the number of execution modules, and each time prediction module corresponds one-to-one with each execution module. In implementation, the determination of the first waiting time of the corresponding execution module by the time prediction module can be found in the specific implementation of the aforementioned step S120.

[0138] The second comparison module can be any suitable hardware module capable of implementing this function, such as a comparator.

[0139] like Figure 6As shown, the arbitration module 21 includes a resource calculation module 211, a first comparison module 212, a distribution module 213, a resource recording module 214, a time prediction module 215, and a second comparison module 216 corresponding to each execution module 22.

[0140] In this embodiment, on the one hand, the time prediction module determines the waiting time based on resource demand and task release volume to accurately predict resource availability time and reduce idle time caused by resource fragmentation; on the other hand, the second comparison module directly selects the execution module with the shortest waiting time as the target execution module to minimize latency and improve overall throughput.

[0141] In some implementations, the resource request of the workgroup includes the demand for at least one resource, and the time prediction module includes a second calculation unit and a second comparison calculation unit for each resource; each second calculation unit is used to determine the second waiting time of the resource based on the demand of the corresponding resource and the resource release amount of the corresponding execution module in the current clock cycle; the second comparison calculation unit is used to compare the second waiting time of each resource and take the largest second waiting time as the first waiting time of the execution module.

[0142] Here, the second computing unit can be any suitable hardware unit capable of implementing this function. The number of the second computing units is adapted to the number of resources. In implementation, the determination of the second waiting time for the corresponding resource by the second computing unit can be found in the specific implementation of the aforementioned step S1201.

[0143] The second comparison calculation unit can be any suitable hardware unit capable of implementing this function, such as a comparator. In implementation, the second comparison calculation unit is connected to each second calculation unit, and outputs the maximum second waiting time by comparing the second waiting times output by each second calculation unit.

[0144] like Figure 7 As shown, the time prediction module 215 includes a second calculation unit 71 and a second comparison calculation unit 72 corresponding to each resource.

[0145] In this embodiment, on the one hand, the second calculation unit determines the waiting time of resources based on the amount of tasks released and the average release amount of the previous cycle, so as to accurately predict the resource release trend in the next cycle, realize intelligent load prediction and quantify the resource release pattern, and improve the dynamic resource prediction capability; on the other hand, the second comparison calculation unit uses the maximum waiting time of each resource as the waiting time of the execution module to ensure that all resources of the execution module can meet the demand, so as to accurately quantify the resource supply and demand relationship. At the same time, the waiting time of each execution module can guide the working group to run on the optimal execution module, so as to more dynamically measure the scheduling delay and realize multiple optimizations among performance, energy efficiency and latency.

[0146] In some implementations, the resource release amount of the execution module in the current clock cycle includes the amount of tasks released in the current clock cycle. The second calculation unit includes a moving average calculation unit, a reciprocal calculation unit, and a second multiplication calculation unit. The moving average calculation unit is used to determine the average release amount of resources in the current clock cycle based on the amount of tasks released in the current clock cycle and the average release amount of resources in the previous clock cycle. The reciprocal calculation unit is used to determine the reciprocal of the average release amount of resources in the current clock cycle. The second multiplication calculation unit is used to determine the third product between the reciprocal and the first difference, and use the third product as the second waiting time of the resources. The first difference is the difference between the demand for resources and the remaining amount of resources in the execution module.

[0147] Here, the moving average calculation unit can be any suitable hardware unit capable of implementing this function. In implementation, the process by which the moving average calculation unit determines the average release amount of resources in the current cycle can be found in the specific implementation of steps S151 to S154 above.

[0148] The reciprocal calculation unit can be any suitable hardware unit capable of implementing this function. In implementation, the reciprocal calculation unit is connected to the moving average calculation unit, and the average release amount of resources output by the moving average calculation unit in the current cycle is inversely calculated, and the reciprocal is output.

[0149] The second multiplication calculation unit can be any suitable hardware unit capable of implementing this function, such as a multiplier. In implementation, the second multiplication calculation unit is connected to the reciprocal calculation unit. After obtaining the first difference, it calculates the product between the reciprocal output by the reciprocal calculation unit and the first difference, and outputs the product as the second waiting time.

[0150] like Figure 8 As shown, the second calculation unit 71 includes a moving average calculation unit 711, a reciprocal calculation unit 712, and a second multiplication calculation unit 713.

[0151] In some embodiments, the second calculation unit may further include a difference calculation unit, which is used to calculate the difference between the demand for resources and the remaining amount of resources. The difference calculation unit may be any suitable hardware unit capable of performing this function.

[0152] In this embodiment, on the one hand, different resources are assigned different weights, which improves the rationality of the weights and achieves the best optimization effect; on the other hand, the second calculation unit determines the average release amount of the current round based on the weight of the resources, the weight of the task, the average release amount of the resources in the previous round, and the release amount of the task, so as to quantify the relative importance of resources and tasks and improve the ability to accurately predict resources.

[0153] The technical solution of this disclosure is described in detail below.

[0154] In modern general-purpose GPU hardware design, for use cases such as general computing, AI (Artificial Intelligence) training, and inference, the GPU usually receives kernels one by one. The GPU's front-end module splits a kernel into multiple workgroups. Multiple workgroups can be distributed to different execution modules (such as SM) to execute the instructions of the compute shaders and output the results to memory.

[0155] Efficient and balanced distribution of workgroups across multiple task managers (SMs) largely determines the overall performance of the GPU. Since the overall runtime is limited by the SM with the longest runtime, uneven distribution can lead to significant differences in runtime between different SMs.

[0156] Currently, commonly used load balancing methods include: (1) Round-robin scheduling assigns the workgroups to be executed to each SM in sequence, and the process is repeated. The advantage is that the hardware implementation is simple, no need to save additional running state or workgroup information, and no additional area overhead. However, the disadvantages are also obvious. It cannot handle scenarios where there are large differences in resources between workgroups and large differences in the number of compute shader instructions. It is only suitable for scenarios where the execution time of workgroups in the kernel is similar and the resource usage is uniform.

[0157] (2) Feedback-controlled scheduling: Downstream SMs typically have a fixed-depth buffer at their entry point, and the scheduler records the available buffer counts for all SMs. Initially, the available buffer count for each SM is at its maximum depth. Each time a workgroup is assigned to an SM, the available buffer count for that SM decreases by 1. When a downstream SM consumes a workgroup from its input buffer, it feeds back to the scheduler via an interface, and the scheduler increments the available buffer count for that SM by 1. During arbitration, the scheduler selects the SM with the largest available buffer count for assignment. The advantage is that it can select the SM that can currently accept workgroups based on real-time hardware feedback, ensuring the saturation of downstream load. The disadvantage is that this method does not consider the differences between different kernels, resource consumption, and runtime.

[0158] (3) Resource-aware scheduling: The scheduler determines the next workgroup assignment target based on the current resource usage of each SM, such as the number of remaining GPRs, the amount of shared access memory available, and the number of remaining runnable tasks. For example, based on the key resources configured in the software, such as the number of GPRs, the remaining resources of different SMs are compared to find the SM with the largest remaining resources. The advantage of this method is that it can dynamically determine the situation based on the remaining resources. The disadvantage is that it requires maintaining the state of each SM, and the judgment will not be accurate when resources are saturated. This is because the available resources of each SM are close to 0 at this time, and it is impossible to predict which SM will release resources first.

[0159] Therefore, most existing technologies rely on polling or resource-aware methods to arbitrate among multiple shader execution modules. Polling-based arbitration has relatively low hardware overhead and performs well when the size of the working cores and the shader complexity are similar, but it performs poorly in scenarios with significant differences in the number of working cores. Resource-aware methods select based on the remaining resources of each shader execution module, but after a certain period, when resources reach saturation, accurate judgment becomes impossible. This is because once multiple shader execution modules are resource-saturated, the resource differences between them become smaller. At this point, arbitrating based on remaining resources is susceptible to resource fluctuations or fails to detect which execution module will release resources faster, leading to the selection of a suboptimal execution module.

[0160] This solution aims to combine the advantages of resource awareness and time prediction methods to implement an algorithm and related devices for arbitration selection of workgroups in scenarios of resource idleness and resource saturation.

[0161] When resources are sufficient, the bottleneck resource algorithm is used to find the optimal choice for the arbitration workgroup among multiple shader execution modules. By comparing the bottleneck resource scores of each shader execution module (corresponding to the aforementioned first score), the execution module with the most abundant resources is selected for assignment. Simultaneously, when resources are saturated, time feedback and time prediction methods are used to perceive and predict the resource release time of each shader execution module, and the optimal solution is selected by comparing the waiting times of different execution modules (corresponding to the aforementioned first waiting time).

[0162] Under certain hardware constraints, a design scheme for a GPU workgroup arbitration module is proposed to distribute multiple workgroups to different execution units and achieve good load balancing.

[0163] This solution aims to propose a working group distribution module (corresponding to the aforementioned arbitration module) that predicts the resource status and waiting time of different shader execution modules based on some static characteristic information of the working core, dynamic resource information and running status feedback from the hardware in real time, and finds the most suitable shader execution module (such as the one with the most sufficient resources or the shortest waiting time) based on the above information.

[0164] like Figure 9 As shown, the workgroup arbitration module 91 (corresponding to the aforementioned arbitration module) is connected to the upstream workgroup splitting module 92 and the downstream shader execution module 93 (corresponding to the aforementioned execution module), respectively. The workgroup arbitration module 91 includes four bottleneck resource calculation modules 911 (corresponding to the aforementioned resource calculation modules), a first comparison calculation module 212, four time prediction modules 215, a second comparison calculation module 216, a resource recording module 214, and a distribution module 213.

[0165] First, we need to define the information that the hardware and software can obtain through the arbitration module of this working group: The software can primarily provide information about the working core's characteristics, including: (1) The number of workgroups contained in the work core; (2) The number of tasks that each work group can be broken down into; (3) The amount of resources used by each workgroup, including: the size of shared memory, the size of general-purpose registers (GPR), the number of waves in the workgroup, the coefficient for calculating shared memory resources (corresponding to the coefficient for shared memory), the coefficient for calculating general-purpose register resources (corresponding to the coefficient for GPR), and the coefficient for calculating the number of waves in the workgroup (corresponding to the coefficient for the wave in the wave group).

[0166] The downstream shader execution module in the hardware can provide feedback on the following information: the working core ID corresponding to the task, the working group ID corresponding to the task, and the task completion information.

[0167] The hardware working group arbitration module can statistically or calculate the following information: the number of resources released per cycle based on the moving average.

[0168] from Figure 9 As can be seen, the software driver distributes several working kernels to the hardware. The hardware's front-end module then divides these working kernels into multiple workgroups and distributes the workgroups to the workgroup arbitration module. The workgroup arbitration module can obtain the characteristic information of the working kernels belonging to that workgroup from upstream, including the resource information required by the working kernels. Simultaneously, the workgroup arbitration module can receive feedback information from the downstream shader execution module, including the working kernel information, workgroup information, and completion information of a specific task.

[0169] The workflow of the arbitration module in this working group is divided into two types, corresponding to two different scenarios: (1) When the resources of one or more shader execution modules can satisfy the resource requests of the workgroup, the process will proceed. Figure 9 The first workflow (shown by the long dashed line) selects the best shader execution module through the bottleneck resource calculation module and the first comparison calculation module, and sends the work group information to the distribution module for dispatch.

[0170] (2) When the resources of all shader execution modules are insufficient, a process will occur. Figure 9 The second workflow (shown by the short dashed line) uses the time prediction module and the second comparison calculation module to predict the shader execution module with the shortest waiting time for resources, and then sends the work group information to the distribution module for dispatch.

[0171] First workflow: The bottleneck calculation module first compares whether the resources required by the work group can be satisfied by a certain calculation shader execution module. If they are satisfied, the bottleneck resource score is calculated, and the shader execution module with the highest score is selected as the target result of arbitration.

[0172] The second workflow: If the bottleneck calculation module finds that the resources cannot be met after comparing resources, it will send the request to the first comparison calculation module. When the first comparison calculation module sees that the resources of all shader execution modules cannot be met, it will rebroadcast the request of the work group to several time prediction modules to perform time prediction calculations. The calculation results will be given to the second comparison calculation module, which will select the shader execution module with the shortest waiting time as the target result of arbitration.

[0173] The next step is to explain the working principle of the working group arbitration module: The first step is to select the shader execution module with the highest score that meets the resource requirements of the workgroup from all shader execution modules and assign it to the workgroup.

[0174] The resource recording module records the remaining size of various resources for each shader execution module. The bottleneck resource calculation module calculates a score for each shader execution module regarding bottleneck resources (corresponding to the aforementioned second score) through logical and multiplicative calculations, and sends this score to the first comparison calculation module. The first comparison calculation module selects the shader execution module number with the highest score as the target for the current workgroup assignment and assigns the workgroup to the distribution module. The distribution module assigns the workgroup to the shader execution modules and simultaneously sends the workgroup's resource size to the resource recording module, updating the remaining resource information as follows: R_available = R_available - R_workgroup. Here, R_available represents the remaining resource information, and R_workgroup represents the workgroup's resource size.

[0175] If the remaining resources of the shader execution modules cannot meet the needs of the current workgroup, the bottleneck resource calculation module will output a score of 0. Furthermore, when the first comparison calculation module receives scores of 0 from all shader execution modules, it will rebroadcast the current workgroup to multiple time prediction modules, each corresponding to a downstream shader execution module. This time prediction module will calculate the waiting time for the corresponding shader execution module to release resources and output it to the second comparison calculation module. The second comparison calculation module will select the shader execution module with the shortest waiting time and output it to the distribution module.

[0176] The following details the design of the bottleneck resource computing module, with the structure as follows: Figure 5 As shown.

[0177] The bottleneck resource calculation module takes into account the resource requirements of the work group and the remaining resource information of the current shader execution module issued by the resource recording module, and outputs the minimum score of all resources to the first comparison calculation module.

[0178] The bottleneck resource calculation module internally contains three primary calculation units, each corresponding to one of the three resources of the workgroup (shared memory, general-purpose registers, and the number of tasks in the workgroup). Each primary calculation unit consists of a comparison calculation unit, a shift calculation unit, and a primary multiplication calculation unit. The bottleneck resource calculation module also includes a multiplexer (corresponding to the aforementioned selection unit) and a primary comparison calculation unit.

[0179] The detailed processing steps are as follows: The first step is for the resource management module to maintain a table that records the remaining resources for each shader execution module, including the amount of shared memory available, the amount of general-purpose registers available, and the number of tasks that the shader execution module can run. Assuming there are four shader execution modules, the table would contain the following:

[0180] The initial value of the remaining resources for each shader execution module is the maximum value of the resources for that shader execution module.

[0181] For each workgroup assigned to a shader execution module, the available resources of that shader execution module need to be subtracted from the resources required to arbitrate the workgroup to that shader execution module. Rshared_mem_avalible = Rshared_mem_available – Rshared_mem_req; Rgpr_available = Rgpr_available – Rgpr_req; Rtask_available = Rtask_available – Rtask_req; The first comparison calculation module calculates the bottleneck resource score (corresponding to the aforementioned first score) of the working group for different shader execution modules based on the remaining available resources of each shader execution module and the requested resources of the working group to be arbitrated. Then, it finds the shader execution module with the highest score among all shader execution modules as the target of the working group's arbitration.

[0182] Specifically as follows: = MIN( , ); in, The bottleneck resource score of the i-th shader execution module for the arbitration working group is calculated by taking the minimum value of the three resources within the shader execution module.

[0183] Then, the maximum value of the bottleneck resource score among all shader execution modules is found and used as the target result for arbitration.

[0184] Because division incurs excessive overhead for the shader execution module, it needs to be converted, i.e.: ; in, as well as These are the characteristic information of the working core, which can be pre-calculated on the CPU by software. However, the hardware overhead and area of ​​the multiplier may still be relatively large.

[0185] Therefore, further modifications are made here, redefining the three parameters calculated by the software as follows: shared_mem_coeff = gpr_req * wave_req; gpr_coeff = shared_mem_req * wave_req; wave_coeff = shared_mem_req * gpr_req; ; Here, I, J, and K represent the number of bits shifted right by the shifter, which reduces the number of bits in the multiplier input parameters to save some overhead. However, these three values ​​need to be weighed and implemented between the actual arbitration result and the area overhead to select the optimal values.

[0186] The number of shifts determined by the shifter determines the input bit width of the multiplier and also affects the accuracy of the score calculation for each resource. For resources with larger allocation granularity and a larger total resource volume, such as shared memory (shared_mem), a larger shift value can be chosen; for resources with smaller allocation granularity and a smaller total resource volume, such as thread bundle resource (wave_id), a smaller shift value can be chosen. In some implementations, the shift amount can be calculated using a coefficient, as shown in the aforementioned formula (1-1). Assuming the first coefficient is 0.2, then shared memory has a total of 1024 allocation units, which, after taking the logarithm, equals 10. Multiplying by 0.2 equals 2, so the final right shift is 2 bits. Assuming thread bundle resources have a total of 64 allocation units, after taking the logarithm, equals 6. Multiplying by 0.2 equals 1.2, rounded down to 1, so the final right shift is 1 bit.

[0187] When performing the above calculations, it is first necessary to compare the size of the requested resources with the size of the remaining available resources of a certain shader execution module. If the remaining resources cannot meet the needs of the workgroup, there is no need to perform resource multiplication to obtain the bottleneck resource score of that shader execution module. Instead, a fixed score, such as 0, can be directly output to allow downstream systems to determine that the shader execution module cannot currently meet the requirements, and it is necessary to check the resource status of other shader execution modules or wait for the resources of a certain shader execution module to meet the requirements.

[0188] The second step involves estimating the waiting time when the resources of all shader execution modules are insufficient to meet the needs of the workgroup. At this point, the first comparison calculation module does not send the workgroup information to the distribution module; instead, it broadcasts it to the downstream time prediction module.

[0189] The design details of the time prediction module are explained in detail below, such as... Figure 7 As shown: The input information for the time prediction module includes the workgroup requirement information forwarded by the first comparison calculation module, as well as the task completion and resource release information reported by the resource recording module.

[0190] The moving average calculation unit will use the number of resources released in each round to calculate the number of resources released in each round by moving average. Specifically, it can be calculated using the aforementioned formula (1-2).

[0191] The reciprocal calculation unit takes the resource release rate of a single task obtained by the moving average calculation unit, calculates the reciprocal value, and obtains 1 / .

[0192] The second multiplication calculation unit subtracts the remaining resource quantity from the requested resource quantity to obtain the resource quantity that needs to be waited for. It then uses the reciprocal of the resource release rate (1 / ...) obtained from the interpolation and reciprocal calculation unit... Multiply them to get the average waiting time for resources.

[0193] The detailed processing procedure is as follows: First, a table needs to be managed for each shader execution module in the resource management module. The depth of this table is the maximum number of tasks that the shader execution module can execute in parallel. The table content includes: table entry validity, kernel ID of the task, workgroup ID of the task, shared memory request size of the workgroup, gprrequest size of the workgroup, number of tasks in the workgroup, and number of tasks remaining in the workgroup.

[0194] When the resource management module receives a workgroup information update request from the distribution module, it will add or update the kernel id, workgroup id, shared mem request size, gpr request size, tasks in workgroup, and tasks remaining in workgroup to the corresponding table in the shader execution module.

[0195] If the kernel id and workgroup id do not match in the table, a new table entry is added, the valid field of the entry is set to 1, and the other fields are assigned values ​​according to the characteristics of the workgroup.

[0196] If the number of remaining tasks in the workgroup is 0, then the validity of the table entry is set to 0.

[0197] At the same time, the resource recording module will also receive task completion information reported by the shader execution module. Upon receiving this information, the resource recording module will perform the following actions: After the shader execution module completes its task, it sends the kernel ID and workgroup ID to the resource recording module. The resource recording module will query the table entries corresponding to the current shader execution module to find the amount of resources that the current task will release; If the amount of resources released is greater than 0, the value will be reported to the time prediction module. Decrease the number of remaining tasks to be executed in the workgroup in the table by 1.

[0198] The time prediction module will receive the resource quantity information for the current task release from the resource management module in each cycle.

[0199] The time prediction module calculates the amount of resources released in each cycle using a moving average method, as shown in formula (1-2). In formula (1-2)... It is a small constant used to protect hardware from overflow during initialization or when resources are not released for a long time. It is a weighting factor that can be used to optimize and select different resource types, and its value range is (0, 1).

[0200] Typically, a workgroup is divided into multiple thread bundles that are executed in the shader execution module, and shared memory is only released after all the thread bundles of the entire workgroup have completed. Therefore, the frequency of changes to shared memory is lower than that of general-purpose registers and thread bundle IDs. Thus, a smaller weighting factor can be chosen to calculate the moving average for shared memory.

[0201] For the three resources, the selectable values ​​are as follows: Shared memory uses a small weighting factor to smooth out fluctuations, with values ​​ranging from 0.05 to 0.2.

[0202] General-purpose registers and thread bundle IDs use a large weighting factor, with values ​​ranging from 0.3 to 0.8.

[0203] In each cycle, there are two possibilities: The current task terminates from the shader execution module, and the resource management module can simultaneously locate and report the amount of resources released by the task. At this point, the above formula (1-2)... If the result is greater than 0, a moving average will be obtained.

[0204] When no task ends from the shader execution module in the current frame, the resource management module will report that the number of released resources is 0. At this time, the result of the above formula (1-2) is the result of the historical release amount gradually decreasing, which is used to achieve the effect of moving average.

[0205] Here, the time prediction module performs parallel calculations on three resources: shared memory, GPR, and task number. After obtaining the average resource release amount, the value is sent to the reciprocal calculation module to calculate the reciprocal result. Then, the reciprocal calculation unit sends the result to the multiplication calculator to calculate the waiting time for the resource, as shown in the formula (1-3) above.

[0206] After obtaining the waiting times for the three resources, the data is sent to the second comparison and calculation unit to retrieve the maximum value of the three resource waiting times. This maximum value is then sent to the second comparison and calculation module. The second comparison and calculation module selects the execution module number with the shortest waiting time from all time prediction modules and outputs it to the distribution module.

[0207] In summary, this scheme proposes an arbitration algorithm fusion mechanism: a) Resource availability is quantified and calculated using the bottleneck resource ratio, and a hardware-friendly design and implementation method is proposed; b) The prediction of resource release rate is quantified and calculated using the moving average method, and a hardware-friendly design and implementation method is proposed; c) A fusion judgment mechanism is defined, which can evaluate the current state of the workgroup and shader execution module when resources are sufficient or insufficient, and select the best shader execution module for deployment; d) This method can be used to address the difficulty of traditional methods in dynamically balancing short-term / long-term resource status for working group arbitration; This solution proposes a method for performing calculations separately for different resource types: a) A shift calculation unit has been added to the implementation of the most bottleneck resource calculation, which can select different shift amounts for different resource types and perform different optimization processes. b) The moving average calculation also uses an independent calculation unit for different resource rationalities, and different weighting factors can be used for different resource types to optimize the best prediction effect.

[0208] Therefore, to improve resource utilization and scheduling decision quality among multiple shader execution modules in a GPU for workgroup scheduling, this paper proposes a scheduling scheme that integrates a bottleneck resource arbitration algorithm with a resource release time estimation mechanism based on moving average. This scheme comprehensively considers the current resource usage status and the dynamic trend of future resource release of the shader execution modules, thereby achieving more accurate and timely scheduling selection.

[0209] Specifically, the bottleneck resource arbitration algorithm determines the resource capacity of each shader execution module by evaluating the relatively scarcest resources (such as GPR, Shared Memory, and Warp Slot) and uses this to guide workgroup allocation, exhibiting advantages such as high responsiveness and resource awareness. Meanwhile, the moving average resource release time prediction mechanism uses historical task completion data to estimate the rate at which shader execution modules release resources in future cycles, predicting the time required for each execution module to release sufficient resources, thus measuring scheduling latency more dynamically.

[0210] The combined strategy can simultaneously take into account the current resource availability (bottleneck resources) and resource reclamation speed (release rate), effectively reducing the probability of workgroup blocking under resource shortages or uneven loads, and improving the overall system throughput and scheduling fairness.

[0211] This approach is particularly suitable for scenarios with multiple working cores running in a mixed manner or severe resource fragmentation, and can significantly improve the blindness of traditional static or greedy arbitration methods.

[0212] The description of the graphics processor embodiments above is similar to the description of the method embodiments above, and has similar beneficial effects. For technical details not disclosed in the graphics processor embodiments of this disclosure, please refer to the description of the method embodiments of this disclosure for understanding.

[0213] It should be noted that, in the embodiments of this disclosure, if the above methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this disclosure, or the parts that contribute to related technologies, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this disclosure are not limited to any specific hardware and software combination.

[0214] This disclosure provides an electronic device including any of the processors described above.

[0215] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this disclosure. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this disclosure, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure. The sequence numbers of the above-described embodiments are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0216] In the several embodiments provided in this disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0217] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0218] In addition, each functional unit in the embodiments of this disclosure can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0219] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0220] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, magnetic disks, or optical disks.

[0221] The above description is merely an embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for scheduling workgroups, characterized in that, In the arbitration module applied to the graphics processor, the graphics processor further includes at least two execution modules, and the scheduling method includes: For each execution module, a first score is determined based on the resource request of the work group and the remaining resource information of the execution module. The first score of the execution module indicates whether the remaining resource information of the execution module can meet the resource request of the work group. If the first score of each execution module is less than the first score threshold, a target execution module is determined from the at least two execution modules based on the first waiting time of each execution module. The first waiting time of the execution module is the predicted time required for the execution module to release target resource information. The target resource information is determined based on the resource request of the workgroup and the remaining resource information of the execution module. The workgroup is scheduled to the target execution module.

2. The scheduling method according to claim 1, characterized in that, The resource request of the working group includes the demand for at least one resource, and the remaining resource information of the execution module includes the remaining amount of each of the resources. The determination of the first score for the execution module based on the resource requests of the workgroup and the remaining resource information of the execution module includes: For each resource, a second score is determined based on the demand for the resource and the remaining quantity of the resource; A first score for the execution module is determined based on a second score for each of the resources.

3. The scheduling method according to claim 2, characterized in that, The determination of the second rating of the resource based on the demand and remaining quantity of the resource includes: If the demand for a resource is not greater than the remaining amount of the resource, the remaining amount of the resource is shifted based on the shift number corresponding to the resource to obtain the shift result of the resource. Based on the shift result of the resource, a second score for the resource is determined; wherein, the shift number corresponding to the resource is determined based on the total amount of the resource in the execution module. If the demand for a resource exceeds the remaining amount of the resource, a preset first target score is used as the second score for the resource.

4. The scheduling method according to claim 3, characterized in that, Determining the second score of the resource based on the shift result includes: Obtain the coefficient corresponding to the resource; wherein the coefficient corresponding to the resource is determined based on the demand for other resources; A second score for the resource is determined based on the first product between the shift result and the coefficient corresponding to the resource.

5. The scheduling method according to claim 2, characterized in that, The determination of the first score of the execution module based on the second score of each of the resources includes: If the second score of each of the resources is greater than the set second score threshold, the lowest second score will be used as the first score of the execution module. If the second score of at least one of the resources is not greater than the second score threshold, the preset second target score is used as the first score of the execution module.

6. The scheduling method according to claim 1, characterized in that, The step of determining the target execution module from the at least two execution modules based on a first waiting time for each of the execution modules includes: The target first waiting time is determined based on the first waiting time of each of the aforementioned execution modules; From the at least two execution modules, the execution module that is adapted to the target first waiting time is selected as the target execution module.

7. The scheduling method according to any one of claims 1 to 6, characterized in that, Before determining the target execution module, the scheduling method further includes: For each execution module, the first waiting time is determined based on the resource requests of the workgroup and the amount of tasks released by the execution module in the current clock cycle.

8. The scheduling method according to claim 7, characterized in that, The resource requests of the working group include the demand for at least one resource; The determination of the first waiting time for the execution module based on the resource requests of the workgroup and the amount of tasks released by the execution module in the current clock cycle includes: For each resource, based on the amount of tasks released in the current clock cycle and the average amount of resources released in the previous clock cycle, the average release amount of the resource in the current clock cycle is determined, and based on the average release amount of the resource in the current clock cycle, the second waiting time of the resource is determined. The first waiting time of the execution module is determined based on the second waiting time of each of the resources.

9. The scheduling method according to claim 8, characterized in that, Determining the average release amount of the resource in the current clock cycle based on the amount of tasks released in the current clock cycle and the average release amount of the resource in the previous clock cycle includes: Based on the first weight corresponding to the resource, a second weight corresponding to the task volume is determined; wherein, different resources correspond to different first weights; Determine the first product between the first weight and the average release amount of the resource in the previous clock cycle; Determine the second weight and the second product between the amount of task released in the current clock cycle; The average release amount of the resource in the current clock cycle is determined based on the sum between the first product and the second product.

10. The scheduling method according to claim 8, characterized in that, Determining the second waiting time of the resource based on the average release amount of the resource in the current clock cycle includes: Determine the first difference between the required amount of the resource and the remaining amount of the resource in the execution module; The second waiting time of the resource is determined based on the ratio between the first difference and the average release amount of the resource in the current clock cycle.

11. A graphics processor, characterized in that, It includes an arbitration module and at least two enforcement modules, wherein: The arbitration module is used to determine a first score for each execution module based on the resource request of the workgroup and the remaining resource information of the execution module. The first score of the execution module indicates whether the remaining resource information of the execution module can satisfy the resource request of the workgroup. If the first score of each execution module is less than a first score threshold, a target execution module is determined from the at least two execution modules based on a first waiting time of each execution module. The first waiting time of the execution module is the predicted time required for the execution module to release the target resource information, and the target resource information is determined based on the resource request of the workgroup and the remaining resource information of the execution module. The workgroup is then scheduled to the target execution module. The target execution module is used to execute each task in the workgroup.

12. The graphics processor according to claim 11, characterized in that, The arbitration module includes a resource calculation module, a first comparison module, a distribution module, and a resource recording module corresponding to each of the execution modules, wherein: The resource recording module is used to record the remaining resource information of each of the execution modules; Each of the resource calculation modules is used to determine the first score of the corresponding execution module based on the resource request of the work group and the remaining resource information of the corresponding execution module; The first comparison module is used to select the execution module corresponding to the maximum first score as the target execution module when the first score of at least one execution module is not less than the first score threshold, and to pass the identification information of the working group and the target execution module to the distribution module. The distribution module is used to schedule the workgroup according to the identification information of the target execution module; and send the resource request of the workgroup and the identification information of the target execution module to the resource recording module. The resource recording module is also used to update the remaining resource information of the target execution module based on the resource requests of the workgroup.

13. The graphics processor according to claim 12, characterized in that, The resource request of the working group includes the demand for at least one resource, the remaining resource information of the execution module includes the remaining amount of each of the resources, and the resource calculation module includes a first calculation unit and a first comparison calculation unit corresponding to each of the resources, wherein: Each of the first calculation units is used to determine the second score of the resource based on the corresponding resource demand and the remaining amount of the resource in the execution module corresponding to the resource calculation module; The first comparison calculation unit is used to determine the first score of the execution module corresponding to the resource calculation module based on the second score of each resource.

14. The graphics processor according to claim 13, characterized in that, The first calculation unit includes a comparison calculation unit, a shift calculation unit, and a first multiplication calculation unit; The comparison calculation unit is used to compare the demand for the resource with the remaining amount of the resource and output the comparison result; The shift calculation unit is used to shift the remaining amount of the resource according to the shift number corresponding to the resource when the comparison result is the first comparison result, to obtain the shift result; wherein, the first comparison result indicates that the remaining amount of the resource is not less than the required amount of the resource, and the shift number corresponding to the resource is determined based on the total amount of the resource in the execution module; The first multiplication calculation unit is used to determine the first product between the shift result and the coefficient corresponding to the resource, and to use the first product as the second score of the resource.

15. The graphics processor according to claim 14, characterized in that, The resource calculation module further includes a selection unit, wherein: The selection unit is used to determine the selection result based on the comparison result output by each of the comparison calculation units; The first comparison calculation unit is further configured to, when the selection result is the first selection result, use a preset second target score as the first score of the execution module; wherein, the first selection result indicates that the comparison result output by at least one comparison calculation unit is the second comparison result, and the second comparison result indicates that the remaining amount of the resource is less than the demand amount of the resource; and when the selection result is the second selection result, use the smallest second score as the first score of the execution module.

16. The graphics processor according to any one of claims 12 to 15, characterized in that, The arbitration module further includes a time prediction module and a second comparison module corresponding to each of the execution modules, wherein: The first comparison module is further configured to broadcast the resource request of the working group if the first score of each of the execution modules is less than the first score threshold. The resource recording module is further configured to receive task information corresponding to each clock cycle sent by each of the execution modules; and update the remaining resource information of each execution module based on the task information corresponding to each clock cycle sent by each execution module. Each of the time prediction modules is used to determine the first waiting time of the execution module based on the resource requests of the workgroup and the amount of tasks released by the corresponding execution module in the current clock cycle; The second comparison module is used to compare the first waiting time of each of the execution modules, and select the execution module with the smallest first waiting time as the target execution module.

17. The graphics processor according to claim 16, characterized in that, The resource request of the working group includes the demand for at least one resource, and the time prediction module includes a second calculation unit and a second comparison calculation unit corresponding to each resource, wherein: Each of the second computing units is used to determine the second waiting time of the resource based on the corresponding resource demand and the resource release amount of the corresponding execution module in the current clock cycle; The second comparison calculation unit is used to compare the second waiting time of each resource and take the largest second waiting time as the first waiting time of the execution module.

18. The graphics processor according to claim 17, characterized in that, The resource release amount of the execution module in the current clock cycle includes the amount of tasks released in the current clock cycle. The second calculation unit includes a moving average calculation unit, a reciprocal calculation unit, and a second multiplication calculation unit, wherein: The moving average calculation unit is used to determine the average release amount of the resource in the current clock cycle based on the amount of tasks released in the current clock cycle and the average release amount of the resource in the previous clock cycle. The reciprocal calculation unit is used to determine the reciprocal of the average release amount of the resource in the current clock cycle; The second multiplication calculation unit is used to determine the third product between the reciprocal and the first difference, and to use the third product as the second waiting time of the resource; wherein, the first difference is the difference between the demand of the resource and the remaining amount of the resource in the execution module.

19. An electronic device, characterized in that, The graphics processor included in any one of claims 11 to 18.

Images