A Multi-Queue Adaptive Scheduling Method and System for GPUs

Abstract

A multi-queue adaptive scheduling method and system for GPUs is disclosed. The method includes: obtaining a queue affinity matrix, which indicates the overall parallel efficiency index, resource conflict probability, and context switching overhead among queues; determining the current running status of each queue in the GPU and the task characteristic fingerprint of at least one reference task; and using a multi-objective optimization algorithm to determine the optimal assignment scheme for at least one reference task based on the current running status of each queue in the GPU, the queue affinity matrix, and the characteristic fingerprint of at least one reference task. Thus, by using the queue affinity matrix, the queue parallelism behavior of domestic GPUs is transformed from a black box to a white box, making task assignment no longer a guess-based static strategy, but a data-driven decision based on the actual hardware characteristics.

Description

Technical Field

[0001] This application relates to the fields of computer graphics and high-performance computing technology, and in particular to a multi-queue adaptive scheduling method and system for GPUs. Background Technology

[0002] With the widespread support of domestically produced graphics processing units (GPUs) for the cross-platform graphics application programming interface (Vulkan API), the key is to fully utilize their hardware potential.

[0003] Vulkan's multi-queue model is core to improving parallelism, but in the existing drivers and hardware implementations of domestic GPUs, directly applying traditional solutions faces the problem of uncertain queue parallelism efficiency. Whether different queue families (transfer, computation, graphics) are truly hardware parallel or time-division multiplexed is a black box for application developers. Due to the lack of knowledge about the actual scheduling mechanism of the GPU hardware, fixed task allocation strategies are difficult to optimize based on hardware characteristics, resulting in the inability to fully utilize the GPU's parallel processing capabilities and causing low hardware resource utilization.

[0004] Therefore, how to provide a solution that can automatically sense hardware status and dynamically adjust scheduling strategies is a key research topic for those skilled in the art. Summary of the Invention

[0005] Firstly, this application provides a multi-queue adaptive scheduling method for GPUs. The method includes: obtaining a queue affinity matrix, wherein the queue affinity matrix is ​​determined based on the actual parallel processing performance characteristics of a GPU for a test queue combination, the test queue combination being a combination of arbitrary elements from a multiset of all queues supported by the GPU, and the queue affinity matrix being used to indicate the overall parallel efficiency index, resource conflict probability, and context switching overhead among the queues; determining the current operating status of each queue in the GPU and the task characteristic fingerprint of at least one reference task, wherein the current operating status of the queues includes load rate and backlog of pending tasks. The task fingerprint includes at least one of the following: memory utilization and power consumption status. The task characteristic fingerprint includes task type attributes, resource dependencies, and priority information. The reference task is any one of the tasks waiting to be assigned and the tasks that have been assigned and are waiting to be processed. Based on the current running status of each queue in the GPU, the queue affinity matrix, and the characteristic fingerprint of the at least one reference task, a multi-objective optimization algorithm is used to determine the optimal assignment scheme for the at least one reference task. The optimization objectives of the multi-objective optimization algorithm include at least one of the following: optimizing load balancing, minimizing synchronization overhead, minimizing power consumption, and minimizing resource conflicts while satisfying task priority requirements as much as possible.

[0006] The multi-queue adaptive scheduling method for GPUs provided in this application transforms the queue parallelism behavior of domestic GPUs from a black box to a white box through the queue affinity matrix, making task assignment no longer a static strategy based on guesswork, but a data-driven decision based on the actual characteristics of the hardware.

[0007] On the other hand, by comprehensively considering real-time operational status and task characteristic fingerprints, dynamic load balancing can be achieved. For example, tasks of a specific type can be automatically migrated from the busy queue to an idle queue that has the ability to process tasks of that specific type and has high parallel efficiency, in order to maximize GPU utilization.

[0008] On the other hand, multi-objective optimization not only includes performance optimization (load balancing, minimizing synchronization overhead), but also power consumption and resource conflict optimization. While pursuing high performance, it can also meet energy efficiency requirements, and improve system stability by avoiding resource conflicts.

[0009] In some possible implementations, obtaining the queue affinity matrix includes: enumerating the queue families supported by the GPU through a standard interface, and obtaining the basic attributes and functional flags of each queue family, the basic attributes including the number of queues; determining a queue set based on the queue families and the number of queues supported by each queue family, the queue set being a multiset containing all queues supported by each queue family; taking any combination of elements in the queue set as a test queue combination, and submitting a corresponding test task sequence to each queue in the test queue combination based on the functional tags of the queues in the test queue combination; measuring the execution time and resource consumption of the test queue combination for individual and parallel execution of the test task sequence, and obtaining the test data of the test queue combination; and constructing the queue affinity matrix based on the test data of the test queue combination.

[0010] By using this method, through comparative tests of individual and parallel execution, the efficiency gains brought by parallelism or the performance degradation caused by conflicts can be accurately quantified, providing a highly reliable data foundation for scheduling decisions.

[0011] In some possible implementations, the queues supported by the GPU include a graphics queue, a compute queue, and a transfer queue. The queue affinity matrix indicates that both the graphics queue and the compute queue have compute capabilities. The queue affinity matrix also indicates that the parallel efficiency of the graphics queue and the compute queue, and the parallel efficiency of the transfer queue with the graphics queue and the compute queue, respectively, is greater than or equal to a preset parallel efficiency threshold (e.g., the preset parallel efficiency threshold is 0.5). The step of using a multi-objective optimization algorithm to determine the optimal allocation scheme for the at least one reference task includes: when the load state of the graphics queue meets a first preset high load state and the compute queue is relatively idle, dynamically migrating a first target task to the compute queue based on the queue affinity matrix, wherein the first target task is a separable compute shader task; when the utilization of the transfer queue is insufficient, rerouting a second target task in the graphics queue and / or the compute queue to the transfer queue, wherein the second target task is a data preprocessing task; and when the overall GPU load meets a second preset high load state, implementing differentiated scheduling based on task priority to ensure that a third target task, wherein the third target task is a critical path task.

[0012] This approach employs various adaptive strategies to offload computational tasks to dedicated computing units, resolving bottlenecks caused by graphics queue overload. When the transfer queue is underutilized, the transfer queue's DMA engine is fully utilized to route non-urgent data preprocessing tasks to the transfer queue, freeing up the graphics / computation queue to focus on core rendering and computation. Furthermore, it ensures the responsiveness of critical tasks under high system load, preventing performance degradation across all tasks during system congestion.

[0013] In some possible implementations, the tasks currently awaiting assignment and those already assigned and waiting to be processed in the GPU include at least two tasks. The method further includes: determining a first task and a second task that belong to the same queue and have resource dependencies, wherein the first task and the second task are any two tasks belonging to the same queue and having resource dependencies among the at least two tasks; determining an optimal pipeline barrier configuration between the first task and the second task based on the resource dependencies of the first task and the second task and a preset pipeline barrier configuration rule, wherein the optimal pipeline barrier configuration includes an execution barrier, or the optimal pipeline barrier configuration includes an execution barrier and a memory barrier, wherein the memory barrier includes a buffer memory barrier and / or an image memory barrier, and the preset pipeline barrier configuration rule is set with the goal of minimizing synchronization overhead while ensuring accurate satisfaction of synchronization requirements; setting the synchronization barrier between the first task and the second task as the optimal pipeline barrier configuration to narrow the synchronization scope to the actual producer pipeline stage and consumer pipeline stage involved, and, in the case where the optimal pipeline barrier configuration includes a memory barrier, to limit global memory synchronization to local buffers or image resources.

[0014] Generally, to ensure data consistency, developers often use conservative global memory barriers for synchronization within queues. This may introduce huge performance overhead, especially since domestic GPU drivers are not yet fully mature.

[0015] By replacing global memory barriers with specific pipeline stage barriers or specific resource barriers, this approach significantly reduces the scope of GPU pauses and cache flushes, thereby reducing cavitation in the GPU pipeline and improving the consistency and efficiency of task execution.

[0016] In some possible implementations, the preset pipeline barrier configuration rules include: when the resource dependency between the first task and the second task is a read-after-write dependency in read-write dependencies, the optimal pipeline barrier configuration includes an execution barrier and a preset resource memory barrier, wherein the preset resource memory barrier is a buffer memory barrier and / or an image memory barrier determined based on the synchronization resource type; when the resource dependency between the first task and the second task is a read-after-write dependency in read-write dependencies, the optimal pipeline barrier configuration includes an execution barrier; when the resource dependency between the first task and the second task is an image layout transformation dependency, the optimal pipeline barrier configuration includes an execution barrier and an image memory barrier.

[0017] In some possible implementations, the tasks currently to be assigned and those already assigned and waiting to be processed in the GPU include at least two tasks. The method further includes: determining a third task and a fourth task belonging to different queues and having resource dependencies, wherein the third task and the fourth task are any two tasks belonging to different queues and having resource dependencies among the at least two tasks; determining the dependency strength between the third task and the fourth task, wherein the dependency strength is related to at least one of the resource overlap, shared data volume, task timeliness, and queue parallelism of the third task and the fourth task; and determining the optimal synchronization primitive configuration between the third task and the fourth task based on the dependency strength of the dependency relationship between the third task and the fourth task and a preset synchronization primitive configuration rule, wherein the optimal synchronization primitive configuration is a semaphore, a fence, or an event, and the preset synchronization primitive configuration rule is set with the goal of minimizing synchronization overhead while ensuring accurate satisfaction of synchronization requirements.

[0018] In some possible implementations, the preset synchronization primitive configuration rules include: when the dependency relationship between the third task and the fourth task involves CPU awareness, selecting a barrier for synchronization; when the dependency relationship between the third task and the fourth task does not involve CPU awareness, if the dependency strength between the third task and the fourth task is greater than or equal to a preset strength threshold, or the parallel efficiency index of the queue combination corresponding to the third task and the fourth task is less than a preset efficiency threshold, then selecting a semaphore for synchronization; when the dependency strength between the third task and the fourth task is less than the preset strength threshold, and the parallel efficiency index of the queue combination corresponding to the third task and the fourth task is greater than or equal to the preset efficiency threshold, and the GPU supports cross-queue synchronization of event types, selecting an event for synchronization.

[0019] Generally, to ensure data consistency, developers often use conservative semaphores for cross-queue synchronization, which introduces huge performance overhead when domestic GPU drivers are not yet extremely mature.

[0020] This approach intelligently selects the most suitable synchronization primitive (fence, semaphore, event) based on the strength of dependencies between cross-queue tasks. Fences are used when CPU awareness is required, avoiding CPU busy waiting. For dependencies between pure GPUs, semaphores (strong dependencies, low parallel efficiency) or events (weak dependencies, high parallel efficiency) are selected based on dependency strength. While ensuring correctness, this approach chooses the lightest synchronization method for combinations of tasks with weak dependencies and good parallelism, fully leveraging the advantages of hardware synchronization mechanisms.

[0021] Secondly, this application also provides a multi-queue adaptive scheduling system for GPUs. The system includes: a hardware capability awareness and performance analysis module for acquiring a queue affinity matrix. This queue affinity matrix is ​​determined based on the actual parallel processing performance characteristics of the GPU for a test queue combination. The test queue combination is a combination of any elements from a multiset of all queues supported by the GPU. The queue affinity matrix is ​​used to indicate the overall parallel efficiency index, resource conflict probability, and context switching overhead among the queues. A resource load profile construction module is also provided for determining the current operating status of each queue in the GPU. The current operating status of the queues includes load rate, backlog of pending tasks, memory utilization, and power consumption status. The system comprises at least one of the following: a task characteristic fingerprint analysis module, used to determine the task characteristic fingerprint of at least one reference task, the task characteristic fingerprint including task type attributes, resource dependencies, and priority information, wherein the reference task is any one of tasks waiting to be assigned and tasks already assigned and waiting to be processed; and a dynamic task dispatcher, used to determine the optimal assignment scheme for the at least one reference task based on the current running status of each queue in the GPU, the queue affinity matrix, and the characteristic fingerprint of the at least one reference task, using a multi-objective optimization algorithm, wherein the optimization objectives of the multi-objective optimization algorithm include at least one of optimizing load balancing, minimizing synchronization overhead, minimizing power consumption, and minimizing resource conflicts while satisfying task priority requirements as much as possible.

[0022] Thirdly, this application also provides a GPU-oriented multi-queue adaptive scheduling apparatus, including a unit for executing any of the GPU-oriented multi-queue adaptive scheduling methods in the first aspect.

[0023] Fourthly, this application also provides a computer storage medium that can store multiple instructions, which are adapted to be loaded and executed by a processor using any of the GPU-oriented multi-queue adaptive scheduling methods described in the first aspect.

[0024] Fifthly, embodiments of this application also provide a computer program product containing instructions that, when the computer program product is run on an electronic device, cause the electronic device to execute any of the GPU-oriented multi-queue adaptive scheduling methods in the first aspect.

[0025] In a sixth aspect, embodiments of this application also provide a chip module, including a transceiver component and a chip, wherein the chip is used to execute any of the GPU-oriented multi-queue adaptive scheduling methods in the first aspect.

[0026] It is understood that the aforementioned GPU-oriented multi-queue adaptive scheduling device, computer storage medium, computer program, computer program product, and chip system are all used to execute the method shown in any implementation of the first aspect of the embodiments of this application. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding methods, and will not be repeated here. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating a multi-queue adaptive scheduling method for GPUs provided in an embodiment of this application.

[0028] Figure 2 This is a flowchart illustrating a method for determining the optimal pipeline barrier configuration for resource-dependent synchronization tasks in the same queue, as provided in an embodiment of this application.

[0029] Figure 3 This is a flowchart illustrating a method for determining the optimal synchronization primitive configuration for cross-queue resource-dependent synchronization tasks, as provided in an embodiment of this application.

[0030] Figure 4 This is a schematic diagram of the architecture of a multi-queue adaptive scheduling system for GPUs provided in an embodiment of this application;

[0031] Figure 5 This is a schematic diagram of a multi-queue adaptive scheduling device for GPUs provided in an embodiment of this application. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described below in conjunction with the accompanying drawings.

[0033] It should be noted that the embodiments of this application use an electronic device as an example to illustrate the execution subject of the GPU-oriented multi-queue adaptive scheduling method provided in this application. The electronic device can also be understood as the GPU-oriented multi-queue adaptive scheduling system or device shown in the embodiments of this application. In the embodiments of this application, the electronic device can be a microprocessor, GPU, computer, etc., used to execute program code. Any electronic device that can be used to execute the method provided in the embodiments of this application is within the protection scope of the embodiments of this application, and this application does not impose any limitations.

[0034] Please see Figure 1 , Figure 1 This document provides a flowchart of a GPU-oriented multi-queue adaptive scheduling method as an embodiment of this application. Figure 1 As shown, this GPU-oriented multi-queue adaptive scheduling method includes the following steps:

[0035] S101, Electronic device acquires queue affinity matrix.

[0036] In this embodiment, the queue affinity matrix is ​​determined based on the actual parallel processing performance characteristics of the GPU for the test queue combination. The test queue combination is a combination of any elements in the queue set, which is a multiset of all queues supported by the GPU, and the queue set includes at least three queues.

[0037] As an example, step S101 above specifically includes:

[0038] 1) Enumerate the queue families supported by the GPU through the Vulkan standard interface, and obtain the basic attributes and functional flags of each queue family. The basic attributes include the number of queues and the queue description.

[0039] For example, the queue families supported by the GPU, and the basic attributes (number of queues and description) and feature flags of each queue family are included, but are not limited to, those shown in Table 1 below.

[0040] Table 1

[0041]

[0042] 2) Determine the queue set based on the queue family and the number of queues supported by each queue family. This queue set is a multiset that includes all queues supported by each queue family.

[0043] For example, the queue families supported by the GPU are shown in Table 1 above. The above queue set is: {Image Queue, Compute Queue 1, Compute Queue 2, Transfer Queue}. The specific identifier of the queue is not limited in this article.

[0044] 3) Take any combination of elements in the queue set as a test queue combination, and submit the corresponding test task sequence to each queue in the test queue combination based on the function labels of the queues in the test queue combination.

[0045] In the embodiments of this application, the test queue combination may include two or more queues.

[0046] In this embodiment of the application, the queue affinity test is not only tested on different queues across families (e.g., graph queues and computation queues), but also on different queues within the same family (e.g., computation queues and computation queues) when multiple queues are supported within the same family.

[0047] In the embodiments of this application, in the test configuration related to queue affinity testing, the test task sequence corresponding to each queue in the test queue combination is different, and each test task sequence includes one or more lightweight tasks (the task sequence of each queue can reflect the typical workload of the queue).

[0048] For example, a test task sequence for a target queue may include one or more specific tasks corresponding to the hardware units associated with the function tags supported by the target queue. For instance, if the target queue is a graphics queue, the function tags corresponding to the graphics queue include graphics and computation. The hardware units associated with the graphics function tags include, but are not limited to, rasterization units, shader units, and memory units. The hardware units associated with the computation function tags include, but are not limited to, computationally intensive fragment shaders. Therefore, a test task sequence for the graphics queue may include one or more of the following tasks corresponding to the hardware units associated with the graphics function tags: rasterization-intensive tasks, vertex shader-intensive tasks, memory bandwidth-intensive tasks, and one or more of the following tasks corresponding to the hardware units associated with the computation function tags: computationally intensive fragment shader tasks.

[0049] As an example, the relevant test metrics for parallel performance testing of test queue combinations include, but are not limited to, one or more of the following: overall parallel efficiency metrics, computational resource conflict metrics, memory resource conflict metrics, rasterization resource conflict metrics, and cache resource conflict metrics.

[0050] In the embodiments of this application, the same queue in the same test queue combination can correspond to different test task sequences for different test metrics.

[0051] For example, for the same target queue in the same test queue combination, the test task sequence corresponding to the overall parallel efficiency index includes a first task sequence (which includes one or more tasks). The hardware units involved in the execution of the first task sequence should cover all hardware units involved in the function tags supported by the target queue as much as possible. The test task sequence corresponding to other specific resource conflict indices (such as computing resource conflict indices, memory resource conflict indices, rasterization resource conflict indices, or cache resource conflict indices) includes a second task sequence. The hardware units involved in the execution of the second task sequence should only cover the hardware units involved in the specific resource.

[0052] Ideally, there should be no data dependency between the test task sequences corresponding to different queues in the same test queue combination, so as to ensure that the two test task sequences can be executed in parallel except for potential resource competition factors, thus playing the role of controlling variables.

[0053] 4) Measure the execution time and resource consumption of the test queue combination for the individual and parallel execution of the test task sequence, and obtain the parallel test data of the test queue combination.

[0054] For example, resource usage includes, but is not limited to, resource usage related to one or more of the following shared underlying hardware resources: computing units, rasterization units, memory controllers (the main source of conflict), and cache.

[0055] For example, the overall parallel efficiency metric can be, but is not limited to, the ratio of parallel speedup to N (where N is the number of queues in the test queue combination). The parallel speedup is the ratio of the first duration to the second duration. The first duration is the sum of the durations of each of the N queues in the test queue combination executing its corresponding test task sequence individually. The second duration is the duration of executing the corresponding test task sequence in parallel with the N queues in the test queue combination. A larger overall parallel efficiency metric value indicates less parallel conflict.

[0056] For example, the memory resource conflict index can be, but is not limited to, the difference between 1 and a first ratio, where the first ratio is the ratio of the memory bandwidth of N tasks executed in parallel to the sum of the memory bandwidth of N tasks executed individually. The larger the index value, the greater the probability of parallel conflict.

[0057] For example, the computational resource conflict metric can be, but is not limited to, the difference between 1 and a second ratio, where the second ratio is the ratio of the utilization rate of computational units executing N tasks in parallel to the sum of the utilization rates of computational units executing N tasks individually. A larger metric value indicates a higher probability of conflict. Computational unit utilization refers to the proportion of time a computational unit actually spends executing instructions within a specific time window to the total time. For example, hardware resource utilization can be obtained through performance counters.

[0058] For example, the rasterization resource conflict index can be, but is not limited to, the difference between 1 and the third ratio, where the third ratio is the ratio of the raster unit utilization rate of N tasks executed in parallel to the sum of the raster unit utilization rates of N tasks executed individually. The larger the index value, the greater the probability of conflict.

[0059] For example, the cache resource conflict metric can be, but is not limited to, the ratio of a first difference to a first value. The first difference is the difference between the sum of the cache hit rates when N tasks are executed individually and the cache hit rate when N tasks are executed individually. The first value is the sum of the cache hit rates when N tasks are executed individually. The larger the metric value, the greater the probability of conflict.

[0060] 5) Based on the parallel test data of the test queue combination, construct a queue affinity matrix. This queue affinity matrix is ​​used to indicate the overall parallel efficiency index, resource conflict probability, and context switching overhead among the queues.

[0061] In this embodiment, the context switching overhead is determined by rapidly switching tasks between queues and measuring the time loss.

[0062] As an example, the queue affinity matrix includes, but is not limited to, the following Table 2.

[0063] Table 2

[0064]

[0065] S102, the electronic device determines the current running status of each queue in the GPU and the task characteristic fingerprint of at least one reference task.

[0066] In this embodiment, the reference task is any one of the tasks waiting to be assigned and the tasks that have been assigned and are waiting to be processed. The current operating status of the queue includes, but is not limited to, load rate, backlog of tasks to be processed, memory utilization, and power consumption status. The task characteristic fingerprint includes, but is not limited to, task type attribute, resource dependency, priority information (specifying priority level and / or timeliness requirements, which can be converted into priority level), and timeliness requirement information.

[0067] The task type attribute of the reference task includes graphics rendering, general computing, or data transfer task types. Resource dependencies include, but are not limited to, one or more of buffer binding and texture references. Load rate refers to the current busy level (percentage) of the queue, which can be estimated by detecting the task execution status of the queue (e.g., the proportion of the queue's maximum processing capacity being used by the currently executing tasks). The backlog of tasks to be executed can include one or more of the following metrics: queue depth (the number of tasks waiting to be processed in the queue, also reflecting the idle status of the computing queue), task complexity (the total estimated execution time of the backlog of tasks), and average waiting time (the average waiting time of a task in the queue). Memory utilization includes, but is not limited to, the proportion of allocated memory to total memory and / or the frequency of memory access (bandwidth utilization). Power consumption status is the power consumption (watts) at the current moment.

[0068] S103, the electronic device determines the optimal assignment scheme for the at least one reference task based on the current running status of each queue in the GPU, the queue affinity matrix, and the characteristic fingerprint of at least one reference task using a multi-objective optimization algorithm.

[0069] In the embodiments of this application, the optimization objectives of the multi-objective optimization algorithm include at least one of optimizing load balancing, minimizing synchronization overhead, minimizing power consumption, and minimizing resource conflicts while satisfying task priority requirements as much as possible.

[0070] Understandably, once the electronic device has determined the optimal assignment scheme for at least one of the aforementioned reference tasks, it will perform task assignment based on that optimal assignment scheme.

[0071] In some possible implementations, the queue affinity matrix indicates the parallel efficiency of the graphics queue and the computation queue, and that the parallel efficiency of the transmission queue with respect to the graphics queue and the computation queue is greater than or equal to a preset parallel efficiency threshold (e.g., 0.5). The electronic device uses a multi-objective optimization algorithm to determine the optimal assignment scheme for the at least one reference task, including the following adaptive strategies:

[0072] Adaptive Strategy 1: When the graphics queue is in a load state that meets the first preset high load state (e.g., load rate greater than or equal to 80%) and the computation queue is relatively idle (e.g., load rate less than 50%), the first target task is dynamically migrated to the computation queue. The first target task is a separable computation shader task.

[0073] Adaptive Strategy 2: When the transmission queue is underutilized, the second target task, which is a non-urgent data preprocessing task, is rerouted to the transmission queue.

[0074] Adaptive Strategy 3: When the overall GPU load meets the second preset high load state (e.g., the load rate of each queue is greater than or equal to 80%), differentiated scheduling is implemented based on task priority to ensure that the third target task is given priority execution right. The third target task is a critical path task.

[0075] In some possible implementations, after step S103, the electronic device continues to collect key performance indicators while running, such as the utilization rate of each queue, task execution time, and synchronization operation waiting time. Based on real-time data, the task assignment strategy and synchronization granularity are dynamically adjusted. For example, if frequent and time-consuming synchronization waiting is detected between two queues, an attempt is made to reduce their task dependency in the next assignment.

[0076] Using the method provided in this solution, based on the measured queue affinity matrix, we can understand the actual parallel capabilities and conflict situation at the underlying hardware level, and detect the current system status (queue load, memory) and task characteristics in real time. Based on the above information, we can perform intelligent task assignment through a multi-objective optimization algorithm.

[0077] On the one hand, by using the queue affinity matrix, the queue parallelism behavior of domestic GPUs is transformed from a black box to a white box, so that task assignment is no longer a static strategy based on guesswork, but a data-driven decision based on the actual characteristics of the hardware.

[0078] On the other hand, by comprehensively considering real-time operational status and task characteristic fingerprints, dynamic load balancing can be achieved. For example, tasks can be automatically migrated from busy queues to idle queues, maximizing GPU utilization.

[0079] On the other hand, the optimization objectives of multi-objective optimization include not only performance (load balancing, minimizing synchronization overhead), but also power consumption and resource conflicts. While pursuing high performance, it can also meet energy efficiency requirements and improve system stability by avoiding resource conflicts.

[0080] In some possible implementations, when there are pending tasks on the current GPU that are not assigned using this approach, the electronic device uses an initial allocation plus local search method and a multi-objective optimization algorithm to determine the optimal allocation scheme for at least one of the reference tasks.

[0081] If there are no tasks in the GPU that are not assigned using this method, and the task to be assigned received by the electronic device is a batch task (the number of tasks to be assigned is greater than or equal to 3), and the number of currently assigned tasks waiting to be processed is less than a preset threshold, then it is determined that the above-mentioned at least one reference task includes the task to be assigned and the assigned tasks waiting to be processed. If the task to be assigned is a batch task, and the number of already assigned tasks waiting to be processed is greater than or equal to a preset threshold, then it is determined that the at least one reference task only includes the batch task (considering real-time performance). Furthermore, the optimal assignment scheme for the at least one reference task is determined by using an initial allocation plus local search method and a multi-objective optimization algorithm.

[0082] If there are no tasks in the GPU that are not assigned using this method, and the task to be assigned received by the electronic device is a dynamic task (the number of tasks to be assigned is less than 3), then the above-mentioned at least one reference task includes the task to be assigned. The optimal assignment scheme for the at least one reference task is determined by using an incremental allocation method and a multi-objective optimization algorithm.

[0083] In some possible implementations, an initial assignment plus local search method is used, employing a multi-objective optimization algorithm to determine the optimal assignment scheme for at least one reference task, including the following steps:

[0084] 1) Initial allocation: Each reference task is assigned to the queue with the corresponding function that corresponds to the task type attribute of the reference task and the optimal preset index in order of priority from high to low.

[0085] For example, the optimal preset index can refer to the queue currently having the lowest load. For instance, if the queue functional attributes are as shown in Table 1 above, then graphics rendering tasks can be assigned to the queue with the lowest load in the graphics queue, general computing tasks can be assigned to the computing queue or the queue with the lowest load in the graphics queue, and data transmission tasks can be assigned to the transmission queue, computing queue, or the queue with the lowest load in the graphics queue.

[0086] Alternatively, the optimal preset metric can also refer to the optimal backlog of tasks currently waiting to be executed in the queue, including queue depth being less than a preset depth threshold, task complexity being less than a preset complexity threshold, and average waiting time being less than a preset waiting time threshold.

[0087] Alternatively, the optimal preset metric could also refer to the optimal current memory utilization of the queue.

[0088] Alternatively, the optimal preset metric could also refer to the optimal current power consumption state of the queue.

[0089] 2) Calculate the objective function value of the current allocation scheme relative to the multi-optimization objective.

[0090] For example, the objective function value is a weighted sum of the load balancing score, the minimum synchronization overhead score, the power consumption score, and the total conflict avoidance score.

[0091] Regarding the load balancing score, electronic devices can measure the degree of load balancing based on the variance or standard deviation of the expected load for each queue in the current allocation scheme, and then convert it into a score. The higher the score, the better the balance.

[0092] Regarding minimizing the synchronization overhead score, for each cross-queue dependency of the electronic device in the current allocation scheme, the synchronization overhead is estimated based on the average switching overhead recorded in the dependency strength and affinity matrix between queues. The estimated synchronization overhead of all cross-queue dependencies is summed, and the total synchronization overhead is mapped to a score. A higher score indicates a smaller synchronization overhead.

[0093] Regarding power consumption scores, electronic devices can determine power consumption scores based on the expected total power consumption of each queue for the current allocation scheme, the GPU idle power consumption, and the maximum safe power consumption. A higher score indicates lower power consumption. For example, the power consumption score is the ratio of the functional score to the second difference and the third difference. The second difference is the difference between the expected total power consumption and the idle power consumption, and the third difference is the difference between the maximum safe power consumption and the idle power consumption.

[0094] Regarding the overall conflict avoidance score, the electronic device can determine the probability of conflict between hardware units corresponding to that functional type attribute in the resource conflicts of all task pairs belonging to the same functional type attribute in the reference task, based on the queue affinity matrix. For example, referring to Table 2 above again, if the task pair {a,b}, which belongs to the computing functional type, is assigned to the {graphics & computing queue} combination, the corresponding conflict probability is 0.6. The conflict risks of all task pairs are summed, and the total conflict risk is mapped to a score. A higher score indicates a lower conflict risk and better conflict avoidance. Alternatively, the overall conflict avoidance score can also include an overall parallel risk score, determined based on the parallel efficiency corresponding to the queue combinations to which task pairs belonging to different queues are assigned in the queue affinity matrix.

[0095] Optionally, for the purpose of standardized measurement, the scores of multiple sub-objectives of the objective function value can be normalized.

[0096] Optionally, different metric weighting schemes can be selected based on the current load status of the GPU and / or the number of dependencies of the tasks to be assigned (which can also be understood as the estimated synchronization overhead).

[0097] Specifically, when the GPU is under high load, the focus is on load balancing, that is, increasing the weight of the load balancing score and decreasing the weight of other scores; when the number of dependencies of the tasks to be assigned is greater than or equal to the preset dependency number threshold, the focus is on reducing synchronization overhead, that is, increasing the weight of the synchronization overhead score and decreasing the weight of other scores. For example, the default weighting is {load balancing score 0.25, minimize synchronization overhead score 0.25, power consumption score 0.25, total conflict avoidance score 0.25}. If the GPU's current load is high, a weighting of {load balancing score 0.4, minimize synchronization overhead score 0.2, power consumption score 0.2, total conflict avoidance score 0.2} is used. If the number of dependencies for the tasks to be assigned is greater than or equal to a preset dependency threshold, a weighting of {load balancing score 0.2, minimize synchronization overhead score 0.4, power consumption score 0.2, total conflict avoidance score 0.2} is used. If the GPU's current load is high and the number of dependencies for the tasks to be assigned is greater than or equal to a preset dependency threshold, a weighting of {load balancing score 0.35, minimize synchronization overhead score 0.35, power consumption score 0.15, total conflict avoidance score 0.15} is used.

[0098] Optionally, when the overall GPU load meets the second preset high load state, the objective function value may also include a priority score item, which represents the degree to which the priority requirements are met. The higher the score, the more the priority requirements are met, and a higher weight (e.g., 0.8) is assigned to that priority.

[0099] 3) For each task i, try to move it from the current queue j to another queue k (queue k supports the task type of task i), and recalculate the objective function value after the move (note that each time a task is moved, it is necessary to recalculate the synchronization overhead and conflict risk contributed by other tasks that depend on or compete with the task). If the objective function value increases after the move, accept the move and update the allocation scheme.

[0100] Repeat step 3 until there is no improvement or the maximum number of iterations is reached.

[0101] Optionally, considering real-time requirements, if the total number of reference tasks is very large (greater than or equal to a preset threshold), then in step 3, only the tasks currently to be assigned can be moved and optimized, while the tasks that have already been assigned will no longer be moved.

[0102] In some other possible implementations, the optimization objective of the multi-objective optimization algorithm may also include maximizing the overall parallel efficiency, with the objective function being a weighted sum of the load balancing score, the synchronization overhead score, the power consumption score, the total conflict avoidance score, and the overall parallel efficiency score.

[0103] Regarding maximizing the overall parallel efficiency score, electronic devices can traverse all task pairs assigned to different queues in the current allocation scheme, determine the overall parallel efficiency of the queue combination corresponding to each task pair based on the queue affinity matrix, sum and average the overall parallel efficiencies corresponding to all task pairs, and map them to a score. The higher the score, the higher the overall parallel efficiency.

[0104] It should be noted that the objective function value can be a weighted sum of one or more of the following: load balancing score, minimizing synchronization overhead score, power consumption score, total conflict avoidance score, and maximizing overall parallel efficiency score. The aforementioned combinations are merely examples, and this article does not limit them. For example, it can also be a weighted sum of the load balancing score, minimizing synchronization overhead score, total conflict avoidance score, and maximizing overall parallel efficiency score.

[0105] In some possible implementations, an incremental allocation method is used, employing a multi-objective optimization algorithm to determine the optimal allocation scheme for the tasks to be assigned, including the following steps:

[0106] 1) Determine the candidate queue for each new task: For each new task, determine the candidate queue sequentially in descending order of priority. The candidate queue must have the corresponding functionality that corresponds to the task type attribute of the new task. For example, for new task i, the candidate selection queues could include j and k.

[0107] 2) Calculate the objective function value for each candidate queue of new task i, and select the candidate queue with the largest objective function value as the allocation queue for new task i.

[0108] For an explanation of the objective function value, please refer to the relevant explanation above; it will not be elaborated upon here.

[0109] In this way, we only need to calculate the objective function value of assigning the new task to each available queue, and then select the candidate queue with the largest objective function value as the assignment queue for the new task i, without having to perform local search iterations.

[0110] It should be noted that the adaptive strategy described above can be covered by using an initial allocation plus local search method, or by using an incremental allocation method and employing a multi-objective optimization algorithm to determine the optimal allocation scheme for at least one reference task.

[0111] In some possible implementations, if the tasks currently awaiting assignment and those already assigned and waiting to be processed in the GPU include at least two tasks, then for the first and second tasks in the same queue that have resource synchronization dependencies, the electronic device also determines the optimal pipeline barrier configuration based on the resource dependencies and preset pipeline barrier configuration rules. Specifically, such as... Figure 2 As shown, after obtaining the task characteristic fingerprint of the reference task in step S102 above, the electronic device further performs the following steps:

[0112] S201, the electronic device identifies the first task and the second task, which belong to the same queue and have resource dependencies.

[0113] Among them, the first task and the second task are any two tasks that belong to the same queue and have resource dependencies among the tasks currently to be assigned and those that have been assigned and are waiting to be processed (which can also be understood as at least two tasks mentioned above).

[0114] In this embodiment of the application, the resource dependency relationship between the first task and the second task may include at least one of read-write dependency relationship and layout transformation dependency relationship.

[0115] For example, an electronic device determines the dependency between tasks by checking whether two tasks (or operations) access the same resource. For instance, if task A writes to buffer X and task B reads from buffer X, and task A is submitted to the same queue before task B, then the dependency is determined to be write-after-read, and the resource involves buffer X.

[0116] S202, the electronic device determines the optimal pipeline barrier configuration between the first task and the second task based on the resource dependency relationship between the first task and the second task and the preset pipeline barrier configuration rules.

[0117] The optimal pipeline barrier configuration includes an execution barrier, or the optimal pipeline barrier configuration includes an execution barrier and a memory barrier, wherein the memory barrier includes a buffer memory barrier and / or an image memory barrier.

[0118] The preset pipeline barrier configuration rules are set with the goal of minimizing synchronization overhead, while ensuring that synchronization requirements are accurately met.

[0119] In some possible implementations, the preset pipeline barrier configuration rules include:

[0120] 1) If the resource dependency relationship between the first task and the second task is a "write-after-read dependency" in read-write dependency (meaning a read-write dependency relationship that needs to ensure that the write operation is completed and the written data is visible to the read operation), then both execution barriers and memory barriers (which can also be understood as pipeline barriers for complete execution) are used simultaneously, and one or both of the buffer memory barrier and image memory barrier in the preset resource memory barriers are determined according to the synchronization resource type.

[0121] To configure a complete pipeline barrier, the electronic device can perform the barrier configuration by creating instances of the pipeline barrier and configuring producers and consumers. Furthermore, based on the synchronization range, instances of buffer memory barriers and / or image memory barriers for creating specific resources can be selected. The source and target phases involved in resource synchronization can be configured in the corresponding memory barrier instances (e.g., if buffer memory synchronization and image memory synchronization are included, separate buffer memory barriers and image memory barriers are created, and the synchronization source and target phases involved in each barrier are set separately). Finally, the memory barrier instances are configured into the pipeline barrier instances, thus completing the complete pipeline barrier configuration.

[0122] As an example, if the synchronization operation actually only involves specific buffer resources (e.g., vertex buffers, index buffers, uniform buffers, etc.), then a buffer memory barrier (VkBufferMemoryBarrier) is used, and the source and target phases involved are configured within the buffer memory barrier. If the synchronization operation actually only involves specific image resources (e.g., textures, attachments, etc.), then an image memory barrier (VkImageMemoryBarrier) is used, and the source and target phases involved are configured within the image memory barrier. If the synchronization resource actually involves both specific buffer resources and specific image resources, then both buffer memory barriers and image memory barriers are used simultaneously. It should be noted that this paper does not limit the number of specific resource memory barriers that the global memory barrier is broken down into.

[0123] For example, if an electronic device determines that an image exists that needs to be written as a color appendage and then read as a texture in a fragment shader, it will use an image memory barrier to synchronize this transition. Similarly, if an electronic device determines that a buffer exists that needs to be written by a compute shader and then read by a vertex shader, it will use a buffer memory barrier to synchronize it.

[0124] In some possible implementations, the preset pipeline barrier configuration rules also include:

[0125] 2) If the resource dependency between the first task and the second task is a "read-after-write dependency" in read-write dependency, then an execution barrier (which can also be understood as a pipeline barrier without a memory barrier) will be used (only the execution order is guaranteed).

[0126] For "read-after-write dependencies" in read-write dependencies, you can set a complete pipeline barrier or a pipeline barrier without memory barriers, depending on the specific requirements. This article does not impose any restrictions on this.

[0127] In some possible implementations, the preset pipeline barrier configuration rules also include:

[0128] 3) If the resource dependency between the first task and the second task is an image layout transformation dependency, then a complete pipeline barrier, including execution barriers and image memory barriers, is used for synchronization.

[0129] In some possible implementations, if the electronic device cannot accurately determine the resources involved in synchronizing the first and second tasks, which can also be understood as being unable to determine any of the detachable configurations that satisfy the preset pipeline barrier configuration rules, then it is determined to use a global memory barrier.

[0130] This can also be understood as follows: For task synchronization barrier data in the GPU that has been allocated a global memory barrier but not using this proposed method, the electronic device can, based on the resource read / write dependencies between the first and second tasks and preset pipeline barrier configuration rules, attempt to decompose the coarse-grained global memory barrier (VkMemoryBarrier) between the first and second tasks into fine-grained synchronization primitives to determine the optimal pipeline barrier configuration between the first and second tasks. For a new task currently received by the electronic device, the optimal pipeline barrier configuration between the first and second tasks is directly determined based on the resource read / write dependencies between the first and second tasks and preset pipeline barrier configuration rules.

[0131] S203, the electronic device sets the synchronization barrier between the first and second tasks to the optimal pipeline barrier configuration.

[0132] In the embodiments of this application, based on the optimal pipeline barrier configuration, the synchronization range can be narrowed down to the actual producer pipeline stage and consumer pipeline stage involved, and when the optimal pipeline barrier configuration includes a memory barrier, global memory synchronization can be limited to a local buffer or image resource.

[0133] In some possible implementations, if the tasks currently to be assigned and those already assigned in the GPU include at least two tasks, then for the third and fourth tasks in different queues and with resource synchronization dependencies, the electronic device also determines the optimal synchronization primitive configuration between the third and fourth tasks based on the strength of the dependency and preset synchronization primitive configuration rules.

[0134] Specifically, such as Figure 3As shown, after obtaining the task characteristic fingerprint of the reference task in step S102 above, the electronic device further performs the following steps:

[0135] S301, the electronic device identifies the third and fourth tasks, which belong to different queues and have resource dependencies.

[0136] In this embodiment of the application, the third task and the fourth task are any two tasks that belong to different queues and have resource dependencies.

[0137] S302, the electronic device determines the dependency strength of the dependency relationship between the third task and the fourth task.

[0138] For example, an electronic device can determine the dependency strength of the dependency relationship between the third task and the fourth task based on a weighted sum of at least one of the following: resource overlap, amount of shared data, timeliness, and parallelism between queues.

[0139] Resource overlap refers to the degree to which two tasks access the same resources, and it is directly proportional to the degree of dependency. For example, resource overlap can be determined by analyzing the sets of resources accessed by two tasks and calculating the proportion of the overlapping portion. For instance, resource overlap is the ratio of the number of resources in the intersection to the number of resources in the union.

[0140] Shared data volume refers to the amount of data shared between two tasks (e.g., the amount of shared data written by task A and read by task B). The amount of shared data volume is proportional to the degree of dependency. The size of shared data volume is generally greater than or equal to 0 bytes. For ease of statistics, the data volume can also be normalized.

[0141] Timeliness refers to the sensitivity of a subsequent task to latency, and the sensitivity is directly proportional to the degree of dependency. For example, a subsequent task must start immediately after the preceding task is completed, which is considered highly sensitive; a subsequent task can start within a few milliseconds after the preceding task is completed, which is considered moderately sensitive; and a subsequent task can start tens of milliseconds or longer after the preceding task is completed, which is considered low sensitive.

[0142] Inter-queue parallelism refers to the overall parallel efficiency between the queues corresponding to the third and fourth tasks, respectively. Parallel efficiency is inversely proportional to the degree of dependency. Electronic devices can determine the overall parallel efficiency between queues based on the queue affinity matrix.

[0143] S303, the electronic device determines the optimal synchronization primitive configuration between the third task and the fourth task based on the dependency strength of the dependency relationship between the third task and the fourth task and the preset synchronization primitive configuration rules.

[0144] In this embodiment, the optimal synchronization primitive configuration is a semaphore, a fence, or an event. The preset synchronization primitive configuration rules are set with the goal of minimizing synchronization overhead, while ensuring that the synchronization requirements are accurately met.

[0145] In some possible implementations, the default synchronization primitive configuration rules include:

[0146] 1) When the dependency between the third and fourth tasks involves CPU awareness, select the fence for synchronization.

[0147] For example, an electronic device determines whether the synchronization between a third and fourth task requires CPU awareness by analyzing the task's data flow, control flow, resource lifecycle, and task completion progress notification information. For instance, if the data flow shows that the output of the third task is the input of the fourth task, and this data needs to be prepared or consumed by the CPU; or if the control flow shows that the execution order between the third and fourth tasks requires CPU involvement in decision-making; or if the resource lifecycle shows that the resources used by the third and fourth tasks need to be managed by the CPU (e.g., allocated, released, mapped, etc.); or if the task completion progress notification message shows that the CPU needs to know the completion status of the third task to start the fourth task, then the electronic device determines that the synchronization between the third and fourth tasks requires CPU awareness.

[0148] 2) If the dependency relationship between the third and fourth tasks does not involve CPU awareness, and the dependency strength between the third and fourth tasks is greater than or equal to a preset strength threshold, or the parallel efficiency index of the queue combination corresponding to the third and fourth tasks is less than a preset efficiency threshold, then a semaphore is selected for synchronization.

[0149] For example, if the maximum dependency strength is 1.0, the preset strength threshold can be 0.7. For example, if the maximum queue parallel efficiency in the affinity matrix is ​​1.0, the preset efficiency threshold is 0.5.

[0150] 3) If the dependency strength between the third and fourth tasks is less than a preset strength threshold, the parallel efficiency index of the queue combination corresponding to the third and fourth tasks is greater than or equal to a preset efficiency threshold, and the GPU supports cross-queue synchronization of event types, then select the event for synchronization. In other cases, no synchronization primitive is set.

[0151] This application also provides a multi-queue adaptive scheduling system for GPUs, such as... Figure 4 As shown, the system includes:

[0152] The hardware capability awareness and performance analysis module 401 is used to obtain the queue affinity matrix, which is determined based on the actual parallel processing performance characteristics of the graphics processing unit (GPU) for the test queue combination.

[0153] Resource load profiling module 402 is used to determine the current operating status of each queue in the GPU;

[0154] Task characteristic fingerprint analysis module 403 is used to determine the task characteristic fingerprint of at least one reference task;

[0155] The dynamic task dispatcher 404 is used to determine the optimal dispatch scheme for the at least one reference task based on the current running status of each queue in the GPU, the queue affinity matrix, and the characteristic fingerprint of the at least one reference task, using a multi-objective optimization algorithm.

[0156] In some possible implementations, the hardware capability awareness and performance analysis module 401 is specifically used to enumerate the queue families supported by the GPU through a standard interface, and obtain the basic attributes and functional flags of each queue family, the basic attributes including the number of queues; determine a queue set based on the queue families and the number of queues supported by each queue family, the queue set being a multiset containing all queues supported by each queue family; take any combination of elements in the queue set as a test queue combination, and submit a corresponding test task sequence to each queue in the test queue combination based on the functional tags of the queues in the test queue combination; measure the execution time and resource consumption of the test queue combination for the test task sequence for individual and parallel execution, and obtain the test data of the test queue combination; and construct the queue affinity matrix based on the test data of the test queue combination.

[0157] In some possible implementations, the dynamic task dispatcher 404 is specifically used to dynamically migrate a first target task to the computation queue based on the queue affinity matrix when the graphics queue is in a load state that meets a first preset high load state and the computation queue is relatively idle. The first target task is a separable computation shader task. When the utilization of the transmission queue is insufficient, the second target task in the graphics queue and / or the computation queue is rerouted to the transmission queue. The second target task is a data preprocessing task. When the overall GPU load meets a second preset high load state, differentiated scheduling is implemented based on task priority to ensure that a third target task, which is a critical path task, is given priority execution rights.

[0158] In some possible implementations, the above system also includes:

[0159] An adaptive synchronization optimizer 405 is used to identify a first task and a second task that belong to the same queue and have resource dependencies; based on the resource dependencies of the first task and the second task and preset pipeline barrier configuration rules, it determines the optimal pipeline barrier configuration between the first task and the second task; and sets the synchronization barrier between the first task and the second task as the optimal pipeline barrier configuration.

[0160] In some possible implementations, the adaptive synchronization optimizer 405 is further configured to determine a third task and a fourth task that belong to different queues and have resource dependencies; determine the dependency strength between the third task and the fourth task; and determine the optimal synchronization primitive configuration between the third task and the fourth task based on the dependency strength between the third task and the fourth task and a preset synchronization primitive configuration rule.

[0161] In some possible implementations, the above system also includes:

[0162] The performance monitoring and feedback loop 406 is used to continuously collect key performance indicators during runtime and dynamically adjust task assignment strategies and synchronization granularity.

[0163] It should be noted that any implementation method mentioned in the method embodiments is also applicable to the GPU-oriented multi-queue adaptive scheduling system provided in this application. For related terminology and specific execution corrections, please refer to the description of the foregoing method embodiments, which will not be elaborated here.

[0164] This application also provides a GPU-oriented multi-queue adaptive scheduling device, including a unit for executing any of the GPU-oriented multi-queue adaptive scheduling methods in the above method embodiments.

[0165] This application also provides a GPU-oriented multi-queue adaptive scheduling device, including a processor, which is used to execute any one of the GPU-oriented multi-queue adaptive scheduling methods in the above method embodiments.

[0166] Please refer to Figure 5 This is another embodiment provided in this application. A schematic diagram of the device structure is shown below. Figure 5As shown, the GPU-oriented multi-queue adaptive scheduling device 500 may include: at least one processor 501, such as a CPU, at least one communication interface 503, a memory 504, and at least one communication bus 502. The communication bus 502 is used to implement communication between these components. The communication interface 503 may optionally include a standard wired interface, a wireless interface (such as a Wi-Fi interface or a Bluetooth interface), etc. The memory 504 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 504 may also be at least one storage device located remotely from the aforementioned processor 501. Figure 5 As shown, the memory 504, which serves as a computer storage medium, may include an operating system, a network communication module, and program instructions.

[0167] exist Figure 5 In the GPU-oriented multi-queue adaptive scheduling device 500 shown, the processor 501 can be used to load program instructions stored in the memory 504 and specifically execute any of the GPU-oriented multi-queue adaptive scheduling methods in the above method embodiments.

[0168] It should be noted that the specific execution process can be found in the detailed description of the above method embodiments, and will not be elaborated here.

[0169] For specific execution steps, please refer to the description of the foregoing method embodiments, which will not be detailed here.

[0170] This application also provides a computer storage medium that can store multiple instructions. These instructions are adapted to be loaded and executed by a processor using the GPU-oriented multi-queue adaptive scheduling method provided in this application. For details of the execution process, please refer to the specific description of the method embodiments shown above, which will not be elaborated here.

[0171] This application also provides a computer program product containing instructions that, when run on an electronic device, cause the electronic device to execute the method steps of the method embodiments shown above.

[0172] This application also provides a chip module, including a transceiver component and a chip, wherein the chip is used to execute the method steps of the above-described method embodiments.

[0173] It is understood that the GPU-oriented multi-queue adaptive scheduling system, GPU-oriented multi-queue adaptive scheduling device, computer storage medium, computer program, computer program product, and chip provided above are all used to execute the method shown in any implementation of the corresponding aspect of the embodiments of this application. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding method, and will not be detailed here.

[0174] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes the processes of the embodiments of the above methods.

[0175] The term "at least one" in this application refers to one or more items. "More than one item" means two or more items. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, it should be understood that although the terms "first," "second," etc., may be used to describe objects in this application, these objects should not be limited to these terms. These terms are only used to distinguish the objects from each other.

[0176] The terms “including” and “having” mentioned above, and any variations thereof, are intended to cover non-exclusive inclusion.

[0177] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A multi-queue adaptive scheduling method for GPUs, characterized in that, The method includes: Obtain the queue affinity matrix, which is determined based on the actual parallel processing performance characteristics of the graphics processing unit (GPU) for the test queue combination. The test queue combination is a combination of any elements from the multiset of all queues supported by the GPU. The queue affinity matrix is ​​used to indicate the overall parallel efficiency index, resource conflict probability, and context switching overhead among the queues. The current running status of each queue in the GPU and the task characteristic fingerprint of at least one reference task are determined. The current running status of the queue includes at least one of the following: load rate, backlog of tasks to be processed, memory utilization, and power consumption status. The task characteristic fingerprint includes task type attribute, resource dependency relationship, and priority information. The reference task is any one of the tasks waiting to be assigned and tasks that have been assigned and are waiting to be processed. Based on the current running status of each queue in the GPU, the queue affinity matrix, and the characteristic fingerprint of the at least one reference task, a multi-objective optimization algorithm is used to determine the optimal allocation scheme for the at least one reference task. The optimization objectives of the multi-objective optimization algorithm include optimizing load balancing, minimizing synchronization overhead, minimizing power consumption, and minimizing resource conflicts while satisfying task priority requirements as much as possible.

2. The method as described in claim 1, characterized in that, The acquisition queue affinity matrix includes: The queue families supported by the GPU are enumerated through the standard interface, and the basic attributes and functional flags of each queue family are obtained. The basic attributes include the number of queues. The queue set is determined based on the queue family and the number of queues supported by each queue family, wherein the queue set is a multiset containing all queues supported by each queue family; The combination of any elements in the queue set is taken as the test queue combination, and the corresponding test task sequence is submitted to each queue in the test queue combination based on the function tags of the queues in the test queue combination. Measure the execution time and resource consumption of the test queue combination for individual and parallel execution of the test task sequence, and obtain the test data of the test queue combination; Based on the test data of the test queue combination, the queue affinity matrix is ​​constructed.

3. The method as described in claim 1 or 2, characterized in that, The queues supported by the GPU include a graphics queue, a compute queue, and a transfer queue. The queue affinity matrix indicates that the parallel efficiency of the graphics queue and the compute queue, and that the parallel efficiency of the transfer queue with the graphics queue and the compute queue is greater than or equal to a preset parallel efficiency threshold, respectively. The step of using a multi-objective optimization algorithm to determine the optimal assignment scheme for the at least one reference task includes: When the load state of the graphics queue meets the first preset high load state and the computation queue is relatively idle, the first target task is dynamically migrated to the computation queue based on the queue affinity matrix. The first target task is a separable computation shader task. In the event of insufficient utilization of the transmission queue, the second target task in the graphics queue and / or computation queue, which is a data preprocessing task, will be rerouted to the transmission queue. When the overall GPU load meets the second preset high load state, differentiated scheduling is implemented based on task priority to ensure that the third target task, which is a critical path task, is given priority execution.

4. The method as described in claim 1 or 2, characterized in that, The tasks currently awaiting assignment and those already assigned and awaiting processing in the GPU include at least two tasks, and the method further includes: Identify a first task and a second task that belong to the same queue and have a resource dependency relationship, wherein the first task and the second task are any two tasks that belong to the same queue and have a resource dependency relationship among the at least two tasks; Based on the resource dependencies between the first task and the second task and the preset pipeline barrier configuration rules, the optimal pipeline barrier configuration between the first task and the second task is determined. The optimal pipeline barrier configuration includes an execution barrier, or the optimal pipeline barrier configuration includes an execution barrier and a memory barrier. The memory barrier includes a buffer memory barrier and / or an image memory barrier. The preset pipeline barrier configuration rules are set with the goal of minimizing synchronization overhead, while ensuring that the synchronization requirements are accurately met. The synchronization barrier between the first task and the second task is set to the optimal pipeline barrier configuration to narrow the synchronization scope to the actual producer pipeline stage and consumer pipeline stage involved, and, if the optimal pipeline barrier configuration includes a memory barrier, to limit global memory synchronization to a local buffer or image resource.

5. The method as described in claim 4, characterized in that, The preset pipeline barrier configuration rules include: When the resource dependency relationship between the first task and the second task is a read-after-write dependency relationship in read-write dependency, the optimal pipeline barrier configuration includes an execution barrier and a preset resource memory barrier. The preset resource memory barrier is a buffer memory barrier and / or an image memory barrier determined according to the synchronization resource type. When the resource dependency between the first task and the second task is a read-after-write dependency in read-write dependencies, the optimal pipeline barrier configuration includes an execution barrier; When the resource dependency between the first task and the second task is an image layout transformation dependency, the optimal pipeline barrier configuration includes an execution barrier and an image memory barrier.

6. The method according to any one of claims 1, 2, and 5, characterized in that, The tasks currently awaiting assignment and those already assigned and awaiting processing in the GPU include at least two tasks, and the method further includes: Identify a third task and a fourth task that belong to different queues and have resource dependencies, wherein the third task and the fourth task are any two tasks that belong to different queues and have resource dependencies among the at least two tasks; Determine the dependency strength of the dependency relationship between the third task and the fourth task, wherein the dependency strength is related to at least one of the resource overlap between the third task and the fourth task, the amount of shared data, the timeliness of the task, and the parallelism of the queue; Based on the dependency strength between the third task and the fourth task and the preset synchronization primitive configuration rules, the optimal synchronization primitive configuration between the third task and the fourth task is determined. The optimal synchronization primitive configuration is a semaphore, a fence, or an event. The preset synchronization primitive configuration rules are set with the goal of minimizing synchronization overhead, while ensuring that the synchronization requirements are accurately met.

7. The method as described in claim 6, characterized in that, The preset synchronization primitive configuration rules include: In cases where the dependency between the third and fourth tasks involves CPU awareness, a fence is selected for synchronization. If the dependency relationship between the third task and the fourth task does not involve CPU awareness, and the dependency strength between the third task and the fourth task is greater than or equal to a preset strength threshold, or the parallel efficiency index of the queue combination corresponding to the third task and the fourth task is less than a preset efficiency threshold, then a semaphore is selected for synchronization. If the dependency strength between the third task and the fourth task is less than a preset strength threshold, and the parallel efficiency index of the queue combination corresponding to the third task and the fourth task is greater than or equal to a preset efficiency threshold, and the GPU supports cross-queue synchronization of event types, then the event is selected for synchronization.

8. The method according to any one of claims 1, 2, 5, and 7, characterized in that, The method further includes: During GPU runtime, real-time data is continuously collected, including key performance indicators such as the utilization rate of each queue, task execution time, and synchronization operation waiting time. Based on the real-time data, the task assignment strategy and synchronization granularity are dynamically adjusted.

9. A multi-queue adaptive scheduling system for GPUs, characterized in that, The system includes: The hardware capability awareness and performance analysis module is used to obtain the queue affinity matrix. The queue affinity matrix is ​​determined based on the actual parallel processing performance characteristics of the graphics processing unit (GPU) for the test queue combination. The test queue combination is a combination of any elements in the multiset of all queues supported by the GPU. The queue affinity matrix is ​​used to indicate the overall parallel efficiency index, resource conflict probability, and context switching overhead among the queues. The resource load profile building module is used to determine the current operating status of each queue in the GPU. The current operating status of the queue includes at least one of the following: load rate, backlog of pending tasks, memory utilization, and power consumption status. The task characteristic fingerprint analysis module is used to determine the task characteristic fingerprint of at least one reference task. The task characteristic fingerprint includes task type attributes, resource dependencies, and priority information. The reference task is any one of the tasks waiting to be assigned and tasks that have been assigned and are waiting to be processed. A dynamic task dispatcher is used to determine the optimal dispatch scheme for the at least one reference task based on the current running state of each queue in the GPU, the queue affinity matrix, and the characteristic fingerprint of the at least one reference task, using a multi-objective optimization algorithm. The optimization objectives of the multi-objective optimization algorithm include optimizing load balancing, minimizing synchronization overhead, minimizing power consumption, and minimizing resource conflicts while satisfying task priority requirements as much as possible.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, which, when executed, performs the method according to any one of claims 1 to 8.

Images