Machine learning-based graphics card task resource scheduling method and system

Abstract

The application discloses a GPU task resource scheduling method and system based on machine learning, relates to the technical field of machine learning and GPU scheduling, and comprises the following steps: continuously monitoring the task flow of an input GPU, extracting a task descriptor in real time and generating an initial multi-dimensional task portrait, and distributing the computing unit and the video memory resource according to the task portrait. In the task execution, the resource consumption deviation is generated by comparing the actual resource consumption track with the expected value, and the deviation is used to drive the online correction of the task portrait model by using the online correction of the task portrait model. Based on the latest task portrait after correction, the optimal resource scheduling strategy is dynamically recalculated, and real-time resource redistribution is implemented for the task being executed. The method realizes online self-correction of the model and dynamic scheduling of the resource in the whole cycle, and improves the resource utilization efficiency and the task execution performance.

Description

Technical Field

[0001] This invention belongs to the field of machine learning and GPU scheduling technology, specifically a method and system for scheduling GPU task resources based on machine learning. Background Technology

[0002] Graphics card task scheduling often relies on static rules or offline prediction models based on historical data. These methods allocate resources in a one-time manner based on limited static parameters of the task, lacking a precise characterization of the dynamic behavior of the task during runtime. Once a task starts, the allocation is fixed, making it difficult for the scheduler to intervene and adjust, resulting in low resource utilization or interference between tasks.

[0003] Existing machine learning-based solutions build task profiles through offline training to predict resource requirements. However, these models remain fixed after deployment and cannot adapt to behavioral drift caused by changes in input data and system state. Prediction bias persists and cannot be corrected, affecting scheduling accuracy. Furthermore, even if the system detects discrepancies between actual and expected consumption, it lacks the closed-loop control capability to proactively adjust resource allocation for running tasks in real time based on these biases.

[0004] There is a need for a scheduling method that can correct the task profile prediction model online and dynamically reallocate resources to tasks in execution based on the correction results, in order to cope with the dynamic uncertainty of task behavior and improve resource utilization efficiency. Summary of the Invention

[0005] This invention aims to solve at least one of the technical problems existing in the prior art;

[0006] To this end, this invention proposes a machine learning-based graphics card task resource scheduling method, including:

[0007] The task flow input to the graphics card is continuously monitored, and the static and dynamic descriptors of each task in the task flow are extracted in real time.

[0008] The static and dynamic descriptors are input into a pre-trained task profile generation model, which outputs a multi-dimensional task profile for each task.

[0009] The initial resource scheduling strategy for each task is calculated based on the multi-dimensional task profile, and computing units and video memory resources are allocated on the graphics card hardware according to the initial resource scheduling strategy.

[0010] During task execution, the actual resource consumption trajectory of each task on the graphics card hardware is monitored synchronously.

[0011] By comparing the actual resource consumption trajectory with the corresponding expected consumption in the multi-dimensional task profile, the resource consumption deviation of each task is generated.

[0012] The resource consumption deviation is fed back to the adversarial learning correction module to drive the task profile generation model to perform adversarial training in order to correct the multi-dimensional task profile.

[0013] The optimal resource scheduling strategy for each task is recalculated based on the revised multi-dimensional task profile.

[0014] Based on the optimal resource scheduling strategy, the tasks currently being executed on the graphics card hardware are dynamically reallocated to complete the scheduling optimization.

[0015] Furthermore, the continuous monitoring of the task flow input to the graphics card and the real-time extraction of the static and dynamic descriptors of each task in the task flow includes:

[0016] Capture the task submission command, and parse the application interface identifier, shader program characteristics and initial buffer configuration information from the task submission command as a static descriptor;

[0017] At the beginning of the task, the instruction throughput, register file pressure and shared memory access patterns of the sampled task in the first computing cycle are used as the initial dynamic descriptor.

[0018] During the stable execution phase of the task, the computational unit utilization rate, memory bandwidth utilization rate, and cache hit rate curves of the periodically sampled task are used as dynamic descriptors for the steady-state phase.

[0019] Furthermore, the step of inputting the static descriptor and dynamic descriptor into the pre-trained task profile generation model to output a multi-dimensional task profile for each task includes:

[0020] The task profile generation model consists of a feature fusion layer and a multi-task prediction layer;

[0021] The feature fusion layer uses an attention mechanism to weight and fuse the static descriptor, the initial stage dynamic descriptor, and the steady-state stage dynamic descriptor to generate a unified task feature vector.

[0022] The multi-task prediction layer takes the unified task feature vector as input and outputs in parallel the predicted computing unit requirement profile, memory bandwidth requirement profile, and cache sensitivity profile, which together constitute the multi-dimensional task profile.

[0023] Furthermore, the calculation of the initial resource scheduling strategy for each task based on the multi-dimensional task profile includes:

[0024] Read the computing unit requirement profile and cache sensitivity profile from the multi-dimensional task profile;

[0025] Based on the computing unit requirement profile, allocate a corresponding number of streaming multiprocessor execution partitions to the task;

[0026] Based on the cache sensitivity profile, configure texture cache and constant cache of appropriate level and size for the task;

[0027] Based on the current global load of the graphics card hardware, dedicated or shared video memory channels are allocated for tasks to form the initial resource scheduling strategy.

[0028] Furthermore, the comparison of the actual resource consumption trajectory with the corresponding expected consumption in the multi-dimensional task profile to generate the resource consumption deviation for each task includes:

[0029] Extract the actual utilization rate sequence of computing units, the actual utilization rate sequence of video memory bandwidth, and the actual cache miss rate sequence from the actual resource consumption trajectory;

[0030] Calculate the difference between the actual utilization rate sequence of the computing unit and the demand profile of the computing unit, the difference between the actual utilization rate sequence of the video memory bandwidth and the demand profile of the video memory bandwidth, and the difference between the actual cache miss rate sequence and the cache sensitivity profile, respectively.

[0031] By weighted fusion of the aforementioned differences, a comprehensive scalar value is generated as the resource consumption deviation, specifically including:

[0032] The first weight is assigned to the difference in computing units, the second weight is assigned to the difference in video memory bandwidth, and the third weight is assigned to the difference in cache. The weights are set based on the sensitivity analysis of the impact of each resource on the overall performance.

[0033] Sum the weighted differences in computing units, memory bandwidth, and cache size;

[0034] The summation result is normalized, and the processed value is limited to between zero and one to obtain the resource consumption deviation.

[0035] Furthermore, feeding back the resource consumption deviation to the adversarial learning correction module to drive the task profile generation model to perform adversarial training, in order to correct the multi-dimensional task profile, includes:

[0036] The adversarial learning correction module includes a discriminator network used to determine the prediction accuracy of the current multi-dimensional task profile.

[0037] The resource consumption deviation is converted into a training signal for the discriminator network;

[0038] During adversarial training, the task profile generation model attempts to generate a new multi-dimensional task profile that minimizes the resource consumption deviation, while the discriminator network attempts to identify the gap between the new multi-dimensional task profile and the ideal profile.

[0039] Through iterative adversarial training, the multi-dimensional task profiles output by the task profile generation model continuously approximate the characteristics of real tasks.

[0040] Furthermore, the recalculation of the optimal resource scheduling strategy for each task based on the revised multi-dimensional task profile includes:

[0041] Obtain the corrected multi-dimensional task profile output by the task profile generation model after adversarial training;

[0042] A heuristic scheduling algorithm is adopted, with the global graphics card resource utilization rate as the optimization goal and the modified multi-dimensional task profile as the constraint condition, to resolve the computing unit allocation scheme, cache configuration scheme and video memory channel binding scheme for each task.

[0043] The re-solved allocation scheme, configuration scheme, and binding scheme are integrated into the optimal resource scheduling strategy.

[0044] Furthermore, during the stable execution phase of the task, the computational unit utilization rate, memory bandwidth utilization rate, and cache hit rate curves of the periodically sampled task include:

[0045] Set a fixed sampling period, and synchronously read the value of the graphics card performance counter at the time point of the sampling period;

[0046] The performance counters include a stream multiprocessor active thread bundle ratio counter, a video memory controller read / write transaction counter, and cache access and miss counters at all levels.

[0047] The values ​​read from multiple consecutive sampling cycles are smoothed to form the computing unit occupancy curve, the memory bandwidth utilization curve, and the cache hit rate curve.

[0048] Furthermore, the feature fusion layer employs an attention mechanism to perform weighted fusion of the static descriptor, the initial-stage dynamic descriptor, and the steady-state-stage dynamic descriptor, including:

[0049] Initialize a learnable attention weight vector for the static descriptor, the initial stage dynamic descriptor, and the steady-state stage dynamic descriptor respectively;

[0050] Calculate the relevance score of each descriptor to the current task scheduling context;

[0051] The attention weight vectors are scaled according to the relevance scores.

[0052] The scaled attention weight vector is multiplied by the corresponding descriptor and then summed to obtain the unified task feature vector.

[0053] Furthermore, the present invention also includes a machine learning-based graphics card task resource scheduling system, the system including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, it implements the steps of the machine learning-based graphics card task resource scheduling method described above.

[0054] Compared with the prior art, the beneficial effects of the present invention are:

[0055] By continuously monitoring the actual resource consumption trajectory during task execution and comparing it in real time with the expected consumption generated by the pre-trained model, the deviation in resource consumption is quantified. This deviation is used as a key signal to feed back to the adversarial learning correction module, driving the task profile generation model to perform online adversarial training. This allows the model to continuously learn and adapt to the behavioral pattern variations of the task in real, dynamic environments, rather than relying on one-time offline training results. The model itself possesses the adaptive ability to continuously correct its prediction accuracy during operation, thereby reducing long-term prediction bias caused by the uncertainty of task behavior or environmental changes, ensuring that the task profile always maintains a keen depiction of the current load characteristics.

[0056] Based on the more accurate and up-to-date task profile generated after online correction, the optimal resource scheduling strategy for each executing task is dynamically recalculated. Based on this new strategy, the system can proactively reallocate the computing units and video memory resources already allocated on the graphics card hardware throughout the task's lifecycle. This mechanism breaks the traditional scheduling model where resource allocation is fixed at task startup. It allows the system to immediately correct and optimize when it detects unreasonable initial allocation or drifting task requirements. This not only promptly reclaims idle or surplus resources and allocates them to tasks that need them more, improving the overall utilization efficiency of hardware resources, but also effectively alleviates the problems of prolonged task execution time or performance fluctuations caused by resource contention or improper allocation, achieving fine-grained, full-lifecycle dynamic control of resource allocation. Attached Figure Description

[0057] Figure 1 This is a flowchart illustrating the steps of the machine learning-based graphics card task resource scheduling method described in this invention.

[0058] Figure 2 A flowchart for task flow monitoring and descriptor extraction;

[0059] Figure 3 Flowchart of the process of generating a model for task profiling;

[0060] Figure 4 A graph showing the trend of graphics card task resource consumption;

[0061] Figure 5 A bar chart comparing resource requirements for multiple task types. Detailed Implementation

[0062] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] See Figure 1 The system continuously monitors the task flow input to the graphics card and extracts the static and dynamic descriptors of each task in real time. The extracted static and dynamic descriptors are then input into a pre-trained task profile generation model, which outputs a multi-dimensional task profile for each task. This profile characterizes the resource requirements of the task from multiple perspectives. Based on the generated multi-dimensional task profile, the system calculates the initial resource scheduling strategy for each task and allocates specific computing units and video memory resources to it on the graphics card hardware according to this strategy. After the task starts execution, the system synchronously monitors the actual resource consumption trajectory of each task on the graphics card hardware. By comparing the actual resource consumption trajectory with the corresponding expected consumption in the multi-dimensional task profile, the system generates the resource consumption deviation for each task. This deviation is fed back to an adversarial learning correction module, which drives the task profile generation model to perform adversarial training, thereby correcting the initially generated multi-dimensional task profile to better reflect the actual behavior of the task. Based on the corrected and more accurate multi-dimensional task profile, the system recalculates the optimal resource scheduling strategy for each task. The system then dynamically reallocates resources to the tasks currently executing on the graphics card hardware according to this optimal resource scheduling strategy, achieving scheduling optimization.

[0064] See Figure 2In one embodiment of the present invention, taking a graphics card performing a hybrid workload of matrix multiplication and ray tracing as an example, the task flow includes alternately submitted graphics rendering tasks and general computing tasks. The monitoring module intercepts the task submission instruction queues from the graphics application programming interface (API) and the computing API. In some embodiments, after capturing the task submission instructions, the process of parsing the static descriptor from the task submission instructions is as follows: For a graphics rendering task, the parsed API identifier is "Vulkan", the shader program characteristics include the number of instructions for the vertex shader and fragment shader and the number of texture sampling instructions, and the initial buffer configuration information includes a frame buffer size of 2048x2048 and a depth buffer format of 32-bit floating-point numbers; for a general computing task, the parsed API identifier is "CUDA", the shader program characteristics include the number of registers used by the kernel function and the size of the shared memory declaration, and the initial buffer configuration information includes the dimension of the input / output matrix in global memory as 1024x1024. This parsed information constitutes the static descriptor of the task.

[0065] In practice, at the initial stage of task startup, the initial dynamic descriptor of the task is sampled within the first computation cycle. When the aforementioned general-purpose computing task is assigned to a streaming multiprocessor for execution, the hardware performance monitoring unit samples the instruction throughput as 1.8 instructions per streaming multiprocessor clock cycle, register file pressure as an average register occupancy rate of 85% for active thread bundles, and shared memory access mode as a memory conflict rate of 15% within the first computation cycle. This data sampled within the first computation cycle is recorded as the initial dynamic descriptor of the general-purpose computing task.

[0066] Optionally, during the stable execution phase of the task, the computation unit utilization, memory bandwidth utilization, and cache hit rate curves of the task are periodically sampled. A fixed sampling period of 1 millisecond is set, and at each 1-millisecond time point, the system synchronously reads the values ​​of multiple performance counters integrated within the graphics card. These performance counters include a streaming multiprocessor active thread bundle ratio counter for measuring computation unit utilization, a memory controller read / write transaction counter for measuring memory bandwidth utilization, and L1 cache access and miss counters and L2 cache access and miss counters for measuring cache efficiency.

[0067] It's understandable that the raw values ​​read from the performance counter need to be processed to form a curve. For the active thread bundle ratio counter in a streaming multiprocessor, the sequence of values ​​read over 10 consecutive sampling periods is... To smooth these values ​​and form a computational unit occupancy curve, one approach is to apply a weighted moving average formula:

[0068]

[0069] in: This indicates the utilization rate of the computational units after smoothing in the current sampling period. This represents the original value of the computing unit occupancy rate read directly from the counter during the current sampling period. This indicates the occupancy rate of the computational unit after smoothing in the previous sampling period. This is a smoothing coefficient with a value between 0 and 1. Applying this formula to the above sequence yields the smoothed computational unit utilization curve data. The values ​​read by the memory controller's read / write transaction counter are converted to units and smoothed to form the memory bandwidth utilization curve. The values ​​read by the cache access and miss counters at each level are calculated by smoothing the ratio of misses to total accesses to form the cache hit rate curve. These processed curves together constitute the dynamic descriptor of the task's steady-state phase.

[0070] See Figure 3 In one embodiment of the present invention, static descriptors and dynamic descriptors are input into a pre-trained task profile generation model to output a multi-dimensional task profile for each task. The task profile generation model consists of a feature fusion layer and a multi-task prediction layer. The feature fusion layer receives static descriptors, initial-stage dynamic descriptors, and steady-state dynamic descriptors from the task flow monitoring module as input. These descriptors are represented as high-dimensional vectors. The feature fusion layer uses an attention mechanism to weightedly fuse the three types of descriptors to generate a unified task feature vector. In some embodiments, the feature fusion layer initializes a learnable attention weight vector for each of the static descriptors, initial-stage dynamic descriptors, and steady-state dynamic descriptors. The dimension of each attention weight vector is the same as the dimension of the corresponding descriptor vector. The system updates these attention weight vectors using a gradient descent algorithm during model training. It can be understood that the feature fusion layer needs to calculate the relevance score between each descriptor and the current task scheduling context. The task scheduling context is a dynamically encoded vector that encapsulates the current resource state of the graphics card hardware and the macroscopic features of the queue of tasks to be scheduled. The relevance score is obtained through the dot product operation between the descriptor vector and the task scheduling context vector.

[0071] In practice, scaling the attention weight vectors based on their relevance scores is accomplished using a scaling function. This function maps the relevance scores to a scaling factor and applies it to the corresponding attention weight vectors. The unified task feature vector is obtained by multiplying the scaled attention weight vectors by their corresponding descriptor vectors and then summing the results. This weighted fusion process can be represented by the following formula:

[0072]

[0073] in: Represents generating a unified task feature vector, symbol Represents the summation operation, set These represent the indices of the static descriptor, the initial-stage dynamic descriptor, and the steady-state-stage dynamic descriptor, respectively. Represents the softmax activation function. Represents the task scheduling context vector. This represents the transformation from the descriptor vector through a linear transformation. Exported key vector, symbol The dot product operation represents vectors, with the symbol... This indicates the transpose operation. Representatives and descriptors The corresponding learnable attention weight vector, symbol This represents element-wise multiplication. The descriptor vector representing the input feature fusion layer. Optionally, during model initialization, the attention weight vector. Set to a random decimal value, task scheduling context vector It is generated from the current system state through an independent fully connected neural network.

[0074] In some embodiments, the multi-task prediction layer takes a unified task feature vector output by the feature fusion layer as input. The multi-task prediction layer consists of multiple parallel fully connected sub-networks, each responsible for outputting a specific dimension of the multi-dimensional task profile. The first fully connected sub-network receives the unified task feature vector and outputs a predicted computational unit requirement profile, a vector that quantifies the prediction task's demand for streaming multiprocessors and thread bundle scheduling efficiency. The second fully connected sub-network receives the unified task feature vector and outputs a predicted memory bandwidth requirement profile, a vector that characterizes the prediction task's sensitivity to memory controller bandwidth and access latency. The third fully connected sub-network receives the unified task feature vector and outputs a predicted cache sensitivity profile, a vector that describes the prediction task's dependence on different levels of cache capacity and expected hit rate. It is understandable that these three fully connected subnetworks share a unified task feature vector as input during training, but each has independent weight parameters and activation functions, thereby generating a computing unit requirement profile, a memory bandwidth requirement profile, and a cache sensitivity profile in parallel. These profiles together constitute a multi-dimensional task profile for use by the subsequent resource scheduling strategy calculation module.

[0075] In one embodiment of the present invention, the process of calculating the initial resource scheduling strategy for each task based on the multi-dimensional task profile is completed by a strategy calculation module. The input of the strategy calculation module is the multi-dimensional task profile output by the task profile generation model, and the output is a specific hardware resource allocation scheme. The strategy calculation module first reads the computing unit demand profile and cache sensitivity profile from the multi-dimensional task profile. The computing unit demand profile is a numerical vector, and the cache sensitivity profile is also a numerical vector. These vectors encode the predicted demand characteristics of the task for specific hardware resources. In some embodiments, the system processes a general computing task, and the read computing unit demand profile vector is [0.85, 0.60, 0.30], where the first dimension represents the predicted demand intensity for the streaming multiprocessor computing core, the second dimension represents the predicted demand intensity for the thread bundle scheduler throughput, and the third dimension represents the predicted demand intensity for the dedicated function computing unit; the read cache sensitivity profile vector is [0.90, 0.45, 0.10], where the first dimension represents the predicted sensitivity to the L1 cache, the second dimension represents the predicted sensitivity to the texture cache or constant cache, and the third dimension represents the predicted sensitivity to the shared part of the L2 cache.

[0076] Based on the compute unit demand profile, the allocation of a corresponding number of streaming multiprocessor execution partitions to tasks is performed according to a mapping function. The dimension representing the core demand intensity in the compute unit demand profile vector is input to the mapping function, which combines the total number of physical streaming multiprocessors in the GPU hardware with the current idle state to output a suggested number of streaming multiprocessor partitions. This allocation process can be represented by the following formula:

[0077]

[0078] in: Represents the number of streaming multiprocessor execution partitions allocated to the current task, symbol This indicates the floor function. This represents a scalar value extracted from the computing unit demand profile vector, characterizing the intensity of predicted demand for streaming multiprocessor computing cores. The current global load factor represents the graphics card hardware. This represents the total number of physical stream multiprocessors in the graphics card hardware. If a graphics card has 80 physical stream multiprocessors and the current global load factor is 0.2, the value extracted from the compute unit demand profile... If the value is 0.85, then the calculated allocation quantity is... Execute partitions for 54 streaming multiprocessors.

[0079] Based on the cache sensitivity profile, the system configures appropriate levels and sizes of texture and constant caches for the task. The dimension value representing the predicted sensitivity to texture or constant cache in the cache sensitivity profile vector is used to query a preset configuration table, which defines cache configuration strategies corresponding to different sensitivity ranges. For a task with a dimension value of 0.45 in the cache sensitivity profile vector, querying the configuration table might yield a configuration strategy of "Enable 48KB texture cache and 8KB constant cache". Based on the configuration strategy, the system issues instructions to the hardware via the graphics card driver interface to set the specified cache capacity and access strategy for the task on its allocated streaming multiprocessor execution partition. Combined with the current global load of the graphics card hardware, the system allocates exclusive or shared video memory channels for the task. Optionally, the system checks the current bandwidth utilization of all memory controllers. If the bandwidth utilization of a memory controller is below a threshold and the predicted sensitivity dimension value of the corresponding L2 cache shared portion in the cache sensitivity profile vector is low, then the task is allocated exclusive access to the channel of that memory controller. If all memory controllers are under high load or the cache sensitivity profile of the task indicates that it is not sensitive to cache shared access, then the system allocates a shared memory channel among multiple high-load controllers for the task. The initial resource scheduling policy is ultimately expressed as a structured configuration file, which records the list of allocated streaming multiprocessor execution partition identifiers, the configured texture cache and constant cache sizes, and the list of bound memory channel controller identifiers.

[0080] In one embodiment of the present invention, the process of generating the resource consumption deviation for each task by comparing the actual resource consumption trajectory with the expected consumption corresponding to the multi-dimensional task profile is performed by the deviation analysis module. The input to the deviation analysis module is the actual resource consumption trajectory monitored during task execution and the multi-dimensional task profile previously output by the task profile generation model. In an example scenario, for a graphics processing task that performs convolution operations, the system extracts the actual utilization rate sequence of computing units, the actual utilization rate sequence of video memory bandwidth, and the actual cache miss rate sequence from the actual resource consumption trajectory. The actual utilization rate sequence of computing units is the proportion of active thread bundles of streaming multiprocessors collected in each of the 20 sampling periods of task execution, and the sequence data is [65%, 68%, 70%, 72%, 75%, 77%, 78%, 76%, 79%, 80%, 78%, 76%, 77%, 79%, 81%, 82%, 80%, 78%, 76%, 74%]. The actual memory bandwidth utilization sequence is the memory controller bandwidth utilization rate collected within the same sampling period, with the sequence data being [45%, 48%, 50%, 52%, 55%, 58%, 60%, 61%, 59%, 57%, 60%, 62%, 58%, 56%, 54%, 52%, 50%, 48%, 46%, 44%]. The actual cache miss rate sequence is the L1 cache miss rate collected within the same sampling period, with the sequence data being [12%, 11%, 10%, 10%, 9%, 8%, 8%, 7%, 7%, 6%, 6%, 5%, 5%, 5%, 4%, 4%, 4%, 5%, 6%, 7%].

[0081] In some embodiments, the difference between the actual computing unit occupancy rate sequence and the computing unit demand profile is calculated. The computing unit demand profile is a predictive scalar value, such as 0.75, representing a predicted average computing unit occupancy rate of 75%. The difference is obtained by calculating the absolute difference between the arithmetic mean of the actual sequence and the predicted scalar value. For the above-mentioned actual computing unit occupancy rate sequence, its arithmetic mean is 75.4%, and the absolute difference from the predicted value of 0.75 (i.e., 75%) is 0.4%. This absolute difference is normalized to a difference score. The normalization process can be achieved by dividing the absolute difference by a baseline value, such as 100%, to obtain the computing unit difference score. The calculation formula is:

[0082]

[0083] in: Represents the degree of difference between computational units, symbol This indicates the absolute value operation. The arithmetic mean of the sequence of actual occupancy rates of computing units. This represents a predicted scalar value extracted from the computational unit demand profile. Similarly, this can be understood as the difference in memory bandwidth. The cache variance is obtained by calculating the absolute difference between the average of the actual memory bandwidth utilization sequence and the predicted scalar value extracted from the memory bandwidth demand profile. The difference is obtained by calculating the absolute difference between the average of the actual cache miss rate sequence and the predicted scalar value extracted from the cache sensitivity profile. See Table 1 for an example comparison of the average of the actual consumption sequence and the predicted value for a task over a specific time period.

[0084] Table 1: Example Table Comparing Actual Average and Predicted Values ​​of Task Resource Consumption

[0085] Resource types Actual consumption sequence average Multi-dimensional task profile prediction value absolute difference Computing unit utilization 75.4 75.0 0.4 Video memory bandwidth utilization 54.3 50.0 4.3 Cache miss rate 6.9 5.0 1.9

[0086] Optionally, the process of weighted fusion of dissimilarity degrees to generate a comprehensive scalar value of resource consumption deviation degree assigns a first weight to the dissimilarity degree of the computing unit. Assign a second weight to the difference in video memory bandwidth. Assign a third weight to cache difference The weights are set based on a sensitivity analysis of the impact of various graphics card hardware resources on overall performance. This sensitivity analysis is determined through offline benchmark testing. For example, weights can be set... , , Resource consumption deviation Calculated using the following formula:

[0087]

[0088] in: It represents the deviation of the final generated resource consumption, and is a scalar value. Represents the degree of difference in computational units. This represents the difference in video memory bandwidth. Represents cache difference. , , These represent the weights assigned to the three variance values. In some embodiments, based on the data in the table above, it is assumed that the normalized variance values ​​are respectively... , , Substituting the weights into the calculation, the weighted sum is obtained as follows: Since the sum of the weights is 1, the normalized resource consumption deviation is... It is understandable that the summation result is normalized. When the sum of weights is 1, the weighted sum is the normalized value. The system limits this value to between zero and one, which is used as the resource consumption deviation of the task.

[0089] See Figure 4 This is a graphics card task resource consumption trend chart, showing the dynamic changes of three types of core resources over 20 sampling periods. This chart is the core raw data in the "deviation analysis phase" of graphics card task resource scheduling, and it can be directly used to calculate the actual average consumption of each resource. It completely records the resource consumption trajectory of the task from startup to steady state and then to termination, accurately depicting the resource demand pattern of the task. For example, the characteristics of high memory bandwidth pressure in the early stage of convolution operation tasks and stable computing unit load in the later stage are intuitively presented in this chart. The steady-state level of computing unit utilization and the peak range of memory bandwidth in the chart are the core inputs for the task profile generation model to calculate the initial resource allocation scheme, which can help the system to more rationally allocate computing units, memory channels, and cache configurations.

[0090] In one embodiment of the present invention, the process of feeding back the resource consumption deviation to the adversarial learning correction module to drive the task profile generation model to perform adversarial training to correct the multi-dimensional task profile is initiated. The adversarial learning correction module includes a discriminator network, which is a multilayer perceptron. Its input is the multi-dimensional task profile vector to be evaluated, and its output is a scalar value between 0 and 1, used to judge the prediction accuracy of the current multi-dimensional task profile. The closer the output value is to 1, the closer the discriminator network considers the profile to be the ideal profile. The resource consumption deviation is converted into a training signal for the discriminator network. Specifically, for the actual resource consumption deviation of a task, the system compares it with a preset threshold. If the resource consumption deviation exceeds the threshold, a label is generated to indicate that the corresponding multi-dimensional task profile is "inaccurate," and this label and the multi-dimensional task profile form a sample pair for training the discriminator network.

[0091] In practical implementation, during adversarial training, the task profile generation model attempts to generate new multi-dimensional task profiles that minimize resource consumption deviation. The model receives the original static and dynamic descriptors as input and generates a new set of multi-dimensional task profile vectors. These vectors are fed into the discriminator network for evaluation, and the system simultaneously calculates the simulated resource consumption deviation based on the new multi-dimensional task profiles. The training objective of the task profile generation model is to minimize the weighted sum of the discriminator network's probability of "inaccurate" judgments of its output profiles and the simulated resource consumption deviation. Understandably, simultaneously, the discriminator network attempts to identify the gap between the new multi-dimensional task profiles and the ideal profiles. The training objective of the discriminator network is to accurately distinguish between the new multi-dimensional task profiles generated by the task profile generation model and a pre-collected set of ideal task profiles considered accurate. Through iterative adversarial training, the task profile generation model and the discriminator network compete and optimize with each other, causing the multi-dimensional task profiles output by the task profile generation model to continuously approach the characteristics of the real task. In one iterative example, the resource consumption deviation of the initial multi-dimensional task portrait generated by the task portrait generation model for a ray tracing task was 0.25. After one adversarial training, the new multi-dimensional task portrait generated by the discriminator network had its "accuracy" probability increased from 0.3 to 0.6, and the resource consumption deviation simulated based on the new portrait decreased to 0.18.

[0092] In some embodiments, the optimal resource scheduling strategy for each task is recalculated based on the corrected multi-dimensional task profile. The system obtains the corrected multi-dimensional task profile output by the task profile generation model after adversarial training. The corrected multi-dimensional task profile may change numerically. For example, the corrected computation unit requirement profile value for the aforementioned ray tracing task is adjusted from 0.70 to 0.78, the memory bandwidth requirement profile value is adjusted from 0.65 to 0.72, and the cache sensitivity profile value is adjusted from 0.40 to 0.35. A heuristic scheduling algorithm is adopted, with the global GPU resource utilization as the optimization objective and the corrected multi-dimensional task profile as the constraint, to resolve the computation unit allocation scheme, cache configuration scheme, and memory channel binding scheme for each task. A specific implementation of the heuristic scheduling algorithm is a greedy algorithm combined with backtracking search. The algorithm traverses all currently executing tasks, sorts the tasks according to the values ​​of each dimension in the corrected multi-dimensional task profile, and then attempts to place the tasks on hardware resource partitions that meet their corrected resource profile constraints according to the sorted order. The objective function is to maximize the average utilization of all physical multiprocessors and the average bandwidth utilization of the memory controller.

[0093] It's understandable that the process of resolving allocation, configuration, and binding schemes requires consideration of resource conflicts. The greedy algorithm first attempts to allocate the required number of streaming multiprocessor partitions to the task with the highest revised computational unit requirement profile. If there are insufficient available partitions, the algorithm backtracks and attempts to adjust the resource allocation of previously assigned tasks, or considers allocating slightly fewer partitions to the task while meeting performance constraints. The cache configuration scheme is directly mapped to a preset configuration policy table based on the revised cache sensitivity profile. The memory channel binding scheme is dynamically determined based on the revised memory bandwidth requirement profile and the current remaining bandwidth of each memory controller. This process can be evaluated using the following formula to assess the objective function value of a resource allocation scheme:

[0094]

[0095] in: This represents the target value for global resource utilization to be optimized. This represents the total number of physical stream multiprocessors in the graphics card. Representing the Projected utilization of physical stream multiprocessors under the current allocation scheme This represents the total number of video memory controllers. Representing the The expected bandwidth utilization of each memory controller under the current allocation scheme. A heuristic algorithm search is used to find the objective function value. Maximize resource allocation. Optionally, the re-solved computing unit allocation scheme, cache configuration scheme, and video memory channel binding scheme can be integrated into an optimal resource scheduling strategy. The optimal resource scheduling strategy is encoded into a new configuration file or applied directly to the graphics card hardware through the driver interface, overriding the original initial resource scheduling strategy of the task, thereby realizing dynamic resource reallocation for the currently executing task.

[0096] See Figure 5This is a bar chart comparing resource requirements across multiple task types. It clearly quantifies the demand for computing units, memory bandwidth, and cache in five typical GPU tasks, including ray tracing and deep learning. This chart provides a quantitative basis for multi-dimensional task profiling and can be directly used for training and validating task profiling models, allowing models to more accurately characterize the resource requirements of various tasks. The scheduling system can dynamically allocate computing units, memory bandwidth, and cache resources based on the differences in resource requirements for various tasks shown in the chart. For example, more computing units can be prioritized for graphics rendering tasks, while high memory bandwidth can be guaranteed for deep learning tasks. Hardware manufacturers can optimize GPU hardware designs based on these requirements, such as strengthening the cache architecture for tasks with high cache sensitivity and improving memory controller performance for tasks with high memory bandwidth requirements. In mixed-task scenarios, this chart helps the system predict the resource contention risks of different tasks, thereby formulating more reasonable task scheduling and resource isolation strategies to avoid performance bottlenecks.

[0097] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A machine learning-based GPU task resource scheduling method, characterized in that, The method includes: Continuously monitor the task flow input to the graphics card and extract the static and dynamic descriptors of each task in the task flow in real time, including: Capture the task submission command, and parse the application interface identifier, shader program characteristics and initial buffer configuration information from the task submission command as a static descriptor; At the beginning of the task, the instruction throughput, register file pressure and shared memory access patterns of the sampled task in the first computing cycle are used as the initial dynamic descriptor. During the stable execution phase of the task, the computation unit utilization rate, memory bandwidth utilization rate, and cache hit rate curves of the periodic sampling task are used as dynamic descriptors for the steady-state phase. The static and dynamic descriptors are input into a pre-trained task profile generation model, which outputs a multi-dimensional task profile for each task, including: The task profile generation model consists of a feature fusion layer and a multi-task prediction layer; The feature fusion layer uses an attention mechanism to weight and fuse the static descriptor, the initial stage dynamic descriptor, and the steady-state stage dynamic descriptor to generate a unified task feature vector. The multi-task prediction layer takes the unified task feature vector as input and outputs in parallel the predicted computing unit demand profile, memory bandwidth demand profile, and cache sensitivity profile, which together constitute the multi-dimensional task profile. The initial resource scheduling strategy for each task is calculated based on the multi-dimensional task profile, and computing units and video memory resources are allocated on the graphics card hardware according to the initial resource scheduling strategy. During task execution, the actual resource consumption trajectory of each task on the graphics card hardware is monitored synchronously. By comparing the actual resource consumption trajectory with the corresponding expected consumption in the multi-dimensional task profile, the resource consumption deviation of each task is generated. The resource consumption deviation is fed back to the adversarial learning correction module to drive the task profile generation model to perform adversarial training in order to correct the multi-dimensional task profile. The optimal resource scheduling strategy for each task is recalculated based on the revised multi-dimensional task profile. Based on the optimal resource scheduling strategy, the tasks currently being executed on the graphics card hardware are dynamically reallocated to complete the scheduling optimization.

2. The machine learning-based graphics card task resource scheduling method as described in claim 1, characterized in that, The calculation of the initial resource scheduling strategy for each task based on the multi-dimensional task profile includes: Read the computing unit requirement profile and cache sensitivity profile from the multi-dimensional task profile; Based on the computing unit requirement profile, allocate a corresponding number of streaming multiprocessor execution partitions to the task; Based on the cache sensitivity profile, configure texture cache and constant cache of appropriate level and size for the task; Based on the current global load of the graphics card hardware, dedicated or shared video memory channels are allocated for tasks to form the initial resource scheduling strategy.

3. The machine learning-based graphics card task resource scheduling method as described in claim 2, characterized in that, The comparison between the actual resource consumption trajectory and the corresponding expected consumption in the multi-dimensional task profile, generating the resource consumption deviation for each task, includes: Extract the actual utilization rate sequence of computing units, the actual utilization rate sequence of video memory bandwidth, and the actual cache miss rate sequence from the actual resource consumption trajectory; Calculate the difference between the actual utilization rate sequence of the computing unit and the demand profile of the computing unit, the difference between the actual utilization rate sequence of the video memory bandwidth and the demand profile of the video memory bandwidth, and the difference between the actual cache miss rate sequence and the cache sensitivity profile, respectively. By weighted fusion of the aforementioned differences, a comprehensive scalar value is generated as the resource consumption deviation, specifically including: The first weight is assigned to the difference in computing units, the second weight is assigned to the difference in video memory bandwidth, and the third weight is assigned to the difference in cache. The weights are set based on the sensitivity analysis of the impact of each resource on the overall performance. Sum the weighted differences in computing units, memory bandwidth, and cache size; The summation result is normalized, and the processed value is limited to between zero and one to obtain the resource consumption deviation.

4. The machine learning-based graphics card task resource scheduling method as described in claim 3, characterized in that, The step of feeding back the resource consumption deviation to the adversarial learning correction module to drive the task profile generation model to perform adversarial training in order to correct the multi-dimensional task profile includes: The adversarial learning correction module includes a discriminator network used to determine the prediction accuracy of the current multi-dimensional task profile. The resource consumption deviation is converted into a training signal for the discriminator network; During adversarial training, the task profile generation model attempts to generate a new multi-dimensional task profile that minimizes the resource consumption deviation, while the discriminator network attempts to identify the gap between the new multi-dimensional task profile and the ideal profile. Through iterative adversarial training, the multi-dimensional task profiles output by the task profile generation model continuously approximate the characteristics of real tasks.

5. The machine learning-based graphics card task resource scheduling method as described in claim 4, characterized in that, The recalculation of the optimal resource scheduling strategy for each task based on the revised multi-dimensional task profile includes: Obtain the corrected multi-dimensional task profile output by the task profile generation model after adversarial training; A heuristic scheduling algorithm is adopted, with the global graphics card resource utilization rate as the optimization goal and the modified multi-dimensional task profile as the constraint condition, to resolve the computing unit allocation scheme, cache configuration scheme and video memory channel binding scheme for each task. The re-solved allocation scheme, configuration scheme, and binding scheme are integrated into the optimal resource scheduling strategy.

6. The machine learning-based graphics card task resource scheduling method as described in claim 5, characterized in that, During the stable execution phase of the task, the curves of computational unit utilization, video memory bandwidth utilization, and cache hit rate of the periodically sampled task include: Set a fixed sampling period, and synchronously read the value of the graphics card performance counter at the time point of the sampling period; The performance counters include a stream multiprocessor active thread bundle ratio counter, a video memory controller read / write transaction counter, and cache access and miss counters at all levels. The values ​​read from multiple consecutive sampling cycles are smoothed to form the computing unit occupancy curve, the memory bandwidth utilization curve, and the cache hit rate curve.

7. The machine learning-based graphics card task resource scheduling method as described in claim 6, characterized in that, The feature fusion layer employs an attention mechanism to perform weighted fusion of the static descriptor, the initial-stage dynamic descriptor, and the steady-state-stage dynamic descriptor, including: Initialize a learnable attention weight vector for the static descriptor, the initial stage dynamic descriptor, and the steady-state stage dynamic descriptor respectively; Calculate the relevance score of each descriptor to the current task scheduling context; The attention weight vectors are scaled according to the relevance scores. The scaled attention weight vector is multiplied by the corresponding descriptor and then summed to obtain the unified task feature vector.

8. A machine learning-based graphics card task resource scheduling system, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the machine learning-based graphics card task resource scheduling method as described in any one of claims 1 to 7.

Images