A GPU resource intelligent scheduling method and system based on reinforcement learning

Abstract

The application provides a GPU resource intelligent scheduling method and system based on reinforcement learning, relates to the field of cloud computing resource scheduling, and comprises the following steps: collecting resource state data of a GPU cluster and attribute data of a to-be-scheduled task, pre-processing, constructing a reinforcement learning model, and training the reinforcement learning model by using historical scheduling records; inputting the resource state data of the current GPU cluster and the task attribute data of the to-be-scheduled task into the trained reinforcement learning model to obtain a preliminary scheduling scheme; generating a candidate scheduling scheme set based on the preliminary scheduling scheme, selecting a final scheduling scheme from the candidate scheduling scheme set by using a multi-objective optimization algorithm; performing resource slice configuration on a target GPU node in the GPU cluster according to the final scheduling scheme, and starting the to-be-scheduled task in a task execution container; and updating the parameters of the reinforcement learning model by using experience data. The application realizes adaptive evolution and multi-objective balanced optimization of a scheduling strategy.

Description

Technical Field

[0001] This invention relates to the field of cloud computing resource scheduling technology, and in particular to a method and system for intelligent scheduling of GPU resources based on reinforcement learning. Background Technology

[0002] With the development of industrial design and simulation technologies, the demand for GPU computing power in CAD design and CAE simulation tasks is increasing daily, making design simulation cloud platforms a crucial infrastructure for enterprises and research institutions. However, existing GPU resource scheduling methods mainly employ rule-based static resource allocation strategies, which allocate GPU resources to tasks according to preset fixed rules in a fixed proportion or with a fixed priority. This static scheduling method has significant technical drawbacks. Although GPU virtualization technologies such as MIG and time slicing have achieved physical or logical partitioning of GPU resources, the granularity of the slices and the allocation ratio are usually preset during deployment and cannot be dynamically adjusted according to the actual computing power requirements of different tasks. This leads to a mismatch where small tasks consume too many resources while large tasks lack sufficient resources, causing GPU resource utilization to remain at a low level for a long time.

[0003] Furthermore, CAD design tasks and CAE simulation tasks in design simulation scenarios exhibit significant differences in their computing power requirements. CAD design tasks typically require high-frequency GPU response but have fluctuating computing power demands, while CAE simulation tasks require continuous and stable large-scale computing power. Existing static scheduling cannot identify and adapt to these heterogeneous task characteristics, resulting in inefficient resource allocation. Traditional schedulers rely on manually set rule parameters, which need to be readjusted when task types increase or load patterns change. They lack adaptive learning capabilities and struggle to maintain long-term high-efficiency scheduling performance in dynamically changing application scenarios.

[0004] Chinese patent CN114443249A discloses a container cluster resource scheduling method and system based on deep reinforcement learning. This method establishes a deep reinforcement learning agent, inputting the resource usage status of container cluster nodes and the feature values ​​of tasks to be scheduled into the agent to obtain the action probability distribution for task scheduling. The container cluster scheduler schedules tasks to be scheduled to the corresponding container cluster nodes according to the action probability distribution and calculates rewards. The network parameters of the deep reinforcement learning agent are updated based on the rewards for continuous learning and training. However, when the task types increase or the load mode changes, the rule parameters need to be readjusted, lacking adaptive learning capabilities and making it difficult to maintain long-term high-efficiency scheduling performance in dynamically changing application scenarios. Summary of the Invention

[0005] In view of this, the present invention provides a GPU resource intelligent scheduling method and system based on reinforcement learning, which can solve the technical problem in the prior art that the rules cannot be adjusted according to the task type mechanism, and realize the adaptive evolution of scheduling strategy and multi-objective balance optimization.

[0006] The technical solution of this invention is implemented as follows:

[0007] On the one hand, this invention provides a GPU resource intelligent scheduling method based on reinforcement learning, comprising:

[0008] S1. Collect resource status data of the GPU cluster and attribute data of the tasks to be scheduled, and preprocess them to obtain standardized resource status data and task attribute data.

[0009] S2. Construct a reinforcement learning model that includes an online network and a target network. Train the reinforcement learning model using historical scheduling records to obtain the trained reinforcement learning model. The online network and the target network determine the Q-value of each action based on resource status data and task attribute data. The actions include the GPU resource slice ratio and GPU node allocation mode.

[0010] S3. Input the current GPU cluster resource status data and the task attribute data of the task to be scheduled into the trained reinforcement learning model to obtain a preliminary scheduling scheme; determine the neighborhood search range based on the decision quality evaluation results of the preliminary scheduling scheme, generate a set of candidate scheduling schemes in the neighborhood, calculate multiple optimization objective values ​​for each candidate scheme in the set of candidate scheduling schemes, and use a multi-objective optimization algorithm to select the final scheduling scheme from the set of candidate scheduling schemes.

[0011] S4. Based on the final scheduling scheme, configure the target GPU node in the GPU cluster to obtain GPU resource instances by performing resource slicing, bind the task execution container to the GPU resource instance, and start the scheduled task in the task execution container.

[0012] S5. Collect performance data of task execution, construct and store experience data based on the performance data, and update the reinforcement learning model parameters when the accumulated experience data reaches a preset threshold.

[0013] Based on the above technical solutions, preferably, step S1 specifically includes:

[0014] The GPU utilization and video memory usage of each GPU node in the GPU cluster are collected to obtain the task type, task priority and task time limit requirements of the task to be scheduled, and to obtain the original resource status data and original task attribute data. The task type includes CAD design tasks and CAE simulation tasks, and the task attribute parameters include the model file size and number of parts for CAD tasks and the number of simulation meshes for CAE tasks.

[0015] The original resource status data and original task attribute data are filtered and normalized to obtain standardized resource status data and standardized task attribute data.

[0016] The static complexity of the task is calculated based on the task attribute parameters. A set of historical tasks similar to the task to be scheduled is retrieved from the historical task database. A resource performance prediction model is established based on the correlation between resource configuration and execution time of the historical task set. The dynamic complexity of the task is calculated based on the resource performance prediction model and the static complexity. The static complexity and dynamic complexity are merged to obtain the final complexity of the task as a component of the task attribute data.

[0017] Based on the above technical solutions, preferably, the dynamic complexity of the computational task specifically includes:

[0018] Calculate the feature vector distance between the task to be scheduled and each historical task in the historical task database, calculate the similarity based on the feature vector distance, and select a preset number of historical tasks with the highest similarity to form a similar task set.

[0019] A segmented resource performance prediction model is established by fitting the resource allocation and execution time data of historical tasks in a set of similar tasks.

[0020] The dynamic complexity is calculated based on the predicted execution time of the segmented resource performance prediction model and the set of similar tasks.

[0021] The final complexity is obtained by fusing the static and dynamic complexity using dynamic weighting coefficients.

[0022] ;

[0023] in, Indicate the final complexity, Represents static complexity. Indicates dynamic complexity. This represents the dynamic weighting coefficient, which is determined based on the number of similar task sets and the average similarity.

[0024] Based on the above technical solutions, preferably, step S2 specifically includes:

[0025] Training samples are extracted from historical scheduling records. The training samples include the state at the time of scheduling, the action performed, the reward obtained, and the state after the task is completed. Each training sample is assigned a priority based on the time difference error and stored in the experience replay memory.

[0026] Set up a multi-objective reward function that includes resource utilization, task response speed, and energy consumption control;

[0027] Training samples are sampled from the experience replay memory according to priority. The optimal action for the next state is selected using the online network, and the Q value of the optimal action is evaluated using the target network to calculate the target Q value. The parameters of the online network are updated by minimizing the mean square error between the current Q value and the target Q value, and the parameters of the target network are made to track the parameters of the online network, thus obtaining the trained reinforcement learning model.

[0028] Based on the above technical solutions, preferably, step S3 specifically includes:

[0029] The average resource utilization of the current GPU cluster and the combined priority of the tasks to be scheduled are used to construct a state vector. The state vector is then input into the trained reinforcement learning model to output the Q value corresponding to each action. The action with the largest Q value is selected as the initial scheduling scheme.

[0030] The decision quality of the preliminary scheduling scheme is evaluated based on the Q value of each action and the similar historical states in the experience replay memory. The neighborhood search range is dynamically determined based on the decision quality evaluation results, and a set of candidate scheduling schemes is generated within the neighborhood search range.

[0031] For each candidate scheduling scheme in the candidate scheduling scheme set, multiple optimization objective values ​​are calculated. A multi-objective optimization algorithm is used to perform non-dominated sorting on the candidate scheduling scheme set, and the final scheduling scheme is determined based on the non-dominated sorting results.

[0032] Based on the above technical solutions, preferably, the quality assessment of the preliminary scheduling scheme specifically includes:

[0033] Extract the Q-values ​​of each action output by the trained reinforcement learning model, calculate the difference between the Q-value of the best action and the Q-value of the second-best action, and calculate the Q-value confidence based on the difference;

[0034] Retrieve historical states from the experience replay memory whose distance from the current state vector is less than a preset threshold, and calculate the historical experience support degree based on the number of historical states that meet the conditions and the average distance.

[0035] The decision quality assessment value is obtained by weighting and summing the Q-value confidence level and the historical experience support level.

[0036] When the decision quality assessment value is higher than the first threshold, the narrowed neighborhood search range is determined; when the decision quality assessment value is lower than the second threshold, the expanded neighborhood search range is determined, wherein the first threshold is greater than the second threshold.

[0037] Based on the above technical solutions, preferably, the step of calculating multiple optimization target values ​​for each candidate scheme in the candidate scheduling scheme set specifically includes:

[0038] Calculate the target resource utilization value based on the GPU resource slicing ratio of the candidate solutions and the average utilization of historical tasks under the corresponding resource configurations:

[0039] ;

[0040] in This represents the target value for resource utilization. Indicates the GPU resource slice ratio. This represents the average GPU utilization of historical tasks under the same slice ratio;

[0041] Based on the GPU resource slice ratio of the candidate solutions, the expected completion time of the candidate solutions is calculated using a segmented resource performance prediction model. The expected latency target value is then calculated based on the difference between the expected completion time of the candidate solutions and the task time limit.

[0042] ;

[0043] in This indicates the expected delay target value. This indicates the expected completion time based on the candidate solutions. Indicates the time limit set by the user;

[0044] Calculate the expected energy consumption target value based on the number of GPUs allocated to the candidate solutions, the average power of a single GPU, and the expected completion time:

[0045] ;

[0046] in This indicates the expected energy consumption target value. Indicates the number of GPUs allocated. This represents the average power of a single GPU.

[0047] Based on the above technical solutions, preferably, step S4 specifically includes:

[0048] Select the target GPU node from the GPU cluster according to the GPU node allocation mode in the final scheduling scheme;

[0049] Based on the GPU resource slicing ratio in the final scheduling scheme, the target GPU node is configured with resource slicing to obtain the GPU resources available for binding;

[0050] Create a task execution container, associate the task execution container with available GPU resources, and start the scheduled task in the task execution container.

[0051] Furthermore, step S5 specifically includes:

[0052] During task execution, execution metrics data are collected, and anomaly detection is performed on the execution metrics data; wherein the execution metrics data includes execution time, GPU utilization, video memory usage, and GPU temperature data;

[0053] After the task is completed, the actual performance metrics of the task are calculated, and the actual performance metrics are substituted into the multi-objective reward function to calculate the actual reward value; wherein the actual performance metrics include actual completion time, average GPU utilization, and total energy consumption data.

[0054] Construct empirical data containing scheduling state vectors, executed actions, actual reward values, and task completion state vectors; assign priorities to the empirical data based on temporal differential errors and store them in an empirical replay memory and a historical task database.

[0055] When the accumulated new experience data in the experience replay memory reaches a preset threshold, training samples are sampled from the experience replay memory according to priority, and the parameters of the online network and the target network are updated using incremental training.

[0056] In addition, the present invention also provides a GPU resource intelligent scheduling system based on reinforcement learning to realize the GPU resource intelligent scheduling method based on reinforcement learning described above.

[0057] The present invention has the following advantages over the prior art:

[0058] (1) By using reinforcement learning models and multi-objective optimization algorithms, the GPU resource slicing ratio and node allocation mode are intelligently determined based on the GPU cluster status and task attributes. The scheduling strategy is continuously optimized through online learning mechanisms to adapt to the dynamic changes in task modes, thus realizing the adaptive evolution and multi-objective balance optimization of the scheduling strategy.

[0059] (2) By establishing a nonlinear response relationship between resource allocation and execution time through similar task retrieval and segmented resource performance prediction model, the accuracy of task complexity measurement and the reliability of scheduling decisions are improved.

[0060] (3) The reliability of the initial decision of reinforcement learning is comprehensively evaluated by Q-value confidence and historical experience support. The neighborhood search range of multi-objective optimization is dynamically adjusted according to the decision quality, thereby improving the decision accuracy. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a flowchart of a GPU resource intelligent scheduling method based on reinforcement learning according to the present invention.

[0063] Figure 2 This is a flowchart illustrating the reinforcement learning scheduling process of a GPU resource intelligent scheduling method based on reinforcement learning, as described in this invention.

[0064] Figure 3 This is a resource allocation diagram illustrating a GPU resource intelligent scheduling method based on reinforcement learning according to the present invention. Detailed Implementation

[0065] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0066] like Figure 1 As shown, this invention provides a GPU resource intelligent scheduling method based on reinforcement learning, comprising:

[0067] S1. Collect resource status data of the GPU cluster and attribute data of the tasks to be scheduled, and preprocess them to obtain standardized resource status data and task attribute data.

[0068] Specifically, step S1 includes:

[0069] The GPU utilization and video memory usage of each GPU node in the GPU cluster are collected to obtain the task type, task priority and task time limit requirements of the task to be scheduled, and to obtain the original resource status data and original task attribute data. The task type includes CAD design tasks and CAE simulation tasks, and the task attribute parameters include the model file size and number of parts for CAD tasks and the number of simulation meshes for CAE tasks.

[0070] The original resource status data and original task attribute data are filtered and normalized to obtain standardized resource status data and standardized task attribute data.

[0071] The static complexity of the task is calculated based on the task attribute parameters. A set of historical tasks similar to the task to be scheduled is retrieved from the historical task database. A resource performance prediction model is established based on the correlation between resource configuration and execution time of the historical task set. The dynamic complexity of the task is calculated based on the resource performance prediction model and the static complexity. The static complexity and dynamic complexity are merged to obtain the final complexity of the task as a component of the task attribute data.

[0072] Understandably, the scheduling system deploys resource monitoring agents on each node of the GPU server cluster. These agents collect real-time utilization, memory usage, and core temperature data for each GPU once per second via NVIDIA's NVML interface, and upload the collected data to the scheduling system's data acquisition module in real time. When a user submits a design or simulation task through the cloud platform, the scheduling system automatically extracts task attribute information, including task type, quantitative indicators of task complexity, user-defined task time limits, and task priority determined based on the user's level, thus obtaining raw resource status data and raw task attribute data.

[0073] Static complexity refers to a quantified metric of complexity calculated based on the task's inherent characteristics at the time of task submission. This metric does not rely on historical execution data or resource allocation; it is calculated solely by analyzing the task's inherent attributes. For CAD design tasks, static complexity is quantified using a weighted normalized value of the model file size and the number of parts. For CAE simulation tasks, static complexity is quantified using a normalized value of the number of simulation meshes.

[0074] In one embodiment of the present invention, the historical task database for CAD tasks and CAE tasks includes a task static feature vector, a resource configuration vector, and actual execution performance for each record.

[0075] In one embodiment of the present invention, the dynamic complexity of the computation task specifically includes:

[0076] Calculate the feature vector distance between the task to be scheduled and each historical task in the historical task database, calculate the similarity based on the feature vector distance, and select a preset number of historical tasks with the highest similarity to form a similar task set.

[0077] A segmented resource performance prediction model is established by fitting resource allocation and execution time data of historical tasks in a set of similar tasks:

[0078] ;

[0079] ;

[0080] ;

[0081] in, Indicates the proportion of a given GPU resource slice. The expected completion time for the assigned task. Indicates reference resource configuration The historical average execution time This represents the resource saturation threshold, used to identify the resource allocation point where GPU utilization begins to decline significantly by analyzing execution data of similar tasks. This indicates the GPU resource slice ratio, which is the proportion of GPU computing power allocated to a task relative to the total computing power of a single GPU. This represents the highest average GPU utilization for similar tasks. This represents the resource insufficiency penalty coefficient, ranging from 2.0 to 3.5, reflecting the deterioration of nonlinear performance caused by resource insufficiency. Represents a set of similar tasks. This represents the similarity between the i-th historical task and the task to be scheduled. This represents the actual execution time of the i-th historical task. This represents the average GPU utilization for similar tasks when the resource allocation ratio is r. Indicates the reference resource allocation ratio;

[0082] The dynamic complexity is calculated based on the predicted execution time of the segmented resource performance prediction model and the set of similar tasks:

[0083] ;

[0084] ;

[0085] ;

[0086] in, Indicates tasks to be scheduled In resource allocation ratio The dynamic complexity is as follows. The median historical execution time of similar tasks is used for normalization. Indicates the prediction confidence level. Represents the stability coefficient. This represents the standard deviation of the average utilization rate for similar tasks. This represents the average GPU utilization for similar tasks. Indicates the stability influencing factor. To minimize the number of trustworthy similar tasks, The average similarity of similar tasks;

[0087] The final complexity is obtained by fusing the static and dynamic complexity using dynamic weighting coefficients.

[0088] ;

[0089] in, Indicate the final complexity, Represents static complexity. Indicates dynamic complexity. This represents the dynamic weighting coefficient, which is determined based on the number of similar task sets and the average similarity.

[0090] Understandably, the segmented resource performance prediction model reflects the characteristic that execution time is inversely proportional to resource allocation when resources are sufficient, and increases non-linearly when resources are insufficient. The dynamic weight coefficient can be set according to actual usage requirements, increasing with the number of similar tasks and the average similarity. When the number of similar tasks is less than a preset threshold or the average similarity is lower than a preset threshold, the dynamic weight coefficient takes a smaller value to reduce the impact of dynamic complexity.

[0091] When historical data is abundant and highly similar, dynamic complexity dominates; when historical data is lacking, it reverts to static complexity. Final complexity As a component of task attribute data, the resource-performance response relationship is learned through historical similar tasks. The piecewise linear model captures the non-linear performance degradation when resources are insufficient, and the confidence mechanism ensures robust regression when historical data is insufficient.

[0092] This invention establishes a nonlinear response relationship between resource configuration and execution time through similar task retrieval and segmented resource performance prediction models. Compared with simple static feature normalization, it can more accurately predict task execution performance under different resource configurations, effectively capturing the nonlinear performance degradation characteristics of GPU resource saturation effect and insufficient resources. At the same time, through the confidence mechanism, it robustly backs to static complexity when historical data is insufficient, improving the accuracy of task complexity quantification and the reliability of scheduling decisions.

[0093] Specifically, for CAD design tasks, the complexity calculation formula is:

[0094] ;

[0095] in, A quantification representing the complexity of a CAD task. This indicates the size of the model file for the current task. and These represent the minimum and maximum model file sizes in historical tasks, respectively. This indicates the number of model parts for the current task. and These represent the minimum and maximum number of parts in the historical mission, respectively. and The weighting coefficients are satisfied. ;

[0096] For CAE simulation tasks, the complexity calculation formula is:

[0097] ;

[0098] in, A quantification representing the complexity of a CAE task. This indicates the number of simulation grids for the current task. and These represent the minimum and maximum number of grid cells in historical tasks, respectively.

[0099] S2. Construct a reinforcement learning model that includes an online network and a target network. Train the reinforcement learning model using historical scheduling records to obtain the trained reinforcement learning model. The online network and the target network are based on the Q-values ​​of each action according to resource status data and task attribute data. The actions include the GPU resource slice ratio and GPU node allocation mode.

[0100] Specifically, step S2 includes:

[0101] Training samples are extracted from historical scheduling records. These training samples include the scheduling state, executed actions, reward received, and task completion state. Each training sample is assigned a priority based on temporal difference error and stored in an experience replay memory. The experience replay memory is a buffer for storing historical training samples, employing a first-in-first-out circular buffer mechanism. When the storage space reaches its capacity limit, the oldest sample data is automatically overwritten, thus maintaining a fixed storage size and ensuring the timeliness of the training samples. Temporal difference error reflects the gap between the reinforcement learning model's predicted Q-value and the actual execution reward. The larger the gap, the more information the sample contains and the higher its learning value.

[0102] Configure a multi-objective reward function that includes resource utilization, task response speed, and energy consumption control:

[0103] ;

[0104] ;

[0105] ;

[0106] ;

[0107] ;

[0108] ;

[0109] ;

[0110] in Represents the total reward value. This indicates a reward item for resource utilization. This indicates a reward item for task response speed. This indicates an energy consumption control reward item. This indicates the penalty for violating the constraints. , , , The weighting coefficients are satisfied. , This indicates the GPU computing power allocated to this task. This represents the total computing power of the GPU cluster. This indicates the actual average GPU utilization for the task. Indicates the actual time taken to complete the task. This indicates the time limit set by the user. This indicates the actual total energy consumption of the task. This indicates the maximum power allocated to the GPU when running at full power for this task. This indicates the timeout penalty. This is the timeout penalty coefficient. This is the timeout tolerance threshold. This indicates a penalty for load imbalance. The imbalance penalty coefficient, This indicates the GPU utilization rate for the task. This represents the average GPU utilization of the cluster. The threshold for tolerance of imbalance;

[0111] Training samples are sampled from the experience replay memory according to priority. The optimal action for the next state is selected using the online network, and the Q value of the optimal action is evaluated using the target network to calculate the target Q value. The parameters of the online network are updated by minimizing the mean square error between the current Q value and the target Q value, and the parameters of the target network are made to track the parameters of the online network, thus obtaining the trained reinforcement learning model.

[0112] In one embodiment of the present invention, the target Q value is calculated using a double Q network method, and the calculation formula is as follows:

[0113] ;

[0114] in, Indicates the target Q value. Indicates the actual reward. Indicates the discount factor. Indicates the target network. Indicates the next state. Indicates online network, This refers to actions, i.e., all possible GPU resource allocation schemes. Parameters representing online networks, The parameters representing the target network, This indicates taking the action that maximizes the Q value;

[0115] The gradient of the loss function with respect to the network parameters is calculated using the backpropagation algorithm. The network parameters are then updated using the optimizer. The loss function is:

[0116] ;

[0117] in, Represents the loss function. Indicates the current state. Indicates the next state. This indicates the experience playback memory. This represents the expectation of samples taken from the experience playback memory D. Indicates the target network's next state Execute action Q-value prediction, This represents the Q-value prediction of the online network for the action a to be performed in the current state s. Indicates all possible next actions Find the maximum value;

[0118] The parameter update formula is:

[0119] ;

[0120] in, For learning rate, This represents the gradient of the loss function with respect to the parameters. It is used during training. - A greedy exploration strategy is used, where the parameters of the target network are updated softly. After a certain number of rounds of parameter updates, the update formula is: ,in, This is the soft update coefficient, which is usually taken as a small value so that the target network parameters slowly track the online network parameters, resulting in a trained reinforcement learning model.

[0121] like Figure 2 As shown, S3, the resource status data of the current GPU cluster and the task attribute data of the task to be scheduled are input into the trained reinforcement learning model to obtain a preliminary scheduling scheme; the neighborhood search range is determined based on the decision quality evaluation results of the preliminary scheduling scheme, a set of candidate scheduling schemes is generated in the neighborhood, multiple optimization objective values ​​are calculated for each candidate scheme in the set of candidate scheduling schemes, and a multi-objective optimization algorithm is used to select the final scheduling scheme from the set of candidate scheduling schemes;

[0122] Understandably, a reinforcement learning framework is used for GPU resource scheduling decisions. This framework comprises four core elements: state space, action space, reward function, and policy network. The state space defines the environmental information that the scheduling system is aware of; the action space defines all possible decisions that the scheduling system can execute; the reward function defines the evaluation criteria for the decisions; and the policy network maps states to actions to achieve the optimal decision.

[0123] Specifically, step S3 includes:

[0124] The average resource utilization of the current GPU cluster and the combined priority of the tasks to be scheduled are used to construct a state vector. The state vector is input into the trained reinforcement learning model, which outputs the Q-values ​​for each action. The action with the largest Q-value is selected as the initial scheduling scheme. This represents the average real-time utilization of each GPU node. This represents the average memory usage of each GPU node. Indicates the priority of the task to be scheduled. This indicates the remaining time limit percentage; the formula for calculating the overall priority is:

[0125] ;

[0126] in, Indicates the overall priority of the task. This represents a priority component based on user level. This indicates a priority component based on the urgency of the deadline. This represents a priority component based on task type. , , The weighting coefficients are satisfied. The formula for calculating the priority of time constraints is as follows: Task type priority is set according to task characteristics and resource requirements;

[0127] The decision quality of the preliminary scheduling scheme is evaluated based on the Q value of each action and the similar historical states in the experience replay memory. The neighborhood search range is dynamically determined based on the decision quality evaluation results, and a set of candidate scheduling schemes is generated within the neighborhood search range.

[0128] For each candidate scheduling scheme in the candidate scheduling scheme set, multiple optimization objective values ​​are calculated. A multi-objective optimization algorithm is used to perform non-dominated sorting of the candidate scheduling scheme set, and the final scheduling scheme is determined based on the non-dominated sorting results.

[0129] In one embodiment of the present invention, The calculation method is the arithmetic mean of the utilization of all GPU nodes. The calculation method is the arithmetic mean of the memory usage of all GPU nodes. The calculation formula is ,in This indicates the deadline set by the user for the task. Indicates the current system time. This represents the task submission time. The action space is defined as a combination of GPU slice ratio and node allocation strategy, forming the action vector. ,in This indicates the GPU slice ratio selection, which includes several discrete levels. This indicates the node allocation strategy, including single-node deployment mode, dual-node collaborative mode, and multi-node collaborative mode.

[0130] In one embodiment of the present invention, a quality assessment of the preliminary scheduling scheme is performed, specifically including:

[0131] Extract the Q-values ​​of each action output by the trained reinforcement learning model, calculate the difference between the Q-value of the best action and the Q-value of the second-best action, and calculate the Q-value confidence based on the difference:

[0132] ；

[0133] ;

[0134] in, Indicates the confidence level of the Q-value. Indicates the difference in Q values. The Q value of the second-best action. This is a temperature coefficient used to map the Q-value difference to the 0-1 range, adjusting the sensitivity of the confidence level. This indicates that the online network performs the initial scheduling scheme actions in state S. Q-value prediction;

[0135] Retrieve historical states from the experience replay memory whose distance from the current state vector is less than a preset threshold, and calculate the historical experience support strength based on the number of historical states that meet the criteria and the average distance:

[0136] ;

[0137] in, Indicates the degree of support from historical experience. Represents the support set, The minimum support experience number represents the benchmark value for judging whether historical data is sufficient. To support the average distance of the concentrated experience;

[0138] The decision quality assessment value is obtained by weighting and summing the Q-value confidence level and the historical experience support level.

[0139] ；

[0140] in, Indicates the decision quality assessment value. This represents the confidence weighting coefficient for the Q-value. This represents the weighting coefficient for the support level of historical experience;

[0141] When the decision quality assessment value is higher than the first threshold, the narrowed neighborhood search range is determined; when the decision quality assessment value is lower than the second threshold, the expanded neighborhood search range is determined, wherein the first threshold is greater than the second threshold.

[0142] The first and second thresholds are set according to the actual usage scenario.

[0143] This invention comprehensively evaluates the reliability of initial decisions made by reinforcement learning by combining Q-value confidence and historical experience support. It dynamically adjusts the neighborhood search range of multi-objective optimization based on the decision quality, thereby improving decision accuracy compared to optimization methods with fixed neighborhoods.

[0144] In one embodiment of the present invention, regarding the GPU slice ratio and node allocation mode Define the neighborhood radius:

[0145] ；

[0146] ;

[0147] in, The neighborhood radius representing the GPU slice ratio. The neighborhood radius represents the node allocation pattern. Represents the radius of the basic neighborhood. Represents the neighborhood expansion coefficient. Indicates the high-quality threshold. This indicates a low-quality threshold. When... When the time complexity is high, multi-objective optimization algorithms can achieve fine-grained optimization within a small neighborhood; when... When the time is low, expand the search space to explore more possibilities. Generate a set of candidate scheduling schemes within the neighborhood search range.

[0148] Furthermore, multiple optimization objective values ​​are calculated for each candidate scheme in the candidate scheduling scheme set, specifically including:

[0149] Calculate the target resource utilization value based on the GPU resource slicing ratio of the candidate solutions and the average utilization of historical tasks under the corresponding resource configurations:

[0150] ;

[0151] in This represents the target value for resource utilization. This indicates the GPU resource slice ratio of the candidate scheme. This represents the average GPU utilization of historical tasks under the same slice ratio;

[0152] Based on the GPU resource slice ratio of the candidate solutions, the expected completion time of the candidate solutions is calculated using a segmented resource performance prediction model. The expected latency target value is then calculated based on the difference between the expected completion time of the candidate solutions and the task time limit.

[0153] ;

[0154] in This indicates the expected delay target value. This indicates the expected completion time based on the candidate solutions. Indicates the time limit set by the user;

[0155] Calculate the target energy consumption based on the number of GPUs allocated to the candidate solutions, the average power of a single GPU, and the expected completion time:

[0156] ;

[0157] in This indicates the expected energy consumption target value. Indicates the number of GPUs allocated. This represents the average power of a single GPU.

[0158] In one embodiment of the present invention, a multi-objective optimization algorithm is used to perform non-dominated sorting on the candidate scheduling scheme set to identify the Pareto front solution set. The multi-objective optimization algorithm employs the NSGA-II genetic algorithm or its improved version based on non-dominated sorting, and screens candidate schemes through non-dominated sorting and crowding calculation, specifically including:

[0159] If the current average GPU utilization is below a set threshold, increase the weight of the resource utilization target; if the proportion of urgent tasks in the scheduling queue is high, increase the weight of the task response speed target; if the current energy consumption accounts for a high proportion of the total energy budget, increase the weight of the energy consumption control target; otherwise, balance the weights of each target. A Pareto front solution set is identified through non-dominated sorting, and a weighted comprehensive score is calculated from this solution set (not from all candidate solutions) based on dynamically determined weight coefficients according to the current cluster state. ,in, The weighting coefficients representing the resource utilization rate target. The weighting coefficients represent the expected delayed targets. The weight coefficients represent the expected energy consumption targets; the scheme with the highest score is selected as the final scheduling scheme, and the final output scheduling scheme is the GPU slice ratio and node allocation decision after multi-objective optimization and correction. This scheme is better in terms of multi-objective balance than the initial decision of reinforcement learning.

[0160] This invention effectively alleviates the Q-value overestimation problem of traditional DQN by decoupling the online network's action selection and the target network's Q-value evaluation. Prioritizing experience replay allows the model to learn more from information-rich samples to accelerate convergence. The online learning mechanism enables the model to continuously learn from actual scheduling experience and adapt to dynamic changes in task patterns, demonstrating adaptive capabilities compared to static rule-based scheduling methods. The NSGA-III multi-objective optimization algorithm searches for Pareto optimal solutions in the neighborhood of the initial reinforcement learning decision and dynamically determines the weight coefficients of each optimization objective based on the current GPU cluster load and task queue status, allowing for flexible trade-offs between different objectives to adapt to the needs of various application scenarios.

[0161] S4. Based on the final scheduling scheme, configure the target GPU node in the GPU cluster to obtain GPU resource instances, bind the task execution container to the GPU resource instances, and start the scheduled task in the task execution container.

[0162] Specifically, step S4 includes:

[0163] Select the target GPU node from the GPU cluster according to the GPU node allocation mode in the final scheduling scheme;

[0164] Based on the GPU resource slicing ratio in the final scheduling scheme, the target GPU node is configured with resource slicing. For GPU nodes that support MIG multi-instance GPU technology, the GPU management interface is called to divide the physical GPU into GPU instances with independent hardware resources. For GPU nodes that do not support MIG multi-instance GPU technology, the time slice quota is calculated based on the GPU resource slicing ratio and the time slice quota is allocated to the task through the operating system resource management mechanism to obtain GPU resources available for binding.

[0165] Create a task execution container, associate the task execution container with available GPU resources, and start the scheduled task in the task execution container.

[0166] Understandably, time-slice quotas are implemented by setting GPU access priority and time quota parameters for task processes. The scheduling system calculates the percentage of computing power for each task based on the GPU resource slice ratio and sets the corresponding GPU time-slice allocation parameters through the resource limitation interface of the container runtime. When the task execution container starts, GPU resource allocation information, including GPU device ID, memory limits, and compute unit quotas, is injected through environment variables and configuration files. The task execution container is then associated with available GPU resources, and an access channel from the process within the container to the physical or virtual GPU instance is established through the device mapping mechanism of the container runtime, ensuring that the application within the container can correctly identify and use the allocated GPU resources. The task to be scheduled is started in the task execution container, the user-submitted design file or simulation configuration file is mounted to the working directory within the container, the corresponding CAD design software or CAE simulation software is launched, and the task execution begins.

[0167] like Figure 3 As shown, the GPU cluster consists of several GPU nodes, each employing different resource slicing methods based on its hardware characteristics to achieve fine-grained resource allocation. For nodes supporting MIG (Multi-Instance GPU) technology, the scheduling system uses the GPU management interface to divide a single physical GPU into multiple GPU instances with independent hardware isolation. Each GPU instance has independent memory space and computing units, and different instances are completely isolated at the hardware level, thus achieving true physical resource isolation. For nodes not supporting MIG technology, the scheduling system uses a time-slice quota mechanism to logically divide resources, calculating the computing power quota for each task based on the GPU resource slicing ratio. The operating system's resource management mechanism controls the access time ratio of different tasks to GPU resources, thus achieving software-level resource isolation. The scheduling system determines the resource allocation strategy for each task based on the final scheduling scheme generated in step S3, issuing specific resource allocation instructions to the corresponding GPU nodes for resource slicing configuration. After configuration, the GPU resource instances are bound to the task execution containers. The task execution containers obtain exclusive access to the allocated GPU resources through the container runtime device mapping mechanism. The application processes within the containers access the bound GPU resource instances through the standard GPU programming interface to complete the actual computation task execution. This layered architecture design creates a clear hierarchical relationship between the three stages of resource scheduling decision-making, resource slicing configuration, and task execution isolation. It ensures both the flexibility and efficiency of resource allocation, as well as resource isolation and execution stability between different tasks, providing a reliable resource guarantee mechanism for multi-tenant concurrent design simulation tasks on the cloud platform.

[0168] In one embodiment of the present invention, the invention can dynamically adjust GPU operating parameters to optimize energy consumption based on task load. It monitors the GPU's load status and power consumption data in real time through NVIDIA's NVML API interface, and dynamically adjusts the GPU's core frequency and voltage based on the monitoring results. For light-load tasks with low GPU utilization, the GPU core frequency can be reduced to a lower level to reduce power consumption; for heavy-load tasks with high GPU utilization, the GPU core frequency can be increased to a higher level to ensure computing performance. This reduces the overall energy consumption of the GPU cluster while ensuring task execution efficiency, effectively controlling the cloud platform's PUE (Power Usage Effectiveness) and achieving a green and energy-saving scheduling effect.

[0169] S5. Collect performance data of task execution, construct and store experience data based on the performance data, and update the reinforcement learning model parameters when the accumulated experience data reaches a preset threshold.

[0170] This invention uses a reinforcement learning model and a multi-objective optimization algorithm to intelligently determine the GPU resource slicing ratio and node allocation mode based on the GPU cluster status and task attributes. It continuously optimizes the scheduling strategy through an online learning mechanism to adapt to the dynamic changes in task modes. Compared with traditional static rule scheduling methods, it significantly improves resource utilization efficiency and task response speed, while realizing adaptive evolution of the scheduling strategy and multi-objective balanced optimization.

[0171] Specifically, step S5 includes:

[0172] During task execution, execution metrics data are collected, and anomaly detection is performed on the execution metrics data; wherein the execution metrics data includes execution time, GPU utilization, video memory usage, and GPU temperature data;

[0173] After the task is completed, the actual performance metrics of the task are calculated, and the actual performance metrics are substituted into the multi-objective reward function to calculate the actual reward value; wherein the actual performance metrics include actual completion time, average GPU utilization, and total energy consumption data.

[0174] Empirical data is constructed, including scheduling state vectors, executed actions, actual reward values, and task completion state vectors. Priorities based on temporal difference errors are assigned to the empirical data, and the data is stored in an experience replay memory and a historical task database. The formula for calculating the temporal difference error is as follows:

[0175] ;

[0176] in, As a discount factor, this error reflects the learning value of the sample. Indicates timing difference error. This represents the actual reward value. This represents the maximum Q-value of the network in state S′ after the task is completed. This indicates that the final scheduling scheme action is executed under state S during scheduling in the online network. The Q-value prediction of l, where S represents the scheduling state vector. This represents the state vector after the task is completed. This indicates the action corresponding to the final scheduling plan;

[0177] When the accumulated new experience data in the experience replay memory reaches a preset threshold, training samples are sampled from the experience replay memory according to priority, and the parameters of the online network and the target network are updated using incremental training. The priority is set to [priority value]. ,in To prevent zero priority for small constants, This is the priority index.

[0178] In one embodiment of the present invention, the anomaly detection includes: if the GPU utilization rate is lower than a set threshold for a certain period of time, it is determined to be an abnormal resource idleness; if the video memory occupancy rate exceeds the allocation limit and continues to increase, it is determined to be a video memory overflow risk; if the GPU temperature exceeds a safe threshold, it is determined to be an abnormal overheating.

[0179] In one embodiment of the invention, incremental training is used to update the parameters of the online network and the target network. Incremental training refers to fine-tuning the parameters in a small number of rounds using newly added experience data while retaining the original model parameters, avoiding the computational overhead and forgetting of historical knowledge caused by complete retraining. Specifically, this includes using a smaller learning rate and fewer training rounds to ensure that the model can learn new scheduling experience while retaining its memory of historical experience. A certain proportion of samples are randomly selected from the experience replay memory to form a validation sample set, which does not overlap with the training sample set, and is used to evaluate the performance of the updated reinforcement learning model. Performance evaluation metrics include the average Q-value prediction error on the validation set and the average reward value of simulated scheduling. When the performance improvement exceeds a preset threshold, the updated model parameters replace the currently running model parameters, completing the online evolution of the model; if the performance does not improve or even declines, the original model parameters are retained. This online learning mechanism enables the scheduling system to continuously learn from actual operating experience and adapt to dynamic changes in task modes and cluster states.

[0180] The present invention also provides a GPU resource intelligent scheduling system based on reinforcement learning, for implementing the GPU resource intelligent scheduling method based on reinforcement learning as described above.

[0181] In one embodiment of the present invention, the scheduling system builds a model training and deployment environment, constructs a task execution environment based on the Kubernetes container orchestration system, configures the YouBox modular application delivery system for containerized encapsulation of design simulation software, and configures the OpenHPC job scheduling module for task queue management. For required CAD design software such as SolidWorks and AutoCAD, and CAE simulation software such as ANSYS and ABAQUS, container images containing these software programs, necessary GPU drivers, and CUDA runtime libraries are pre-created and stored in an image repository. The GPU virtualization interface is configured. For NVIDIA A100 or H100 GPUs that support MIG multi-instance GPU technology, dynamic partitioning and management of GPU instances are achieved through nvidia-smi and NVML APIs. For Quadro RTX or Tesla V100 GPUs that do not support MIG, a time-slicing virtualization mechanism is configured to achieve logical partitioning of GPU resources.

[0182] In one embodiment of the present invention, the scheduling system can also be configured with a cross-process collaboration interface module to achieve seamless integration between CAD design tasks and CAE simulation tasks. This collaboration interface module adopts a RESTful API architecture and supports automatic conversion and transmission of mainstream model file formats such as STEP, IGES, and STL. After the design task is completed, the user can import the design model into the simulation module with a single click. The collaboration interface automatically identifies the model attributes and calls the intelligent scheduling method of the present invention to allocate appropriate GPU resources to the simulation task. This collaboration mechanism is compatible with mainstream CAD design software such as SolidWorks and AutoCAD, as well as CAE simulation software such as ANSYS and ABAQUS, forming a closed loop between the design and simulation processes, reducing manual intervention, shortening the development cycle, and automatically adjusting the GPU slicing ratio and node allocation scheme according to the complexity of the simulation task, ensuring precise matching between GPU resource configuration and task requirements.

[0183] In one embodiment of the present invention, the scheduling system can also be configured with a data security module to ensure data security during the design simulation process. This data security module uses the AES-256 encryption algorithm to encrypt and protect data during task submission, model file transmission, and result download. It implements hierarchical management of user permissions based on the RBAC permission model, including different levels such as administrator permissions, designer permissions, and visitor permissions, ensuring that different users can only access data and functions within their authorized scope. The scheduling system records the operation logs of all users and retains them for a set period of time for security auditing and problem tracing.

[0184] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A GPU resource intelligent scheduling method based on reinforcement learning, characterized in that, include: S1. Collect resource status data of the GPU cluster and attribute data of the tasks to be scheduled, and preprocess them to obtain standardized resource status data and task attribute data. S2. Construct a reinforcement learning model that includes an online network and a target network, and train the reinforcement learning model using historical scheduling records to obtain the trained reinforcement learning model. The online network and the target network determine the Q value of each action based on resource status data and task attribute data. The actions include the GPU resource slicing ratio and the GPU node allocation mode. S3. Input the current GPU cluster resource status data and the task attribute data of the task to be scheduled into the trained reinforcement learning model to obtain a preliminary scheduling scheme. Based on the decision quality evaluation results of the preliminary scheduling scheme, the neighborhood search range is determined, a set of candidate scheduling schemes is generated within the neighborhood, multiple optimization objective values ​​are calculated for each candidate scheme in the set of candidate scheduling schemes, and a multi-objective optimization algorithm is used to select the final scheduling scheme from the set of candidate scheduling schemes. Step S3 specifically includes: The average resource utilization of the current GPU cluster and the combined priority of the tasks to be scheduled are used to construct a state vector. The state vector is then input into the trained reinforcement learning model to output the Q value corresponding to each action. The action with the largest Q value is selected as the initial scheduling scheme. The decision quality of the preliminary scheduling scheme is evaluated based on the Q value of each action and the similar historical states in the experience replay memory. The neighborhood search range is dynamically determined based on the decision quality evaluation results, and a set of candidate scheduling schemes is generated within the neighborhood search range. For each candidate scheduling scheme in the candidate scheduling scheme set, multiple optimization objective values ​​are calculated. A multi-objective optimization algorithm is used to perform non-dominated sorting of the candidate scheduling scheme set. The final scheduling scheme is determined based on the non-dominated sorting result. The quality assessment of the preliminary scheduling plan specifically includes: Extract the Q-values ​​of each action output by the trained reinforcement learning model, calculate the difference between the Q-value of the best action and the Q-value of the second-best action, and calculate the Q-value confidence based on the difference; Retrieve historical states from the experience replay memory whose distance from the current state vector is less than a preset threshold, and calculate the historical experience support degree based on the number of historical states that meet the conditions and the average distance. The decision quality assessment value is obtained by weighting and summing the Q-value confidence level and the historical experience support level. When the decision quality assessment value is higher than the first threshold, the narrowed neighborhood search range is determined; when the decision quality assessment value is lower than the second threshold, the expanded neighborhood search range is determined, wherein the first threshold is greater than the second threshold. S4. Based on the final scheduling scheme, configure the target GPU node in the GPU cluster to obtain GPU resource instances by performing resource slicing, bind the task execution container to the GPU resource instance, and start the scheduled task in the task execution container. S5. Collect performance data of task execution, construct and store experience data based on the performance data, and update the reinforcement learning model parameters when the accumulated experience data reaches a preset threshold.

2. The GPU resource intelligent scheduling method based on reinforcement learning as described in claim 1, characterized in that: Step S1 specifically includes: The GPU utilization and video memory usage of each GPU node in the GPU cluster are collected to obtain the task type, task priority and task time limit requirements of the task to be scheduled, and to obtain the original resource status data and original task attribute data. The task type includes CAD design tasks and CAE simulation tasks, and the task attribute parameters include the model file size and number of parts for CAD tasks and the number of simulation meshes for CAE tasks. The original resource status data and original task attribute data are filtered and normalized to obtain standardized resource status data and standardized task attribute data. The static complexity of the task is calculated based on the task attribute parameters. A set of historical tasks similar to the task to be scheduled is retrieved from the historical task database. A resource performance prediction model is established based on the correlation between resource configuration and execution time of the historical task set. The dynamic complexity of the task is calculated based on the resource performance prediction model and the static complexity. The static complexity and dynamic complexity are merged to obtain the final complexity of the task as a component of the task attribute data.

3. The GPU resource intelligent scheduling method based on reinforcement learning as described in claim 2, characterized in that: The dynamic complexity of the computational task specifically includes: Calculate the feature vector distance between the task to be scheduled and each historical task in the historical task database, calculate the similarity based on the feature vector distance, and select a preset number of historical tasks with the highest similarity to form a similar task set. A segmented resource performance prediction model is established by fitting the resource allocation and execution time data of historical tasks in a set of similar tasks. The dynamic complexity is calculated based on the predicted execution time of the segmented resource performance prediction model and the set of similar tasks. The final complexity is obtained by fusing the static and dynamic complexity using dynamic weighting coefficients. ； in, Indicate the final complexity, Represents static complexity. Indicates dynamic complexity. This represents the dynamic weighting coefficient, which is determined based on the number of similar task sets and the average similarity.

4. The GPU resource intelligent scheduling method based on reinforcement learning as described in claim 1, characterized in that: Step S2 specifically includes: Training samples are extracted from historical scheduling records. The training samples include the state at the time of scheduling, the action performed, the reward obtained, and the state after the task is completed. Each training sample is assigned a priority based on the time difference error and stored in the experience replay memory. Set up a multi-objective reward function that includes resource utilization, task response speed, and energy consumption control; Training samples are sampled from the experience replay memory according to priority. The optimal action for the next state is selected using the online network, and the Q value of the optimal action is evaluated using the target network to calculate the target Q value. The parameters of the online network are updated by minimizing the mean square error between the current Q value and the target Q value, and the parameters of the target network are made to track the parameters of the online network, thus obtaining the trained reinforcement learning model.

5. The GPU resource intelligent scheduling method based on reinforcement learning as described in claim 4, characterized in that: The calculation of multiple optimization objective values ​​for each candidate scheme in the candidate scheduling scheme set specifically includes: Calculate the target resource utilization value based on the GPU resource slicing ratio of the candidate solutions and the average utilization of historical tasks under the corresponding resource configurations: ； in This represents the target value for resource utilization. Indicates the GPU resource slice ratio. This represents the average GPU utilization of historical tasks under the same slice ratio; Based on the GPU resource slice ratio of the candidate solutions, the expected completion time of the candidate solutions is calculated using a segmented resource performance prediction model. The expected latency target value is then calculated based on the difference between the expected completion time of the candidate solutions and the task time limit. ； in This indicates the expected delay target value. This indicates the expected completion time based on the candidate solutions. Indicates the time limit set by the user; Calculate the expected energy consumption target value based on the number of GPUs allocated to the candidate solutions, the average power of a single GPU, and the expected completion time: ； in This indicates the expected energy consumption target value. Indicates the number of GPUs allocated. This represents the average power of a single GPU.

6. The GPU resource intelligent scheduling method based on reinforcement learning as described in claim 5, characterized in that: Step S4 specifically includes: Select the target GPU node from the GPU cluster according to the GPU node allocation mode in the final scheduling scheme; Based on the GPU resource slicing ratio in the final scheduling scheme, the target GPU node is configured with resource slicing to obtain the GPU resources available for binding; Create a task execution container, associate the task execution container with available GPU resources, and start the scheduled task in the task execution container.

7. The GPU resource intelligent scheduling method based on reinforcement learning as described in claim 6, characterized in that: Step S5 specifically includes: During task execution, execution metrics data are collected, and anomaly detection is performed on the execution metrics data; wherein the execution metrics data includes execution time, GPU utilization, video memory usage, and GPU temperature data; After the task is completed, the actual performance metrics of the task are calculated, and the actual performance metrics are substituted into the multi-objective reward function to calculate the actual reward value; wherein the actual performance metrics include actual completion time, average GPU utilization, and total energy consumption data. Construct empirical data containing scheduling state vectors, executed actions, actual reward values, and task completion state vectors; assign priorities to the empirical data based on temporal differential errors and store them in an empirical replay memory and a historical task database. When the accumulated new experience data in the experience replay memory reaches a preset threshold, training samples are sampled from the experience replay memory according to priority, and the parameters of the online network and the target network are updated using incremental training.

8. A GPU resource intelligent scheduling system based on reinforcement learning, characterized in that, The system is used to implement the GPU resource intelligent scheduling method based on reinforcement learning as described in any one of claims 1-7.

Images