A server GPU computing power allocation method and system, and a server

Abstract

This application provides a server GPU computing power allocation method, system, and server, belonging to the field of server resource management, and solves the problems of low resource utilization and inflexible decision-making in traditional allocation methods. The method includes: a central agent constructing a directed weighted game graph of tasks and GPU clusters, and solving a global allocation strategy based on Nash equilibrium; a local agent allocating tasks to specific GPUs through reinforcement learning, achieving intelligent migration by combining dynamic thresholds and anomaly detection, and iteratively optimizing the strategy through federated learning. The system includes a central agent, local agents, GPU clusters, and an experience pool module, and the server runs the system to execute the method. The solution, through two-layer agent collaboration and multi-algorithm fusion, quantifies the task cluster adaptation relationship, dynamically adjusts the allocation strategy, significantly improves resource utilization and task processing efficiency, and enhances adaptability and reliability in complex scenarios.

Description

Technical Field

[0001] This application relates to the field of server resource management, and more particularly to a server GPU computing power allocation method, system, and server. Background Technology

[0002] In scenarios such as artificial intelligence and big data processing, efficient allocation of server GPU computing power is crucial for task execution efficiency and resource utilization. Traditional allocation methods often employ static weight allocation or fixed threshold monitoring, which cannot dynamically adapt to changes in task load and resource heterogeneity, resulting in low resource utilization and inefficient task migration. For example, static allocation ignores real-time task requirements and cluster status, fixed threshold monitoring lacks awareness of task priority and system fluctuations, and a single migration algorithm struggles to balance migration costs and execution efficiency globally. These issues make servers prone to load imbalance, resource waste, or task timeouts when facing dynamic task flows, necessitating a GPU computing power allocation solution capable of real-time awareness and intelligent decision-making. Summary of the Invention

[0003] This application provides a server GPU computing power allocation method, system, and server, which can achieve globally optimal allocation of GPU computing power in a dynamic and intelligent manner, solving the problems of low resource utilization, inflexible decision-making, and low migration efficiency of traditional methods.

[0004] Firstly, this application provides a method for allocating server GPU computing power. The method includes: a central intelligent agent module constructing a directed weighted game graph with tasks and GPU clusters as nodes; calculating edge weights based on the resource matching degree between tasks and clusters, communication overhead, and data locality coefficients; and solving a global allocation strategy based on Nash equilibrium; each local intelligent agent module receiving allocation instructions and allocating tasks to specific GPUs based on a reinforcement learning algorithm; and real-time monitoring of task execution status and GPU status within the cluster, wherein the task execution status includes task completion efficiency and task queue length, and the GPU status includes GPU utilization, memory occupancy rate, and memory fragmentation rate.

[0005] By adopting the above technical solutions, a two-layer intelligent agent collaborative architecture is constructed. The upper layer achieves global optimal allocation through game theory, while the lower layer achieves local fine-grained scheduling through reinforcement learning. Combined with real-time monitoring, a closed-loop optimization is formed to ensure that the allocation of computing power takes into account both global balance and local efficiency.

[0006] Furthermore, the edge weights are determined by the formula... Calculation, where This is a function for matching the resource availability between the task and the cluster. For communication overhead functions, The locality coefficient of the task's data. Preset weight parameters and satisfy .

[0007] By adopting the above technical solutions, the compatibility between tasks and clusters is quantified, and resource matching, communication costs, and data localization are comprehensively considered to provide a scientific basis for global allocation and improve the rationality of allocation strategies.

[0008] Furthermore, the resource matching degree function ,in For the task For the first The demand for similar resources For GPU clusters The Middle The availability of class resources; the communication overhead function The inter-node delay matrix, For the task The amount of data.

[0009] By adopting the above technical solutions, the edge weight calculation process is refined, the quantification method of resource matching and communication overhead is clarified, the accuracy of game graph construction is ensured, and the reliability of the global allocation strategy is improved.

[0010] Furthermore, the state space of the reinforcement learning algorithm includes the source GPU state, the target GPU state, task features, and the network state, and the reward function is: Latency ,in This represents the change in average cluster utilization after migration. Latency is the amount of change in task latency. This represents the change in migration costs.

[0011] By adopting the above technical solutions, local intelligent agents can dynamically adjust allocation strategies based on real-time status, finding a balance between task execution efficiency, resource utilization, and migration costs, thereby improving the flexibility and efficiency of task scheduling within the cluster.

[0012] Furthermore, the real-time monitoring process utilizes the formula AnomalyScore. Calculate the anomaly detection index, among which For real-time indicators, For LSTM predictions, The weights are the indicator weights.

[0013] By adopting the above technical solutions, combined with time series prediction and weighted deviation calculation, we can achieve precise monitoring of task execution and resource status, promptly detect anomalies and trigger migration decisions, and improve the stability and fault tolerance of the system.

[0014] Furthermore, when the anomaly detection index exceeds the value specified by the formula... When the calculated dynamic threshold is reached, the task migration decision is initiated, where This is the historical average utilization rate. The standard deviation of historical utilization rate For dynamic coefficients, This determines the task priority.

[0015] By adopting the above technical solutions, dynamic thresholds, combined with historical data, task priorities, and system fluctuations, make migration decisions more adaptable, avoid the limitations of fixed thresholds, ensure that high-priority tasks are prioritized, and improve the overall reliability of task execution.

[0016] Furthermore, when the reinforcement learning algorithm assigns tasks to specific GPUs, a dual deep Q network (DDQN) is used for policy optimization, and network parameters are updated through an experience replay mechanism.

[0017] By adopting the above technical solutions, the system can achieve synergistic optimization of global strategies and local experience, enabling it to learn from historical data from multiple clusters and improve the generalization ability and long-term optimization effect of the strategies.

[0018] Secondly, this application provides a server GPU computing power allocation system. The system includes multiple GPU clusters, multiple local intelligent agent modules, and a central intelligent agent module;

[0019] The plurality of local agent modules, each for managing one of the GPU clusters, are configured to perform the steps of the local agent module in the server GPU computing power allocation method as described in any of the first aspects above.

[0020] The central agent module is used to manage all the local agent modules and is configured to perform the steps of the central agent module in the server GPU computing power allocation method as described in any of the first aspects above.

[0021] By adopting the above technical solutions, a hardware-based two-layer intelligent agent architecture is implemented, clarifying the division of labor and collaboration mechanisms among modules, providing system support for the practical implementation of the method, and ensuring efficient collaboration between global and local decisions.

[0022] Thirdly, this application provides a server. The server includes a memory and a processor, the memory storing a computer program executable on the processor, and the processor executing the computer program implementing the steps of the server GPU computing power allocation method described in any one of the first aspects above.

[0023] By adopting the above technical solution and integrating the method into the server hardware, it is equipped with dynamic computing power allocation capability, which is suitable for scenarios such as cloud computing and AI training that require efficient GPU resource management, thereby improving the overall performance of the server.

[0024] In summary, this application has at least the following beneficial effects:

[0025] 1. A dynamic and intelligent GPU computing power allocation scheme is provided, which significantly improves resource utilization and task processing efficiency through two-layer intelligent agent collaboration and multi-algorithm fusion;

[0026] 2. By quantizing edge weight calculation and using a dynamic threshold mechanism, precise adaptation of allocation strategies and intelligent response to abnormal scenarios are achieved;

[0027] 3. By utilizing reinforcement learning and federated learning, the system's self-optimization capability in complex environments is ensured, meeting the scheduling requirements of high load and dynamic task flows.

[0028] It should be understood that the description in the Summary Section is not intended to limit the key or essential features of the embodiments of this application, nor is it intended to restrict the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0029] The above and other features, advantages, and aspects of the embodiments of this application will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0030] Figure 1 A flowchart illustrating a server GPU computing power allocation method according to an embodiment of this application is shown.

[0031] Figure 2 An architecture diagram of a server GPU computing power allocation system according to an embodiment of this application is shown. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0033] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0034] This application provides a server GPU computing power allocation method, system, and server, which achieves dynamic intelligent allocation of GPU computing power through two-layer intelligent agent collaboration and multi-algorithm fusion, thereby improving resource utilization and task processing efficiency.

[0035] In a first aspect, embodiments of this application disclose a method for allocating server GPU computing power.

[0036] Figure 1 A flowchart illustrating a server GPU computing power allocation method according to an embodiment of this application is shown.

[0037] Reference Figure 1 The method specifically includes the following steps:

[0038] S1: The central intelligent agent module constructs the game graph and solves the global allocation strategy.

[0039] The central agent module (Agent-C), as the core of global resource scheduling, primarily abstracts tasks and GPU clusters into a directed weighted game graph, thereby achieving optimal task allocation to the cluster. When constructing the game graph, the central agent first needs to acquire multi-dimensional state data: for the set of task nodes... Each task eigenvectors middle, This represents the computational mode, such as using the LLVM compiler plugin to perform static analysis on the task code and extract the proportion of instructions such as matrix operations and loop operations. The memory access mode is determined by dynamically sampling the address distribution trajectory of memory accesses during task runtime using the NVIDIA Nsight Systems tool; Reflecting the type of resource demand, the normalized ratio of computing power and video memory usage is calculated through time series analysis of historical task execution data. Priority is determined by the business configuration file when the task is submitted, and its value ranges from 1 to [value missing]. It is the data locality coefficient, which is calculated by querying the block location mapping table of the HDFS file system to determine the proportion of task input data stored locally on the target cluster.

[0040] For GPU cluster node set Each cluster state matrix middle, This is a computing power utilization vector, which collects the SM unit utilization of each GPU in real time through the NVIDIA DCGM (DataCenterGPUManager) API; It is a memory usage vector, obtained from the memory allocation data returned by the NVIDIA-SMI interface of the GPU driver; To determine the video memory fragmentation rate, the First-Fit algorithm is used to analyze the distribution of video memory blocks and calculate the proportion of fragments smaller than 1MB in the available space. The power consumption status is reported to the system management controller (BMC) via the I2C bus by the temperature and power sensors on the GPU board.

[0041] After obtaining the above data, the central agent uses the formula... Compute task nodes to cluster node The edge weights. Wherein, the resource matching degree function... middle, Typically, 3 is chosen (computing power, video memory, bandwidth). For the task to the first Normalized requirements for class resources, such as computing power requirements, are obtained by the ratio of the FLOPS metric declared in the task to the total computing power of the system (the sum of the computing power of all GPUs). For the cluster Normalized availability of resources, such as the ratio of remaining computing power to total computing power. Communication overhead function. middle, It is obtained by periodically sending ICMPEcho requests and recording the RTT (Round Trip Time). Read the input file size field from the task metadata, in GB. Weight parameters. Default is 0.2, determined through offline training: using task execution data from the past week, with global resource utilization (objective function 1) and task completion time (objective function 2) as optimization objectives, a Bayesian optimization algorithm is employed. Search for the optimal combination under constraints.

[0042] After constructing the Boyi graph, the central agent uses a hypothetical Boyi algorithm to solve for the Nash equilibrium. Initially, it is assumed that each cluster has an equal probability of assigning tasks. In each iteration, the central agent first calculates the historical average policy of each cluster. Then based on the profit matrix Calculate the optimal response strategy for the current iteration The iteration terminates when the rate of change of the policy distribution between two consecutive iterations is less than 1%. The strategy obtained at this time That is, the task To cluster The allocation scheme. To handle large-scale task scenarios, the central intelligent agent, based on the Dask distributed framework, divides the reward matrix into... The submatrices are distributed to the 48-core CPUs of the management node for parallel computation. Intermediate results are transmitted between nodes via the RoCE protocol of the InfiniBand network, ensuring that the iterative convergence time does not exceed 500ms under a scale of 1000 tasks.

[0043] S2: The local agent module assigns tasks to specific GPUs based on reinforcement learning.

[0044] After receiving the allocation instructions from the central agent, the local agent (Agent-L) of the target cluster uses a Dual Deep Q Network (DDQN) to achieve fine-grained task allocation to specific GPUs. The construction of the state space S integrates multi-source data: the source GPU state (if migration scenarios exist) includes four metrics: utilization, memory usage, temperature, and power consumption, collected in real-time by the monitoring SDK provided by the GPU manufacturer; the target GPU state is obtained in the same way; and task characteristics... middle, and Sampling is performed by inserting probes into dynamic libraries (such as NVIDIA's nvtx library) loaded at task startup; network status is determined by the remaining bandwidth of the switch ports in the cluster where the target GPU resides, obtained by querying Cisco Nexus switch interface statistics via SNMP. All status characteristics are normalized using Z-Score. ,in and The mean and standard deviation of the historical data.

[0045] The DDQN network employs a three-layer fully connected structure: the input layer receives 13-dimensional features ( The network consists of a state (4 task features + 1 network state), two hidden layers of 128 neurons each (with ReLU activation function), and an output layer with the number of nodes equal to the number of GPUs in the cluster, corresponding to the Q-value of each GPU. The network is implemented using the PyTorch framework, with hidden layer weights initialized using the Xavier initialization method and a bias set to 0.1. The reward function... Latency middle, The difference in average cluster utilization before and after allocation is calculated using GPU utilization data collected by Prometheus. Latency is the difference between the actual execution time of a task and the estimated time. The estimated time is the median of the historical execution times of similar tasks (stored in the Redis cache). Cost includes data transmission time. Cost of competing for resources The link bandwidth between the source and target GPUs is obtained by querying the network interface card counter using the `ethtool -S` command. The resource preemption cost is based on the priority coefficient of the preempted task (preemption cost for high-priority tasks). (Low priority) calculation.

[0046] During training, the experience pool stores... A priority experience replay mechanism is adopted, and the sample weights and TD error are compared. It is directly proportional to the absolute value, and the specific calculation formula is: ,in Each time, 32 experiences are sampled from the experience pool to form a mini-batch. The optimizer uses Adam (learning rate). The loss function is the mean squared error. The target network replicates parameters from the evaluation network every 100 steps, with a discount factor... For A100 GPUs that support MIG technology, the local agent also needs to adjust the GPU memory requirements based on the task. Choose the appropriate MIG instance type: If Then select a 1g.5gb instance. Choose a 2GB or 10GB instance, and so on, to maximize the utilization of video memory.

[0047] S3: The local agent module monitors the status in real time and triggers migration decisions.

[0048] The local agent achieves real-time monitoring of the cluster status through a collaborative mechanism of dynamic thresholds and anomaly detection metrics. The anomaly detection metric is AnomalyScore. In the middle, real-time indicators Data including GPU utilization, memory usage, temperature, and task queue length are collected by Prometheus at 10-second intervals and stored in InfluxDB; predicted values... Generated by an LSTM model, which takes data from the past 10 time steps (100 seconds) as input, has 2 hidden layers with 128 units each, and outputs a predicted value for the next time step. During training, it uses the MSE loss function and the RMSprop optimizer (learning rate...) ); Indicator weights The default values ​​are 0.4 for utilization, 0.3 for video memory, and 0.3 for temperature. These values ​​can be dynamically adjusted through the configuration file of the local agent.

[0049] Dynamic threshold The calculations are based on historical statistics and task characteristics: This represents the average utilization rate over the past 30 minutes. To correspond to the standard deviation, it is updated in real time using a sliding window (window size 180 time steps); dynamic coefficients Related to task priority, when Time to take Pick Pick The priority value set when the task is submitted.

[0050] AnomalyScore At that time, the local agent initiates a hybrid intelligent transfer decision, which combines reinforcement learning and genetic algorithms to determine the transfer target.

[0051] The reinforcement learning module expands the state space of the original DDQN to include the global states of the source and target clusters (including the load and memory usage of all clusters), and the action space to include all available clusters. A new reward function is added to account for global load balancing. Balance, where Balance represents the change in the Jain index before and after the migration. The Jain index is calculated using the formula: Jain... For cluster utilization. In the genetic algorithm module, chromosomes are encoded as triplets [dst_cluster, data_path, resource_slice], where the target cluster number is 0 to... The integer represents the shortest path in the network topology graph calculated using Dijkstra's algorithm (the network topology is stored in the Neo4j graph database). Resource slices are floating-point numbers between 0 and 1, representing the proportion of computing power allocated to the task from the target cluster. The fitness function is... In LoadBalanceFactor, PerformanceGain represents the percentage improvement in task execution efficiency after migration, obtained by comparing the execution time of similar tasks on different clusters in historical data; MigrationCost is the migration time (in seconds), including data transfer time and GPU context switching time; LoadBalanceFactor is the Jain index after migration to the target cluster. The genetic algorithm employs roulette wheel selection (fitness ratio selection), single-point crossover (probability 0.8), and uniform mutation (probability 0.1), iterating for 100 generations to select the chromosome with the highest fitness. The final migration decision is determined by weighted voting, with the reinforcement learning scheme having a weight of 0.6 and the genetic algorithm scheme having a weight of 0.4.

[0052] During migration, the local agent utilizes RDMA technology to transfer task data over the InfiniBand network. For GPUs supporting GPUDirectP2P, NVIDIA's CUDA API is used to implement direct copying of video memory, avoiding the additional overhead of data passing through host memory. During migration, the local agent dynamically adjusts the data block size (default 128MB) by monitoring network throughput and GPU context switching latency to ensure that migration bandwidth utilization exceeds [a certain threshold]. .

[0053] S4: Experience feedback and strategy iteration.

[0054] The central agent and local agents share experiences and optimize strategies through a federated learning mechanism, building a continuously evolving intelligent allocation system. The experience pool management adopts a hierarchical storage architecture: the local experience pool is deployed on the management node of each cluster, using a RocksDB database to store 100,000 experience records in a specific format. ,in The global policy serves as the central agent's strategy. The global experience pool, based on Ceph distributed storage, periodically (hourly) aggregates local experiences, removes duplicates using a Bloom filter, and stores them categorized by task type (training, inference, rendering, etc.). In the priority replay mechanism, sample weights consider not only TD error but also task priority, specifically... This enables the acquisition of experience from high-priority tasks. The weighting has been increased.

[0055] Federated learning parameter aggregation uses the FedAvg algorithm. The specific process is as follows: each local agent performs 50 rounds of training based on its local experience pool (batch size 32) to generate parameters. Parameters are uploaded to the central agent via the gRPC protocol, with gzip compression and TLS encryption used during transmission; the central agent adjusts the processing volume according to the task load of each local agent. Calculate the weighted average and update the global parameters. ,in For the number of clusters, The central agent distributes the updated parameters to each local agent, triggering fine-tuning of the local model (training 10 epochs). To reduce communication overhead, gradient compression is used for parameter transmission, sending only parameters with changes exceeding a certain threshold. With these parameters, the average compression ratio can reach 10:1.

[0056] The central agent also iteratively optimizes the Boyi graph edge weights using gradient descent based on historical allocation records in the global experience pool. Specifically, for each edge weight... Its update formula is The learning rate The gradient is the average value of global resource utilization. Calculations are performed using TensorFlow's automatic differentiation mechanism. This is especially important during peak task periods (e.g., 9:00 AM to 6:00 PM daily).

[0057] The system will automatically increase the load balancing weight. Up to 0.7, increase the resource matching weight during periods of resource scarcity (such as nighttime). Up to 0.3, enabling dynamic adaptation of strategies.

[0058] In a practical application at a cloud computing center, this method improved GPU resource utilization from traditional solutions. Upgraded to The average completion time for high-priority tasks has been shortened. The misjudgment rate of migration decisions has increased from Down to Through continuous feedback, after 30 days of operation, the average time for task migration decreased from 120ms to 75ms, demonstrating the self-optimization capability of the method.

[0059] In summary, this method achieves optimal global resource allocation through game theory modeling by a central intelligent agent, utilizes reinforcement learning by local intelligent agents for fine-grained GPU scheduling, combines dynamic thresholds and anomaly detection for intelligent migration, and continuously optimizes the strategy through a federated learning mechanism, forming a complete closed-loop intelligent allocation system. This system maximizes cluster resource utilization while ensuring QoS for high-priority tasks, making it suitable for scenarios with stringent requirements for GPU resource efficiency, such as AI training and cloud computing.

[0060] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0061] Secondly, embodiments of this application disclose a server GPU computing power allocation system.

[0062] Figure 2 An architecture diagram of a server GPU computing power allocation system according to an embodiment of this application is shown.

[0063] The system includes multiple GPU clusters, multiple local intelligent agent modules, and a central intelligent agent module;

[0064] The plurality of local agent modules, each for managing one of the GPU clusters, are configured to perform the steps of the local agent modules as disclosed in the server GPU computing power allocation method of the first aspect above.

[0065] The central agent module, which manages all the local agent modules, is configured to execute the steps of the central agent module in the server GPU computing power allocation method disclosed in the first aspect above.

[0066] Secondly, embodiments of this application disclose a server. The server includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of the server GPU computing power allocation method described in any one of the first aspects above.

[0067] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system and server described herein can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0068] In summary, this application has at least the following beneficial effects:

[0069] 1. By deeply integrating a two-layer intelligent agent architecture with game theory and reinforcement learning, global optimization and local fine-grained scheduling of GPU computing power allocation are achieved, which improves resource utilization by 30%-40% and reduces the average task completion time by 25%-35% in dynamic task flow, which is significantly better than the traditional static allocation scheme.

[0070] 2. Based on the dynamic threshold mechanism of LSTM prediction and fuzzy logic, combined with the hybrid transfer decision of reinforcement learning-genetic algorithm, the false positive rate of anomaly detection is reduced by 40%, and the transfer decision response time is controlled within 50ms, effectively coping with the load change and fault tolerance requirements in heterogeneous hardware environment.

[0071] 3. By leveraging federated learning and the dynamic optimization strategy of prioritizing experience replay, the system can automatically aggregate historical experience from multiple clusters to achieve continuous evolution of allocation strategies. In long-term operation, this reduces migration overhead by 50%, improves resource matching accuracy by 60%, and significantly enhances adaptability and reliability in complex scenarios.

[0072] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the foregoing disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for allocating server GPU computing power, characterized in that, Includes the following steps: The central intelligent agent module constructs a directed weighted game graph with tasks and GPU clusters as nodes. Based on the resource matching degree between tasks and GPU clusters, communication overhead, and the data locality coefficient of the tasks, it calculates the edge weights from task nodes to GPU cluster nodes in the game graph. Then, it solves for the global allocation strategy from tasks to GPU clusters based on Nash equilibrium. The edge weights are expressed by the formula... Calculation, where This is a function for matching the resource availability between the task and the cluster. For communication overhead functions, The locality coefficient for the task data. , , Preset weight parameters and satisfy The resource matching degree function ,in, For the i-th task node, For the task For the first The demand for similar resources For the j-th GPU cluster, For GPU clusters The Middle The availability of class resources; the communication overhead function The inter-node delay matrix, For the task The amount of data; Each local agent module receives the allocation instructions from the central agent module and, based on a reinforcement learning algorithm, allocates the tasks assigned to its cluster to specific GPUs. The state space of the reinforcement learning algorithm includes the source GPU state, the target GPU state, task features, and the network state, and the reward function is... ,in This represents the change in average cluster utilization after migration. This represents the change in task delay. This represents the change in migration costs. Each local agent module monitors the task execution status and GPU status within its cluster in real time. The task execution status includes task completion efficiency and task queue length, while the GPU status includes GPU utilization, memory usage, and memory fragmentation rate. This real-time monitoring is achieved using formulas... Calculate the anomaly detection index, among which For real-time indicators, These are the predicted values ​​generated by the LSTM model based on monitoring indicator data over a past period. The weight of the indicator; when the anomaly detection indicator exceeds the weight of the indicator by the formula... When the calculated dynamic threshold is reached, the task migration decision is initiated; whereby This is the historical average utilization rate. The standard deviation of historical utilization rate For dynamic coefficients, the For the task eigenvectors In the priority component, where, Represents the computing mode. For video memory access mode, Reflects the type of resource demand. It is the locality coefficient of the data; It also includes an experience feedback step: the central agent module and the local agent module aggregate experience through federated learning to update the global policy parameters, and the update formula for the global policy parameters is as follows: ,in For cluster The workload, The total number of clusters, , These are the parameters of the pre-trained local agent.

2. The server GPU computing power allocation method according to claim 1, characterized in that, When assigning tasks to specific GPUs based on reinforcement learning algorithms, a dual deep Q network (DDQN) is used for policy optimization, and network parameters are updated through an experience replay mechanism.

3. A server GPU computing power allocation system, characterized in that, It includes multiple GPU clusters, multiple local intelligent agent modules, and a central intelligent agent module; The plurality of local agent modules, each for managing one of the GPU clusters, are configured to perform the steps of the local agent module in the server GPU computing power allocation method of claim 1 or 2. The central agent module is used to manage all the local agent modules and is configured to execute the steps of the central agent module in the server GPU computing power allocation method of claim 1 or 2.

4. A server, characterized in that, It includes a memory and a processor, the memory storing a computer program that can run on the processor, and the processor executing the computer program to implement the steps of the server GPU computing power allocation method of claim 1 or 2.

Images