A GPU virtualization resource scheduling method based on multi-agent reinforcement learning

Abstract

The application discloses a GPU virtualization resource scheduling method based on multi-agent reinforcement learning, relates to the technical field of computers, and comprises the following steps: S1, memory usage data, temperature sensor data and task exception rate records in a GPU cluster are collected in real time, a multidimensional time series dataset is constructed, and each dimension of data is subjected to smoothing processing to filter noise, so that a preliminary signal change characteristic set is obtained; and S2, according to the signal change characteristic set, the correlation between memory usage trends, temperature change rates and abnormal rate fluctuation values is analyzed by using a preset sliding window method, and the dynamic linkage mode between signals is determined; the GPU virtualization resource scheduling method based on multi-agent reinforcement learning can avoid task interruption and system performance decline caused by scheduling lag, and has good practical value and engineering application effect.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to a GPU virtualization resource scheduling method based on multi-agent reinforcement learning. Background Technology

[0002] With the rapid development of cloud computing platforms and large-scale AI model training tasks, GPUs, as core computing resources, are typically deployed in cluster environments via virtualization to support concurrent operation by multiple users and tasks. Within GPU virtualization clusters, different tasks have significantly different requirements for memory capacity, computing performance, and operational stability. The rationality of resource scheduling strategies directly affects task completion efficiency, system throughput, and hardware operational security. Therefore, how to achieve efficient and stable GPU virtualization resource scheduling under conditions of multi-tasking parallelism and resource constraints has become a key issue of concern in related technical fields.

[0003] In existing technologies, GPU resource scheduling typically relies on real-time monitoring of operational metrics such as memory usage, temperature, or task load, and makes scheduling decisions based on fixed thresholds or preset rules. For example, when memory usage or temperature exceeds a set threshold, intervention measures such as task migration, rate limiting, or pause are triggered. These methods are simple to implement, but their scheduling decisions are mostly based on single-moment or static threshold judgments, making it difficult to reflect the changing trends of the GPU's operating state over time. In real-world operating environments, GPU resource anomalies often exhibit gradual and interconnected characteristics. For example, memory usage may continuously increase when multiple tasks are running concurrently, the rate of temperature change may increase accordingly, and the task anomaly rate may gradually accumulate as the running time increases. These operational metrics usually have strong correlations and dynamic coupling relationships, but before triggering a clear threshold, existing scheduling methods struggle to identify in time that the system is gradually approaching an unstable state, thus missing the opportunity for early intervention. Furthermore, in GPU virtualization scenarios, the management overhead introduced by the virtualization layer fluctuates dynamically with changes in page replacement frequency and resource contention levels. When memory pressure and temperature load rise simultaneously, virtualization overhead can be further amplified, leading to a decrease in the actual usable computing performance of the GPU, creating a vicious cycle of performance degradation and resource strain. Traditional rule-based or single-point monitoring-based scheduling strategies can only be reactively addressed after problems have become apparent, making it difficult to make proactive judgments based on the overall operational situation. Furthermore, in cluster environments consisting of multiple GPU nodes or multiple virtual GPU instances, the operating status, resource availability, and task load distribution of different nodes vary significantly. Existing centralized or single-policy scheduling methods often lack support for parallel decision-making capabilities of multiple decision-makers, making it difficult to achieve fine-grained and adaptive scheduling of resources on each GPU node while ensuring overall stability. This makes scheduling strategies prone to response lag or local optima under complex load conditions. Summary of the Invention

[0004] The purpose of this invention is to provide a GPU virtualization resource scheduling method based on multi-agent reinforcement learning, thereby solving the problems existing in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a GPU virtualization resource scheduling method based on multi-agent reinforcement learning, comprising: S1. By collecting real-time data on GPU memory usage, temperature sensor data and task anomaly rate records in the GPU cluster, a multi-dimensional time series dataset is constructed. The data in each dimension is smoothed to filter noise, and a preliminary set of signal change features is obtained. S2. Based on the set of signal change characteristics, the correlation between memory usage trends, temperature change rate and anomaly rate fluctuation values ​​is analyzed using a preset sliding window method to determine the dynamic linkage mode between each signal. S3. If the dynamic linkage mode shows that the memory usage trend continues to rise and the temperature change rate exceeds the preset threshold, then the correlation analysis is used to determine whether the system is close to the critical state and obtain the potential risk assessment result. S4. Based on the potential risk assessment results, obtain the priority distribution and resource usage of the current tasks, and determine the list of tasks to be adjusted and the target GPU nodes by combining the task migration timing and resource throttling strategy. S5. Through the load balancing mechanism, priority tasks in the task list that need to be adjusted that are higher than the preset threshold are migrated to the target GPU node with available resources. At the same time, rate limiting is performed on priority tasks that are lower than the preset threshold to obtain the updated resource allocation scheme.

[0006] As can be seen from the above technical solution, the present invention has the following beneficial effects: This GPU virtualization resource scheduling method based on multi-agent reinforcement learning continuously perceives and correlates multi-dimensional operational data such as memory usage, temperature changes, and task anomalies in the GPU cluster. It can identify potential risk trends in the system from the perspective of temporal evolution and signal linkage, rather than passively responding solely to static thresholds. On this basis, it dynamically schedules task migration and resource throttling by combining task priority and resource occupancy, and continuously corrects the scheduling results through load balancing among multiple GPU nodes. This enables the resource allocation process to have continuous feedback and adaptive optimization capabilities, thereby effectively reducing the probability of anomalies such as memory overflow and temperature overload, suppressing the vicious cycle of performance caused by virtualization overhead amplification, improving the operational stability and resource utilization efficiency of the GPU virtualization cluster under complex load conditions, and avoiding task interruptions and system performance degradation caused by scheduling lag. It has good practical value and engineering application effects. Attached Figure Description

[0007] Figure 1 This is a flowchart of the GPU virtualization resource scheduling method of the present invention; Figure 2 This is a schematic diagram of the dynamic linkage mode and critical state evaluation of the present invention. Detailed Implementation

[0008] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0009] like Figure 1 and Figure 2 As shown, this invention provides a technical solution: a GPU virtualization resource scheduling method based on multi-agent reinforcement learning, comprising: S1. By collecting real-time data on GPU memory usage, temperature sensor data and task anomaly rate records in the GPU cluster, a multi-dimensional time series dataset is constructed. The data in each dimension is smoothed to filter noise, and a preliminary set of signal change features is obtained. S2. Based on the set of signal change characteristics, the correlation between memory usage trends, temperature change rate and anomaly rate fluctuation values ​​is analyzed using a preset sliding window method to determine the dynamic linkage mode between each signal. S3. If the dynamic linkage mode shows that the memory usage trend continues to rise and the temperature change rate exceeds the preset threshold, then the correlation analysis is used to determine whether the system is close to the critical state and obtain the potential risk assessment result. S4. Based on the potential risk assessment results, obtain the priority distribution and resource usage of the current tasks, and determine the list of tasks to be adjusted and the target GPU nodes by combining the task migration timing and resource throttling strategy. S5. Through the load balancing mechanism, the priority tasks in the task list that need to be adjusted that are higher than the preset threshold are migrated to the target GPU node with the available resources. At the same time, the rate limiting operation is performed on the priority tasks that are lower than the preset threshold to obtain the updated resource allocation scheme. S6. Based on the updated resource allocation scheme, monitor the feedback data on the trend of video memory usage and the rate of temperature change after migration, analyze the changes in page replacement frequency and virtualization output, and determine whether the intervention measures can alleviate the vicious cycle in signal linkage. S7. If the feedback data indicates that the vicious cycle has not been alleviated, the task will be redistributed to a better GPU node by adjusting the load balancing mechanism and resource throttling strategy to obtain the final resource scheduling configuration.

[0010] In this embodiment, the GPU virtualization resource scheduling method based on multi-agent reinforcement learning is characterized by the unified modeling of multi-source heterogeneous state information generated during the operation of the GPU cluster, and the characterization of the inherent coupling mechanism between GPU resource pressure, thermal state and task stability through time series analysis and linkage identification, thereby providing a reliable basis for subsequent adaptive scheduling decisions.

[0011] During system operation, GPU memory usage data reflects the continuous occupancy of storage resources by computing tasks, temperature sensor data characterizes the thermal load changes of GPU nodes, and task anomaly rate records reflect the changing trends in task execution stability in the virtualization environment. Since all of these data exhibit high-frequency fluctuations and transient noise, smoothing processes are first used to reduce occasional interference, allowing signal changes to more accurately reflect the long-term trends and periodic changes in the GPU's operating status.

[0012] Building upon this foundation, a sliding window analysis mechanism is introduced to perform correlation analysis on memory usage trends, temperature change rates, and anomaly rate fluctuations over continuous time periods within a unified time scale. By jointly modeling the direction, magnitude, and rate of change of multidimensional signals within the same window, it is possible to identify whether an increase in memory pressure is accompanied by a rapid rise in temperature and whether the anomaly rate amplifies synchronously, thereby revealing the dynamic linkage patterns that may exist in GPU nodes under virtualization operation.

[0013] When the aforementioned dynamic linkage mode exhibits characteristics such as continuously increasing memory usage and exceeding temperature change limits, it indicates that the GPU node may be gradually approaching a critical state of resource or thermal stability. At this point, by judging the correlation results, a potential risk assessment result is formed to quantify the risk level of GPU node performance degradation, frequent anomalies, or amplified virtualization overhead under the current scheduling state.

[0014] In this embodiment, based on the potential risk assessment results, the system further incorporates task priority distribution and resource usage information, matching and analyzing tasks of different business importance with GPU resource status. During this process, considering the impact of task migration timing on system performance and the constraints of resource throttling strategies, tasks with a significant impact on the current risk status are identified, and reasonable migration or throttling adjustment paths are planned for them.

[0015] During the scheduling and execution phase, a load balancing mechanism is used to migrate high-priority and performance-sensitive tasks to GPU nodes where memory and temperature are within safe margins, thereby quickly alleviating the resource and thermal pressure on the original nodes. At the same time, resource throttling is implemented for low-priority tasks to reduce their competition for memory bandwidth and computing resources, so as to avoid further aggravating the negative feedback loop in signal linkage.

[0016] After the scheduling policy takes effect, the actual impact of the scheduling intervention on the system state is evaluated by continuously monitoring metrics such as memory usage trends, temperature change rates, page replacement frequency, and virtualization overhead. If the feedback data shows that the strengthening relationship between memory pressure and temperature increase has been weakened, and virtualization overhead has stabilized, the current scheduling policy is deemed effective; otherwise, the task allocation is further optimized by adjusting load balancing parameters and resource throttling strength, so that the GPU cluster gradually converges to a balance between resource utilization, thermal status, and task stability.

[0017] Specifically, S1 includes real-time acquisition of GPU memory usage data, temperature sensor data, and task anomaly rate records from the GPU cluster to construct a multi-dimensional time series dataset; for each dimension of the multi-dimensional time series dataset, a moving average method is used for smoothing to obtain a set of smoothed sequences after filtering out noise; signal change features are extracted from the smoothed sequence set, the slope is obtained by calculating the difference between adjacent data points, and local maxima are detected as peaks to generate a preliminary set of signal change features; the set of signal change features is combined with preset historical cluster load data, and a resource pre-allocation prediction matrix is ​​determined by matrix multiplication; the state input of the multi-agent reinforcement learning environment is updated using the resource pre-allocation prediction matrix to obtain the preliminary set of signal change features as the input environment state.

[0018] In this embodiment, the process of this step starts with the collection of multi-source data on the running status of the GPU cluster. Through layer-by-layer processing and step-by-step calculation, the raw monitoring data is transformed into state inputs that can be directly used in a multi-agent reinforcement learning environment, thereby providing an accurate, stable and predictive state description for subsequent resource scheduling decisions.

[0019] In actual operation, the system first synchronously collects memory usage data, temperature sensor data, and task failure rate records from each GPU node in the GPU cluster at a preset fixed sampling period. Memory usage data specifically reflects the relationship between the allocated and available GPU memory capacity at each moment; temperature sensor data comes from the GPU's built-in or external temperature detection units and reflects the thermal state changes during chip operation; task failure rate records are obtained by counting the number of task execution failures, retries, or interruptions per unit time, and are used to characterize the stability of task operation in the virtualization environment. All of the above data are stored aligned according to a unified timestamp, thus constructing a multi-dimensional time-series dataset that is strictly synchronized in the time dimension.

[0020] After obtaining the multidimensional time series dataset, to avoid misjudgments in subsequent analysis caused by instantaneous load fluctuations, electromagnetic interference, or sampling jitter, the system performs smoothing processing on the data for each dimension. Specifically, for time series data of the same dimension, a continuous time window is selected. The length of this window is determined jointly by the GPU cluster's scheduling cycle and data sampling frequency, typically set to cover a time span that can cover multiple scheduling decision intervals without introducing significant latency. Within this window, the average of all sampling points is taken, and this average is used as the smoothed result at the center time point of the window. By continuously sliding the window along the time axis, the original data sequence is replaced with a smoothed sequence set with more continuous changes and more obvious trends. This window length parameter is usually determined by offline statistical analysis of historical data fluctuation amplitudes to ensure that the true trend of change is preserved while suppressing noise.

[0021] After obtaining the set of smoothed sequences, the system begins to extract signal change features. Specifically, the difference between adjacent time points in each smoothed sequence is calculated. By calculating the difference between the current and previous time points, the corresponding change amplitude is obtained, reflecting the direction and rate of change of the monitored indicator per unit time. This difference calculation process is performed point-by-point across the entire time series, forming a slope sequence reflecting the overall trend. Subsequently, within the same smoothed sequence, the numerical relationships between multiple adjacent time points are compared. When the data at a certain time point is simultaneously greater than the data at its immediate and subsequent time points, that time point is marked as a local maximum, representing the peak state of the monitored indicator within a short period. By summarizing the slope and peak information, a preliminary set of signal change features, including rate of change and sudden change characteristics, is formed.

[0022] After obtaining a preliminary set of signal change features, the system further incorporates pre-defined historical load data from the GPU cluster. This historical load data originates from scheduling logs recorded during long-term operation, including memory usage levels, temperature variations, and corresponding scheduling results for each GPU node under different load conditions. This historical load data is obtained offline during system deployment and organized according to a dimensional structure consistent with the current signal change features. Subsequently, the current set of signal change features and the historical load data are combined in matrix form. Specifically, based on the correspondence between the dimensions of each feature, a weighted superposition operation is performed on the current and historical features to generate a resource pre-allocation prediction matrix reflecting the possible evolution direction of the current state in the historical load space. The weight values ​​of each feature are determined through statistical analysis of historical scheduling effects, ensuring that features with a greater impact on system stability have a higher weight in the calculation.

[0023] After the resource pre-allocation prediction matrix is ​​generated, the system uses this matrix as the basis for state updates, refreshing the state input of the multi-agent reinforcement learning environment. Specifically, the values ​​in the prediction matrix are mapped to corresponding state variables in the reinforcement learning environment, enabling each agent to perceive the current resource change trends, potential load pressure, and possible risk evolution directions of the GPU cluster when making policy decisions. Through this state update process, the environment state, which originally only reflected real-time monitoring data, is expanded to include information states that simultaneously incorporate historical experience and trend predictions, thereby significantly improving the learning efficiency and decision stability of multi-agent reinforcement learning in resource scheduling scenarios.

[0024] Furthermore, S2 includes obtaining the memory usage trend and temperature change rate from the signal change feature set, and using a preset sliding window method to calculate the Pearson correlation coefficient between the abnormality rate fluctuation values ​​by taking the product of the covariance and the standard deviation, thus obtaining a preliminary set of correlation indicators. Based on the preliminary set of correlation indicators, and combined with the energy consumption fluctuation monitoring data obtained from the signal change feature set, the basic linkage parameters between signals are determined by fusing the temperature change rate and anomaly rate fluctuation values ​​through matrix operations. Based on the basic linkage parameters, peak offset features are extracted from the time series data derived from the signal change feature set to generate an extended dynamic linkage mode description. From the extended dynamic linkage mode description, the cluster historical load information extracted from the signal change feature set is integrated, and the state evolution feature components are updated by vector embedding. By constructing the reinforcement learning environment input through state evolution feature components, the dynamic linkage pattern between various signals is determined as part of the reinforcement learning state evolution features.

[0025] In this embodiment, the system first extracts memory usage trend and temperature change rate data from the signal change feature set. The memory usage trend is obtained by comparing a smoothed memory usage sequence over continuous time, reflecting the overall upward or downward direction of memory usage. The temperature change rate is calculated by measuring the change in temperature data at adjacent time points, characterizing the speed of changes in the thermal state of the GPU node. Simultaneously, the system acquires a task anomaly rate fluctuation value aligned with the above data time. This anomaly rate fluctuation value is obtained by statistically analyzing the changes in the frequency of anomalies per unit time. All three types of data use a unified timestamp rule during acquisition and processing to ensure time consistency in subsequent calculations.

[0026] After obtaining the above data, the system uses a pre-defined sliding window method to segment the time series. The length of the sliding window is determined based on a combination of GPU scheduling cycles and data sampling density, typically choosing a time span that covers multiple consecutive scheduling decision cycles to ensure the statistical significance of the correlation analysis. The sliding window moves along the time axis in fixed steps, and within each window, the changes in the rate of temperature change and the fluctuation of the anomaly rate are statistically analyzed. Subsequently, the correlation result, used to measure the strength of the association between the two types of data, is obtained by calculating the degree of synchronous change of the two types of data within the window. This correlation result is obtained by normalizing the overall consistency of the changes of the two types of data within the window, thus forming a preliminary set of correlation indicators. The value range of the correlation indicators is verified through historical data statistics to distinguish between weak, moderate, and strong correlation states.

[0027] After forming a preliminary set of relevant indicators, the system further incorporates energy consumption fluctuation monitoring data. This data originates from the power consumption statistics of GPU nodes during operation, reflecting the impact of load changes on energy consumption. The system arranges the temperature change rate, anomaly rate fluctuation value, and energy consumption fluctuation monitoring data according to a unified time window and performs combined calculations in matrix form. This combined calculation process essentially involves weighted fusion of the changing trends of multiple signals within the same time period, where the weights of each signal are determined by statistical results of the signal's impact on system stability in historical operating data. Through this fusion process, basic linkage parameters are generated to describe the mutual influence relationships of multiple signals. These parameters characterize the comprehensive coupling degree between resource pressure, thermal state, and abnormal behavior under the current operating state.

[0028] After obtaining the basic linkage parameters, the system further performs time-series analysis on the time series corresponding to the signal change feature set to extract peak offset features. Specifically, the system identifies the peak occurrence times in memory usage trends, temperature change rates, and anomaly rate fluctuations, and compares the temporal relationships between different signal peaks. When the peak of one signal consistently leads or lags behind another signal in time, the system records this time difference as a peak offset feature, reflecting the triggering sequence relationship between different signals. By integrating the peak offset feature with the basic linkage parameters, the system generates an extended dynamic linkage mode description, enabling the signal linkage relationship to simultaneously include intensity and temporal evolution information.

[0029] Based on this, the system extracts historical load information of the cluster from the signal change feature set. This historical load information includes the GPU cluster's memory usage distribution, temperature change characteristics, and anomaly occurrences at different load stages. This information is continuously accumulated and updated periodically during system operation. The system employs vector embedding to jointly encode the current dynamic linkage mode description with the historical load information, mapping features of different time scales and dimensions to a unified state representation space, thus forming state evolution feature components. The embedding dimension used in the vector embedding process is determined based on the number of GPU nodes and feature complexity to ensure improved compactness of the state representation without losing key information.

[0030] Finally, the system uses the state evolution feature components as input states for the reinforcement learning environment, enabling the multi-agent reinforcement learning model to simultaneously perceive the current GPU operating state, the dynamic interaction between signals, and the historical load evolution trend when making policy selections. Through the above state construction method, the dynamic interaction patterns between various signals are explicitly incorporated into the reinforcement learning state evolution features, thereby providing a reliable state foundation for the adaptive optimization of subsequent resource scheduling strategies.

[0031] Specifically, S3 includes obtaining the rising trend of video memory and accelerated temperature changes from the dynamic linkage mode, and combining system load monitoring with energy consumption fluctuations to obtain comprehensive monitoring indicators. For comprehensive monitoring indicators, threshold over-limit detection is used to compare with preset thresholds. If the temperature accelerates and changes beyond the preset threshold, an over-limit event flag is determined. Based on the over-limit event flag, the linkage mode analysis is performed by calculating the correlation coefficient, and the network latency index is integrated to determine the critical proximity and obtain the state proximity value. Risk assessment results are extracted from state proximity values ​​and fused with state evolution updates to generate a description of potential risk levels. Based on the description of potential risk levels, a feedback mechanism is constructed to generate reward signals, which serve as reward feedback signals for reinforcement learning.

[0032] In the specific implementation process, the system first extracts information on the upward trend of video memory usage and the accelerated change in temperature from the dynamic linkage mode. The upward trend of video memory usage is obtained by statistically analyzing the direction of change in video memory usage over multiple consecutive time windows. When video memory usage continuously increases over multiple adjacent time periods, the system determines that the video memory is in an upward trend state. This judgment is based on the statistical results of video memory changes within historical stable operating intervals and is used to distinguish between short-term fluctuations and continuous accumulated pressure. The accelerated change in temperature is obtained by comparing the increase or decrease in the rate of temperature change over adjacent time periods, based on the existing rate of temperature change. This is used to reflect whether the GPU node is experiencing a continuously accelerating rate of increase in thermal load, thereby identifying potential risks of thermal instability.

[0033] After extracting the above information, the system further incorporates system load monitoring data and integrates it with energy consumption fluctuation information. System load monitoring data includes GPU computing load, memory usage load, and task queuing status at the scheduling level. This data comprehensively reflects the current operating pressure level of GPU nodes. Energy consumption fluctuation information is obtained by statistically analyzing the change in GPU power consumption per unit time, reflecting the impact of load changes on hardware energy consumption. During the integration process, the system first normalizes various data types to ensure comparability across different dimensions. Then, it weights and amplifies the data according to pre-defined weights to form a comprehensive monitoring index. Each weight parameter is determined by analyzing the correlation between index changes and the probability of anomalies in historical operating data, ensuring that indicators with a greater impact on system stability have a higher weight in the comprehensive monitoring index.

[0034] After obtaining the comprehensive monitoring indicators, the system performs threshold over-limit detection on these indicators. The threshold is a preset threshold, determined based on GPU hardware specifications, thermal design capabilities, and historical long-term stable operating data. Specifically, during the deployment phase, the system statistically analyzes the GPU's temperature change rate, memory usage growth, and power consumption fluctuation range under normal operating conditions. It selects the maximum safe value under conditions without anomalies, performance degradation, or hardware protection triggering as a threshold reference, and sets a safety margin based on this, thus obtaining the final preset threshold. During real-time operation, the system compares the accelerated temperature change component in the comprehensive monitoring indicators with this preset threshold hourly. When the accelerated temperature change continuously exceeds the preset threshold, the system determines that the current state has exceeded the safe operating range and generates a corresponding over-limit event flag. This over-limit event flag clearly indicates that the GPU node has entered a high-risk state requiring close monitoring.

[0035] After generating an over-limit event flag, the system further performs linkage mode analysis based on this flag. Specifically, the system comprehensively evaluates the correlation results between various signals in the dynamic linkage mode to determine whether memory pressure, temperature changes, anomaly rate fluctuations, and energy consumption changes show a synchronous increasing trend. Simultaneously, the system introduces a network latency indicator, obtained by statistically analyzing the average latency and latency fluctuations during task communication, data access, or migration between GPU nodes, reflecting changes in system pressure at the communication and scheduling levels. The system performs a unified analysis of the network latency indicator and the aforementioned signal linkage results. When multiple indicators simultaneously exhibit characteristics of moving towards a critical direction, the system considers the current operating state to be approaching the system's tolerable limits in multiple dimensions. Based on the above multi-dimensional analysis results, the system generates a state proximity value, which quantifies the relative distance between the current operating state and the critical state, reflecting the level of system risk. The interval division of the state proximity value is also determined based on historical operating data statistics, used to distinguish between safe, alert, and critical states.

[0036] After obtaining the state proximity value, the system extracts the risk assessment result from it. Specifically, the system determines the risk level corresponding to the current operating state based on the interval where the state proximity value falls, and then merges and updates this risk level with the state evolution result from the previous moment to form a potential risk level description. This potential risk level description not only reflects the risk level at the current moment but also reflects the trend of risk changes over time, used to distinguish between different scenarios where the risk is mitigating, remaining stable, or continuously escalating.

[0037] Finally, the system constructs a feedback mechanism based on the potential risk level description to generate reward signals for multi-agent reinforcement learning. Specifically, when the potential risk level description indicates that the system risk has decreased, remained within a safe range, or shifted from high risk to low risk, the system generates a positive reward signal to reinforce the behavior of the current scheduling strategy. When the potential risk level description indicates that the system risk has increased, is approaching a critical state, or remains in a high-risk range, the system generates a negative reward signal to suppress scheduling behaviors that exacerbate the risk. The generated reward signals are input into the multi-agent reinforcement learning model as reward feedback signals during the reinforcement learning process, enabling each agent to gradually learn scheduling decisions that are beneficial for reducing system risk and balancing resources and hot load during policy updates.

[0038] S4 includes obtaining the priority distribution from the risk assessment results, determining the migration timing by integrating resource usage, and obtaining the set of tasks that need to be rate-limited; For the set of tasks that require rate limiting, a rate limiting strategy is applied and the target nodes are determined. The load balancing optimization needs are assessed, and an adjusted resource allocation scheme is obtained. Based on the adjusted resource allocation scheme, a virtual resource scheduling process is constructed, in which the intelligent agent makes decisions on the allocation of node resources, resulting in a scheduling decision sequence. Learning process modeling parameters are extracted from the scheduling decision sequence. These parameters refer to quantitative indicators in the sequence that reflect the decision state and reward. The updated multi-agent reinforcement learning model is generated by combining the feedback signal. The input of the updated multi-agent reinforcement learning model is the scheduling decision sequence and the feedback signal. The output is the resource allocation decision, which determines the optimized virtual GPU resource allocation. For the optimized virtual GPU resource configuration, the task list is integrated and the migration operation is adjusted to obtain the final task distribution of the target GPU node.

[0039] In the specific implementation of this method, the priority distribution information of each task in the current GPU cluster is first extracted from the risk assessment results. This priority distribution is pre-set based on the task's business importance, service level requirements, and real-time needs, and is dynamically updated during task creation or execution. Simultaneously, the system obtains the resource usage of each task on the current GPU node, including the proportion of video memory usage, the intensity of computing resource usage, and the duration of continuous task execution. By fusing the priority distribution with resource usage analysis, the system can identify tasks that significantly contribute to increased video memory pressure, temperature rise, or anomaly rate at the current risk level. Furthermore, the system combines the risk level change trend reflected in the risk assessment results to determine whether task migration or rate limiting is appropriate at the current time, thereby determining a reasonable migration timing and filtering out tasks with lower priority and higher resource usage to form a set of tasks requiring rate limiting.

[0040] After obtaining the set of tasks requiring rate limiting, the system applies rate limiting strategies to each task in the set. The specific form of the rate limiting strategy is determined based on the task type and resource usage characteristics, including limiting the upper limit of GPU memory that a task can use, reducing its computing resource allocation ratio, or extending its scheduling time slice interval. The values ​​of each rate limiting parameter are determined through statistical analysis of the relationship between task performance degradation and resource limitation magnitude in historical running data, to ensure effective reduction of resource pressure without significantly affecting task availability. While executing the rate limiting strategy, the system evaluates the resource reserves of each node in the GPU cluster to determine if there is a need to further optimize load balancing through task migration. When it detects that the resource utilization of some nodes is significantly higher than the cluster average, while other nodes still have a lot of available resources, the system determines that there is a need for load balancing optimization, and generates an adjusted resource allocation scheme based on the GPU memory reserves, temperature margin, and current load of the target nodes.

[0041] After the adjusted resource allocation scheme is determined, the system constructs a virtual resource scheduling process based on this scheme as constraints. Specifically, the available resource status, task demand status, and rate limiting constraints of each node in the GPU cluster are mapped to the state space of a multi-agent reinforcement learning environment, and executable resource allocation and task migration actions are mapped to the decision space. Multiple agents are responsible for making resource allocation decisions for different GPU nodes or different task subsets, with each agent independently selecting its scheduling action based on the current environment state. By executing the above decision-making process in consecutive time steps, the system forms a time-ordered sequence of scheduling decisions, which fully records the decision-making process for resource allocation and task adjustment.

[0042] After obtaining the scheduling decision sequence, the system extracts learning process modeling parameters from it. These parameters quantify the state changes and effect feedback corresponding to each decision during the scheduling process, specifically including the resource state before the decision execution, changes in risk level, and reward feedback signals generated from previous steps. The system uses the scheduling decision sequence and feedback signals as input to update the multi-agent reinforcement learning model, enabling the model to gradually establish the correlation between scheduling decisions and changes in risk and resource utilization efficiency during the learning process. Through multiple rounds of updates, the reinforcement learning model outputs new resource allocation decisions that reflect the optimized results that more effectively balance risk control and resource utilization in the current operating environment, thereby determining the optimized virtual GPU resource allocation.

[0043] Finally, based on the optimized virtual GPU resource configuration, the system uniformly adjusts the task list and performs task migration operations according to the optimization results. Specifically, the system determines the target GPU node for each task based on the resource configuration decision and completes task migration or rebinding operations when the migration timing conditions are met. After migration, the task distribution and resource allocation status on each GPU node are updated, forming the final task distribution result for the target GPU node, providing a new operational basis for subsequent monitoring and feedback analysis.

[0044] S5 includes obtaining a set of priority tasks with a priority higher than a preset threshold from the task list, determining the target GPU node through resource availability assessment, and executing a migration mechanism to achieve an initial distribution adjustment. For the initial distribution adjustment, rate limiting is used to handle priority tasks below a preset threshold, load balancing requirements are determined, and the set of tasks after rate limiting is obtained. Based on the task set after rate limiting, construct the agent's action space, integrate task migration and resource rate limiting as action options, and determine the decision space parameters; The reinforcement learning input is obtained from the decision space parameters, and the agent model is updated by the Q-learning algorithm through the state-action value function. The input is the state and reward signal corresponding to the decision space parameters, and the output is the Q-value table to obtain the optimized action sequence. To optimize the action sequence, integrate the load redistribution execution virtual resource pool management, determine dynamic priority adjustments, and obtain an updated resource allocation scheme.

[0045] In this implementation, the system first reads the priority information for each task from the current task list. This priority information is initially set during task creation based on business importance, service level requirements, and real-time needs, and can be dynamically adjusted according to system status during operation. The system compares each task priority with a preset threshold, which distinguishes between critical tasks requiring priority resource allocation and non-critical tasks whose resource allocation can be adjusted. This preset threshold is determined based on statistical results of the impact of tasks on system stability and performance under high load conditions in historical operational data, selecting a priority boundary point with a significant difference in impact before the risk occurs as the threshold. Through this comparison, the system filters out tasks with priorities higher than the preset threshold, forming a high-priority task set.

[0046] After obtaining the set of high-priority tasks, the system comprehensively evaluates the resource availability of each GPU node in the GPU cluster. This resource availability evaluation includes a comprehensive analysis of remaining memory capacity, temperature safety margin, and current computing load levels. The system first calculates the difference between the currently allocated resources and the maximum available resources of each node, and then considers the recent temperature trends of the nodes to determine their capacity to handle new tasks. Based on this, the system sorts the GPU nodes from highest to lowest resource availability, selecting nodes that meet the resource requirements of high-priority tasks and have sufficient safety margins as target GPU nodes. Subsequently, the system executes a migration mechanism for the high-priority task set, migrating the corresponding tasks from their original nodes to the target GPU nodes, thus forming a preliminary adjusted task distribution. During the migration process, the system controls the timing of the migration to ensure that the migration operation occurs within a relatively stable task load window, avoiding unnecessary interruptions or performance fluctuations in task execution.

[0047] After migrating high-priority tasks, the system performs resource rate limiting on the remaining tasks. Specifically, the system identifies tasks in the task list with priorities below a preset threshold and categorizes them as a low-priority task set. For this set, the system applies rate limiting operations to each task, including limiting the maximum amount of GPU memory they can use, reducing their computational resource allocation ratio, and extending their scheduling time slice interval. The specific values ​​of each rate limiting parameter are determined through statistical analysis of the performance changes of low-priority tasks under different rate limiting intensities in historical running data, ensuring that resource consumption is reduced without causing task anomalies or prolonged blocking. After the rate limiting operation is completed, the system re-calculates the resource usage of each GPU node and determines whether there is still a significant imbalance in the current resource distribution, thus obtaining the result of the rate-limited task set and the corresponding load balancing requirements.

[0048] After obtaining the task set after rate limiting, the system begins to construct the agent's action space. In its implementation, the system abstracts executable task migration operations and resource rate limiting operations into unified action options, and determines decision space parameters based on the current number of GPU nodes, the number of tasks, and the constraints on simultaneous execution. These decision space parameters describe the range of all legal scheduling actions the agent can choose under the current system state, and eliminate action combinations that may lead to resource conflicts or violate safety thresholds through constraints, thereby ensuring the feasibility and security of the decision space.

[0049] Building upon this foundation, the system constructs the input states for reinforcement learning from the decision space parameters and updates the agent model using the Q-learning algorithm. Specifically, the system takes the current resource distribution state, task priority distribution state, and reward signals generated from previous steps as input, records the specific action chosen by the agent in the current state, and updates the value of the action based on the system state change after the action is executed. By repeatedly executing this process over multiple consecutive scheduling cycles, the system gradually forms a Q-value table describing the merits of each action in different states. This Q-value table, by accumulating historical decision-making effects, enables the agent to identify more advantageous scheduling actions under different resource pressures and risk levels, thereby generating an optimized action sequence.

[0050] After obtaining the optimized action sequence, the system performs load redistribution operations on the GPU cluster according to the sequence and manages the virtual resource pool uniformly. Specifically, the system performs task migration or rate limiting intensity adjustment in the order of the optimized action sequence, and synchronously updates the resource allocation records of each node in the virtual resource pool. Simultaneously, the system dynamically adjusts task priorities based on the task's performance under the new resource configuration, ensuring that task priorities more accurately reflect their actual importance in the current system state. Through these adjustments, the system ultimately generates an updated resource allocation scheme that effectively alleviates resource concentration and load imbalance issues while ensuring the performance of high-priority tasks.

[0051] S6 includes acquiring real-time data on memory usage trends and temperature change rates for each node from the migrated resource allocation scheme. For memory usage trends, it records the fluctuation in the percentage of memory usage per unit time; for temperature change rates, it collects the temperature increase / decrease values ​​per unit time. Combining these two data points, a node operating status table is constructed by recording the correspondence between trend fluctuations and temperature increases / decreases, yielding dynamic trend analysis results for each node. Based on the dynamic trend analysis results, it extracts data on page replacement frequency and virtualization overhead changes. When the page replacement frequency exceeds a preset threshold, it records the corresponding time period and node location. Simultaneously, it acquires the peak distribution of virtualization overhead. Integrating these two sets of data, it forms a resource consumption feature set by merging frequency records and peak distributions. This resource consumption feature set includes frequency peak pairs and overhead distribution pairs to determine the load pressure distribution for each node. Finally, it analyzes signal linkage based on the load pressure distribution. The system identifies abnormal behavior within a loop, obtains task scheduling dependencies between nodes, and determines whether vicious cycles of task blocking or resource contention exist. By comparing data changes before and after intervention measures, it assesses the degree of mitigation of the vicious cycle. Based on this assessment, it collects task execution delay monitoring indicators from feedback data and combines them with node health assessment parameters to construct a reward evaluation basis. A pre-established mapping rule is used, where the mapping rule transforms feedback data into reward values ​​by assigning corresponding weights to delay indicators and health parameters, thus determining the reference basis for updating the scheduling strategy. Based on the reward evaluation basis, the system adjusts the scheduling strategy parameters of the reinforcement learning agent, extracts the weight ratios of positive and negative feedback from the reward values, updates task allocation priorities and node selection rules, and obtains an optimized scheduling strategy configuration, thus completing the dynamic adjustment of the resource allocation scheme.

[0052] In the specific implementation process of this embodiment, the operating status of each node in the GPU cluster is first collected in real time based on the resource allocation scheme after migration. The collected data includes the memory usage trend and temperature change rate of each node. The memory usage trend is obtained by collecting the memory occupancy ratio over a continuous time period and recording the fluctuation of the occupancy ratio per unit time. This fluctuation reflects whether the memory pressure has stabilized or is accumulating again. The temperature change rate is obtained by collecting the increase or decrease of GPU node temperature at the same time scale and recording the temperature change value per unit time, used to determine the direction and intensity of the node's thermal state change. The system records the memory occupancy ratio fluctuation and temperature increase / decrease values ​​in chronological order and constructs a node operating status table with GPU nodes as the basic unit. This node operating status table is used to fully describe the resource usage changes and thermal state changes of each node in the post-migration stage, thereby forming the dynamic trend analysis results of each node.

[0053] After obtaining the dynamic trend analysis results, the system further extracts data on changes in page replacement frequency and virtualization overhead. Page replacement frequency is obtained by statistically analyzing the number of memory page replacements per unit time, reflecting whether memory pressure leads to frequent memory scheduling. Virtualization overhead is obtained by statistically analyzing the additional management overhead generated by the GPU virtualization layer during resource management, task scheduling, and context switching, reflecting the impact of the virtualization environment itself on system performance. The system sets a preset threshold for page replacement frequency. This threshold is determined through statistical analysis of page replacement behavior of the GPU under long-term stable operation, selecting the maximum safe frequency without causing significant performance degradation or anomalies, and setting a safety margin as the threshold standard. When the page replacement frequency exceeds the preset threshold, the system records the corresponding time period and the location of the GPU node where the situation occurred. Simultaneously, the system obtains the peak distribution of virtualization overhead within the same monitoring period. Subsequently, the system integrates the page replacement frequency records with the virtualization load distribution peaks. By merging the frequency records and peak distributions, a resource consumption feature set is formed. The resource consumption feature set includes frequency peak pairs reflecting the intensity of page replacement and load distribution pairs reflecting the virtualization management burden, which are used to characterize the load pressure distribution of each GPU node.

[0054] After obtaining the load distribution of each node, the system further analyzes the abnormal behavior in the signal linkage loop based on this distribution. Specifically, the system combines the dynamic linkage mode established in previous steps to comprehensively analyze the relationship between memory usage trends, temperature changes, page replacement frequency, and virtualization overhead. Simultaneously, the system obtains the task scheduling dependencies between each GPU node. This dependency is obtained by analyzing the resource requests, waiting, and migration records of tasks across different nodes, used to determine whether there are blocking phenomena caused by mutual waiting, resource contention, or improper migration order. When multiple nodes are detected to exhibit mutually reinforcing linkages in terms of memory pressure, temperature increases, and virtualization overhead increases, accompanied by increased task scheduling dependencies, the system determines that there is a risk of a vicious cycle caused by task blocking or resource contention. Subsequently, the system evaluates whether the vicious cycle has been alleviated by comparing the dynamic trend analysis results, resource consumption feature sets, and changes in task dependencies before and after the implementation of scheduling intervention measures, obtaining an evaluation result of the degree of vicious cycle alleviation. This evaluation result reflects the suppression effect of the current scheduling strategy on abnormal linkages.

[0055] After obtaining the assessment results of the degree of mitigation of the vicious cycle, the system further collects task execution delay monitoring indicators from the feedback data. These indicators are obtained by statistically analyzing the actual execution time of tasks from scheduling to completion, reflecting the impact of the current resource allocation scheme on task response performance. Simultaneously, the system evaluates the overall operating status of GPU nodes in conjunction with node health assessment parameters. These parameters include indicators such as node temperature stability, resource utilization fluctuation range, and anomaly frequency. The normal range for each parameter is determined by statistically analyzing data from historical stable operating conditions. Based on the above feedback data, the system constructs a reward result evaluation basis and uses a pre-established mapping rule to convert the feedback data into reward values. This mapping rule combines task execution delay indicators and node health assessment parameters according to corresponding weights, where each weight is determined based on the statistical analysis results of the impact of performance indicator changes on system stability during historical operation. Through this mapping process, the system generates reward values ​​that comprehensively reflect the quality of the scheduling strategy, serving as the basis for subsequent scheduling strategy updates.

[0056] Finally, based on the evaluation criteria for the reward results, the scheduling strategy parameters of the reinforcement learning agent are adjusted. Specifically, the system extracts the weight ratios corresponding to positive and negative feedback from the reward values ​​and updates the task allocation priority and GPU node selection rules according to these weight ratios. When the weight of positive feedback is high, the system strengthens decision-making behaviors in the current scheduling strategy that are beneficial to reducing task latency and improving node health; when the weight of negative feedback is high, the system suppresses scheduling behaviors that lead to increased resource contention or exacerbate vicious cycles. Through the above strategy parameter update process, the system generates an optimized scheduling strategy configuration and dynamically adjusts the GPU virtualization resource allocation scheme accordingly.

[0057] S7 includes obtaining a vicious cycle indicator from feedback data. This indicator is formed by statistically analyzing the task blocking frequency and resource contention ratio between nodes. The mitigation status is determined by comparing the vicious cycle indicator with a preset threshold, resulting in a list of unmitigated nodes. For the unmitigated node list, the load balancing mechanism is adjusted by recalculating node weights to obtain the current task density of each node. The load pressure distribution is determined by merging task density with resource usage monitoring data, which is extracted from node real-time logs. Based on the load pressure distribution, a resource rate limiting strategy is adopted. This strategy is executed by setting a traffic threshold to obtain task priority ranking. The traffic of priority tasks exceeding the preset threshold is limited to obtain a rate-limited task queue. For the rate-limited task queue, tasks are reallocated to GPU nodes, and node performance evaluation results are obtained. These results are generated by comparing processing speed and available memory. The optimal allocation scheme is determined by selecting the node with the lowest load. Based on the optimal allocation scheme, the final resource scheduling configuration is generated by integrating node load prediction data to obtain optimized configuration parameters, which are calculated based on historical trends.

[0058] In the specific implementation of this method, the system first extracts a vicious cycle indicator from the feedback data after scheduling execution. This vicious cycle indicator is composed of the task blocking frequency and resource contention ratio between nodes. The task blocking frequency is obtained by statistically analyzing the number of times a task enters a waiting state per unit time due to resource unavailability, unmet scheduling dependencies, or migration conflicts. The resource contention ratio is obtained by statistically analyzing the proportion of multiple tasks simultaneously contending for GPU memory, computing resources, or bandwidth resources within the same time period. The system summarizes these two types of indicators on the same time scale to form a vicious cycle indicator used to characterize the intensity of abnormal node linkage. Subsequently, the system compares this vicious cycle indicator with a preset threshold. The preset threshold is determined by statistical analysis of the vicious cycle indicator during the historical stable operation phase of the GPU cluster, selecting the maximum safe value without causing significant performance degradation, abnormal spread, or scheduling failure as the threshold benchmark, and setting a safety margin based on this, combined with system redundancy capabilities. When the vicious cycle indicator corresponding to a node continuously exceeds the preset threshold, the system determines that the abnormal state of that node has not been effectively mitigated and records the node in the unresolved node list.

[0059] After obtaining the list of unresolved nodes, the load balancing mechanism is adjusted accordingly. Specifically, the system updates the load balancing mechanism by recalculating the scheduling weights of each GPU node. The calculation of node weights is based on the node's current task density and resource usage monitoring data. Task density is obtained by statistically analyzing the number of running tasks, queued tasks, and average task occupancy time on a node, reflecting the node's pressure level at the scheduling level. Resource usage monitoring data is extracted from the node's real-time logs, including memory usage ratio, computing resource utilization, and node temperature load. The system integrates the task density and resource usage monitoring data, comprehensively considering the importance of different indicators to form a load pressure distribution reflecting the overall load level of the nodes, thus clarifying the relative load of each node in its current state.

[0060] After obtaining the load distribution, a resource rate limiting strategy is further employed to constrain task execution behavior. This strategy is implemented by setting a traffic threshold, which limits the intensity of task consumption on memory bandwidth, computing resources, or scheduling requests per unit time. The traffic threshold is determined based on GPU hardware specifications, virtualization management capabilities, and historical load levels when resources are close to saturation but no anomalies occur. A safe upper limit is obtained through statistical analysis. When executing the rate limiting strategy, the system first obtains the priority ranking of all current tasks, determined based on task importance and service level requirements. Subsequently, for tasks with priorities higher than the preset threshold, rate limiting is implemented by restricting their resource request frequency or concurrent execution scale, thus forming a rate-limited task queue. This method prevents high-priority tasks from excessively consuming resources and further amplifying node load differences when anomalies are not completely eliminated.

[0061] After the rate-limited task queue is formed, tasks are reallocated. Specifically, the system performs performance evaluations on each node in the GPU cluster, generating node performance evaluation results. These results are obtained by comparing the task processing speed and available memory of each node in its current running state. Task processing speed is determined by counting the number of tasks completed or the computation progress per unit time; available memory is determined by counting the remaining GPU memory capacity of each node. These two indicators are used as important evaluation criteria for node performance. The nodes are comprehensively ranked, and the node with the lowest load and sufficient resource safety margin is selected as the better target node. Subsequently, based on the better allocation scheme, tasks in the rate-limited task queue are reallocated to the target node, thereby effectively transferring the load pressure from unrelieved nodes.

[0062] After task reallocation is completed, a final resource scheduling configuration is generated based on a better allocation scheme. Specifically, the system further integrates node load prediction data to correct the current scheduling results. This node load prediction data is calculated based on the historical operating trends of nodes. By analyzing changes in resource usage, temperature variations, and task completion efficiency under similar load conditions, the system predicts the load evolution direction in subsequent scheduling cycles. This prediction result is then integrated with the current scheduling scheme, and the resource configuration parameters are refined to ensure that the final resource scheduling configuration not only adapts to the current operating state but also has the ability to predict short-term load fluctuations.

[0063] Table 1. Comparison of performance data between the present invention and fixed threshold / single-point monitoring scheduling. As shown in Table 1, this application can identify potential risks in GPU clusters in advance. Existing technologies typically trigger scheduling only after memory usage, temperature, or task anomalies have exceeded thresholds. However, this application, through time-series analysis of memory trends, temperature change rates, and anomaly rate fluctuations, can identify risks before the system reaches a critical state. Therefore, the risk identification lead time in the table is increased from 0-1 min in existing technologies to 8-15 min in this application, demonstrating the proactive scheduling capability of this application. Simultaneously, this invention can reduce the probability of memory overflow, temperature overload, and task anomalies. The table shows a significant reduction in memory overflow events, temperature overload events, and task failure rates. This indicates that this application does not wait for GPU resource failures to occur before passively addressing them, but rather proactively migrates tasks or limits flow when memory pressure and temperature load show a correlated upward trend, thereby reducing failure events. This invention better ensures the operation of high-priority tasks; the high-priority task interruption rate in the table decreased from 4.5% to 0.8%, indicating that this application considers task priority distribution and resource occupancy during scheduling, prioritizing the migration of high-priority tasks to GPU nodes with sufficient resource reserves, preventing critical tasks from being interrupted due to memory shortages, excessive temperature, or node overload. Furthermore, this invention improves GPU resource utilization and load balancing; the average GPU utilization in the table increased from 68% to 82%, and node load imbalance was reduced. The reduction from 31% to 12% indicates that this application, through target GPU node selection and load balancing mechanisms, makes the task distribution among different GPU nodes more reasonable, reducing the situation where some nodes are overloaded while others are idle. This invention can suppress the performance degradation caused by the amplification of GPU virtualization overhead. The virtualization management overhead ratio in the table has decreased from 18% to 10%, which shows that this application can reduce page replacement, resource contention, and virtualization layer scheduling overhead by alleviating memory pressure and temperature load in advance, thereby avoiding the mutual reinforcement of performance degradation and resource shortage. The system stable running time in the table has increased from 72h to 120h, indicating that this invention, through continuous perception, dynamic analysis, and adaptive scheduling, can reduce task interruption and system performance degradation caused by scheduling lag, and improve the stability of the GPU virtualization cluster under complex load conditions.

[0064] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A GPU virtualization resource scheduling method based on multi-agent reinforcement learning, characterized in that, include: S1. By collecting real-time data on GPU memory usage, temperature sensor data and task anomaly rate records in the GPU cluster, a multi-dimensional time series dataset is constructed. The data in each dimension is smoothed to filter noise, and a preliminary set of signal change features is obtained. S2. Based on the set of signal change characteristics, the correlation between memory usage trends, temperature change rate and anomaly rate fluctuation values ​​is analyzed using a preset sliding window method to determine the dynamic linkage mode between each signal. S3. If the dynamic linkage mode shows that the memory usage trend continues to rise and the temperature change rate exceeds the preset threshold, then the correlation analysis is used to determine whether the system is close to the critical state and obtain the potential risk assessment result. S4. Based on the potential risk assessment results, obtain the priority distribution and resource usage of the current tasks, and determine the list of tasks to be adjusted and the target GPU nodes by combining the task migration timing and resource throttling strategy. S5. Through the load balancing mechanism, priority tasks in the task list that need to be adjusted that are higher than the preset threshold are migrated to the target GPU node with available resources. At the same time, rate limiting is performed on priority tasks that are lower than the preset threshold to obtain the updated resource allocation scheme.

2. The GPU virtualization resource scheduling method based on multi-agent reinforcement learning according to claim 1, characterized in that: S1 includes: A multidimensional time series dataset is constructed by collecting real-time data on GPU memory usage, temperature sensor data, and task failure rate records from the GPU cluster. For each dimension of the multidimensional time series dataset, a moving average method is used for smoothing to obtain a set of smoothed sequences after filtering out noise. The signal change features are extracted from the set of smooth sequences. The slope is obtained by calculating the difference between adjacent data points and the local maximum value is detected as the peak value to generate a preliminary set of signal change features. By combining the set of signal change features with the preset historical load data of the cluster, the resource pre-allocation prediction matrix is ​​determined through matrix multiplication. The state input of the multi-agent reinforcement learning environment is updated by pre-allocating the resource prediction matrix, and a preliminary set of signal change features is obtained as the environment state after input.

3. The GPU virtualization resource scheduling method based on multi-agent reinforcement learning according to claim 1, characterized in that: S2 includes: The usage trend and temperature change rate of the video memory are obtained from the signal change feature set. The Pearson correlation coefficient between the abnormality rate fluctuation values ​​is calculated by taking the product of the covariance and the standard deviation using the preset sliding window method, and a preliminary set of correlation indicators is obtained. Based on the preliminary set of correlation indicators, and combined with the energy consumption fluctuation monitoring data obtained from the signal change feature set, the basic linkage parameters between signals are determined by fusing the temperature change rate and anomaly rate fluctuation values ​​through matrix operations. Based on the basic linkage parameters, peak offset features are extracted from the time series data derived from the signal change feature set to generate an extended dynamic linkage mode description. From the extended dynamic linkage mode description, the cluster historical load information extracted from the signal change feature set is integrated, and the state evolution feature components are updated by vector embedding. By constructing the reinforcement learning environment input through state evolution feature components, the dynamic linkage pattern between various signals is determined as part of the reinforcement learning state evolution features.

4. The GPU virtualization resource scheduling method based on multi-agent reinforcement learning according to claim 1, characterized in that: S3 includes: The dynamic linkage mode is used to obtain the upward trend of video memory and the accelerated change of temperature. Combined with system load monitoring and energy consumption fluctuation, a comprehensive monitoring index is obtained. For comprehensive monitoring indicators, threshold over-limit detection is used to compare with preset thresholds. If the temperature accelerates and changes beyond the preset threshold, an over-limit event flag is determined. Based on the over-limit event flag, the linkage mode analysis is performed by calculating the correlation coefficient, and the network latency index is integrated to determine the critical proximity and obtain the state proximity value. Risk assessment results are extracted from state proximity values ​​and fused with state evolution updates to generate a description of potential risk levels. Based on the description of potential risk levels, a feedback mechanism is constructed to generate reward signals, which serve as reward feedback signals for reinforcement learning.

5. The GPU virtualization resource scheduling method based on multi-agent reinforcement learning according to claim 1, characterized in that: S4 includes: The priority distribution is obtained from the risk assessment results, and the migration timing is determined by integrating resource usage data to obtain the set of tasks that need to be rate-limited. For the set of tasks that require rate limiting, a rate limiting strategy is applied and the target nodes are determined. The load balancing optimization needs are assessed, and an adjusted resource allocation scheme is obtained. Based on the adjusted resource allocation scheme, a virtual resource scheduling process is constructed, in which the intelligent agent makes decisions on the allocation of node resources, resulting in a scheduling decision sequence. Learning process modeling parameters are extracted from the scheduling decision sequence. These parameters refer to quantitative indicators in the sequence that reflect the decision state and reward. The updated multi-agent reinforcement learning model is generated by combining the feedback signal. The input of the updated multi-agent reinforcement learning model is the scheduling decision sequence and the feedback signal. The output is the resource allocation decision, which determines the optimized virtual GPU resource allocation. For the optimized virtual GPU resource configuration, the task list is integrated and the migration operation is adjusted to obtain the final task distribution of the target GPU node.

6. The GPU virtualization resource scheduling method based on multi-agent reinforcement learning according to claim 1, characterized in that: S5 includes: The set of priority tasks with a priority higher than a preset threshold is obtained from the task list, the target GPU node is determined by resource availability assessment, and the migration mechanism is executed to achieve an initial adjustment of the distribution; For the initial distribution adjustment, rate limiting is used to handle priority tasks below a preset threshold, load balancing requirements are determined, and the set of tasks after rate limiting is obtained. Based on the task set after rate limiting, construct the agent's action space, integrate task migration and resource rate limiting as action options, and determine the decision space parameters; The reinforcement learning input is obtained from the decision space parameters, and the agent model is updated by the Q-learning algorithm through the state-action value function. The input is the state and reward signal corresponding to the decision space parameters, and the output is the Q-value table to obtain the optimized action sequence. To optimize the action sequence, integrate the load redistribution execution virtual resource pool management, determine dynamic priority adjustments, and obtain an updated resource allocation scheme.

7. The GPU virtualization resource scheduling method based on multi-agent reinforcement learning according to claim 1, characterized in that, This also includes S6, monitoring feedback data on memory usage trends and temperature change rates after migration based on the updated resource allocation scheme, analyzing changes in page replacement frequency and virtualization usage, and determining whether intervention measures alleviate the vicious cycle in signal linkage, specifically including: From the migrated resource allocation scheme, real-time data on the memory usage trend and temperature change rate of each node are obtained. For the memory usage trend, the fluctuation of the occupancy ratio per unit time is recorded. For the temperature change rate, the temperature increase or decrease value per unit time is collected. Combining the two, a node operation status table is constructed by recording the correspondence between trend fluctuation and temperature increase or decrease, and the dynamic trend analysis results of each node are obtained. Based on the dynamic trend analysis results, the change data of page replacement frequency and virtualization expenditure are extracted. When the page replacement frequency exceeds the preset threshold, the corresponding time period and node position are recorded. At the same time, the peak distribution of virtualization expenditure is obtained. The two data are integrated by merging the frequency records and peak distribution to form a resource consumption feature set. The resource consumption feature set includes frequency peak pairs and expenditure distribution pairs to determine the load pressure distribution of each node. To analyze the abnormal behavior in the signal linkage loop based on the load pressure distribution, obtain the task scheduling dependency relationship between each node, determine whether there is a vicious cycle of task blocking or resource competition, and obtain the assessment result of the degree of mitigation of the vicious cycle by comparing the data changes before and after the intervention measures are adjusted.

8. The GPU virtualization resource scheduling method based on multi-agent reinforcement learning according to claim 7, characterized in that: S6 further includes: Based on the assessment results of the degree of mitigation of the vicious cycle, the task execution delay monitoring indicators in the collected feedback data are combined with the node health assessment parameters to construct the evaluation basis for the reward results. A pre-established mapping rule is adopted, in which the mapping rule transforms the feedback data into reward values ​​through the corresponding weights of the delay indicators and health parameters, and determines the reference basis for updating the scheduling strategy. Based on the evaluation criteria for reward results, the scheduling strategy parameters of the reinforcement learning agent are adjusted. The weight ratio of positive and negative feedback is extracted from the reward value, the task allocation priority and node selection rules are updated, and the optimized scheduling strategy configuration is obtained, thus completing the dynamic adjustment of the resource allocation scheme.

9. A GPU virtualization resource scheduling method based on multi-agent reinforcement learning according to claim 7, characterized in that, This also includes S7. If feedback data indicates that the vicious cycle has not been alleviated, tasks are reallocated to better GPU nodes by adjusting the load balancing mechanism and resource throttling strategy to obtain the final resource scheduling configuration, specifically including: The vicious cycle indicator is obtained from the feedback data. The vicious cycle indicator is formed by statistically analyzing the task blocking frequency and resource competition ratio between nodes. The mitigation status is determined by comparing the vicious cycle indicator with a preset threshold, and a list of unmitigated nodes is obtained. For the list of unrelieved nodes, the load balancing mechanism is adjusted. The adjustment is achieved by recalculating the node weights, obtaining the current task density of each node, and determining the load pressure distribution by merging the task density with resource usage monitoring data. The resource usage monitoring data is extracted from the node's real-time logs. Based on the load pressure distribution, a resource rate limiting strategy is adopted. The resource rate limiting strategy is executed by setting a traffic threshold, obtaining the task priority ranking, and obtaining the rate-limited task queue by limiting the traffic of priority tasks that are higher than the preset threshold.

10. A GPU virtualization resource scheduling method based on multi-agent reinforcement learning according to claim 9, characterized in that: The S7 also includes: For the task queue after rate limiting, tasks are reallocated to GPU nodes, and node performance evaluation results are obtained. The node performance evaluation results are generated by comparing processing speed and available memory, and the optimal allocation scheme is determined by selecting the node with the lowest load. Based on the optimal allocation scheme, the final resource scheduling configuration is generated. The optimized configuration parameters are obtained by integrating node load prediction data, where the node load prediction data is calculated based on historical trends.

Images