Method for predicting and scheduling gpu resource usage based on multi-policy reinforcement learning

Abstract

The application discloses a GPU resource use prediction and scheduling method based on multi-strategy reinforcement learning, relates to the technical field of computers, and comprises the following steps: S1, a real-time current load mode is acquired from a GPU cluster through a pre-established monitoring module, the load mode comprises a task type distribution and an arrival frequency, and an initial load vector is constructed based on the current load mode and taken as an initial input of GPU resource use prediction, which is used for subsequent judgment; the GPU resource use prediction and scheduling method based on the multi-strategy reinforcement learning enables the scheduling strategy to be continuously optimized with the change of the load, reduces the need for manual intervention and repeated training, significantly improves the GPU resource utilization, reduces the task waiting time, and enhances the stability and self-adaptive capacity of the scheduling system in the multi-tenant and complex business scenarios.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to a GPU resource usage prediction and scheduling method based on multi-policy reinforcement learning. Background Technology

[0002] In modern data centers and cloud computing platforms, GPU resources have become critical computing resources supporting large-scale model training, high-concurrency inference services, and various compute-intensive applications. With the continuous expansion of artificial intelligence applications, GPU clusters exhibit characteristics such as diversified task types, significant fluctuations in arrival frequency, and rapid changes in resource usage status. Therefore, the rational prediction and scheduling of GPU resources has become a crucial foundation for ensuring system performance and resource utilization.

[0003] Most existing GPU scheduling systems focus on allocating resources based on current or historical statistics, typically relying on manually designed scheduling rules or a single reinforcement learning model for decision-making. While these methods can meet basic resource scheduling needs to a certain extent under relatively stable load conditions, they still have significant shortcomings in real-world environments. Firstly, GPU cluster load patterns are highly dynamic and uncertain, with different types of tasks exhibiting significant differences at different times. Simply relying on static rules or fixed strategies makes it difficult to reflect changes in GPU resource usage in a timely manner. Secondly, existing methods often lack effective predictive capabilities for GPU resource usage, with scheduling decisions based more on immediate conditions or empirical assumptions, making it difficult to maintain stable scheduling performance when load changes or new load patterns emerge. Furthermore, in multi-tenant and complex business scenarios, GPU clusters frequently experience frequent load pattern switching. When the current load pattern differs significantly from historical operating patterns, the scheduling system faces problems such as incomplete state information and difficulty in accurately obtaining key parameters, such as the uncertainty of remaining task execution time and the dynamic changes in memory fragmentation distribution. In this context, existing scheduling methods typically lack a systematic mechanism for handling uncertainties, easily leading to resource allocation decisions deviating from the optimal state, and consequently causing problems such as decreased GPU utilization and increased task queuing time. Furthermore, existing reinforcement learning scheduling methods often employ a single decision-making strategy, lacking the ability to switch, merge, or evolve strategies across different load phases. When the cluster operating environment undergoes long-term changes, the original strategy struggles to adapt to new load characteristics in a timely manner, often requiring retraining or manual intervention, thus limiting the overall adaptive and continuous optimization capabilities of the scheduling system. Summary of the Invention

[0004] The purpose of this invention is to provide a GPU resource usage prediction and scheduling method based on multi-policy reinforcement learning, thereby solving the problems existing in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a GPU resource usage prediction and scheduling method based on multi-policy reinforcement learning, comprising: S1, obtaining the current load mode from the GPU cluster in real time through a pre-established monitoring module, the load mode including task type distribution and arrival frequency, and constructing an initial load vector based on the current load mode as the initial input for GPU resource usage prediction; S2, calculating a similarity score for the initial load vector using a deep neural network model, if the similarity score exceeds a preset threshold, determining that the current load mode matches a historical load mode, and obtaining the corresponding stable state information as the stable state basis for GPU resource usage prediction; S3, otherwise, marking it as a new load mode. S4. When the state information is incomplete, uncertain parameters are extracted from the remaining execution time estimation module and the memory fragmentation distribution scan. The GPU resource usage prediction results are corrected based on the uncertain parameters to obtain the adjusted resource state matrix. S5. Based on the adjusted resource state matrix and short-term load fluctuation indicators, the optimal scheduling decision sequence is iteratively calculated using the Q-learning algorithm in reinforcement learning to form a short-term scheduling strategy based on the GPU resource usage prediction results. S6. The short-term scheduling strategy is fused with long-term change adaptation historical data through an online update algorithm to form an evolutionary scheduling strategy under multi-strategy reinforcement learning, and then output to the GPU allocation engine for execution.

[0006] Preferably, step S1 includes acquiring real-time data streams from the GPU cluster through a pre-established monitoring module, parsing the current load pattern containing task type distribution and arrival frequency from the real-time data stream; extracting computational density features based on the current load pattern, and concatenating the computational density features with the arrival frequency to construct an initial load vector; using the initial load vector as input for time-series evolution deduction to generate a resource usage prediction value containing memory occupancy, so as to complete the subsequent judgment of GPU resource usage prediction based on the resource usage prediction value.

[0007] Preferably, step S2 includes obtaining an initial load vector to be analyzed, encoding the initial load vector using a deep neural network model to generate a high-dimensional load feature tensor; calculating the similarity score between the high-dimensional load feature tensor and historical standard vectors, and determining a matching historical load pattern if the similarity score exceeds a preset threshold; extracting stable state information containing memory allocation peaks based on the historical load pattern; and correcting the benchmark parameters based on the stable state information to determine the stable state basis for GPU resource usage prediction.

[0008] Preferably, step S3 includes acquiring real-time operating data to extract load features, matching the load features with historical benchmark data to obtain feature deviation; if the feature deviation exceeds a verification threshold, generating a new load mode switching instruction, and capturing a system state snapshot based on the switching instruction; performing integrity verification on the system state snapshot to generate a state missing description file, obtaining a completion strategy based on the state missing description file and synthesizing a state set to be processed; and transmitting the state set to be processed to the recovery queue to start a state information incompleteness processing program for the new load mode switching.

[0009] Preferably, step S4 includes extracting instruction stream execution progress data and a video memory physical address mapping table; calculating the remaining execution time value based on the instruction stream execution progress data; constructing a video memory fragmentation distribution scan map based on the video memory physical address mapping table; performing joint analysis on the remaining execution time value and the video memory fragmentation distribution scan map to output a set of uncertainty parameters; inputting the set of uncertainty parameters into an error evaluation function to obtain a prediction deviation quantization value; generating an adjustment coefficient matrix based on the prediction deviation quantization value; and using the adjustment coefficient matrix to perform weighted correction on the initial prediction data to obtain an adjusted resource state matrix.

[0010] Preferably, step S5 includes obtaining the adjusted resource state matrix and short-term load fluctuation index; performing weighted feature fusion on the resource state matrix and short-term load fluctuation index to construct a multi-dimensional state vector; mapping the multi-dimensional state vector to the state space of reinforcement learning; iteratively calculating using the Q-learning algorithm to obtain a converged Q-value table; performing a greedy optimization operation based on the converged Q-value table to generate an optimal scheduling decision sequence; parsing the optimal scheduling decision sequence and converting it into a resource allocation instruction set; and forming a short-term scheduling strategy based on GPU resource usage prediction results by executing the resource allocation instruction set.

[0011] Preferably, step S6 includes acquiring the instantaneous computing power request and historical job characteristics of the task to be processed, aligning and normalizing the instantaneous computing power request and historical job characteristics to construct a hybrid state tensor, inputting the hybrid state tensor into a short-term decision network and a long-term adaptation model to generate a multi-policy joint distribution, and using the global reward feedback value of the environment feedback to perform online gradient correction on the multi-policy joint distribution to generate policy evolution parameters.

[0012] Preferably, step S6 further includes generating a final scheduling instruction based on the policy evolution parameters, and transmitting the final scheduling instruction to the GPU allocation engine to complete the execution of the evolution scheduling policy under multi-policy reinforcement learning.

[0013] Preferably, it also includes S7, obtaining feedback indicators from performance degradation monitoring after execution, and using the feedback indicators as input for the next initial load vector to realize closed-loop update of GPU resource usage prediction and multi-strategy reinforcement learning scheduling strategy. Specifically, it includes extracting performance degradation features from GPU cluster running status data, extracting performance degradation features when task latency exceeds a preset threshold; quantifying the performance degradation features to obtain quantitative feedback values, and embedding the quantitative feedback values ​​into historical load sequences to construct the initial load vector.

[0014] Preferably, step S7 further includes inputting the initial load vector into the resource prediction model to obtain a prediction deviation matrix, the prediction deviation matrix being generated by comparing the GPU computing power map with the actual results; adjusting the action probability distribution according to the prediction deviation matrix to achieve closed-loop update of GPU resource usage prediction and multi-strategy reinforcement learning scheduling strategy.

[0015] As can be seen from the above technical solution, the present invention has the following beneficial effects: This GPU resource usage prediction and scheduling method based on multi-policy reinforcement learning can effectively predict and adaptively schedule GPU resource usage in complex and dynamically changing GPU cluster environments. On one hand, by modeling the current load pattern and incorporating historical load information, scheduling decisions no longer rely solely on the immediate state but possess the ability to predict GPU resource usage trends, thereby improving the foresight and rationality of resource allocation. On the other hand, in cases of load pattern switching or incomplete state information, uncertainties are introduced for correction, effectively mitigating the performance degradation caused by missing information or sudden environmental changes. Furthermore, by combining short-term scheduling decisions with long-term operational history, an evolutionary mechanism based on multi-policy reinforcement learning is constructed, enabling the scheduling strategy to continuously optimize with load changes, reducing the need for manual intervention and repetitive training, significantly improving GPU resource utilization, reducing task waiting time, and enhancing the stability and adaptability of the scheduling system in multi-tenant and complex business scenarios. Attached Figure Description

[0016] Figure 1 This is a flowchart of the GPU resource usage prediction and scheduling method of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] like Figure 1As shown, this invention provides a technical solution: a GPU resource usage prediction and scheduling method based on multi-policy reinforcement learning, including: S1, obtaining the current load mode from the GPU cluster in real time through a pre-established monitoring module, the load mode including task type distribution and arrival frequency, and constructing an initial load vector based on the current load mode as the initial input for GPU resource usage prediction, used for subsequent judgment; S2, calculating a similarity score for the initial load vector using a deep neural network model, if the similarity score exceeds a preset threshold, determining that the current load mode matches the historical load mode, and obtaining the corresponding stable state information as the stable state basis for GPU resource usage prediction; S3, otherwise, marking it as a new load mode switch, triggering incomplete state information processing; S4, in the case of incomplete state information, from the remaining execution... Uncertainty parameters are extracted from the line time estimation module and memory fragmentation distribution scan. Based on the uncertainty parameters, the GPU resource usage prediction results are corrected to obtain an adjusted resource state matrix. S5. According to the adjusted resource state matrix and short-term load fluctuation indicators, the optimal scheduling decision sequence is iteratively calculated using the Q-learning algorithm in reinforcement learning to form a short-term scheduling strategy based on the GPU resource usage prediction results. S6. The short-term scheduling strategy is fused with long-term change adaptation historical data through an online update algorithm to form an evolutionary scheduling strategy under multi-strategy reinforcement learning, and output to the GPU allocation engine for execution. S7. After execution, feedback indicators are obtained from performance degradation monitoring and used as input for the next initial load vector to realize the closed-loop update of GPU resource usage prediction and multi-strategy reinforcement learning scheduling strategy.

[0019] This implementation method is applied to resource management scenarios of large-scale GPU clusters. It addresses the problems of rapid dynamic changes in task load, severe memory fragmentation, and insufficient adaptability of traditional scheduling strategies to unknown loads by achieving high-precision scheduling through a collaborative mechanism of "prediction-correction-reinforcement learning decision-feedback update".

[0020] Specifically, the monitoring module continuously collects runtime status data from the GPU cluster and encodes features such as task type distribution and task arrival frequency into an initial load vector, thereby achieving a quantitative description of the current load status. Subsequently, a deep neural network model performs similarity calculations based on historical load features to determine whether the current load can be mapped to a known stable runtime state. When a match is successful, historical stable state information can be directly used as a prediction basis, thereby reducing computational overhead.

[0021] When a new load pattern is detected, due to insufficient historical data, the system introduces a remaining execution time estimation module to predict the completion time of running tasks, and obtains memory availability characteristics through a memory fragmentation distribution scanning module, characterizing the current resource uncertainty from multiple dimensions. Based on this, the predicted GPU resource usage is corrected to generate a resource state matrix that more closely reflects the actual situation.

[0022] Furthermore, the system uses the resource state matrix as the input to the reinforcement learning environment state, combines it with short-term load fluctuation indicators, and utilizes the Q-learning algorithm to iteratively evaluate the long-term rewards of different scheduling actions on system performance, thereby generating the optimal scheduling decision sequence. Through an online update algorithm, short-term scheduling strategies are integrated with long-term historical evolution characteristics, enabling the scheduling strategy to continuously adapt over time.

[0023] By introducing a stable state reuse mechanism based on similarity judgment, GPU resource usage prediction can be completed quickly when the load pattern recurs, significantly reducing prediction latency. Under the new load mode, uncertainty correction is performed by combining the remaining execution time and memory fragmentation distribution, which improves the accuracy of resource status modeling and reduces scheduling errors caused by missing information. The scheduling strategy is generated based on the Q-learning algorithm, which enables the GPU resource allocation process to comprehensively consider the current state and long-term benefits, effectively alleviating resource contention and performance degradation problems. Through an online update and feedback closed-loop mechanism, the scheduling strategy can continuously evolve with changes in the GPU cluster's operating environment and task characteristics, thereby improving the overall stability, resource utilization, and scalability of the system.

[0024] S1 includes acquiring real-time data streams from the GPU cluster through a pre-established monitoring module, parsing the current load pattern from the real-time data stream, which includes task type distribution and arrival frequency; extracting computational density features based on the current load pattern, and concatenating the computational density features with the arrival frequency to construct an initial load vector; using the initial load vector as input for time-series evolution deduction to generate a resource usage prediction value including memory occupancy, and making subsequent judgments on GPU resource usage prediction based on the resource usage prediction value.

[0025] In this embodiment, the monitoring module continuously accesses the real-time data stream of the GPU cluster in a streaming manner. The real-time data stream is generated synchronously by the task submission side and the execution side, and at least covers task submission events, task start events, task end events, task queue identifier, task resource request information, task running node identifier, and GPU device running status reporting information. After entering the monitoring module, the real-time data stream first undergoes time alignment processing. The sampling period used for time alignment is determined by closely matching the minimum decision period of the cluster scheduler (the sampling period is determined by reading the minimum scheduling interval in the scheduler configuration and setting the sampling period to an integer multiple of this minimum scheduling interval; the integer multiple is determined by statistical analysis of historical load change rates, ensuring that the load pattern change amplitude between two adjacent sampling points remains distinguishable and does not generate data jitter). Subsequently, deduplication processing is performed. The primary key used for deduplication is determined by a combination of the task's unique identifier and the event type (the primary key is determined by retaining only the first arrival record of the same event type within the task's lifecycle, and discarding subsequent duplicate records). Missing field completion processing is also performed. The mapping relationship used for missing field completion is determined by the field dictionary of the task submission system (the mapping relationship is determined by mapping synonymous fields to standard field names based on the meaning of fields exposed by the submission system).

[0026] After the structured data is generated, the monitoring module parses the current load pattern from the real-time data stream. The parsing process aggregates events according to a preset time window. The length of the time window is determined by the statistical stability of the cluster load and the response requirements of the scheduling decision (the time window length is determined by aggregating task arrival sequences with multiple candidate window lengths based on historical running data, comparing the fluctuation amplitude and latency of arrival frequency under different window lengths, and selecting the window length where the fluctuation amplitude tends to be stable and meets the scheduling response time limit; the lower limit of the candidate window length is determined by multiplying the sampling period by 10, and the upper limit is determined by the scheduling policy update period). Within this time window, the task type distribution is obtained by the task type discrimination rule, which is established based on the frame identifier, mirror identifier, entry command feature, and operator set feature in the task metadata (the discrimination rule is determined by clustering historical task logs, extracting metadata field combinations that can stably distinguish training, inference, data preprocessing, and graphics rendering tasks, and solidifying these field combinations as discrimination conditions; when multiple conditions are met simultaneously, the priority is determined by the dominance of this type of task in resource consumption). After task type labeling is completed, the task type distribution is obtained through count normalization. The denominator of the normalization is the total number of tasks within the time window (the denominator is determined by only counting tasks that have submitted events within the window, filtering out duplicate submission records triggered by retries). The task arrival frequency is calculated from the number of submission events within the time window. The conversion process first counts the number of submission events within the window, and then converts it to the number of arrivals per unit time according to the window length (the value of the unit time is consistent with the sampling period, which is constrained by the scheduler's minimum decision period to ensure that the arrival frequency is consistent with the subsequent scheduling time scale).

[0027] After obtaining the current load pattern, computational intensity features are extracted driven by task type distribution. The extraction process uses task type as an index to link to a type profile library. The type profile library is generated from historical execution records and continuously updated online. The profile content includes at least the average computational unit utilization level, average execution time distribution, average memory allocation distribution, and average memory dwell time distribution for that type of task. The generation process of the type profile library is as follows: when the task ends, the execution statistics of the task are collected and aggregated according to task type. During aggregation, outliers are pruned. The pruning threshold is determined by quantile statistics (the pruning threshold is determined by calculating the upper and lower quantiles for the indicator samples of the same task type, taking the upper and lower quantiles as the boundary of the pruning interval, and the quantile value is determined by the sample size. If the sample size is insufficient, the quantile is increased to avoid over-pruning). The numerical construction of the computational intensity feature adopts a "joint characterization of intensity and persistence": first, the average occupancy level of the computing unit per unit time for this type of task reflects the intensity; then, the average execution time of this type of task reflects the persistence. Both are uniformly scaled, with the scaling boundary determined by the historical minimum and maximum values ​​of this type of task in the profile database (the scaling boundary is determined by maintaining a sliding historical boundary synchronously during profile database updates; the sliding range length is jointly determined by the profile update cycle and the business cycle, with the business cycle being the cluster's working day load cycle length; when abrupt changes occur in the historical boundary, median trend smoothing is used to suppress short-term anomalies). Subsequently, the computational intensity feature is weighted and summarized according to task type components and task type distribution. The weights are taken as the distribution proportion of the corresponding task type within the current window (the weights are determined by directly using the normalized count results from the task type distribution, ensuring that the sum of all weights is 1 and consistent with the current load pattern), thus obtaining the computational intensity feature characterizing the overall computing power pressure of the current window.

[0028] After generating the computational density feature, it is concatenated with the arrival frequency to construct the initial load vector. The concatenation process first scales the arrival frequency, with the scaling boundary determined by the sliding minimum and maximum values ​​of historical arrival frequencies (the boundary is determined by maintaining the arrival frequency sequence at the time window length, with the sliding range consistent with the sliding range of the aforementioned profile boundary, ensuring consistent time coverage of different feature sources). Then, the scaled arrival frequency and computational density feature are combined in a fixed order, determined during system initialization and maintained consistently (the fixed order is determined by placing features reflecting load scale first, followed by features reflecting load intensity; this order is recorded by the monitoring module configuration to ensure consistent input dimensions for subsequent time-series extrapolation). The initial load vector undergoes integrity verification before output, including dimensional consistency and the validity of the value range. The upper and lower bounds of the value range are defined by the scaling boundaries of each feature (the boundary definition is as follows: after scaling, the range is limited to 0 to 1; values ​​exceeding the range are truncated at the boundary, and truncation triggers anomaly counting for subsequent diagnosis).

[0029] The initial load vector is used as input to perform time-series evolution extrapolation to generate resource usage predictions including memory occupancy. The time-series evolution extrapolation uses an initial load vector sequence formed by multiple consecutive time windows as the input sequence. The length of the input sequence is determined by the load autocorrelation length (the input sequence length is determined by calculating the autocorrelation decay interval for historical arrival frequency and computational density sequences, selecting the number of windows before the autocorrelation significance disappears as the length, and trimming it together with system memory budget constraints). The extrapolation process first performs trend extraction and fluctuation extraction on the input sequence. Trend extraction is used to characterize the slow drift of the load, and fluctuation extraction is used to characterize short-term bursts. The separation of trend and fluctuation adopts a fixed-window smoothing and residual extraction process. The smoothing window length is determined by three times the time window length (the smoothing window length is determined by expanding the window to stabilize the trend without crossing the peak-valley transition of the business cycle; the three-fold factor is determined by the statistically stable interval of the kurtosis and skewness of the historical sequence). After obtaining the trends and fluctuations, the inference module generates resource usage predictions for several future time windows based on the correspondence between "load vector changes" and "memory usage changes" in the historical sequence. The prediction time span is consistent with the short-term execution cycle of the scheduling strategy (the prediction time span is determined by reading the shortest resource reallocation cycle of the GPU allocation engine and ensuring the prediction span covers at least one reallocation cycle but does not exceed the online update cycle, avoiding excessively long predictions that could amplify uncertainty). The memory usage rate is calculated by converting the predicted memory usage capacity into the total memory capacity of the device. The memory usage capacity comes from the sum of allocated capacity and cache resident capacity reported by the driver side, and the total memory capacity comes from the device's static attribute table (the static attribute table is determined by reading and recording device information during node registration and triggering a refresh when a node undergoes hardware changes). The converted memory usage rate is also limited to the range of 0 to 1 and is output along with the resource usage predictions.

[0030] The initial load vector sequence refers to a multi-dimensional vector sequence formed by collecting various indicators of the computing task or system load (such as CPU utilization, task queue length, and video memory usage) over multiple consecutive time windows. The length of the input sequence can be determined based on the calculation result of the load autocorrelation function; that is, the number of time windows before the autocorrelation significance disappears is used as the sequence length to ensure that the sequence can reflect the dynamic changes in the system load. The method for determining before the autocorrelation significance disappears includes calculating the autocorrelation coefficient of the load vector sequence. When the autocorrelation coefficient drops below a preset threshold, it is considered that the significance has disappeared, and this data is obtained through conventional statistical analysis methods.

[0031] A historical sequence refers to the data set used to establish the relationship between load vector changes and memory usage changes during the simulation process. To clarify the meaning of "historical sequence," this invention defines it as a time series set containing the initial load vector sequence and its corresponding memory usage records. In this historical sequence, a one-to-one correspondence is formed between the load vector of each time window and the corresponding memory usage value, enabling the simulation module to predict resource usage in future time windows based on historical change patterns. Actual data on the initial load vector and memory usage are obtained through system monitoring tools or log collection modules.

[0032] A computational intensity sequence is a sequence used to evaluate the computational load intensity within each time window. It is typically obtained by statistically analyzing metrics such as the number of task executions, CPU / GPU utilization, or number of operations. This sequence can help determine the length of the input sequence and the trend of load changes, thereby improving prediction accuracy. The necessary data is collected and this sequence is generated using performance monitoring tools.

[0033] S2 includes obtaining the initial load vector to be analyzed, encoding the initial load vector using a deep neural network model to generate a high-dimensional load feature tensor; calculating the similarity score between the high-dimensional load feature tensor and the historical standard vector, and determining the matching historical load pattern if the similarity score exceeds a preset threshold; extracting stable state information containing memory allocation peaks based on the historical load pattern; and correcting the baseline parameters based on the stable state information to determine the stable state basis for GPU resource usage prediction.

[0034] In this implementation, the initial load vector to be analyzed is first received by the resource usage prediction module and preprocessed for consistency. This preprocessing revolves around dimensionality consistency and scale consistency. Dimensionality consistency means that the order of each component of the initial load vector is consistent with the feature order used in the training phase. The order is determined based on the definition order and feature concatenation order of the features output by the monitoring module. These two are bound in the system initialization configuration and remain unchanged in subsequent operation. Scale consistency means that the values ​​of each component are within a uniform numerical range. The range is determined by maintaining the sliding minimum and sliding maximum values ​​of each component based on historical operating data. The sliding range length is an integer multiple of the cluster business cycle. The integer multiple is obtained by statistically analyzing the repetition cycle of historical load within a week or day, so that the scale boundary covers typical peak and valley changes. When a component exceeds the boundary, boundary truncation is performed and an anomaly count is recorded. The anomaly count is used for subsequent threshold adaptive adjustment. After the above preprocessing is completed, the initial load vector enters the encoding path of the deep neural network model. The encoding path consists of multiple layers of feature extraction connected in series. Each layer of feature extraction performs linear mapping, nonlinear activation, and normalization stabilization. The parameters of the linear mapping are taken from the weight set after convergence during the offline training phase. The weight set is determined by using historical load patterns and their corresponding stable state labels as supervision signals, adopting batch training, and using the minimum loss on the validation set as the stopping criterion. The validation set is extracted from samples in multiple recent business cycles in blocks according to time to ensure coverage of peak, off-peak, and trough periods. The selection of nonlinear activation is based on the sparsity and discriminative statistics of the encoded output. Excessive sparsity will cause similarity concentration, and insufficient discriminativeness will cause an increase in mismatches. Therefore, during the training phase, the activation form is constrained by both the mismatch rate and inference latency. The encoded output forms a high-dimensional load feature tensor. The high-dimensional dimension is determined as follows: under the premise of meeting the upper limit of online inference latency, the dimension is gradually increased with the load pattern separability of historical samples as the target until the load pattern discrimination on the validation set slows down. The upper limit of online inference latency is obtained by deducting the time spent on monitoring and data preprocessing from the scheduling decision cycle, ensuring that encoding and similarity calculation are completed within a single decision cycle.

[0035] After obtaining the high-dimensional load feature tensor, the system initiates a similarity retrieval process, aligning the high-dimensional load feature tensor with the historical standard vector library item by item to calculate the similarity score. The historical standard vector library is obtained by summarizing the encoding results of historical load patterns within stable operating intervals. The stable operating interval is determined by: sliding and scanning historical data in a fixed-length time window, and statistically analyzing the fluctuation range of memory usage, computing unit usage, and task completion time within each window. The threshold for judging the fluctuation range is taken as the upper quantile of the historical noise level. The historical noise level is obtained by statistically analyzing periods without sudden events, and periods with sudden events are filtered through alarm records and anomaly counts. When multiple consecutive windows meet the condition that the fluctuation range is below the threshold, the continuous interval is marked as a stable operating interval, and the encoding results within the interval are summed by time weighting to obtain the historical standard vector of that load pattern. The similarity score calculation process employs a consistent normalization alignment strategy: First, amplitude normalization is performed on the current high-dimensional load feature tensor and each historical standard vector, ensuring that different samples can still be compared in terms of structural similarity despite overall intensity differences. The normalization scale is the cumulative intensity of the absolute values ​​of each component, obtained by summing the absolute values ​​of all components. In the engineering implementation, the summation process is completed in blocks, with the block size determined by hardware cache capacity and parallelism constraints. After normalization, the corresponding products are calculated for each component and accumulated to form the original similarity value. The accumulation order adopts a fixed order to avoid numerical drift, arranged in ascending order of component index. Subsequently, the original similarity value is mapped to a unified range to form the similarity score. The upper and lower bounds of the mapping range are determined by the minimum and maximum original similarity values ​​observed on the validation set during the training phase. In online operation, it is updated using a sliding window, with the sliding window length taken as the median of the stable operating interval length, ensuring that the update reflects recent changes while avoiding short-term spikes.

[0036] After the similarity score is calculated, the system performs a matching judgment based on a preset threshold. The threshold is determined as follows: when building the historical standard vector library offline, the similarity score distributions for "pairing under the same load mode" and "pairing under different load modes" are statistically analyzed separately. The threshold is selected with the goal of controlling the risk of mismatches, ensuring that the proportion of different load modes judged as matches is lower than the target upper limit. This target upper limit is determined by the cluster's tolerance for scheduling errors, which is derived by back-calculating the historical number of scheduling rollbacks and the performance degradation alarm trigger threshold. When the business scenario focuses more on recalling stable state information, the threshold is adjusted downwards, and the adjustment range is determined by the trade-off curve between the missed match rate and the mismatch rate on the validation set. During online operation, the threshold also undergoes drift correction. Drift correction is based on the overall distribution change of the similarity score in the most recent business cycle. Correction is triggered when the distribution mean or distribution dispersion exceeds the drift threshold. The drift threshold is determined by statistically analyzing the natural fluctuation range of the similarity score distribution without policy changes and taking its upper quantile as the threshold. If the current similarity score exceeds the threshold, the system determines the historical load pattern corresponding to the historical standard vector with the highest score as the matching result, and locates the index of the historical load pattern to the stable state information set. The location process adopts a one-to-one binding relationship, which is generated synchronously when building the historical standard vector library. The generation is based on the joint key of the time identifier and the load pattern identifier of the stable operation interval. The joint key is the hash result of the task type distribution summary and the arrival frequency summary. The input summary of the hash is obtained by fixed-length bins. The bin length is determined by the time window length to ensure that the summary remains consistent under the same load pattern.

[0037] After matching the historical load pattern, the system extracts stable state information segments containing memory allocation peaks from the stable state information. The calculation process of memory allocation peaks revolves around the "reproducible upper bound of allocation amount": within the stable operating range corresponding to the historical load pattern, the sequence of allocated memory capacity is read according to the sampling period, and peak filtering is performed on the sequence. The threshold for peak filtering is jointly determined by the median and dispersion statistics of the sequence. The dispersion statistics are characterized by quantile differences, and the threshold is the median plus a certain multiple of the quantile difference. The multiple is determined by the historical misjudged peak ratio constraint, so that the number of filtered points does not exceed the set ratio. The set ratio is obtained by jointly evaluating the sampling noise level of the monitoring module and the jitter level reported by the driver. After filtering, the maximum value from the remaining sequence is selected as the memory allocation peak, and the duration of the window before and after the peak is recorded. The duration of the window is used to characterize whether the peak is a short pulse or a sustained pressure. The window length is determined by expanding forward and backward from the peak point until the allocated memory capacity drops below a certain percentage of the peak. This percentage is obtained by statistically analyzing the peak drop rate within the historical stable interval, taking the percentage corresponding to the median level of the drop rate. In addition to the memory allocation peak, the stable state information also includes a summary of the task concurrency scale and a summary of the memory reclamation rate corresponding to the peak. The summaries are calculated from the sequence of concurrent task counts and the sequence of memory release events within the stable interval, respectively. The release event sequence is obtained by jointly determining the task end event and the decrease in memory usage. The time alignment error of the joint determination is half of the sampling period, and the error value is based on the maximum deviation caused by the discretization of the sampling time.

[0038] Based on the aforementioned stable state information, the system corrects the baseline parameters, forming a stable state foundation for subsequent resource usage prediction. The baseline parameters include at least the memory safety margin, the upper bound parameter for memory allocation, and the prediction drift correction parameter. The correction of the memory safety margin is based on the percentage difference between the peak memory allocation and the total memory capacity of the device. The percentage difference is calculated with the total memory capacity of the device as the denominator. The total memory capacity is obtained from the device's static attribute registration, which involves reading the driver report and verifying it before storage when the node comes online. When the percentage difference is lower than the safety threshold, the memory safety margin is increased. The safety threshold is determined by statistically analyzing the percentage difference before historical memory allocation failures or memory jitter alarms, and taking the upper quantile of this percentage as the threshold, thus increasing the safety margin before risks occur. The correction of the upper bound parameter for memory allocation directly uses the peak value of memory allocation as the upper bound reference, and adjusts the conservatism of the upper bound in conjunction with the duration of the peak value window. The longer the duration window, the closer the upper bound is to the peak value itself; the shorter the duration window, the higher the buffer introduced by the upper bound on the peak value. The buffer amplitude is determined by taking the buffer amplitude corresponding to the slower-falling interval based on the distribution of the fall rate of historical short pulse peaks, to avoid short pulses being mistaken for risk-free spikes. The prediction drift correction parameter is used to constrain the offset of subsequent resource usage predictions under the stable boundary. The correction amplitude is determined by statistically analyzing the deviation sequence of "predicted value and actual value" in the stable operating interval of this historical load mode. The deviation sequence is obtained by subtracting the predicted memory utilization rate and the actual memory utilization rate aligned to the same timestamp, with the alignment error taken as half of the sampling period. Then, the upper quantile of the deviation sequence is used as the upper limit of correction, and the lower quantile of the deviation sequence is used as the lower limit of correction, and the correction interval is mapped to the pruning boundary of the subsequent prediction output. After the above corrections are completed, the stable state basis consists of the corrected baseline parameters and stable state information, and serves as the stable constraint input for the subsequent GPU resource usage prediction process.

[0039] S3 includes acquiring real-time operational data to extract load characteristics, matching the load characteristics with historical benchmark data to obtain the characteristic deviation; if the characteristic deviation exceeds the verification threshold, generating a new load mode switching instruction, and capturing a system state snapshot based on the switching instruction; performing integrity verification on the system state snapshot to generate a state missing description file, obtaining a completion strategy based on the state missing description file and synthesizing a state set to be processed; and transmitting the state set to be processed to the recovery queue to start the incomplete state information processing procedure under the new load mode switching.

[0040] In this implementation, real-time operational data of the cluster is first acquired to extract load characteristics. The real-time operational data is aggregated by the monitoring module within a unified sampling period. The sampling period is determined based on the minimum decision interval of the scheduling decision. This is achieved by reading the minimum decision interval in the scheduler configuration and selecting an integer multiple of this interval as the sampling period, ensuring that the sampling period covers a complete scheduling observation window without introducing significant lag. Within each sampling period, the monitoring module collects task submission events, task start events, task end events, queue length, number of concurrent tasks, graphics processor utilization, allocated video memory capacity, reclaimable video memory capacity, video memory fragmentation indicators, node health alarms, and driver reporting latency. After the above data enters the feature extraction process, time alignment is first performed. Time alignment uses the sampling period boundary to merge event timestamps, aggregating events falling within the same sampling period into the same time slice. Then, deduplication and consistency cleaning are performed. Deduplication is based on the combined key formed by the unique task identifier and event type. Cleaning is based on rules for field integrity and value range validity, which are directly derived from the monitoring protocol field definitions. Missing fields trigger a missing flag and enter the subsequent integrity verification stage. After cleaning, the load characteristics are vectorized within the same time slice. Each vector component includes at least the task type percentage, arrival frequency, computational pressure, concurrency, queuing pressure, memory utilization, memory fragmentation, and recycling rate. The calculation of each component uses a unified time slice aggregation method: the task type percentage is obtained by dividing the count of each task type within the time slice by the total task count; the arrival frequency is converted from the submission event count within the time slice to the arrival rate per unit time according to the sampling period; the computational pressure is obtained by weighted summation of the computational density profiles of different task types within the time slice, based on the type percentage. These profiles are derived from statistical aggregation of historical stable periods and updated on a rolling basis according to the business cycle; concurrency is directly obtained by statistically analyzing the number of running tasks within the time slice; queuing pressure is obtained by weighted summation of queue length and queue waiting time within the time slice, with the weighting coefficient determined by the ranking of queuing correlation in historical performance degradation events and normalized to a sum of 1; and memory utilization... The fragmentation rate is obtained by dividing the allocated video memory capacity by the total video memory capacity of the device, which is taken from the static device attributes read during node registration. The fragmentation rate is constructed by the number of free video memory blocks, the scale distribution of free blocks, and the percentage of the largest contiguous free blocks. The construction process first counts the list of free blocks, then calculates the number of free blocks and the percentage of the largest contiguous free blocks, and maps the scale distribution of free blocks into a discrete interval percentage vector. The recycling rate is obtained by dividing the amount of video memory released within the time slice by the sampling period. The amount of video memory released is determined by the decrease in the allocated video memory capacity and the time alignment relationship of the task end event. The maximum allowable deviation of the time alignment is half of the sampling period, and the value is determined based on the maximum deviation limit caused by the discretization of the sampling time.

[0041] After obtaining the load characteristics, the system matches these characteristics with historical benchmark data to determine the feature deviation. The historical benchmark data is organized based on the matching principle of "consistency between business cycle and operating conditions." The benchmark selection process first determines the business cycle segment to which the current time slice belongs. The division of business cycle segments is determined based on the periodic statistics of historical arrival frequencies. The statistical method involves periodically aligning the arrival sequences of multiple consecutive business cycles and extracting stable intervals for peak and valley positions. Subsequently, a benchmark set consistent with the current operating conditions is retrieved within this business cycle segment. Operating conditions include node size, equipment model, driver version, and scheduling strategy version. The matching rules require these conditions to remain consistent, thereby avoiding misjudgments caused by benchmark drift. For each load feature component, the system extracts the center level and fluctuation scale of that component within the benchmark set. The center level is the median level to resist the influence of abnormal peaks, and the fluctuation scale is the quantile difference to characterize typical fluctuation intervals. The upper and lower quantile points of the quantile difference are determined by the sample size. In scenarios with a small sample size, a more conservative quantile difference is used to avoid the scale being amplified by occasional fluctuations. The deviation strength of each component is then calculated. The calculation process involves taking the difference between the current component and the baseline center level, and scaling it using the baseline fluctuation scale to ensure all components are on a comparable scale. After calculating the deviation strength of each component, the system performs a weighted aggregation based on component importance to obtain the feature deviation degree. The component importance is determined by: reviewing historical load mode switching events and performance degradation events, and ranking the contribution of each component's deviation strength to the event triggering. The contribution ranking is calculated based on the correlation between the component's deviation strength and the event occurrence, with the correlation using a joint constraint of monotonicity and consistency to avoid accidental correlations. The ranking is then mapped to weights and normalized to a sum of 1. The normalization process is completed by dividing each weight by the sum of the weights, which is derived from the cumulative value of all component weights.

[0042] When the feature deviation exceeds the verification threshold, the system generates a new load mode switching instruction and captures a system status snapshot based on the switching instruction. The verification threshold is determined as follows: During the offline phase, the feature deviation sequences of the "stable running period" and the "confirmed switching event period" are aggregated respectively. The stable running period is marked by time periods with no alarms and low fluctuations in key indicators. The switching event period is jointly marked by scheduling rollback records, performance degradation alarms, and manual confirmation records. Then, a false trigger risk target is set. The false trigger risk target is taken from the upper limit of the switching instruction trigger frequency allowed by operation and maintenance. The upper limit of the frequency is determined based on the resource overhead of the recovery process and the acceptable disturbance of the business. Then, the upper quantile that meets the false trigger risk target in the stable running deviation sequence is selected as the verification threshold, and the missed detection ratio is verified to meet the requirements on the switching event deviation sequence. The upper limit of the missed detection ratio is determined based on the impact assessment of the prediction error after switching on the scheduling benefits. During the online operation phase, the verification threshold is also subject to drift adjustment. This drift adjustment is based on whether the distribution of stable operational deviations within the most recent business cycle has shifted upwards or spreads overall. The threshold for determining overall upward shift and overall spread is determined by the upper quantile of the natural fluctuation range when there are no version changes in the past. After the switch command is generated, snapshot capture is performed according to the principle of "consistency at the same point in time." The capture trigger time is the end of the time slice where the verification threshold is first exceeded. The captured content includes the scheduling queue status, the list of running tasks, the details of task resource requests, device occupancy mapping, the video memory allocation table, the list of free video memory blocks, node health status, driver reporting latency statistics, and a summary of container runtime status. To ensure consistency, the same time slice identifier is applied to all data sources during snapshot capture. Data sources that fail to return within that time slice are marked as timeout missing and enter the missing description process.

[0043] After obtaining a system state snapshot, the snapshot is subjected to integrity verification to generate a state missing description file. Based on the state missing description file, a completion strategy is obtained and a set of states to be processed is synthesized. Integrity verification is divided into structural integrity and semantic integrity: Structural integrity verification checks whether the snapshot contains a predefined set of fields. The predefined set of fields is determined by compiling a list of minimum inputs required for subsequent state correction and scheduling decisions, constrained by dependency analysis results. Dependency analysis is obtained by tracing input references from the prediction and scheduling modules. Semantic integrity verification checks the field value range, field association consistency, and timestamp consistency. The value range is determined by the device's static attributes and the upper and lower bounds of the monitoring protocol. Association consistency is achieved through cross-checking; for example, the allocated capacity of video memory must not exceed the total capacity of the device's video memory, and the total resources in the running task list must not exceed the total resource usage displayed by the device's occupation mapping. Timestamp consistency requires all fields to belong to the same time slice identifier. After verification, a state missing description file is generated. The file content includes a list of missing fields, the module to which the missing fields belong, the classification of missing reasons, the severity of the missing fields, and the scope of the missing fields. The severity of missing data is calculated based on field importance and missing data type. Field importance is derived from the critical path weights in dependency analysis. The critical path weights are determined by statistically analyzing the historical impact of missing fields on prediction errors and scheduling failures, sorting them by impact magnitude, mapping them to weights, and then normalizing the sum to 1. Missing data types are categorized into complete missing, timed missing, illegal range, and association conflict. The type determination is directly determined by the triggering results of structural integrity and semantic integrity. Missing data severity is obtained by weighted summation of the importance of missing fields under different missing data types. The weighting coefficient is jointly determined by the historical recovery success rate and recovery time statistics, thus reducing the priority of completion paths with high time consumption and low benefits. Based on the missing state description file, the system retrieves and completes the data according to a "backfill first, then infer" approach: For fields with traceable historical sources, the most recent valid snapshot is used for backfilling. The search window length for the most recent valid snapshot is determined by the data freshness requirement, which is a multiple of the scheduling decision cycle. The window length is verified by statistical analysis of historical snapshot drift rates to ensure it does not introduce significant bias. For fields with similar nodes that can be referenced, isomorphic node mapping is used for backfilling. The determination of isomorphic nodes is based on device model, driver version, and load characteristic similarity. The similarity calculation process is consistent with the component scaling of the aforementioned deviation. Several nodes with the highest similarity are selected and backfilled at the median level to suppress single-point anomalies. For fields that cannot be obtained through backfilling, constrained inference is used. The inference process uses the available fields in the current snapshot as the constraint boundary. For example, when the video memory allocation table is missing, the inference interval is jointly limited by the total video memory request amount of the running task list and the known value of the video memory allocated capacity. The inference output is the value that is closest to the historical same load pattern statistics within the interval. The selection criteria for the historical same load pattern are that the task type proportion and the arrival frequency fall into the same business cycle segment.After completion, a set of states to be processed is formed. The set of states to be processed is obtained by merging the original snapshot field and the completed field. Each completed field is attached with a source identifier and a confidence identifier. The source identifier is the backfill source type or the inferred source type. The confidence identifier is determined by the error distribution statistics of this completion strategy in history. The error distribution statistics are obtained by comparing the difference between the completed value and the subsequent true value. Then, the confidence identifiers of each field are aggregated into the confidence level of the state set. The aggregation rule adopts the method of "taking the minimum confidence for key fields and taking the weighted aggregation for non-key fields". The set of key fields is determined by the critical path of the dependency relationship, so that the confidence level of the state set remains sensitive to key missing values.

[0044] When transmitting a set of states to be processed to the recovery queue to initiate the incomplete state information handling procedure, the system first generates a queue priority for the set of states to be processed. The priority is determined based on factors including the magnitude of feature deviation, the severity of missing information, the confidence level of the state set, the number of affected devices, and the number of affected tasks. The priority calculation process first performs scaling on the above indicators, with the scaling boundary determined by the sliding minimum and sliding maximum values ​​from historical operations. The sliding range length is taken as one business cycle to closely reflect the recent situation. Then, a priority score is formed by weighted summation according to the importance of the indicators. The importance of the indicators is determined by ranking them according to the ratio of recovery benefit to recovery time. The recovery benefit is measured by the combined decrease in prediction error and the decrease in scheduling failure rate, while the recovery time is measured by the median processing time of the recovery procedure under the same type of missing information in history. When the priority score is higher, the state set is placed at a higher position in the queue, thus prioritizing the processing of higher-risk state sets during sudden switches. The transmission process adopts a delivery mechanism with acknowledgment. The timeout threshold for delivery acknowledgment is determined by the response latency percentile of the queue service under peak load and is updated with the queue latency distribution during online operation. If a delivery timeout occurs, the system regenerates the status set delivery record and triggers an alarm. The alarm trigger threshold is determined by the upper limit of the number of consecutive delivery failures, and the upper limit of the number of failures is set according to the fault tolerance requirements and alarm noise control requirements.

[0045] S4 includes extracting instruction stream execution progress data and a video memory physical address mapping table; calculating the remaining execution time value based on the instruction stream execution progress data; constructing a video memory fragmentation distribution scan map based on the video memory physical address mapping table; performing joint analysis on the remaining execution time value and the video memory fragmentation distribution scan map to output a set of uncertainty parameters; inputting the set of uncertainty parameters into an error evaluation function to obtain a prediction deviation quantization value; generating an adjustment coefficient matrix based on the prediction deviation quantization value; and using the adjustment coefficient matrix to perform weighted correction on the initial prediction data to obtain the adjusted resource state matrix.

[0046] In this embodiment, instruction stream execution progress data and a physical address mapping table for video memory are first extracted from the graphics processor's runtime environment. The collection of instruction stream execution progress data is organized by task dimension. The collection sources are the driver-side run count, kernel scheduling completion report, task stage identifier, and batch progress count. The collection period is an integer multiple of the scheduling decision period (the method for determining this integer multiple is: statistically analyzing the jitter amplitude of historical progress data at different sampling intervals, and selecting the smallest integer multiple where the jitter amplitude enters a stable range without introducing significant lag). Within the same collection period, data from different collection sources are time-aligned. The time alignment adopts the principle of "taking the end of the collection period as a unified reference point." Data earlier than the reference point is taken as the most recent valid value, and data later than the reference point is marked as timeout and missing, triggering subsequent uncertainty amplification processing. After alignment, consistency checks are performed on the progress data. The consistency checks cover the consistency of task identifiers, the validity of stage identifiers, the monotonically increasing progress, and the interpretability of rollback events. When a progress rollback occurs, the cause of the rollback is attributed based on the logs of three types of events: restart, retry, and migration. Rollbacks that cannot be attributed directly increase the strength of the progress uncertainty marker. The value of the marker strength is determined based on the distribution of the remaining time misjudgment amplitude caused by historical rollback events. The statistical scope of the misjudgment amplitude distribution is the prediction deviation sequence of similar tasks in the same stage.

[0047] After obtaining qualified instruction stream execution progress data, the system calculates the remaining execution time. The calculation process first determines the "executed time", which is the time difference between the task start time and the current reference point. The start time comes from the first valid record of the task start event. In the task migration scenario, the executed time before migration and the executed time after migration are continuously accumulated, with the accumulation rule based on the start and end times of the migration event. Subsequently, the "progress completion ratio" is determined, which is the ratio of the current completed workload to the total expected workload. The completed workload is the weighted sum of the number of batches completed, the number of kernel commits completed, and the number of key operators completed in the current stage. The weighting coefficients are determined by statistically analyzing the correlation strength between each count and the actual time consumption in historical samples, sorting them by correlation strength, mapping them to weights, and normalizing them so that the weights sum to 1. The total expected workload is the joint estimate of the total number of batches declared in the task metadata, the total number of key operators derived from the model structure, and the average total number of kernel commits in historical similar task profiles. The joint estimate is determined by prioritizing the explicit declaration of task metadata. When the explicit declaration is missing or inconsistent with the running count for a long time, the median level of the historical profile is used and proportionally corrected according to the current input scale. The input scale is a combined summary of batch size and sequence length, and the value of the summary comes from the configuration parameters at task startup. After obtaining the completion rate, the system generates the "remaining rate," which is the complement of the completion rate. A minimum effective threshold is set for the completion rate. The minimum effective threshold is determined by statistically analyzing the progress fluctuations caused by compilation and warm-up during the initial task startup. The upper quantile of the completion rate before the fluctuation ends is selected as the threshold to avoid abnormally amplified remaining time under extremely low completion rates. Finally, the remaining execution time value is extrapolated from the executed time using the ratio of the remaining rate to the completion rate. A phase correction is applied to the extrapolation result, based on the difference between the time consumption ratio of the current phase and the historical profile phases. When the time consumption ratio of the current phase is higher than the historical median, the conservatism of the remaining time is increased. The increase is determined by taking the position of the deviation of the time consumption ratio of this phase in the historical distribution, mapping it to a conservative increment, and setting an upper limit for the conservative increment. The upper limit is taken as the upper quantile of the increment when similar tasks in the same phase experienced a long tail at the end, preventing single abnormalities from dominating.

[0048] In parallel with the calculation of remaining execution time, the system constructs a memory fragmentation distribution scan map based on the memory physical address mapping table. The extraction of the memory physical address mapping table covers the start address, length, task identifier, allocation time, release flag, memory pool ownership, and page granularity mapping information of allocated segments, and freezes the acquisition at the same reference point. The freeze acquisition adopts the order of "first pausing the metadata update of the allocation path, then capturing the mapping table snapshot, and then resuming the update". The upper limit of the pause duration is determined by the quantile of the metadata lock holding time tolerated by the driver side. If the upper limit is exceeded, the pause is abandoned and double read verification is used instead. Double read verification obtains the variation range by continuously capturing the mapping table twice and comparing the differences. The variation range is marked as an unstable region and reflected in the subsequent uncertainty parameters. When constructing the scan map, the scan granularity is first determined. The scan granularity is the larger of the memory paging granularity and the allocator's minimum block granularity (this larger granularity is determined by reading the paging granularity from the driver report and the minimum block granularity from the allocator configuration, selecting the coarser granularity to reduce scan overhead and ensure block boundary alignment). Then, the memory space is divided into contiguous segments according to address from low to high. Each segment is marked as occupied or free. The marking rule is based on whether the allocation segment covering the segment exists in the mapping table. After completing the occupancy marking, the free segments are continuously merged. Continuous merging is based on the condition that adjacent segments are all free to form free blocks. The scale statistics of the free blocks are generated: the total number of free blocks, the number of free blocks, and the maximum capacity of the contiguous free block. The free blocks are then binned by capacity to form a capacity distribution vector. The bin boundaries are determined by collecting the distribution of memory requests from recent tasks and taking several percentiles of the request volume as bin boundaries, so that the binning can cover the high-frequency request scale and the large block request scale. Based on the above statistics, fragmentation is characterized by a set of multiple indicators, including "the percentage of the largest consecutive free blocks in the total free space," "the number density of free blocks," "the percentage of large free blocks," "the percentage of small free blocks," and "the percentage of unstable regions." Each indicator is normalized within the range of 0 to 1, with the normalization boundary being the natural upper bound of the total memory capacity and the total free space. To characterize the changes in fragmentation over time, the system repeatedly constructs scan maps at multiple consecutive reference points and calculates the rate of change of the indicators. The time interval for the rate of change is taken from the aforementioned acquisition period, and the abnormal threshold for the rate of change is taken from the upper quantile of the rate of change during historical stable periods. If the threshold is exceeded, the fragmentation state is considered to fluctuate significantly, and the uncertainty intensity is increased.

[0049] After obtaining the remaining execution time and the memory fragmentation distribution map, the system performs joint analysis and outputs a set of uncertainty parameters. The joint analysis unfolds along two main lines: "resource release timing" and "allocatable continuity." First, a resource release schedule is constructed based on the remaining execution time. The resource release schedule records the expected release time on a task-by-task basis. The expected release time is obtained by adding the remaining execution time to the current reference point. The schedule is then corrected by incorporating the memory residency characteristics of the task. The residency characteristics are taken as the median level of memory reclamation latency after the completion of similar tasks in history. The latency statistics are derived from the alignment difference between the task completion event and the memory usage decrease event. Subsequently, a "request feasibility profile" is constructed based on the memory fragmentation scan map. The profile takes typical request scales from recent tasks as input, examines each free block in the current free block set to see if there are any free blocks that meet the continuous capacity requirements, and calculates the feasibility ratio. The calculation method for the feasibility ratio is as follows: the sample of typical request scales is taken from the memory request volume summary of recently submitted tasks. The summary is aggregated according to bin boundaries, and the top several scales are extracted as typical scales according to the aggregation weight. The extraction quantity is determined by ensuring that the cumulative percentage of typical scales covering the total recent requests reaches a preset coverage threshold. The coverage threshold is taken as the median level of the cumulative percentage before the occurrence of historical allocation failure events, making the profile more sensitive to high-risk scales. After aligning the resource release schedule with the request feasibility profile, the set of uncertainty parameters includes at least the following: remaining execution time stability, remaining execution time dispersion, release time congestion, fragmentation severity, fragmentation volatility, continuous feasibility ratio, mapping table instability ratio, and reclamation delay risk. The determination process for each parameter is as follows: The stability of remaining execution time is a comprehensive evaluation of the monotonicity and slope consistency of the remaining execution time series over the most recent sampling periods. The evaluation window length is the median of a typical period for task phase switching, obtained statistically from the historical phase switching intervals of similar tasks. The dispersion of remaining execution time is the fluctuation range of the remaining execution time within the same window, using the upper quantile of the fluctuation in historical stable windows as a reference to ensure the dispersion is close to a low value within the stable range. The congestion of release time is the proportion of tasks expected to fall into the same time slice within the next few sampling periods. The time slice length is the sampling period, and the future outlook span is the short-term decision span of the scheduling engine. The degree of short-term decision span is determined by the scheduling strategy update rhythm; the fragmentation severity is a joint evaluation of the largest consecutive free block ratio and the number density of free blocks, and the joint evaluation weight is the result of the contribution ranking mapping of the two in historical memory allocation failure events; the fragmentation volatility is the fluctuation amplitude of the fragmentation index change rate, and is normalized to the quantile of the stable period; the continuous feasible ratio is the proportion of consecutive free blocks that can be found in the typical application scale; the mapping table instability ratio is the proportion of the aforementioned double read check difference segment to the total memory capacity; the recovery latency risk is the position of the memory usage decrease latency in the recent distribution after the task ends, and is mapped to the risk level, with the mapping boundary being the quantile of the latency distribution before the historical alarm is triggered.

[0050] Subsequently, the system inputs the set of uncertainty parameters into the error evaluation function to obtain the quantified value of the prediction deviation, and generates an adjustment coefficient matrix based on the quantified value of the prediction deviation. In engineering implementation, the error evaluation function is driven by historical error samples. These historical error samples are formed by the alignment differences between the "initial prediction data" and the "actual observation data," with the alignment based on the same reference point and the same acquisition period. The actual observation data comes from the monitoring module's capture of computing unit usage, video memory usage, available continuous video memory capacity, task concurrency, and queue pressure at that reference point. To ensure the evaluation results have directionality, the system decomposes historical errors into two categories: "underestimation of risk" and "overestimation of risk." The classification rule is based on the offset direction of the predicted value relative to the actual value, and statistics are compiled separately according to resource dimensions. The resource dimension includes at least video memory usage, available video memory, available continuous capacity, and computing usage. The error assessment function is determined using two parts: "sensitivity weights" and "benchmark offsets." Sensitivity weights are determined by fixing other uncertainty parameters within their normal range in historical samples, observing only the trend of the absolute magnitude of the deviation when a certain uncertainty parameter increases, assigning weights to the parameter according to the strength of the trend, and then normalizing the weights to a sum of 1. Benchmark offsets are determined by statistically analyzing the median level of the deviation in samples where the uncertainty parameter set is in a low-value range, using this as a systematic offset baseline to offset long-standing observation lags and sampling noise. The output of the predicted deviation quantification value is given separately for each resource dimension. The output process is as follows: for each resource dimension, the uncertainty parameters are weighted according to their corresponding sensitivity weights to form the deviation intensity, which is then superimposed with the benchmark offset. The correction direction is then given based on the classification results of underestimating and overestimating risk. To prevent overshooting due to a single anomaly, the system sets upper and lower bounds for the deviation intensity. The upper bound is taken as the upper quantile of the deviation intensity of similar historical scenarios, and the lower bound is taken as the lower quantile. The quantile values ​​are determined based on the sample size; when the sample size is insufficient, a more conservative quantile range is used to avoid boundary drift.

[0051] The adjustment coefficient matrix is ​​generated based on the principle of "consistency between matrix dimensions and resource status representation." The row and column organization of the adjustment coefficient matrix is ​​consistent with the organization of the initial prediction data, covering at least the device dimension, time slice dimension, and resource dimension. Each matrix element is first set to a baseline coefficient of 1, and then adjusted according to the quantified prediction deviation value of the corresponding resource dimension: when the quantified deviation value points to underestimating the risk, the coefficient of the occupied related item is increased and the coefficient of the available related item is decreased, making the prediction more conservative; when the quantified deviation value points to overestimating the risk, the coefficient of the occupied related item is decreased and the coefficient of the available related item is increased, so that the prediction avoids being overly conservative. The magnitude of the coefficient change is obtained by mapping the deviation intensity, and the mapping adopts a segmented approach: a smaller adjustment magnitude is used when the deviation intensity falls into the low interval, a medium adjustment magnitude is used when it falls into the middle interval, and a larger adjustment magnitude is used when it falls into the high interval; the interval boundaries are determined by statistically analyzing several quantiles of the historical deviation intensity distribution, and using the quantiles as interval boundaries, so that the interval division conforms to the normal and extreme states of the actual deviation. To control numerical stability, upper and lower limits are set for the coefficients, with an upper limit of 1.2 and a lower limit of 0.8 (the upper and lower limits are determined by: reviewing the scheduling failure rate and resource waste rate of historical scheduling strategies under different correction magnitudes, selecting the maximum correction magnitude range where the failure rate does not increase significantly and the waste rate does not worsen significantly, and mapping the range boundary to the upper and lower limit values). The coefficients of multiple consecutive time slices are smoothed, and the smoothing window length is 3 times the acquisition period (the window length is determined by: statistically analyzing the jitter reduction effect and response lag of the coefficient sequence under different window lengths, and selecting the minimum window length where jitter reduction enters the stable range and response lag does not exceed the short-term decision span).

[0052] Finally, the system uses an adjustment coefficient matrix to weight and correct the initial prediction data, resulting in an adjusted resource state matrix. The correction process is performed element-wise: for occupancy-related prediction items, the initial prediction values ​​are amplified or reduced according to the corresponding coefficients; for availability-related prediction items, the initial prediction values ​​are adjusted inversely according to the corresponding coefficients, ensuring that occupancy and availability remain consistent under the same total constraint. After correction, the system performs constraint checks on the adjusted resource state matrix. These checks include at least the following: memory occupancy must not exceed the total memory capacity; available memory must not be negative; allocable contiguous capacity must not exceed the maximum contiguous free block capacity; and the sum of occupancy and availability for the same device in the same time slice must be consistent with the total. When minor boundary violations occur, boundary truncation is used, and relevant items are backfilled for consistency. The backfilling order prioritizes critical scheduling inputs. The set of critical inputs is determined by analyzing the dependencies of subsequent scheduling decisions on resource state matrix fields, prioritizing those with higher dependency strength. After verification, the adjusted resource state matrix is ​​output, enabling subsequent short-term scheduling strategy calculations to obtain state inputs that simultaneously reflect the remaining lifecycle of the task and the true allocability of memory fragments.

[0053] S5 includes acquiring the adjusted resource state matrix and short-term load fluctuation indicators, performing weighted feature fusion of the resource state matrix and short-term load fluctuation indicators to construct a multi-dimensional state vector, mapping the multi-dimensional state vector to the state space of reinforcement learning, iteratively calculating using the Q-learning algorithm to obtain a converged Q-value table, performing a greedy optimization operation based on the converged Q-value table to generate the optimal scheduling decision sequence, parsing the optimal scheduling decision sequence and converting it into a resource allocation instruction set, and forming a short-term scheduling strategy based on the GPU resource usage prediction results by executing the resource allocation instruction set.

[0054] In this embodiment, the adjusted resource state matrix and short-term load fluctuation index are first obtained, and then aligned with the same scheduling decision time point before entering the feature fusion process. The scheduling decision time point is the end time of one decision cycle of the scheduler. The time point is determined by using the internal clock trigger signal of the scheduler as the reference, aligning the monitoring sampling, state correction and scheduling decision to the same trigger boundary, so that the subsequent state vector and execution action are at the same time reference. The resource state matrix is ​​organized by device dimension and resource dimension, and includes at least the memory usage level, available memory level, allocable continuous capacity level, computing usage level and queue pressure summary of each device at the current time point and within several predicted time slices, and is accompanied by the state confidence level generated by the uncertainty correction stage; the short-term load fluctuation index is calculated with the same time granularity as the scheduling cycle, and the index includes at least arrival intensity fluctuation, concurrency fluctuation, memory usage fluctuation, queuing pressure fluctuation and fragmentation fluctuation. The calculation process of the short-term load fluctuation index takes the monitoring sequence of the most recent consecutive scheduling cycles as input. The input window length is the length of one short-term scheduling cycle group (the window length is determined by counting the number of durations from the start to the stabilization of historical load mutation events, and taking the median as the window length, so that the window covers the rising and falling segments of a typical mutation). Within this window, the change amplitude and change rate of each monitoring quantity are calculated: the change amplitude is the difference between the maximum and minimum values ​​within the window, and the change rate is the median level of the absolute value sequence of the differences between adjacent scheduling cycles. Then, the change amplitude and change rate are normalized. The normalization boundary is the sliding minimum and sliding maximum values ​​of the corresponding index during the stable operation period. The sliding range length is one business cycle, so that the normalization result is adapted to the intraday or weekly cycle changes. To avoid abnormal peaks dominating the fluctuation index, the system sets an abnormal pruning threshold for the rate of change. The abnormal pruning threshold is determined as follows: during the stable operation period, the upper quantile of the rate of change is counted and used as the threshold. Values ​​exceeding the threshold are truncated according to the threshold. At the same time, the number of truncations is included in the fluctuation confidence deduction item. The deduction range is determined by the statistical analysis of the prediction error amplification ratio when the historical truncation occurred.

[0055] After obtaining the resource state matrix and short-term load fluctuation indicators, the system performs weighted feature fusion to construct a multi-dimensional state vector. The weighted feature fusion follows the organizational principle of "state constraints first, fluctuation adjustment as a supplement." First, key resource components within the resource state matrix are scaled, with the scaling range uniformly limited to 0 to 1. The scaling boundaries are determined by the upper bound of the device's static capacity and the lower bound of 0. Boundaries involving video memory usage and available video memory are taken as the total video memory capacity of the device; boundaries involving computational usage are taken as the upper bound of computational unit usage, determined by the full load definition of the monitoring interface; boundaries involving allocable continuous capacity are taken as the upper bound of the maximum continuous free block capacity, which is statistically obtained from the maximum continuous free block count in the video memory fragmentation scan map. Subsequently, the short-term load fluctuation indicators are also scaled, with the scaling boundaries taken as the aforementioned fluctuation indicator normalization boundaries. After scaling, the resource state components and fluctuation components are concatenated in a predefined order to form the original state vector. The predefined order is determined as follows: first, the components directly related to security constraints are placed, including memory usage, available memory, allocable continuous capacity, and state confidence level; then, the components related to efficiency are placed, including computational usage and queue pressure summary; and finally, the fluctuation components are placed, so that the learning process prioritizes capturing the risk of hard constraint violation. The weighting process is executed after concatenation, achieved by multiplying each component by its corresponding weight. The weights are determined as follows: a contribution ranking is constructed based on historical scheduling results, and the contribution is measured by a comprehensive index of "the improvement in task completion time after action execution, the reduction in memory allocation failure rate, and the decrease in performance degradation alarms". For each component, the consistency strength between its change and the change in the comprehensive index is calculated, and the consistency strength is mapped to weights according to the ranking and normalized so that the weights sum to 1. A minimum weight lower limit is set for the state confidence level (the minimum weight lower limit is 0.1, determined by: backtracking scheduling failure cases in scenarios with missing information, statistically analyzing the explanatory contribution of the confidence level to the failure, and selecting the minimum lower limit that can significantly reduce the failure recurrence rate), so that low-confidence states are explicitly addressed during learning. After weighting, a multidimensional state vector is obtained. This multidimensional state vector and the reinforcement learning state space are discretely mapped to support tabular value storage. The discrete mapping process bins each component, with the number of bins chosen as the median of the number of resolvable levels for each component in both stable and fluctuating scenarios. (The number of bins is determined by clustering historical sequences, observing the saturation point where the number of cluster centers changes with the number of bins, and selecting a number of bins near the saturation point while maintaining a controlled state space size.) Bin boundaries are defined for each component, taking several quantiles from its historical distribution, allowing for finer resolution of common intervals and coarser resolution of extreme intervals. Bin indices from multiple components are combined in a fixed order to obtain discrete state identifiers, which are used to locate specific state nodes in the state space.

[0056] After mapping in the state space, the system calculates the converged value table using a learning iteration process. The value table initialization combines zero initialization with historical warm-start: when historical value tables exist for the same business cycle, they are loaded first as initial values, with loading conditions constrained by policy version consistency and device configuration consistency. The policy version and device configuration are determined based on the scheduling policy version number and the node device's static attribute summary, respectively. Zero initialization is used when the loading conditions are not met. The learning iteration is driven by a sequence of "state, action, reward, next state," generated from observations after the scheduler executes a resource allocation instruction set. The action set is defined to cover short-term executable resource allocation actions, with action granularity aligned with the GPU allocation engine interface, including at least tasks placed on a specified device, tasks migrated to a specified device, limiting the concurrency of a queue, triggering memory defragmentation, and delaying the execution of a certain type of task. The size of the action set is determined as follows: Atomic instructions supported by the GPU allocation engine are listed, and instructions that are not executable under the current device topology and permission constraints are removed. Removal rules are based on device reachability and resource availability. Device reachability is determined by network and topology policies, and resource availability is determined by the constraint verification results of the resource state matrix. The reward is calculated based on the scheduling target definition. The reward is obtained by a weighted summary of multiple indicators, which at least include changes in task completion time, the number of memory allocation failures, the number of performance degradation alarms, the improvement in device utilization, and changes in queue waiting time. The weights of each indicator are determined by the scheduling target priority configuration, which is determined by the service level target on the business side. Historical scheduling results are used to verify that the weight combination will not cause a deterioration in security indicators. To map the returns to a stable range of values ​​required for learning and updating, the system performs normalization and pruning on the returns. The normalization boundary is determined by the sliding minimum and sliding maximum values ​​of the historical return distribution, and the sliding range length is one business cycle. The upper bound of pruning is the upper quantile, and the lower bound of pruning is the lower quantile. The value of the quantile is determined based on the sample size. When the sample size is insufficient, a more conservative quantile range is used to suppress occasional extreme values.

[0057] The learning and updating process is executed after each observation of a "state, action, reward, next state" sequence. The update magnitude is controlled by the learning rate (the learning rate is determined by trying multiple candidate learning rates when playing back historical sequences offline, and selecting a value that balances the convergence speed of the value table with the stability of the policy. The convergence speed is measured by the number of iterations required for the value table change magnitude to decrease to a stable range, and the policy stability is measured by the difference rate of action selection between adjacent periods. An upper limit for the difference rate is set, which is determined by the policy jitter level allowed by the system). The consideration of future returns is controlled by a discount factor (the discount factor is determined by setting it to a value that makes the cumulative impact within the effective time span of the short-term scheduling strategy dominant, the span being the scheduling strategy update cycle, and the update cycle being configured by the scheduler), and an exploration probability is set to ensure early coverage of the action space (the exploration probability is determined by generating a decreasing curve with the number of state visits as the independent variable, the initial exploration probability being 0.2, and the determination method is: in offline replay, a stable interval of strategy return improvement around 0.2 is statistically analyzed, and the median value of this interval is used as the initial value; the decreasing rate is determined by the inflection point where the return improvement slows down after the number of state visits reaches a certain threshold, and the inflection point threshold is detected by the slope change of the historical replay return curve). The convergence determination of the value table adopts a change magnitude threshold. The change magnitude threshold is determined as follows: within a stable business cycle, the natural decline curve of the update magnitude of value table elements is statistically analyzed. The upper quantile of the update magnitude when the decline curve enters the plateau period is selected as the threshold. Convergence is only determined if the update magnitude is below this threshold for several consecutive iterations. The number of consecutive iterations is set to 3 (the number of consecutive iterations is determined by performing sensitivity analysis on historical replays to observe the impact of the number of consecutive iterations on false convergence and convergence delay, and selecting the minimum number of iterations where the false convergence rate significantly decreases and the delay is acceptable). The value table obtained after convergence is marked as the converged value table and bound to the current strategy version, business cycle, and device configuration summary to ensure consistent referencing in subsequent greedy optimization.

[0058] After obtaining the convergence value table, the system performs a greedy optimization operation to generate the optimal scheduling decision sequence. Greedy optimization starts with the current discrete state identifier, selects the action with the highest value from the action set as the current action, and uses the next state obtained under the simulation or approximate transition model as the input for the next step. This process is repeated until the short-term decision sequence length is reached. The short-term decision sequence length is determined by the number of scheduling cycles covered by the scheduling policy execution window. The execution window is the minimum window length that the GPU allocation engine can stably execute without triggering frequent migrations. This minimum window length is determined through statistical analysis of the performance loss caused by historical migrations. The performance loss is measured by the median increase in task completion time after migration, and an upper limit is set to constrain the migration frequency. To ensure the optimization process operates within safety constraints, the system performs a feasibility screening before each action selection. This screening is based on the constraint verification rules of the resource state matrix and the predicted continuous allocability of video memory. Actions that fail the screening are eliminated, and the next candidate is selected. When all actions are infeasible, the action with the lowest risk is selected as the degradation action. The action with the lowest risk is determined by calculating the historical frequency of each action triggering video memory allocation failure and performance degradation alarms, prioritizing the action with the lowest frequency. If the frequencies are the same, the action with the least impact on queue waiting time is selected, with the impact assessment based on the historical median level. The action sequence obtained through this process constitutes the optimal scheduling decision sequence, along with the target device, target queue, target task set, and execution time for each action.

[0059] After the optimal scheduling decision sequence is generated, it enters the parsing and instruction generation stage to be transformed into a resource allocation instruction set and executed to form a short-term scheduling strategy. The parsing process first performs semantic expansion on the action sequence, breaking down each action into atomic instructions that can be recognized by the GPU allocation engine, and supplementing the parameters required for the atomic instructions, including task identifier, source device identifier, target device identifier, memory quota, concurrency limit, compaction trigger identifier, and number of time slices. Task and device identifiers are derived from the scheduler's task and device tables. The memory quota is determined by using the available memory and allocable contiguous capacity in the resource state matrix as dual constraints, taking the minimum allowed safe quota. The safety margin of the safe quota is determined by the memory safety margin configuration, which is derived from the joint result of stable state correction and uncertainty correction. The concurrency limit is determined by selecting an upper limit value that reduces queuing pressure and ensures computational usage is not lower than the target lower limit, based on queue pressure summary and volatility indicators. The target lower limit is determined by the service-side throughput requirements. The number of delayed time slices is determined by postponing actions on high-risk time slices to time slices where the risk decreases, based on release timing congestion and recovery delay risk. The risk decrease criterion is that congestion and recovery delay risk are below their respective thresholds. The thresholds are determined by statistically analyzing the distribution of congestion and recovery delay risk before historical memory allocation failures, and taking the upper quantile as the threshold. After parameter population, a resource allocation instruction set is formed and sorted by instruction dependencies. This sorting follows the constraints of "release before allocation, consolidate before large-scale allocation, migrate before concurrency boosting," based on root cause statistics of historical execution failures. These statistics show that this order significantly reduces the failure frequency. Instruction set execution employs a receipt-based mechanism. The receipt timeout threshold is set at the upper quantile of the GPU allocation engine's receipt latency under peak load. Exceeding the threshold triggers rollback and alarms, with the rollback path reversing according to instruction dependencies. After instruction execution, the system feeds the execution results and reward calculation results back into the learning iteration loop, providing sequence samples for the next value table update, thus forming a short-term scheduling strategy based on GPU resource usage prediction results.

[0060] S6 includes acquiring the instantaneous computing power request and historical job characteristics of the task to be processed, aligning and normalizing the instantaneous computing power request and historical job characteristics to construct a hybrid state tensor; inputting the hybrid state tensor into a short-term decision network and a long-term adaptation model to generate a multi-policy joint distribution, using the global reward feedback value from the environment to update the gradient correction of the multi-policy joint distribution online to generate policy evolution parameters; generating the final scheduling instruction based on the policy evolution parameters, and transmitting the final scheduling instruction to the GPU allocation engine to complete the execution of the evolution scheduling policy under multi-policy reinforcement learning.

[0061] In this embodiment, the instantaneous computing power request and historical job characteristics of the task to be processed are first obtained, and then aligned at the same scheduling time point. The scheduling time point is the boundary moment of one decision cycle of the scheduler (the scheduling time point is determined by using the internal trigger signal of the scheduler as a reference, and aligning the monitoring sampling, state construction, and policy output to the same trigger boundary to avoid decision deviation caused by data from different sources belonging to different time slices). Instantaneous computing power requests are used to characterize the intensity of computing resource demand for a task at the current moment. An instantaneous computing power request is formed by task submission information, task operation configuration, and resource request summary. It includes at least the scale of requested computing resources, the requested memory capacity, the requested parallel scale, the latency target level, the task priority level, and the expected data throughput level. (The fields are determined as follows: the original fields are extracted from the task submission description and queue strategy. For missing fields, they are backfilled according to the median level of similar tasks in the most recent business cycle of the same queue. The median level is obtained by statistical analysis within the cluster after clustering historical tasks by type. If the number of backfill triggers increases continuously within a short window, the field is marked as low confidence and its fusion weight is reduced. The adjustment range of the fusion weight is determined by statistical analysis of the scheduling error amplification ratio caused by historical backfilling.) Historical job features are used to characterize the long-term execution patterns of jobs with the same origin or type as the current task. Historical job features are extracted from the job profile library, which is organized hierarchically by tenant, queue, model family, and job template (the hierarchical rules are determined by using the business identifier, mirror identifier, entry command summary, and resource request summary in the task metadata as the clustering key to cluster historical jobs. After clustering, the dispersion of execution time and peak memory usage within a group is required to be lower than a threshold. The threshold is determined by using the upper quantile of the dispersion during a stable runtime period as the threshold. Clusters exceeding the threshold trigger further splitting). Historical job characteristics include at least the following: average execution time summary, peak memory allocation summary, stage time consumption percentage summary, failure retry rate summary, migration sensitivity summary, and queue congestion response summary. (Each summary is determined as follows: the statistical range is the samples within the most recent business cycles, and the length of the statistical range is three times the length of the business cycle. The basis for taking three times is that the median level of the profile indicators converges within this length and is not sensitive to short-term anomalies. The summary statistics preferentially use the median level and quantile difference. The median level is used to represent the central trend, and the quantile difference is used to represent the fluctuation scale. The quantile value varies with the sample size. When the sample size is insufficient, the conservatism of the quantile is increased to reduce the impact of occasional values.)

[0062] After data acquisition, the system performs alignment and normalization between instantaneous computing power requests and historical job characteristics to construct a hybrid state tensor. The alignment process first unifies the time granularity, which is taken as the shortest scheduling period (the time granularity is determined by reading the shortest policy update period in the scheduler configuration and using it as the time granularity benchmark; if the monitoring sampling period is finer, aggregation is performed within the time granularity, and the aggregation adopts a combination of median level and latest value, with the median level used to suppress jitter and the latest value used to retain sudden changes). The normalization process is divided into dimension normalization and distribution normalization: Dimension normalization maps different fields to a range of 0 to 1 (the mapping boundary is determined by maintaining a sliding minimum and sliding maximum value for each field, with the sliding range length being one business cycle to ensure the boundary covers recent peaks and valleys; when the sliding boundary changes abruptly, a smoothing window is introduced to smooth the boundary, with the smoothing window length being three time granularities, the value of 3 being based on the fact that boundary changes do not produce significant lag within this window length); Distribution normalization is used to handle long-tail fields, truncating values ​​exceeding the upper bound quantile by the upper bound quantile (the upper bound quantile is determined by statistically analyzing the upper quantile of the field during stable operation and setting this upper quantile as the truncation boundary; if the truncation trigger frequency continues to increase, the truncation boundary is moved upward synchronously, and the upward movement is determined by statistically analyzing the changes in scheduling revenue before and after the truncation trigger). After normalization, the system concatenates the "instantaneous field group" and the "historical field group" in a fixed order and adds a "field confidence group" to form a hybrid state tensor. The field confidence group is used to represent the reliability of each field (the confidence is determined by recording the source type of each field, which is divided into direct observation, rule inference, historical backfilling, and constrained inference; maintaining the historical error distribution for each source type, which is obtained by statistically analyzing the difference between the field's ex-post true value and its value at the time, and mapping the median level of the error distribution to the confidence deduction, and obtaining the field confidence after deduction; setting a confidence lower limit for key fields, with the confidence lower limit set at 0.6. The basis for the value of 0.6 is that when the confidence is lower than 0.6 in historical replay, the fluctuation of the policy output increases significantly, leading to an increase in the migration failure rate). The hybrid state tensor is organized into four dimensions: task, queue, device, and time slice. The task dimension carries instantaneous requests, the queue dimension carries congestion background, the device dimension carries resource supply constraints, and the time slice dimension carries short-term executable windows. (The size of each dimension is determined as follows: the task dimension is the size of the current set of tasks to be processed; the queue dimension is the size of the queue group that is the same as the queue to be processed, and the queue group is defined by the resource isolation policy; the device dimension is the size of the set of devices participating in scheduling, and the set of devices is obtained by filtering based on health status, with the filtering threshold being the lower quantile of the health score, which is composed of the frequency of fault alarms and the driver reporting delay; the time slice dimension is the length of the short-term execution window, and the window length is the number of time slices covered by the policy update cycle, ensuring that instruction execution and benefit feedback are closed within the window.)

[0063] After the hybrid state tensor is constructed, the system inputs it into the short-term decision network and the long-term adaptation model respectively to generate a multi-policy joint distribution. The short-term decision network focuses on evaluating the immediate benefits of the current state. The input tensor is aggregated in the short-term path in the order of "device resource constraint priority". The aggregation process first extracts the sub-tensors directly related to video memory, allocable continuous capacity, computing usage, and queue pressure, and then combines them with the instantaneous request intensity to generate an action tendency distribution (the action set is determined by listing the atomic actions supported by the allocation engine and removing non-executable actions. The criteria for determining non-executable actions are resource constraint verification and topology reachability verification. The resource constraint verification is based on video memory usage not exceeding the upper limit, continuous allocable capacity meeting the request scale, and computing usage not exceeding the upper limit. The topology reachability verification is based on the task migration path and the network bandwidth threshold. The bandwidth threshold is taken as the upper quantile of bandwidth usage before the performance degradation caused by historical migration). The long-term adaptive model focuses on evaluating steady-state returns across business cycles. Input tensors are aggregated in the long-term path in the order of "prioritizing historical operation patterns". The aggregation process extracts execution duration distribution, peak memory distribution, failure retry rate, migration sensitivity, and congestion response curve summary. Combined with the current queue strategy and service level target, a long-term tendency distribution is generated (the service level target is determined by reading the target completion time interval and failure tolerance interval of the queue, and the target interval is configured by the business side; if the target interval deviates significantly from the recent actual value, a target deviation alarm is triggered and the risk penalty weight in the long-term tendency distribution is increased. The increase is obtained by mapping the position of the deviation degree in the historical distribution). The multi-strategy joint distribution is fused across the same action set. The fusion weight is jointly determined by short-term and long-term confidence (short-term confidence is determined by resource status confidence level and short-term fluctuation reliability, which is determined by the number of truncations of fluctuation indicators and monitoring latency; long-term confidence is determined by profile sample size and profile stability, which is determined by the drift amplitude of profile indicators within a continuous business cycle. When the drift amplitude exceeds the drift threshold, the long-term confidence is reduced. The drift threshold is determined by statistically analyzing the upper quantile of natural fluctuations in profile drift without strategy changes and using it as the threshold). The resulting joint distribution provides a selection bias for each action and retains risk markers. Risk markers are obtained by statistically analyzing the frequency of actions triggering memory allocation failures and performance degradation alarms historically. Frequency statistics are filtered based on similar tasks and device configurations to ensure the comparability of risk markers.

[0064] Subsequently, the global reward feedback value from the environment is used to perform online updates and gradient corrections on the multi-strategy joint distribution to generate strategy evolution parameters. The global reward feedback value is based on the cluster-level target and is formed by a weighted aggregation of multiple indicators. The indicators include at least the improvement in task completion time, throughput, change in the number of memory allocation failures, change in the number of performance degradation alarms, improvement in device utilization, and change in queue waiting time (each indicator is calculated by comparing a fixed observation window before and after the execution of the final scheduling instruction, with the observation window length being the short-term execution window length to ensure that reward attribution and action execution are on the same time scale; the improvement in completion time is the median difference in completion time within the window before and after; the improvement in throughput is the difference in the number of tasks completed within the window before and after; the change in the number of failures is the difference in the number of failures within the window before and after; the change in alarms is the difference in the number of alarms within the window before and after; the improvement in utilization is the median difference in device utilization within the window before and after; and the change in waiting time is the median difference in queue waiting time within the window before and after). The weights of each indicator are determined by the target priority (the weights are determined as follows: a priority sequence is set according to the service level target, with security indicators placed at the highest level; security indicators include memory allocation failure and performance degradation alarms; efficiency indicators include throughput and utilization; experience indicators include completion time and waiting time; priorities are mapped to weights and normalized so that the weights sum to 1; a lower limit is set for the security indicators, which is 0.4. The value of 0.4 is based on the fact that the failure rate increases significantly when the security weight is lower than 0.4 in historical replays). Global rewards are normalized and pruned before entering online updates. The normalization boundary is the sliding minimum and sliding maximum value of the reward distribution within the most recent business cycle, and the pruning boundary is the upper and lower quantiles. The quantile values ​​vary with the sample size; when the sample size is insufficient, the pruning interval is tightened to suppress extreme values. The step size for online updates is controlled by the update step size parameter (the update step size parameter is determined by: evaluating the stability of different step sizes in historical replays; stability is measured by the difference rate of output strategies in adjacent windows and the amplitude of profit fluctuations; the step size with the highest average profit and a difference rate not exceeding the upper limit is selected; the upper limit of the difference rate is determined by the level of strategy jitter allowed by the system; the level of jitter is constrained by the threshold of migration frequency and queue oscillation frequency). The direction of gradient correction is jointly determined by the "reward increase / decrease trend" and the "risk label": when the reward increases and the risk label is in the low range, the tendency of the corresponding action in the joint distribution is strengthened; when the reward decreases or the risk label is in the high range, the tendency of the corresponding action is weakened and the tendency is transferred to the alternative action with lower risk; the threshold of the high risk range is taken as the upper quantile of the risk frequency before the historical alarm is triggered (the threshold is determined by: statistically analyzing the risk frequency distribution of historical execution records under the same device configuration, taking the upper quantile of the window before the alarm is triggered as the threshold, ensuring that high-risk actions are suppressed in advance before the alarm).The strategy evolution parameters are formed by summarizing the above update results. The strategy evolution parameters include at least short-term weights, long-term weights, risk penalty intensity, exploration intensity, and action constraint intensity. (The short-term and long-term weights are obtained by smoothing the short-term and long-term confidences. The smoothing window length is 3 short-term execution windows. The value of 3 is based on the fact that the weight change does not produce significant lag under this length.) The risk penalty intensity is obtained by mapping the distribution position of the risk markers. The exploration intensity is determined by the state access coverage. The coverage is determined by: counting the proportion of discrete states accessed in the most recent business cycle to the total number of states. When the proportion is lower than the coverage threshold, the exploration intensity is increased. The coverage threshold is 0.6. The value of 0.6 is based on the fact that when the coverage is lower than 0.6 in historical playback, the strategy benefit is prone to getting trapped in local conditions. The action constraint intensity is jointly determined by the weight of security indicators and the recent failure trend. The failure trend is determined by the sliding median level of the change in the number of failures.

[0065] Based on the strategy evolution parameters, the system generates the final scheduling instructions and transmits them to the graphics processor allocation engine for execution. The generation of the final scheduling instructions first selects the action sequence with the highest tendency and that meets feasibility screening from the joint distribution. This action sequence is then parsed into an engine atomic instruction set, which includes at least task placement instructions, task migration instructions, concurrency constraint instructions, resource quota instructions, and memory management trigger instructions. The resource quota is determined by using the device's available video memory and allocable contiguous capacity as dual constraints, taking the minimum allowable safe quota. The safety margin of the safe quota is jointly determined by the risk penalty intensity and the state confidence level. When the state confidence is low, the safety margin is increased, with the increase amount statistically mapped from the failure amplification ratio in historical low-confidence scenarios. The concurrency constraint is determined by using the queue waiting time target and the throughput target as constraints. The waiting time target is satisfied first, and then throughput is increased with the remaining resources. The waiting time target is set according to the service level target configuration. Instruction ordering follows dependency constraints: release and defragmentation instructions are executed first, followed by allocation and concurrency boosting instructions. The dependency constraints are determined by tracing back historical execution failures and finding that failures occur significantly more frequently in the reverse order; this order is then used as the execution constraint. Transmission and execution employ a receipt mechanism. The receipt timeout threshold is taken from the upper quantile of the engine's receipt latency under peak load (the threshold is determined by statistically analyzing the receipt latency distribution within the most recent business cycle, taking the upper quantile as the threshold, and setting a minimum threshold to avoid false alarms due to an excessively small threshold). Timeouts trigger a degradation path, which selects the lowest-risk, conservative instruction combination and reduces the migration frequency. The upper limit of the migration frequency is derived by working backward from the upper quantile of historical migrations that caused a deterioration in completion time.

[0066] S7 includes extracting performance degradation features from GPU cluster runtime status data, whereby performance degradation features are extracted when task latency exceeds a preset threshold; quantizing the performance degradation features to obtain quantitative feedback values, and embedding the quantized feedback values ​​into historical load sequences to construct an initial load vector; inputting the initial load vector into a resource prediction model to obtain a prediction bias matrix, which is generated by comparing the GPU computing power map with the actual results; and adjusting the action probability distribution based on the prediction bias matrix to achieve a closed-loop update of GPU resource usage prediction and multi-policy reinforcement learning scheduling strategy.

[0067] In this implementation, performance degradation features are first extracted from the GPU cluster runtime status data. The extraction process uses the scheduling decision point as the alignment boundary and task delay triggering as the entry point. The scheduling decision point is the end of one decision cycle of the scheduler (the scheduling decision point is determined by using the scheduler's internal trigger signal as a unified clock, and both monitoring sampling and index aggregation are merged into this trigger boundary to avoid the same task being counted repeatedly in different time slices). At each decision point, the GPU cluster runtime status data is aggregated by the monitoring module to form a time slice snapshot. The snapshot includes at least the task end-to-end latency, queuing time, execution time, GPU utilization, computing unit occupancy level, memory occupancy level, allocable contiguous capacity, memory allocation failure count, memory fragmentation index, task migration count, node health alarm, and driver reporting latency. First, perform field consistency checks and value range checks on the snapshot. Field consistency checks require that the task identifier and device identifier be consistent across all sub-tables. Range checks determine the upper and lower bounds based on the device's static attributes. For video memory usage, the upper bound is the total capacity of the device's video memory, and for utilization, the upper bound is the full load definition. When a check fails, the corresponding field is marked as low confidence and its weight is reduced in subsequent quantization processes. The reduction in weight is determined by the statistical proportion of misjudgments caused by historical check failures.

[0068] Performance degradation features are triggered when task latency exceeds a preset threshold. The threshold is determined based on the service level target tiering. Specifically, a target latency range is maintained at the queue or tenant level, derived from the service level configuration on the business side. Based on this, the preset threshold is the amplification value of the upper bound of the target latency range, with the amplification value set to 1.2 (1.2 is determined by: retrospectively analyzing historical latency distributions, statistically analyzing the proportion of false alarms when latency reaches the target upper bound and the hit rate of performance degradation root causes after latency exceeds the target upper bound, selecting the minimum amplification value that does not significantly increase the false alarm proportion but significantly increases the hit rate, and using the median level of this amplification value as a fixed value). When the end-to-end latency of a task in the current time slice exceeds this preset threshold, the system marks the task as a degradation trigger sample and extracts a subset of the task's running status within the same time slice, including the task's waiting time, execution time, queue congestion level, device memory usage and fragmentation level, whether migration has occurred, and changes in execution rate before and after migration. The calculation process for the change in execution rate before and after migration is as follows: Taking the migration event as the boundary, the processing volume per unit time of the task within the window before and after migration is counted respectively. The processing volume is taken as the number of batches or the number of inference requests. The window length is taken as 3 scheduling cycles (the method of determining 3 is: to count the noise level of the rate change caused by migration under different window lengths, and select the smallest window length where the noise level enters the stable range and can cover the impact of a cold start of migration). The absolute value of the difference between the two is taken as the migration disturbance intensity. In addition to task-level features, cluster-level background features are extracted synchronously, including the count of memory allocation failures, the count of performance degradation alarms, the count of driver-reported latency anomalies, and the count of resource contention events within the current time slice. The method of determining the count of resource contention events is: to count the number of times concurrent tasks on the same device experience memory allocation failures or sudden increases in execution time within the same time slice, and to obtain the contention intensity by normalizing the number of tasks.

[0069] After stripping performance degradation features, the system performs numerical analysis on these features to obtain quantitative feedback values. The numerical analysis process characterizes degradation using three dimensions: "intensity," "range," and "duration." The intensity dimension includes at least the degree of latency exceeding the threshold, the increase in the proportion of waiting time, the increase in execution time, the degree of memory fragmentation deterioration, the increment of memory allocation failures, and the intensity of migration disturbances. The range dimension includes at least the proportion of affected tasks, the proportion of affected devices, and the proportion of affected queues. The duration dimension includes at least the number of consecutive time slices exceeding the threshold and the number of time slices of degradation fallback. The calculation process for latency exceeding the threshold is as follows: Take the difference between the end-to-end latency of the task and a preset threshold, then normalize it using the preset threshold. The normalized value is limited to the range of 0 to 1, with any value exceeding 1 truncated to 1. The calculation process for the increase in waiting time percentage is as follows: Divide the waiting time by the end-to-end latency to obtain the waiting percentage, and compare it with the median level of the waiting percentage for the same type of task in history to obtain the difference. The method for determining the historical similarity is: grouping by task type and queue, with the grouping key derived from task metadata. The calculation process for the increase in execution time is the same as for the waiting percentage, but changed to execution time percentage. The calculation process for the deterioration of memory fragmentation is as follows: Take the difference between the current time slice fragmentation index and the median level of the fragmentation index during the most recent stable period, and normalize it using the percentile difference of the fragmentation index during the stable period. The method for determining the stable period is: when there are no alarms and the utilization fluctuation is below the threshold. For each time period, the threshold is taken as the upper quantile of the utilization fluctuation in the stable operating sample; the calculation process for the memory allocation failure increment is as follows: take the failure count of the current time slice minus the failure count of the previous time slice, and normalize it with the number of tasks arriving in the current time slice to avoid the count amplification caused by the increase in the number of arrivals; the calculation process for the proportion of affected tasks is as follows: take the number of tasks with latency exceeding the threshold in the current time slice divided by the number of completed tasks, and the number of completed tasks is taken as the count of task end events; the proportion of affected devices and the proportion of affected queues are obtained by dividing the number of devices or queues that have degradation trigger samples by the corresponding total number; the number of consecutive time slices exceeding the threshold is obtained by counting the number of slices in which the same task exceeds the threshold in consecutive time slices, and the number of time slices of degradation fallback is obtained by counting the number of time slices from when the task first stops exceeding the threshold to when it recovers to the historical baseline fluctuation range, and the historical baseline fluctuation range is determined by the latency distribution quantile difference of the cluster to which the task belongs. Before the above numerical components are formed, the scale is unified, and the scale boundary adopts the sliding minimum and sliding maximum values. The sliding range length is one business cycle. A truncation boundary is set for the long-tail component. The truncation boundary is taken as the upper quantile of the stable running sample. The number of truncation triggers is used as the confidence deduction item, and the deduction range is determined by the statistical relationship between the historical truncation and misjudgment ratio.

[0070] After generation, the quantized feedback values ​​are embedded into the historical load sequence to construct the initial load vector. The embedding process follows the principle of "fixed position and controllable impact." The historical load sequence consists of load vectors from multiple recent consecutive time slices. The load vector includes task type distribution, arrival frequency, computational pressure intensity, concurrency, queuing pressure, memory usage trend, and fragmentation trend. During embedding, the quantized feedback values ​​are added as new feedback components to the end of each time slice load vector. In the historical sequence, several recent time slices are given higher weights. The weights are determined by using the number of consecutive time slices exceeding the threshold in the degradation duration dimension as the independent variable. The longer the duration, the greater the recent weight. At the same time, a maximum recent weight upper limit is set, which is 0.7 (the 0.7 is determined by: reviewing cases of policy jitter caused by excessively high recent weights in historical closed-loop updates, statistically analyzing the proportion of action distribution fluctuations that significantly increase when the weight exceeds 0.7, and using 0.7 as the upper bound for controlled jitter risk). After embedding, a new initial load vector is formed. The new initial load vector contains load structure information and degradation feedback information, along with the confidence level of the feedback. The confidence level is determined by the aforementioned field verification, truncation triggering, and data latency. The data latency is the sum of the driver reporting latency and the monitoring aggregation latency. The upper quantile of the latency distribution under peak load is used as the low confidence criterion. If the confidence exceeds this criterion, it is reduced. The reduction is determined by the statistical proportion of feedback misleading caused by historical latency.

[0071] After inputting the initial load vector into the resource prediction model, the system obtains a prediction deviation matrix, which is generated by comparing the GPU computing power map with the actual results. The GPU computing power map is organized by device dimension and includes at least device model, computing power level, memory capacity, memory bandwidth level, interconnect topology cost, historical stable throughput level, historical stable latency level, and fragmentation sensitivity level. The fields of the computing power map are derived from device static attributes and historical stable operation statistics, and the time period of the historical stable operation statistics is determined according to the aforementioned stable time period criteria. The prediction results output by the resource prediction model provide resource status predictions by device and time slice, including expected available computing power, expected memory usage, expected allocable contiguous capacity, and expected queuing pressure. The actual results are taken from the monitoring observations under the same device and time slice, and the observations are aggregated and cleaned using the same criteria. The cleaning rules are consistent with the aforementioned consistency and range checks. When generating the prediction deviation matrix, the prediction and actual results are first time-aligned. The upper limit of the time alignment error is half of the scheduling cycle. Observations exceeding the upper limit are marked as low confidence and their weights are reduced in the deviation calculation. Then, the deviation strength and direction are calculated for each matrix element. The deviation strength is the magnitude of the difference between the prediction and the actual value, normalized according to the upper bound of the corresponding resource. The upper bound of the resource is defined as the static capacity or full load of the equipment. The deviation direction is marked based on whether the predicted value is higher or lower than the actual value. To distinguish between systematic deviations and occasional deviations, the system aggregates the deviation matrix within a short window. The window length is 3 scheduling cycles (the value of 3 is based on the significant decrease in autocorrelation of the deviation sequence at this length). The aggregation statistics use the median level and quantile difference. The median level is used as the center of systematic deviation, and the quantile difference is used as the uncertainty scale. When the quantile difference exceeds the uncertainty threshold, the deviation is marked as a high uncertainty deviation. The uncertainty threshold is determined by statistically analyzing the upper quantile of the deviation quantile in the stable operating samples and using this upper quantile as the threshold to prevent stable scenarios from triggering the high uncertainty label. The deviation matrix is ​​also conditionally mapped to the computing power map, and the deviations are grouped and summarized according to device type, interconnect topology level, and fragmentation sensitivity level. The grouping and summarization are used to identify whether the deviations are concentrated in a specific hardware family or a specific topology layer, thereby providing the scope boundary for subsequent action probability adjustment.

[0072] The action probability distribution is adjusted based on the predicted deviation matrix to complete the closed-loop update. The adjustment process is guided by the principles of "deviation-driven, risk-constrained, and degradation-suppressed." The action probability distribution is derived from the joint distribution output by the multi-policy scheduler, which provides a selection probability for the action set in each state. During adjustment, the types of actions that need to be suppressed or strengthened are first determined: when the deviation matrix shows that "available resources are systematically overestimated" and the degradation feedback intensity is high, actions that increase resource consumption are suppressed, including increasing concurrency, centralized placement, and delayed recycling; when the deviation matrix shows that "available resources are systematically underestimated" and the degradation feedback intensity is low, excessively conservative action probabilities are suppressed, including unnecessary delays, excessive migration, and excessive reorganization, while increasing the probabilities of direct allocation and concurrent recovery actions. The magnitude of suppression and reinforcement is jointly determined by the deviation intensity and the degradation feedback intensity: the deviation intensity is taken as the center of systematic deviation in the corresponding resource dimension of the deviation matrix, and the degradation feedback intensity is taken as the weighted sum of the intensity dimensions in the quantified feedback values. The weighting is determined by the historical degradation root cause attribution statistics, which are obtained by associating "degradation trigger samples" with "resource status anomaly types". After the magnitude calculation, an upper limit is set, with the adjustment upper limit set to 0.15 (0.15 is determined by the proportion of a single adjustment exceeding 0.15 in the historical online adjustment that leads to a significant increase in strategy jitter and migration frequency, and 0.15 is used as the controlled upper bound for jitter). To maintain the legality of the probability distribution, the system performs normalization after adjustment, re-normalizing all action probabilities to a sum of 1, and sets a minimum exploration lower limit, with the exploration lower limit set to 0.05 (0.05 is determined by the probability of falling into local action preferences when the exploration is below 0.05 in historical replay, and 0.05 is used as the controlled lower limit for coverage), ensuring that the strategy still retains self-correction space when the environment changes. For states marked with high uncertainty deviation, the system introduces an additional conservative factor. This conservative factor is determined by mapping the position of the deviation quantile in the stable sample distribution to a conservative coefficient; the higher the position, the higher the conservative coefficient. Priority is given to increasing the probability of actions with the lowest risk. The determination of the lowest-risk action is based on the frequency of historical memory allocation failures and performance degradation alarms. Frequency statistics are filtered by grouping devices with the same configuration and load to ensure consistent risk comparison criteria. After these adjustments, the action probability distribution and the initial load vector with embedded feedback jointly enter the next round of prediction and scheduling. This allows the system to identify the direction of prediction distortion through the deviation matrix after detecting latency degradation and to suppress actions that lead to degradation through probability distribution adjustment.

[0073] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A GPU resource utilization prediction and scheduling method based on multi-policy reinforcement learning, characterized in that, include: S1. The current load pattern is obtained in real time from the GPU cluster through a pre-established monitoring module. The load pattern includes task type distribution and arrival frequency. An initial load vector is constructed based on the current load pattern as the initial input for GPU resource usage prediction. S2. For the initial load vector, a deep neural network model is used to calculate the similarity score. If the similarity score exceeds the preset threshold, the current load mode is determined to match the historical load mode, and the corresponding stable state information is obtained as the stable state basis for GPU resource usage prediction. S3. Otherwise, mark it as a new load mode switch and trigger incomplete status information processing; S4. In the case of incomplete state information, extract uncertainty parameters from the remaining execution time estimation module and the memory fragmentation distribution scan, and correct the GPU resource usage prediction results based on the uncertainty parameters to obtain the adjusted resource state matrix. S5. Based on the adjusted resource state matrix and short-term load fluctuation index, the optimal scheduling decision sequence is iteratively calculated using the Q-learning algorithm in reinforcement learning to form a short-term scheduling strategy based on the GPU resource usage prediction results. S6. The short-term scheduling strategy is fused with the long-term change adaptation historical data through an online update algorithm to form an evolutionary scheduling strategy under multi-strategy reinforcement learning, and then output to the GPU allocation engine for execution.

2. The GPU resource utilization prediction and scheduling method based on multi-policy reinforcement learning according to claim 1, characterized in that: S1 includes: The system obtains real-time data streams from the GPU cluster through a pre-established monitoring module, and parses the current load pattern, which includes task type distribution and arrival frequency, from the real-time data stream. The computational density features are extracted based on the current load pattern, and the computational density features are concatenated with the arrival frequency to construct an initial load vector; The initial load vector is used as input to perform time-series evolution simulation, generating a resource usage prediction value that includes memory occupancy, and then the subsequent judgment of GPU resource usage prediction is completed based on the resource usage prediction value.

3. The GPU resource utilization prediction and scheduling method based on multi-policy reinforcement learning according to claim 1, characterized in that: S2 includes: Obtain the initial load vector to be analyzed, and use a deep neural network model to encode the initial load vector to generate a high-dimensional load feature tensor; Calculate the similarity score between the high-dimensional load feature tensor and the historical standard vector. If the similarity score exceeds a preset threshold, the matching historical load pattern is determined. Extract stable state information, including peak memory allocation values, based on historical load patterns; The baseline parameters are adjusted based on the steady-state information to determine the steady-state basis for GPU resource usage prediction.

4. The GPU resource utilization prediction and scheduling method based on multi-policy reinforcement learning according to claim 1, characterized in that: S3 includes: Real-time runtime data is acquired to extract load characteristics, and these load characteristics are matched with historical benchmark data to obtain the characteristic deviation. If the feature deviation exceeds the verification threshold, a new load mode switching instruction is generated, and a system status snapshot is captured based on the switching instruction. Perform integrity verification on system state snapshots to generate state missing description files, obtain completion strategies based on state missing description files, and synthesize a set of states to be processed; The set of states to be processed is transferred to the recovery queue to initiate a process for handling incomplete state information during the switch to a new load mode.

5. The GPU resource utilization prediction and scheduling method based on multi-policy reinforcement learning according to claim 1, characterized in that: S4 includes: Extract instruction stream execution progress data and video memory physical address mapping table; calculate the remaining execution time value based on instruction stream execution progress data; and construct a video memory fragmentation distribution scan map based on video memory physical address mapping table. A joint analysis of the remaining execution time values ​​and the memory fragmentation distribution scan map is performed to output a set of uncertainty parameters. The set of uncertainty parameters is input into the error evaluation function to obtain the quantified value of the prediction deviation, and the adjustment coefficient matrix is ​​generated based on the quantified value of the prediction deviation. The initial forecast data is weighted and corrected using the adjustment coefficient matrix to obtain the adjusted resource status matrix.

6. The GPU resource utilization prediction and scheduling method based on multi-policy reinforcement learning according to claim 1, characterized in that: S5 includes: Obtain the adjusted resource status matrix and short-term load fluctuation index, and perform weighted feature fusion on the resource status matrix and short-term load fluctuation index to construct a multi-dimensional status vector. The multidimensional state vector is mapped to the state space of reinforcement learning, and the convergent Q-value table is obtained by iterative calculation using the Q-learning algorithm. Based on the convergent Q-value table, a greedy optimization operation is performed to generate the optimal scheduling decision sequence; The optimal scheduling decision sequence is analyzed and transformed into a resource allocation instruction set. By executing the resource allocation instruction set, a short-term scheduling strategy based on the GPU resource usage prediction results is formed.

7. The GPU resource utilization prediction and scheduling method based on multi-policy reinforcement learning according to claim 1, characterized in that: S6 includes: Obtain the instantaneous computing power request and historical job characteristics of the task to be processed, and align and normalize the instantaneous computing power request and historical job characteristics to construct a hybrid state tensor; The mixed-state tensor is input into the short-term decision network and the long-term adaptation model to generate a multi-policy joint distribution. The global reward feedback value from the environment is used to update the gradient of the multi-policy joint distribution online to generate policy evolution parameters.

8. The GPU resource usage prediction and scheduling method based on multi-policy reinforcement learning according to claim 7, characterized in that: S6 further includes: The final scheduling instruction is generated based on the policy evolution parameters and transmitted to the GPU allocation engine to complete the execution of the evolution scheduling policy under multi-policy reinforcement learning.

9. The GPU resource usage prediction and scheduling method based on multi-policy reinforcement learning according to claim 7, characterized in that, It also includes S7, which obtains feedback metrics from performance degradation monitoring after execution, and uses these feedback metrics as input for the next initial load vector to achieve closed-loop updates of GPU resource usage prediction and multi-policy reinforcement learning scheduling strategies. Specifically, it includes: Performance degradation features are extracted from GPU cluster runtime status data when task latency exceeds a preset threshold. The performance degradation characteristics are quantified to obtain quantized feedback values, and these quantized feedback values ​​are embedded into the historical load sequence to construct an initial load vector.

10. The GPU resource utilization prediction and scheduling method based on multi-policy reinforcement learning according to claim 9, characterized in that: The S7 also includes: The initial load vector is input into the resource prediction model to obtain the prediction bias matrix, which is generated by comparing the GPU computing power map with the actual results. By adjusting the action probability distribution based on the prediction bias matrix, a closed-loop update of GPU resource usage prediction and multi-policy reinforcement learning scheduling strategy is achieved.

Images