Gpu resource optimization method and system based on dynamic prediction and topology thermogram

Abstract

The application discloses a GPU resource optimization method and system based on dynamic prediction and topology heat map. The method comprises the following steps: constructing a server heat map, the heat value of which is calculated based on historical task characteristics and node topology connection edge quantity weighting; predicting future integrated GPU resource demand based on a to-be-scheduled task queue and historical task data to generate a resource demand heat map; quantitatively identifying GPU resource fragmentation risks based on the resource demand heat map and a dynamic reference value calculated based on the server heat map; selecting an optimal optimization strategy in response to the fragmentation risks; and executing the selected strategy, including dynamically migrating tasks supporting checkpoints and / or scaling resources of tasks supporting elastic scaling, to realize lossless reconstruction of GPU resources. Through the intelligent closed loop of prediction-decision-execution, the application actively resolves resource fragmentation, and significantly improves the resource utilization and task scheduling efficiency of a large-scale GPU cluster.

Description

Technical Field

[0001] This invention relates to the fields of computer resource management and cloud computing technology, and in particular to a GPU resource optimization method and system based on dynamic prediction and topology heatmap. Background Technology

[0002] With the rapid development of artificial intelligence and high-performance computing, large-scale GPU clusters have become the core infrastructure supporting AI training, scientific computing, and cloud services. In such clusters, resource schedulers (such as Kubernetes-based scheduling extensions) play a crucial role. Current mainstream schedulers generally adopt bin packing algorithms based on multi-dimensional resources (such as the number of GPUs, video memory, and CPU), the core goal of which is to treat computing tasks as independent entities and "pack" them into physical server nodes that currently have sufficient idle resources.

[0003] However, this static scheduling strategy based on instantaneous states exposes a serious resource fragmentation problem when facing complex and ever-changing production environments, resulting in overall cluster resource utilization and throughput falling far short of expectations. Specifically, this problem is caused by the following factors:

[0004] First, the resource requirements of the tasks are highly diverse and heterogeneous. The cluster contains everything from lightweight inference tasks that only require a portion of the memory on a single GPU to large-scale distributed training tasks that require multiple nodes and dozens or even hundreds of GPUs. This huge difference in requirements makes the resource allocation pattern naturally fragmented.

[0005] Secondly, the dynamic and random nature of task lifecycles. The start and end times of tasks differ, resulting in asynchronous and discontinuous resource allocation and release within the cluster. After prolonged operation, idle resources inevitably become scattered across different nodes, forming fragmented resource silos.

[0006] The aforementioned factors collectively lead to the cluster frequently falling into a "false full load" dilemma. Monitoring metrics may show overall cluster resource utilization at a low level (e.g., 60%), but the actual available idle resources are fragmented across numerous nodes. When a new task requiring large, consolidated resources (e.g., a continuous 8-GPU server) is submitted, the scheduler cannot find a single node to meet its needs, even though the total amount of fragmented resources far exceeds the task's requirements. The direct consequence is that the task is forced to remain in the waiting queue for an extended period, facing the paradox of having resources but being unable to allocate them, severely impacting the delivery cycle of critical tasks and the overall efficiency of the cluster.

[0007] Currently, some existing solutions in the industry for the problem of resource fragmentation have the following significant shortcomings:

[0008] Passive response: Existing systems only passively trigger fragmentation processing when they receive large-scale resource tasks and immediately fail to schedule them. This reactive approach is slow and cannot intervene in the early stages of fragmentation problems, missing the optimal time for defragmentation.

[0009] Crude resource management methods: While some advanced schedulers offer resource management features, they often rely on forcibly terminating or evicting running low-priority tasks. This approach is extremely user-unfriendly, interrupting critical computing processes, wasting computing resources, and degrading the user experience.

[0010] Lack of predictive capability: Existing scheduling decisions rely entirely on a snapshot of the cluster's resources at the current moment, lacking any foresight regarding task submission trends and resource demand patterns in the near future (next scheduling cycle). Therefore, the system cannot perform proactive, preventative resource allocation optimization.

[0011] The optimization objectives are singular: most existing optimizations are limited to improving the scheduling success rate in the current instant, and lack a global consideration of higher-dimensional objectives such as the waiting time of the entire task queue, the long-term throughput of the system, and the fairness of multi-tenancy.

[0012] Therefore, there is an urgent need in this field for a new resource optimization scheme that can proactively predict, make intelligent decisions, and execute without loss, in order to fundamentally solve the resource fragmentation problem of GPU clusters. Summary of the Invention

[0013] To address the aforementioned technical problems, the technical solution adopted by this invention is as follows:

[0014] According to a first aspect of the present invention, a GPU resource optimization method based on dynamic prediction and topology heatmap is provided, the method comprising the following steps:

[0015] S100, Construct a server heatmap, wherein the heatmap value is calculated based on a weighted average of node feature parameters from multiple dimensions.

[0016] S200 predicts future integrated GPU resource requirements based on the queue of tasks to be scheduled and historical task data, in order to generate a resource demand heatmap.

[0017] S300, based on the resource demand heatmap and the server heatmap, uses comparative analysis to quantify and identify the risk of GPU resource fragmentation.

[0018] S400, in response to the risk of fragmentation of quantized GPU resources, selects the optimal optimization strategy from a pre-built strategy library.

[0019] S500 executes the selected optimization strategy to change the physical layout of GPU resources.

[0020] According to a second aspect of the present invention, a GPU resource optimization system based on dynamic prediction and topology heatmaps is provided, comprising:

[0021] The heatmap construction module is used to build server heatmaps based on historical task characteristics and the number of node topology connection edges.

[0022] The resource demand prediction and risk identification module is connected to the heatmap construction module. It is used to predict future integrated GPU resource demand based on the task queue to be scheduled and historical task data to generate a resource demand heatmap. Based on the resource demand heatmap and the server heatmap, it quantifies and identifies GPU resource fragmentation risk through comparative analysis.

[0023] The strategy decision engine, connected to the resource demand prediction and risk identification module, is used to select the optimal optimization strategy from a pre-set strategy library in response to the quantified GPU resource fragmentation risk.

[0024] A resource reconfiguration executor, connected to the policy decision engine, is used to execute the selected optimization policy to change the physical layout of GPU resources.

[0025] The present invention has at least the following beneficial effects:

[0026] (1) Significantly improved the scheduling success rate and efficiency of large resource tasks: By proactively predicting resource demand and actively managing resources, the problem of "false full load" is fundamentally solved. This greatly improves the scheduling success rate of tasks that require large-scale integration of GPU resources (such as multiple consecutive cards), while effectively shortening the average waiting time of such tasks in the queue.

[0027] (2) User-insensitive resource optimization and improved user experience: The traditional crude task termination or expulsion methods have been abandoned, and lossless reconstruction technologies such as dynamic migration and elastic scaling based on checkpoints have been adopted instead. This makes the resource reorganization process virtually imperceptible to the user's computing tasks, effectively avoiding the waste of computing resources, loss of progress and user complaints caused by task interruption, and decoupling operation and maintenance from user business.

[0028] (3) Achieved a dual improvement in cluster throughput and resource utilization: precisely solved the system paradox of "resource idleness" and "task queuing". By eliminating resource fragmentation, GPU resources can be utilized more fully and continuously, thereby significantly improving the average GPU utilization of the cluster and the number of tasks completed per unit time (system throughput).

[0029] (4) Enhanced system autonomy and reduced operation and maintenance costs: The system has a complete set of intelligent decision-making logic from risk identification to strategy execution, and has the ability to self-diagnose and self-optimize. This greatly reduces the frequency of operation and maintenance personnel manually intervening in resources, handling scheduling failures and performing fault recovery, realizing the automation and refinement of operation and maintenance, thereby reducing operating costs.

[0030] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 A flowchart of a GPU resource optimization method based on dynamic prediction and topology heatmap provided in an embodiment of the present invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0035] It should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. A process can be terminated when its operation is complete, but it may also have additional steps not included in the figures. A process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0036] (Example 1)

[0037] This embodiment provides a GPU resource optimization method based on dynamic prediction and topology heatmaps, applied to large-scale GPU clusters built by multiple servers via high-speed networks (such as InfiniBand or RoCE), aiming to solve the "pseudo-full load" problem caused by resource fragmentation. Figure 1 As shown, the method includes the following steps:

[0038] S100, construct a server heatmap.

[0039] The purpose of this step is to transform the cluster's historical load and topology into a quantifiable and visualized resource status map. The specific implementation is as follows:

[0040] First, data is collected from cluster management systems (such as Kubernetes) and monitoring systems, including:

[0041] Initial server resource configuration data: number of GPU cards, GPU model, video memory capacity, number of CPU cores, memory capacity, storage space, network interface type, bandwidth specifications, pre-packaged software environment and model library version, etc.

[0042] Historical task execution data: Within a preset statistical time window (e.g., 24 hours), the following information is collected:

[0043] Task feature data:

[0044] Task priority: Represented using numerical values ​​(e.g., 0-100) or hierarchical levels (high / medium / low);

[0045] Resource requirements specifications: number of GPUs requested, amount of video memory requested, number of CPU cores requested;

[0046] Actual resource consumption: peak GPU utilization during operation, and video memory usage during operation.

[0047] Execution time data: Task start time, task end time, and actual runtime.

[0048] Task path information: The sequence of server nodes a task traverses throughout its entire lifecycle, and the time it spends on each node.

[0049] Cluster topology data includes network connection relationships (edge ​​information) between server nodes, connection types (such as InfiniBand, Ethernet, etc.), connection bandwidth and latency characteristics, and network topology information. The network connection relationships between server nodes can be obtained from the cluster's network configuration manager, including but not limited to: physical network topology connections (such as InfiniBand, RoCE network links), logical communication relationships (such as MPI communication groups, distributed training task dependencies), and resource sharing relationships (such as NVLink high-speed interconnects, storage network paths).

[0050] Next, for each server node in the cluster, a heat value is calculated based on at least historical task characteristics and the number of node topology connection edges.

[0051] In this embodiment of the invention, the heat values ​​of the server heatmap are obtained through the following steps:

[0052] S101, Obtain task path information and task characteristics from historical task datasets.

[0053] In this embodiment of the invention, the task characteristics include at least task priority, resource requirement specifications (the requested amount of GPU resources and video memory size), actual runtime, and resource usage intensity.

[0054] In this embodiment of the invention, resource occupancy intensity is a comprehensive indicator that quantifies the GPU resource requirements of a single task. Resource occupancy intensity = W gpu ×Standardized value of GPU quantity + W mem × Standardized value of video memory size. W gpu W is the weighting factor for the normalized value of the number of GPUs, and Wmem is the weighting factor for the normalized value of the video memory size. gpu +W mem =1. Wherein, the normalized value of the number of GPUs = (number of GPUs requested by the task - number of GPUs requested by the minimum) / (number of GPUs requested by the maximum - number of GPUs requested by the minimum), and the normalized value of the video memory size = (video memory size requested by the task - video memory requested by the minimum) / (video memory requested by the maximum - video memory requested by the minimum).

[0055] In a preferred embodiment, W is set gpu =0.6, W mem =0.4, to emphasize the dominant role of GPU computing cores in AI workloads, while also taking into account the demand for video memory capacity.

[0056] Suppose a task requests 4 GPUs and 48GB of video memory, and the number of GPUs requested by tasks in the cluster ranges from 1 to 8, and the video memory ranges from 12 to 96GB, then:

[0057] The normalized value for the number of GPUs is approximately (4-1) / (8-1) ≈ 0.429.

[0058] The normalized value of the video memory size = (48-12) / (96-12)≈0.429;

[0059] Resource occupancy intensity = 0.6 × 0.429 + 0.4 × 0.429 = 0.429.

[0060] This calculation method integrates resource requirements of different dimensions into a unified scalar value, providing a reliable data foundation for accurately assessing node load and quantifying fragmentation risk.

[0061] S102, based on the task path information, associate each task with the server nodes it passes through during its operation, and calculate the feature parameters of multiple dimensions corresponding to each server node for a preset statistical time window.

[0062] Based on the task path information, tasks are precisely associated with the server nodes they have run on. For each server node in the cluster, by traversing its task path information, all tasks that have run on that node are selected to form a node-specific task subset.

[0063] Among them, the feature parameters of the multiple dimensions include at least the node load index obtained based on task feature aggregation and the topological connectivity centrality of the node. Each server node specifically includes the following five core parameters:

[0064] The number of tasks processed is determined by counting the total number of all tasks that have run on the server node within a preset statistical time window, based on the task path information.

[0065] The priority weighted value is obtained by multiplying the priority of each task and its resource consumption intensity for all tasks running on the server node and then summing the results.

[0066] Resource utilization level is obtained by calculating the average resource utilization intensity of all tasks on the server node, and is used to characterize the average resource load of the node within a time window.

[0067] The cumulative load over time is calculated by multiplying the actual runtime of each task running on the server node by its resource consumption intensity and then summing the results.

[0068] Topological connectivity centrality is directly taken as the number of edges that the server node is connected to in the cluster topology. It is used to quantify the pivotal position of a node in the network based on the degree centrality principle in graph theory.

[0069] By systematically integrating the feature parameters of the above five dimensions, we can comprehensively and accurately characterize the overall load status of server nodes and their structural importance in the cluster, providing a reliable quantitative basis for subsequent resource optimization decisions.

[0070] S103, perform weighted normalization on the feature parameters of the multiple dimensions to generate thermal values ​​in the range of 0-100.

[0071] In S300, the feature parameters used in multiple dimensions are all normalized parameters. Specifically, the aforementioned five feature parameters are subjected to min-max normalization, which linearly transforms them to the interval [0,1]. The minimum and maximum values ​​required for normalization are taken from the actual minimum and maximum values ​​of all nodes in the cluster for that parameter, respectively.

[0072] The thermal value is calculated using the following weighted fusion formula: Thermal value = 100 × (W1 × N) norm +W2×P norm +W3×R norm +W4×T norm +W5×D norm ), N norm P is the normalized number of tasks processed. norm R is the normalized priority weighted value. norm For normalized resource utilization levels, T norm For normalized time-cumulative load, D norm This represents the normalized topological connectivity centrality. W1 to W5 correspond to the weight coefficients of the five normalization parameters mentioned above, and each weight coefficient is non-negative and satisfies the constraint that the sum of the weights is 1.

[0073] This calculation formula integrates the node features from five different dimensions into a single thermal value index through a weighted fusion method: N norm It mainly reflects the characteristics of a node in terms of task throughput, P norm R reflects the load of high-priority tasks. norm T represents the intensity of resource use. norm Reflecting the cumulative load over time, while D norm This method quantifies the structural importance of nodes within the network topology. Through this weighted fusion model of multi-dimensional features, the resulting heatmap value comprehensively and objectively characterizes the overall load status and criticality of server nodes in the cluster environment, providing accurate quantitative basis for subsequent resource optimization. The advantage of this method lies in considering both the real-time load characteristics of nodes and their structural position within the cluster topology, thus achieving a comprehensive assessment of node status.

[0074] In this embodiment of the invention, the weighted normalization of the feature parameters is determined by a data-driven sensitivity analysis method, specifically including the following steps:

[0075] S1 clusters the historical task datasets according to their task characteristics to form multiple task clusters.

[0076] In this embodiment of the invention, historical task datasets are clustered based on their multidimensional task characteristics to form multiple representative task clusters. The clustering analysis is based on a similarity measure in the task feature space, automatically classifying tasks with similar resource demand patterns and operational characteristics.

[0077] The task features used for clustering include at least two of the following dimensions: task priority, number of GPUs required, required video memory size, and estimated runtime.

[0078] This invention can automatically cluster tasks using clustering algorithms well-known in the art, such as distance-based partitioning methods (e.g., K-means algorithm), density-based clustering methods (e.g., DBSCAN algorithm), or hierarchical clustering methods. The clustering process includes standard steps such as feature standardization, similarity measurement, cluster center determination, and iterative optimization.

[0079] The task clusters formed by clustering collectively constitute a standardized set of workloads for weight sensitivity testing, with each cluster representing a typical computational task pattern. This approach ensures that the weight sensitivity test covers a variety of typical workloads that may be encountered, thereby determining weight combinations with better generalization ability and robustness.

[0080] S2, for the main parameter whose weight is to be determined, fix the weight value of the main parameter, and treat the weights of all other parameters as a whole, and randomly allocate them under preset constraints to form a weight combination.

[0081] For the target parameters (i.e., the main parameters) whose weights need to be determined, perform the following operations:

[0082] Fixed target parameter weights: The weight values ​​of the target parameters are fixed at a specific initial value.

[0083] Randomly assign the remaining weights: Treat the weights of all other parameters as a whole and randomly assign them under the premise of satisfying preset constraints to form a complete weight combination. The preset constraints are: the sum of all weight coefficients must be equal to 1, and each weight coefficient must be a non-negative number.

[0084] This step uses the controlled variable method to generate a large number of random weight combinations that conform to real-world physical meaning for each weight value to be tested for the target parameter. These combinations constitute a statistical sample space for testing the sensitivity of the target parameter, providing a data foundation for subsequent evaluation of the volatility of the heatmap.

[0085] S3, input the data of the multiple task clusters into the heatmap calculation model in sequence, and apply the current weight combination to generate the corresponding heatmap sequence.

[0086] In this embodiment of the invention, the heatmap calculation model is a calculation module encapsulated according to a predefined deterministic algorithm logic. The model is essentially a parameterized calculation function, and its core algorithm is based on the weighted normalization processing method defined in step S102 above, without containing any internal parameters that need to be obtained through training. The technical solution for model construction is as follows:

[0087] a. Model input / output specifications:

[0088] The model input includes two elements:

[0089] Weighted combination vector: a set containing five weight coefficients (W1, W2, W3, W4, W5) that satisfy the constraint that the sum of the weight coefficients is 1 and any weight coefficient is greater than or equal to 0.

[0090] Task dataset: A collection of data containing path information and complete task features of historical tasks.

[0091] The model output is a server heatmap, which is represented by a graph data structure G=(V,E,H), where V represents the set of server nodes, E represents the set of topological connections between nodes, and H is the heatmap value h(v)∈[0,100] for each node v∈V.

[0092] b. Internal calculation logic of the model:

[0093] The model is configured to perform the following deterministic computation process:

[0094] Node task filtering: For each server node v in the cluster, based on the path information in the input task dataset, all tasks that have been run on that node are filtered out to form a node-specific task subset Dv;

[0095] Feature parameter calculation: For the task subset Dv of node v, calculate the feature parameters in five dimensions:

[0096] Number of tasks processed: obtained by counting the total number of tasks in Dv;

[0097] Priority weighted value: obtained by summing the products of the priority and resource consumption intensity of each task in Dv;

[0098] Resource usage level: obtained by calculating the arithmetic mean of the resource usage intensity of all tasks in Dv;

[0099] Cumulative load over time: obtained by summing the products of the runtime of each task in Dv and the resource consumption intensity;

[0100] Topology connectivity importance: Directly obtain the number of connection edges of node v in the cluster topology;

[0101] Parameter normalization: The five calculated feature parameters are subjected to minimum-maximum normalization and mapped to the interval [0, 1];

[0102] Weighted fusion of thermal values: Based on the weight combination of the input, according to the formula h(v)=100×(W1×N) norm +W2×P norm +W3×R norm +W4×T norm +W5×D norm Calculate the final thermal value of node v.

[0103] c. Heatmap sequence generation process:

[0104] After the model is built, generate a heatmap sequence by following these steps:

[0105] Task cluster traversal: The k representative task clusters obtained in step S1 are used as the task dataset input for the model in sequence;

[0106] Weight combination application: Use the current test weight combination as the model parameter input;

[0107] Batch computation execution: The model independently executes the above internal computation process for each task cluster to generate the corresponding heatmap;

[0108] Sequence construction: After processing all k task clusters, output a heatmap sequence consisting of k heatmaps.

[0109] This heatmap sequence objectively reflects the changing characteristics of resource distribution under different types of workloads with the same weight combination configuration, providing accurate quantitative basis for subsequent weight sensitivity analysis. The construction of this deterministic calculation model ensures the reliability of the weight optimization process.

[0110] S4 quantifies the overall fluctuation of the heatmap sequence. S2-S3 are executed multiple times to statistically analyze the heatmap fluctuation distribution characteristics corresponding to the current weight values ​​of the main parameters.

[0111] The purpose of this step is to perform quantitative analysis on the heatmap sequence generated by S3, and to evaluate the stability of the current master parameter weight values ​​by calculating the overall fluctuation of the sequence.

[0112] In one illustrative embodiment, the overall fluctuation level of the heatmap sequence is obtained through the following steps:

[0113] For each server node in the heatmap sequence, calculate the sample variance of its heatmap values ​​across all heatmaps;

[0114] The maximum value of the variance of all server nodes or the average value of the variance of all server nodes is used as the overall volatility of the heatmap sequence.

[0115] In another illustrative embodiment, the overall fluctuation level of the heatmap sequence is obtained through the following steps:

[0116] For each server node in the heatmap sequence, calculate the range of its heatmap values ​​across all heatmaps;

[0117] The average or maximum value of the ranges of all nodes is taken as the overall fluctuation of the heatmap sequence.

[0118] By repeatedly executing S2 to S3 multiple times, the heatmap fluctuation distribution characteristics corresponding to the current weight values ​​of the main parameters are established, specifically including:

[0119] Multiple rounds of testing: Under the condition of fixed main parameter weight values, S2 (generating random weight combination) and S3 (generating heat map sequence) are repeated m times (m≥30) to obtain m corresponding fluctuation degree measurement values, forming a fluctuation degree sample set.

[0120] Distribution feature extraction: Based on m fluctuation degree measurements, calculate its fluctuation distribution characteristics, including:

[0121] Central tendency of distribution: mean, median;

[0122] Distribution dispersion: standard deviation, interquartile range;

[0123] Distribution shape characteristics: skewness (measures the asymmetry of the distribution) and kurtosis (measures the sharpness of the distribution).

[0124] Stability assessment: The statistical characteristics of the degree of fluctuation are used as the basis for evaluating the stability of the current main parameter weight value. A lower average fluctuation and a smaller standard deviation indicate that the weight value can produce a stable heatmap under different weight combinations, that is, the weight value has a relatively robust impact on the overall system performance.

[0125] In this embodiment of the invention, the number of repetitions m can be determined in the following way:

[0126] Preliminary experimental determination method:

[0127] The appropriate number of repetitions, m, is determined through preliminary experiments to ensure the stability of the statistical results. Specifically, tests are conducted under typical workloads, and the number of repetitions is considered sufficient when the standard error of the estimated volatility decreases below a preset threshold.

[0128] Convergence Judgment Method (Preferred Implementation):

[0129] The convergence criterion is used to determine the number of repetitions: when the relative change of the average fluctuation value of 50 consecutive repetitions is less than 1%, the statistical characteristic is considered to have converged, and the repeated testing can be terminated. Through the above-mentioned volatility quantification and statistical analysis methods, we can: provide an objective and quantitative evaluation standard for the stability of the main parameter weight values; ensure the statistical significance of the evaluation results through multiple rounds of repeated testing; provide a reliable decision-making basis for subsequently determining the optimal weight value range; and control computational costs through intelligent termination conditions while ensuring statistical reliability.

[0130] S5, adjust the weight values ​​of the main parameters, repeat S2-S4, and determine the range of main parameter weight values ​​that stabilizes the fluctuation of the heat map within a preset acceptable range.

[0131] This invention provides a systematic method for determining the weight range of master parameters. In this process, the weight values ​​of the master parameters are first adjusted, and steps S2 to S4 are repeated to determine a weight value range that stabilizes the fluctuation of the heatmap within a preset acceptable range. The weight value adjustment strategy can employ a parameter space scanning method or a boundary refinement search method, with the boundary refinement search being the preferred implementation. The parameter space scanning method uses an equidistant sampling method.

[0132] Within the interval [0,1], iterate through all possible weight values ​​of the main parameter with a preset step size (such as 0.05 or 0.1);

[0133] For each candidate weight value, repeat the complete process S2-S4;

[0134] Record the statistical characteristics of the fluctuation level corresponding to each weight value.

[0135] The boundary refinement search method, based on the data obtained from the initial scan test, filters out the preliminary weight intervals with excellent performance from the entire weight value range of the main parameter according to a preset quantification standard. This quantification standard can be selecting consecutive test points that rank in the top percentage (e.g., the top 30%) in terms of volatility, or selecting consecutive test points with volatility below a set threshold. After initially determining the candidate intervals with lower volatility, a further refinement search strategy is employed to precisely define the weight value range of the main parameter. Specifically, within the candidate intervals, dense sampling and testing are performed with smaller step sizes (e.g., 0.01). By observing the trend of volatility changes with weight values, the critical points that exceed the preset acceptable threshold are identified, and these critical points are defined as the upper and lower boundaries of the parameter's weight value range.

[0136] Regarding the criteria for determining the preset acceptable range, this invention provides two implementation schemes: the absolute threshold method and the relative comparison method. The absolute threshold method sets an absolute upper limit threshold for the degree of fluctuation, including weight values ​​that simultaneously satisfy the condition that the average value and standard deviation of the fluctuation do not exceed their respective preset thresholds within the acceptable range. The relative comparison method performs a relative evaluation based on the performance of all tested weight values, allowing weight values ​​whose average fluctuation is at the lowest percentage to be included in the acceptable range, or weight values ​​whose fluctuation does not exceed a certain multiple of the minimum fluctuation. Finally, by analyzing the distribution of acceptable weight values ​​on the number axis, continuous acceptable intervals are identified and merged, while recording the upper and lower boundaries and length of each interval, thereby scientifically determining the range of the main parameter weight values.

[0137] S6, sequentially using other parameters as the main parameters, repeat S2-S5 until the weight range of all parameters is determined.

[0138] Before starting the loop, the order in which the parameters are selected as primary parameters for optimization must be determined. This order can be based on the preset importance of the parameters (e.g., optimizing the number of tasks processed first, then the priority weighting value), or a random order can be used. This defined order ensures the orderly nature of the optimization process.

[0139] In each iteration, a new principal parameter is selected from the feature parameters whose value range has not yet been determined, and then the process from S2 to S5 is repeated in its entirety. It is worth noting that when a new parameter is selected as the principal parameter, the weights of the remaining parameters are randomly reassigned while satisfying the summation constraint. This means that the value range of each parameter is determined independently against the backdrop of the "random fluctuations" of the weights of other parameters, thus ensuring the robustness and universality of the final result.

[0140] Once the value ranges for all five feature parameters (number of tasks processed, priority weighting, resource consumption level, cumulative load over time, and importance of topology connections) have been determined, this step outputs a global set of weight value ranges. This set contains five sub-intervals, each defining the allowable weight values ​​for the corresponding parameter while ensuring the stability of the heatmap.

[0141] Through iterative execution of this step, the local optimization of a single parameter is upgraded to the global coordination of multiple parameters, ensuring that the optimal weight combination in the final solution can simultaneously take into account the stability requirements of all dimensions, thus laying a solid foundation for generating accurate and reliable server heatmaps.

[0142] S7. Within the determined range of parameter weights, with the goal of maximizing the stability of the heatmap, the optimal weight combination that satisfies the conditions is solved by a constraint optimization algorithm.

[0143] This step aims to find the globally optimal weight configuration within the determined range of parameter weights using mathematical optimization methods. The core optimization objective is the stability of the heatmap, and the weight ranges obtained in the preceding steps are used as key constraints, which are then precisely solved using a constrained optimization algorithm.

[0144] Specifically, the optimization process aims to minimize the fluctuation of the heatmap, which can be quantified as the average or maximum variance of the node heatmap values. The constraints of the objective function include two parts: first, a basic probability constraint ensuring the sum of all weight coefficients is 1; and second, requiring each weight coefficient to fall within its respective value range determined in step S6. For algorithm selection, optimizers suitable for boundary-constrained problems, such as sequential least squares programming or interior-point methods, can be used. These algorithms can efficiently navigate complex multi-parameter spaces and find the optimal solution that satisfies all constraints. Through this systematic optimization framework, the final output is a weight combination that combines mathematical optimality and system stability. This combination ensures that the generated heatmap exhibits high stability and consistency under different workloads, thus providing the most reliable data foundation for subsequent GPU resource optimization decisions.

[0145] Those skilled in the art should understand that the mathematical principles and calculation process of the constraint optimization algorithm used in this invention are well-known technologies in the field of optimization theory.

[0146] The optimizer searches for the optimal solution that satisfies the constraints through iterative computation. Its general workflow includes: first, establishing an optimization model based on the objective function and constraints; then, generating an initial solution or initial solution set through an initialization process; next, entering the iterative optimization phase, determining the search direction by calculating the gradient of the objective function, the Hessian matrix, or their approximations, and determining the step size through line search or trust region methods; in each iteration, the algorithm simultaneously handles equality and inequality constraints to ensure that the solution always lies within the feasible region; finally, terminating the iteration based on convergence criteria (such as the change in the objective function being less than a threshold, the change in the solution being less than a threshold, or the gradient norm being less than a threshold).

[0147] Specifically, the sequential quadratic programming algorithm employed in this invention solves the original problem by transforming it into a series of quadratic programming subproblems; while the interior-point method transforms the constrained optimization problem into an unconstrained optimization problem by introducing a barrier function, or iteratively solves the problem in the primal-dual space. The specific implementation details of these algorithms are well documented in classic works such as "Numerical Optimization" and "Optimization Methods," as well as relevant academic literature, and belong to the standard knowledge system of those skilled in the art.

[0148] The contribution of this invention to the prior art lies not in improving the optimization algorithm itself, but in constructing the weight determination problem in GPU resource optimization as a specific constrained optimization problem. This includes defining a mathematical expression with heatmap stability as the objective function and establishing a constraint system centered on the weight value range. This specific problem construction method and application scenario are the innovations of this invention, while the specific solution process of the optimizer belongs to well-known techniques and conventional operating methods in this field.

[0149] This invention further provides a visualization scheme and data structure definition for server heatmaps. For visualization, the calculated heatmap values ​​are presented intuitively using color mapping technology. Cool blue tones represent relatively low heatmap value ranges (0-33), indicating server nodes with relatively relaxed resource utilization; neutral green tones represent medium heatmap value ranges (34-66), indicating server nodes with normal load utilization; and warm red tones represent high heatmap value ranges (67-100), indicating server nodes with high or near-saturated resource utilization. This progressive color mapping scheme allows operations and maintenance personnel to quickly perceive and understand the cluster resource status.

[0150] At the data structure level, the final generated server heatmap is organized using a graph data structure. Nodes represent physical servers or virtualized computing nodes in the cluster, with each node maintaining its independent resource pool status information. Edges represent various network connections between nodes, including physical links, logical channels, and resource-sharing paths. The heat value attached to each node serves as a core attribute, comprehensively representing the node's resource consumption intensity over historical operating cycles and its criticality within the overall cluster topology. This complete data structure and visualization scheme together form the situational awareness foundation for subsequent resource optimization decisions.

[0151] S200 predicts future integrated GPU resource requirements based on the queue of tasks to be scheduled and historical task data, in order to generate a resource demand heatmap.

[0152] This step aims to generate a resource demand heatmap that proactively indicates the demand for integrated GPU resources by intelligently predicting the future workload of the cluster. The specific implementation process is as follows:

[0153] S201, Data Preparation and Feature Engineering.

[0154] The current queue of tasks to be scheduled can be obtained from the cluster scheduler. This queue contains information on all tasks that have been submitted but not yet allocated resources. Simultaneously, historical task data within a preset time window is extracted from the historical database. This data is then processed as follows:

[0155] Task feature extraction: For both scheduled tasks and historical tasks, key features are extracted, including but not limited to task submission time, number of GPUs requested and memory size, task priority, user or queue identifier, and (for historical tasks) actual runtime.

[0156] Time series construction: Historical task data is aggregated at fixed time intervals (e.g., every minute or every 5 minutes) to construct multiple time series. The core time series include:

[0157] A sequence of tasks arriving for specific resource specifications: For example, a sequence of the number of tasks that request “8 or more” GPUs that arrived in each time interval over the past N hours.

[0158] Queue backlog sequence: The sequence of changes over time in the total number of tasks grouped by different resource specifications in the current queue to be scheduled.

[0159] S202, Prediction Model Selection and Training.

[0160] The embodiments of the present invention can use time series prediction models to model and predict the above-mentioned sequences. The models include, but are not limited to, classic statistical models such as the Autoregressive Integrated Moving Average (ARIMA) model, or deep learning models such as the Long Short-Term Memory (LSTM) network.

[0161] The ARIMA model is suitable for stationary time series exhibiting significant autocorrelation and trends. Its model parameters (p, d, q) can be identified and determined using the box-Jenkins method based on the autocorrelation plot (ACF) and partial autocorrelation plot (PACF), and then fitted using maximum likelihood estimation.

[0162] LSTM models, as a type of recurrent neural network (RNN), are particularly adept at capturing long-term dependencies in time series data. Their network structure typically includes an input layer, one or more LSTM layers, optional dropout layers to prevent overfitting, and an output layer. The model uses historical time series data as input for supervised learning, training the network weights by minimizing prediction errors (such as mean squared error) through backpropagation and gradient descent algorithms (e.g., the Adam optimizer).

[0163] Model selection and hyperparameter tuning is an iterative process, with the ultimate goal of achieving the best prediction accuracy on the validation set.

[0164] S203, Execution of Future Demand Forecasts.

[0165] Using a trained model, rolling forecasts of resource demand in the near future (e.g., the next 30 minutes) are made: the latest time-series data is input into the model to predict the probability or expected number of tasks arriving at different integrated resource specifications (e.g., "4 consecutive cards", "8 consecutive cards", "16 cards or more", etc.) at each subsequent time point. This process integrates known information about large tasks that are about to be scheduled from the scheduling queue with the model's probabilistic predictions of unknown tasks that may be submitted in the future.

[0166] S204, Resource Demand Heatmap Generation.

[0167] Finally, the prediction results are combined into a resource demand heatmap. This heatmap is a multi-dimensional data structure, and its core elements include:

[0168] Time dimension: The horizontal axis represents future time points (e.g., t+1, t+2, ..., t+30 minutes).

[0169] Resource Specification Dimension: The vertical axis or different layers represent different integrated resource specification modes.

[0170] Popularity score: The value of each cell (time point, resource specification) in the graph represents the expected demand intensity for a specific resource specification pattern at that future time point. This intensity can be the predicted number of tasks, the probability of arrival, or a normalized comprehensive score.

[0171] The heat index is visualized using a color gradient. For example, a gradient from blue (low demand) to red (high demand) can be used to make it easy for operations and maintenance personnel or subsequent decision-making modules to identify when and what resources will be most urgently needed in the future "demand hotspots".

[0172] Through the above technical solution, the present invention realizes the transformation from passive response to active prediction, providing a key time window and accurate data basis for subsequent fragmentation risk identification and optimization strategy decision-making.

[0173] S300, based on the resource demand heatmap and the server heatmap, uses comparative analysis to quantify and identify the risk of GPU resource fragmentation.

[0174] This step is used to quantitatively identify GPU resource fragmentation risks based on resource demand heatmaps and server heatmaps. This step constitutes the core of the system's intelligent decision-making, enabling proactive risk detection and early warning by forward-lookingly comparing future resource demands with current and future resource potential.

[0175] In this embodiment of the invention, GPU resource fragmentation specifically refers to a systemic resource allocation dilemma that arises in a shared GPU cluster resource environment. Its core characteristics are: from the perspective of overall cluster resource statistics, the total amount of currently available idle GPU resources is numerically sufficient to meet the resource requirements of newly submitted tasks; however, due to the dynamic randomness of historical task scheduling and resource release, these idle resources exhibit highly dispersed fragmentation in physical distribution, distributed across multiple different physical nodes or discontinuous regions within nodes, failing to form continuous, complete resource allocation units with specific topologies that meet task requirements. GPU resource fragmentation risk refers to the potential in this environment where, due to the dynamic nature of resource allocation and release, physically idle GPU resources cannot be effectively integrated and utilized, thus hindering the successful scheduling of tasks requiring continuous, large-scale, or specific topology GPU resources. This risk manifests as a systemic contradiction between resource sufficiency and resource availability: although the cluster as a whole has sufficient resource supply capacity, it cannot provide effective resource allocation for tasks requiring specific resource blocks, ultimately leading to negative impacts such as task scheduling delays and decreased system throughput. This risk is characterized by dynamic evolution, constantly changing with the workload of the cluster, and requires forward-looking prediction and quantitative assessment to achieve effective risk control.

[0176] This invention overcomes the inherent limitations of traditional schedulers by introducing a multi-source information fusion risk assessment mechanism. Specifically, by correlating and comparing server heatmaps representing real-time internal status with resource demand prediction maps representing future external demand, a novel and forward-looking resource fragmentation risk indicator is generated. This innovation expands the perspective of scheduling decisions from a single "current state" to "the degree of matching between the future state and the current state," enabling the system to proactively trigger scheduling strategies to avoid resource fragmentation before it truly becomes a performance bottleneck. This achieves a fundamental shift from passive response to proactive prevention, completely resolving the persistent technical problem of cluster performance fluctuations caused by response lag in traditional systems.

[0177] Furthermore, the S300 specifically includes the following steps:

[0178] S310, compare the resource demand heat map with the future resource distribution map simulated based on the current resource status, and identify server nodes whose idle resource specifications cannot meet the requirements of any integrated task predicted in the resource demand heat map in the future resource distribution by analyzing the degree of matching between the future resource distribution and the predicted demand, as candidate nodes.

[0179] In this embodiment of the invention, the future resource distribution map is obtained by dynamically simulating the current state of the cluster, and its specific generation process includes the following steps:

[0180] (1) Real-time resource status acquisition: Collect real-time resource status data of the cluster. This data includes at least: current resource allocation details of each server node, the predicted remaining running time of the currently executing tasks, and the dependency and priority information between tasks.

[0181] (2) Remaining task time prediction: Based on accumulated historical task data, a task execution model is constructed, and machine learning methods (such as random forest, gradient boosting tree, etc.) are used to predict the remaining running time of each running task. The task execution model can be characterized as: Remaining running time = f(task type, running time, resource consumption pattern, historical similar tasks), where the function f(·) is the prediction function obtained through training, and its specific form is determined by the selected machine learning algorithm and its parameters.

[0182] (3) Future Resource Release Simulation: Based on the predicted remaining runtime of the tasks, simulate the resource release of the cluster at a specific future time point (e.g., 30 minutes in the future). This simulation process specifically includes:

[0183] Simulate the completion time of each running task and the resources it releases in a discrete manner according to the time series.

[0184] The simulation considers the uncertainty of task execution and introduces the probability distribution of early completion or delayed completion of the task.

[0185] Simulate the chain reaction of resource release effects caused by inter-task dependencies to ensure the consistency and realism of resource release logic.

[0186] (4) Future Resource Distribution Map Synthesis: Integrating current instantaneous idle resources with simulated predicted release resources, the resource distribution at future target time points is calculated to generate a future resource distribution map (or future resource snapshot). Specifically:

[0187] This future resource distribution map, in a structured data format, comprehensively describes the expected available resource specifications for each node in the cluster at a future target time, mainly including but not limited to: the number of available GPUs, video memory capacity, number of CPU cores, and memory size. The core synthesis principle of the future resource distribution map is as follows: the total expected available resources at a future time are equal to the currently observed idle resources, plus the sum of all predicted resources to be released from the current time to the future target time.

[0188] Through the above steps, a future resource distribution map reflecting the cluster resource layout at a certain point in the future can be constructed in a forward-looking manner, providing key input for subsequent quantitative identification of resource fragmentation risks.

[0189] In this embodiment of the invention, the matching degree is evaluated through a quantified comprehensive matching degree, calculated as follows: Comprehensive matching degree = k1 × Specification matching degree + k2 × Time matching degree + k3 × Topology matching degree, where k1 to k3 are preset weighting coefficients, satisfying k1 + k2 + k3 = 1. The calculation methods for the matching degree of each dimension are as follows:

[0190] Specification matching degree: Used to evaluate the degree of match between the available resources of a node and the resources required by the task. The calculation formula is: Specification matching degree = 1 - |Available node resource specifications - Required task resource specifications| / Required task resource specifications. The closer this value is to 1, the more perfect the specification matching.

[0191] Time matching degree: Used to evaluate the degree of alignment between the available time of resources and the time window of task requirements. The calculation formula is Time matching degree = Overlap time / Requirement duration.

[0192] Topology matching degree: used to evaluate the ability of node topology to support task execution. Its calculation takes into account the network location advantages of nodes and cluster connectivity. Specifically, it can be calculated by comprehensively considering factors such as node degree centrality, task communication requirements and the matching degree of available bandwidth between nodes.

[0193] Based on the above matching degree calculation, the embodiments of the present invention use the following judgment conditions to mark nodes as candidate nodes:

[0194] When the specification matching degree is lower than the first preset threshold (e.g., 0.8);

[0195] Furthermore, if the overall matching degree based on this is simultaneously lower than the second preset threshold (e.g., 0.7);

[0196] Alternatively, when a node has an integration task requirement that cannot be met at all, i.e., its specification matching degree is 0.

[0197] If any of the above conditions are met, the system will mark the server node as a candidate node and proceed to the subsequent risk quantification process. This decision-making strategy ensures that it can identify nodes with insufficient resource specifications as well as nodes with potential risks due to time or topology mismatches.

[0198] In this embodiment of the invention, the integrated task refers to a computing task with specific integration requirements for GPU resources. Its core characteristics lie in the indivisibility and topological correlation of resource requirements. Such tasks need to acquire resource combinations that conform to a specific organizational structure physically or logically, rather than discretely distributed scattered resources. The integrated task is specifically defined by the following characteristics: in terms of resource scale, it requires a single node to provide no less than four GPUs of continuous computing resources or to coordinate no less than eight GPUs across nodes; in terms of memory capacity, it requires a total of no less than 64GB; in terms of resource continuity, it requires continuously distributed GPU device numbers or specific GPU topology connections; in terms of scheduling characteristics, it exhibits high scheduling difficulty, and its resource requirements are typically met by no more than 10% of the nodes in the cluster individually.

[0199] S320, for each candidate node, calculate the estimated load value of the candidate node in the future resource distribution map, and compare the estimated load value with the dynamic reference value calculated by the server heat map. Based on the comparison result, generate a quantitative fragmentation risk score, which reflects the degree of risk of the node facing insufficient resources in the future.

[0200] First, calculate the dynamic reference value used for risk comparison, which can be calculated using any of the following methods:

[0201] Method A: Based on the heat values ​​of all nodes in the server heatmap, calculate the difference between the highest and lowest heat values ​​of all nodes, and divide this difference by the total number of nodes to obtain the average resource value as a dynamic reference value.

[0202] Method B: Calculate the fluctuation of thermal values ​​at all nodes. When the fluctuation is less than a preset threshold, the average thermal value is used as the dynamic reference value. When the fluctuation is greater than or equal to the preset threshold, the average thermal value after removing the highest and lowest values ​​is used as the dynamic reference value, thereby eliminating the influence of extreme values ​​on the reference benchmark.

[0203] In this embodiment of the invention, the degree of fluctuation is used to quantify the dispersion of node heat values ​​in the server heat map, and can be the variance or the coefficient of variation. The coefficient of variation is the ratio of the standard deviation to the average value of all node heat values.

[0204] The preset threshold can be determined in the following ways:

[0205] For variance metrics, the preset threshold can be set to the median or the 75th percentile of historical variance data;

[0206] For the coefficient of variation, the preset threshold can be set to a value between 0.1 and 0.3, preferably 0.15;

[0207] The specific value of the preset threshold can be determined through historical data analysis or cluster analysis methods, clearly dividing the cluster status into two categories: balanced and unbalanced.

[0208] Method C: The heat values ​​are redistributed based on the topological structure relationship. Specifically, this includes: identifying nodes that are in the core hub position in the topology and have abnormal heat values, weighting and correcting the abnormal heat values ​​based on the number of edges they are connected to, and then using the corrected heat map to calculate reference values.

[0209] Among them, nodes located in the core hub position can be those in the top K% (e.g., the top 20%) of the overall centrality ranking. The overall centrality is a weighted fusion value of betweenness centrality, degree centrality, and eigenvector centrality.

[0210] Nodes with abnormal thermal values ​​are those whose Z-score of thermal value is greater than a preset value, such as 2. Alternatively, using a box plot method, nodes whose thermal values ​​exceed 1.5 times the interquartile range are identified as abnormal.

[0211] For identified anomalous nodes, a thermal value correction is performed based on their topological importance: hc = h × [1 + α × (CC)] min ) / (C max -C min ]], where hc is the corrected thermal value, C is the topological centrality score of the node, C max C min These represent the maximum and minimum centrality scores for all nodes, respectively; α is an adjustment factor ranging from -0.3 to 0.3, with its sign depending on the anomaly direction. Using the corrected heatmap data, calculate the dynamic reference value using any of the following methods:

[0212] Calculate the weighted average of the corrected thermal values ​​for all nodes, with the weights being the topological importance of the nodes;

[0213] The distribution of thermodynamic values ​​is fitted using the kernel density estimation method, and the mode or median of the distribution is taken as the reference value.

[0214] The cluster is divided into multiple sub-communities based on the community detection algorithm. Reference values ​​for each community are calculated and then combined.

[0215] Method C, by incorporating topology information, effectively eliminates thermal value deviations caused by network location differences, making it particularly suitable for heterogeneous cluster environments. When there are obvious topological characteristics within the cluster, this method can provide more accurate dynamic reference values, thereby improving the accuracy of fragmentation risk identification. Furthermore, this method can adapt to clusters of different sizes, maintaining the stability of the calculation results by adjusting parameters.

[0216] Next, the estimated load value of the candidate node in the future resource distribution map is calculated. This load value is obtained by analyzing the resource commitment and capacity ratio of the node at a specific future point in time. Specifically, it is the ratio of the amount of resources that the node has already reserved or is expected to be occupied to its total resource capacity. In the calculation process, the resource requirements of confirmed tasks to be scheduled and the potential task resource requests predicted based on historical patterns are comprehensively considered, thus forming a comprehensive estimate of the node's future load status.

[0217] Then, after obtaining the estimated load value, it is systematically compared with the dynamic reference value calculated in the previous steps. This comparison process considers not only the absolute numerical differences but also analyzes the degree of relative deviation and its statistical significance in the overall cluster environment. Based on the comparison results, a fragmentation risk score is generated through a preset risk quantification function, which weights and integrates factors such as the deviation between the load value and the reference value, the trend of deviation changes, and the topological importance of the node in the cluster.

[0218] The fragmentation risk score generation mechanism employs a non-linear mapping approach. When the estimated load exceeds the dynamic reference value, the risk score increases exponentially to accurately reflect the rapid increase in resource overload risk. The scoring model also incorporates a topology impact factor. For candidate nodes located at network core hubs, the potential cascading effects of resource shortages are included in the score calculation, thereby appropriately increasing their risk level. The final fragmentation risk score is a normalized numerical value, typically ranging from 0 to 100. This score quantitatively characterizes the likelihood and severity of resource shortages faced by candidate nodes within a future time window, providing precise data for subsequent optimization decisions.

[0219] In this embodiment of the invention, the fragmentation risk score is obtained by multiplying a base risk score, a topological importance coefficient, and a time urgency factor. The base risk score is calculated based on the difference between the estimated load value and the dynamic reference value of a candidate node, and its value increases non-linearly with the increase of the difference. The topological importance coefficient is determined based on the relative position of a node's degree centrality within the cluster, and its value increases with the increase of the node's connectivity centrality. The time urgency factor is determined based on the proximity of the predicted risk occurrence time, and its value increases as the risk occurrence time approaches. In an illustrative embodiment, the fragmentation risk score is quantitatively evaluated using the following formula:

[0220] Fragment risk score = Basic risk score × Topological importance coefficient × Time urgency factor.

[0221] Wherein, the basic risk score = 50 × [1 + tanh((LR) / R)], L is the estimated load value of the candidate node in the future resource distribution map, R is the dynamic reference value, and tanh is the hyperbolic tangent function, which smoothly maps the difference value to the interval [-1,1].

[0222] Topological importance coefficient = 1 + 0.5 × (DD) min ) / (D max -D min ), where D is the degree centrality of the node (the number of connecting edges), D min D is the minimum degree centrality of nodes in the cluster. max This represents the maximum degree centrality of nodes in the cluster.

[0223] Time urgency factor = 1 + 0.3 × (T) remaining / T total ), where T remaining T represents the time difference between the current time and the predicted time of risk occurrence. total This represents the total forecast time window length.

[0224] Where L = α × (currently allocated resources / total resource capacity) + β × (predicted new resource demand / total resource capacity), α and β are weighting coefficients, satisfying α + β = 1, with α = 0.6 and β = 0.4 being preferred. The currently allocated resources are calculated based on the resource usage of running tasks, and the predicted new resource demand is based on the expected task demand for this node in the resource demand heatmap.

[0225] The risk levels can be categorized as follows: 0-30 points: low risk (marked in green); 31-70 points: medium risk (marked in yellow); 71-100 points: high risk (marked in red).

[0226] This scoring formula introduces a non-linear function, tanh, to ensure that risk increases gradually when the load is close to the reference value, while it rises sharply when the load significantly exceeds the reference value, accurately reflecting the marginal increasing effect of resource shortage risk. At the same time, the formula comprehensively considers the importance of the node's topology and the urgency of the risk's occurrence, making the scoring results more consistent with actual operation and maintenance needs and providing a precise quantitative basis for the selection of subsequent optimization strategies.

[0227] S330, the candidate nodes whose fragmentation risk scores are higher than a preset threshold are marked as nodes to be optimized, providing a clear processing target for subsequent resource optimization strategy selection.

[0228] In a preferred embodiment, the preset threshold can be set to 60 points. Candidate nodes with a fragmentation risk score higher than this value will be marked as nodes to be optimized, providing a clear processing target for the selection of subsequent resource optimization strategies.

[0229] S400, in response to the risk of fragmentation of quantized GPU resources, selects the optimal optimization strategy from a pre-built strategy library.

[0230] In S400, the optimal optimization strategy is selected from a pre-set strategy library based on a multi-objective optimization function; the multi-objective optimization function aims to balance the execution cost and expected benefits of the optimization strategy.

[0231] The multi-objective optimization function specifically considers the following two core optimization objectives: The primary objective is to minimize the execution performance overhead of the optimization strategy, which includes, but is not limited to, the computational resource consumption, network bandwidth consumption, task interruption time, and potential impact on the normal service quality of the cluster during task migration; the secondary objective is to maximize the system throughput improvement brought about by the successful scheduling of future large tasks, which is measured by estimating the increase in the number of tasks completed per unit time resulting from the execution of previously blocked large-scale resource tasks after resource optimization.

[0232] The pre-built strategy library includes a variety of lossless or lightweight resource management methods, forming a set of selectable behavioral strategies. These mainly include three core strategies: a dynamic migration strategy, suitable for task types that support checkpoint saving and restoration, which achieves task location transfer through state saving and restoration; an elastic scaling strategy, suitable for deep learning tasks that support elastic training, which achieves resource adaptation by dynamically adjusting the scale of computing resources used by the task; and a task packaging strategy, which integrates multiple small tasks running on the same node into fewer nodes through resource reuse technology, improving resource integration.

[0233] When selecting the optimal strategy, a complete evaluation profile is established for each candidate strategy in the strategy library, quantifying its specific performance across two objective dimensions: performance overhead and system throughput improvement. The evaluation results are comprehensively compared and used for decision-making through a weighted summation method or a Pareto optimality method. The weighting coefficients are dynamically adjusted based on the cluster's real-time status and operational strategies—prioritizing performance reduction during peak business periods, while focusing on throughput improvement potential when resources are severely fragmented. By systematically solving this multi-objective optimization problem, the optimization strategy with the highest overall benefit in a specific scenario can be identified, providing clear guidance for subsequent implementation.

[0234] Furthermore, the strategy selection process includes a strategy compatibility assessment mechanism that comprehensively considers the execution dependencies and potential conflicts between different strategies, ensuring that multiple optimization operations can be performed collaboratively and avoiding resource conflicts and state inconsistencies. The strategy execution order has also been optimized and orchestrated to minimize the impact on normal services. The ultimately selected optimal strategy achieves the best balance between short-term execution costs and long-term system benefits, ensuring continuous optimization of cluster resource utilization efficiency and stable improvement in system performance.

[0235] Furthermore, the S400 specifically includes:

[0236] S401, Establish a multi-objective optimization model that includes the following elements:

[0237] Decision variables: Define the strategy selection variable X = [x1, x2, ..., x...] h ], where x r ∈[0,1] represents the execution strength of the r-th strategy, where r ranges from 1 to h, and h is the total number of available strategies in the strategy library. When x r =0 indicates that the strategy is not executed. r =1 indicates that the strategy is fully executed, while intermediate values ​​indicate partial execution strength.

[0238] Objective function:

[0239] Minimize the execution cost function: f1(X)=Σ r (c resource r +c network r +c downtime r )×x r ; where c resource r c represents the computational resource cost during the execution of strategy r. network r c represents the network bandwidth consumption cost of strategy r. downtime rLet Σ represent the service quality loss cost caused by the task interruption time of strategy r, and let Σ represent the summation symbol.

[0240] Maximize throughput revenue function: f2(X) = Σ s=1 Q (throughput gain s ×I s (X) is used to measure the total expected system throughput gain after the execution of strategy combination X, which was previously hindered by resource fragmentation. Where throughput... gain s This represents the throughput increase resulting from the successful scheduling of the s-th initially blocked integration task, where s ranges from 1 to Q, Q is the number of integration tasks, and I... s (X) is a mapping function that returns either 0 or 1, used to determine whether the cluster's resource status can meet the scheduling requirements of task s after the execution of strategy combination X. I returns 0 if and only if the resources released after the execution of strategy combination X can meet the requirements of task s. s (X) = 1. Constraints:

[0241] Resource constraints: Σ r (resource_demand r )×x r ≤available_resources, ensuring that the total resource demand of all policies does not exceed the total available resources of the cluster; where resource_demand r The resource requirement is defined by strategy r, and available_resources represents the total amount of currently available resources.

[0242] Time constraint: Σ r (execution_time r )×x r ≤time_window ensures that the total execution time of the strategy combination is completed within the preset time window. execution_time r The execution time of strategy r is specified by time_window, which is the preset time window.

[0243] Policy Mutual Exclusion Constraint: x a +x b ≤1 (when strategies a and b are technically not feasible to execute simultaneously), to avoid technical conflicts and execution interference between strategies.

[0244] This multi-objective optimization model uses the quantified results of fragmentation risk as a key input parameter. By balancing execution costs and system benefits, it provides a quantitative basis for selecting resource optimization strategies. During the model solution process, the system automatically adjusts the execution intensity configuration of each strategy based on the distribution characteristics and severity of fragmentation risk, ultimately outputting the strategy execution plan with the optimal overall benefits under a specific risk scenario, thereby achieving continuous optimization of resource utilization efficiency.

[0245] S402 performs a multi-dimensional quantitative evaluation of each candidate strategy in the strategy library to obtain the corresponding evaluation results.

[0246] In this embodiment of the invention, the multi-dimensional quantitative evaluation includes two core dimensions: execution cost evaluation and expected benefit evaluation.

[0247] Execution cost assessment is achieved by establishing a cost calculation model that outputs a quantitative value of the comprehensive cost required for policy execution. The specific assessment process includes: 1) Computational resource occupancy cost, obtained by weighted summation of the products of various computing resources (including CPU cores, GPUs, and memory capacity) used during policy execution and their duration of occupancy; 2) Estimating network transmission cost, calculated by multiplying the transmission time by the ratio of the total amount of data migrated by the policy-related tasks to the available bandwidth of the cluster, and then multiplying by the bandwidth cost per unit time; 3) Assessing service impact cost, quantifying the degree of service quality degradation by analyzing the product of task interruption time caused by policy execution and the priority weight of the affected tasks. Finally, the execution cost assessment outputs a standardized cost scalar value that comprehensively reflects the degree of system resource consumption by policy execution.

[0248] The expected benefit assessment is achieved by establishing a benefit prediction model, which outputs a quantifiable value of the system benefits that the strategy execution may bring. The specific assessment process includes: analyzing the resource integration effect by simulating the resource distribution state after strategy execution and calculating the size and quantity distribution of the maximum released contiguous resource blocks; estimating throughput improvement by statistically analyzing the resource integration effect results, counting the number of integrated tasks that become schedulable, and converting the expected runtime of these tasks into system throughput gain; and considering long-term benefits by analyzing the effect of strategy execution on improving the fragmentation of cluster resources and assessing its continuous improvement effect on task scheduling success rate over a future period. Finally, the benefit prediction model outputs a standardized scalar value of benefit, which comprehensively reflects the immediate and long-term benefits that strategy execution brings to the system.

[0249] Through the above systematic evaluation process, each candidate strategy obtains a set of quantitative evaluation results including cost and benefit values, providing accurate numerical basis for subsequent multi-objective optimization decisions. S403, Multi-objective Decision Analysis, transforms the evaluation results of candidate strategies into the selection of the optimal strategy.

[0250] This invention employs a systematic multi-objective decision-making method. First, it performs adaptive weight adjustment, calculating the weight A1 of the execution cost objective based on the real-time cluster load status. The cluster load is quantified by a weighted average of CPU / GPU utilization, with baseline weights set for three time periods: peak, normal, and off-peak. Simultaneously, the weight A2 of the throughput benefit objective is determined based on operational strategy preferences, comprehensively considering optimization bias and SLA level requirements. Finally, the two weights are normalized to ensure A1 + A2 = 1. This adjustment mechanism enables the system to dynamically balance the relative importance of different objectives according to the actual operating environment.

[0251] In one illustrative embodiment, A1 = c1 × L norm +c2×T factor +c3×P bias , where L norm The normalized cluster load level is calculated using the following formula: L norm =(Current CPU utilization × Wcpu + Current GPU utilization × W) gpu +Current memory utilization × W mem ) / (W cpu +W gpu +W mem ), where W cpu W gpu W mem The weighting coefficients for CPU, GPU, and memory are respectively, satisfying W cpu +W gpu +W mem =1; T factor This represents the time period factor, which takes a value based on the time period in which the system operates: Peak business hours: T factor =0.8; During normal business hours: T factor =0.5; Business off-peak period: T factor =0.2. P bias This indicates a strategy preference bias, determined based on the operational strategy configuration: when the optimization objective is biased towards cost control: P bias =+0.2; When the optimization objective is biased towards performance improvement: P bias =-0.2; When the optimization objective remains balanced: P bias =0; the adjustment coefficients from c1 to c3 satisfy c1+c2+c3=1, with the preferred values ​​being c1=0.5, c2=0.3, and c3=0.2.

[0252] The weights are ultimately made reasonable by the following constraints: A1=max(0.1,min(0.9,A1)), A2=1–A1.

[0253] After determining the weights, the NSGA-II algorithm is used to generate a Pareto optimal solution set. This algorithm stratifies the individuals in the solution set according to their quality through fast non-dominated sorting, maintains the diversity of solutions using crowding calculation, and iteratively evolves the population through selection, crossover, and mutation operations, ultimately outputting a set of non-dominated optimal solutions. The specific performance of each solution on the execution cost function f1(X) and throughput benefit function f2(X) is calculated, and based on these values, a non-dominated solution set forming the Pareto front is identified.

[0254] Finally, a final decision is made. Based on weighted Chebyshev distance, the solution closest to the ideal point is selected from the Pareto front. The ideal point is composed of the optimal values ​​achievable by each of the two objective functions, with the distance calculation considering the weighted allocation of each objective. Alternatively, a fuzzy decision-making method can be used, transforming the objective values ​​into satisfaction indicators by constructing membership functions for each objective, and then comprehensively evaluating them based on the decision-maker's preferences. The final selected solution is ensured to satisfy all constraints, including resource constraints, time constraints, and policy exclusivity constraints, thus guaranteeing the feasibility of the solution. This multi-objective decision analysis process ensures that the final selected strategy maximizes overall benefits under complex constraints.

[0255] S404 performs systematic compatibility verification and execution sequence optimization on the preliminary strategy selected through multi-objective decision analysis to ensure the feasibility and efficiency of strategy execution.

[0256] During the resource conflict detection phase, a comprehensive resource feasibility verification mechanism is established. First, the availability of computing resources required for policy execution is verified, including checking whether the target node has sufficient GPU, CPU, and memory resources to support policy operations. Second, network communication requirements are verified, ensuring that the available bandwidth between nodes can meet the transmission requirements of task migration or data synchronization, while confirming that the IOPS capacity of the storage system supports the expected data read / write load. Finally, resource deadlock prevention analysis is performed, using a resource allocation graph algorithm to detect potential circular wait conditions during policy execution, ensuring that no new system deadlock risks are introduced due to policy execution.

[0257] In the execution sequence optimization phase, a policy dependency graph model is constructed, where nodes represent individual policies and directed edges represent the execution order dependencies between policies. Based on this dependency graph, a topological sorting algorithm is used to generate a linear execution sequence that meets all order constraints. During the sorting process, both positive synergistic effects and negative conflict avoidance between policies are considered simultaneously. Policies that can generate resource release effects are prioritized for execution, creating more favorable execution conditions for subsequent policies. For policy pairs with resource contention conflicts, the conflict is eliminated by adjusting the execution order or introducing time intervals.

[0258] Through the above verification and optimization process, a final strategy solution that has been verified for feasibility and optimized for execution sequence is output. This solution effectively ensures the smooth implementation of the strategy, maximizes the overall benefits of the strategy, and avoids unnecessary interference with existing services.

[0259] S405 conducts a comprehensive risk assessment of validated optimization strategies and develops corresponding alternatives to ensure security and robustness during strategy execution.

[0260] In the risk quantification and analysis phase, a multi-dimensional risk assessment model is established. First, a probabilistic risk assessment method is employed to quantify the potential probability of strategy execution failure based on historical execution data and the current system state, and to assess the specific scope and extent of the impact of failure scenarios on system service quality and task progress. Simultaneously, the complexity and operational costs of the rollback plan are evaluated in detail, including analyzing the resource overhead, time consumption, and complexity of cleaning up incomplete operations required for the rollback process. Furthermore, the worst-case system state evolution path is simulated to identify potential cascading failure risks, and corresponding isolation and recovery contingency plans are developed.

[0261] During the alternative strategy preparation phase, a tiered emergency response mechanism is established. Based on multi-objective optimization results, the second-best strategy, ranking second only to the primary strategy in comprehensive evaluation, is selected as a backup plan to ensure rapid switching should the primary strategy encounter obstacles. For complex or high-risk primary strategies, they are decomposed into multiple phases, and progressive execution plans are developed. By setting intermediate checkpoints and acceptance criteria, phased risk control and timely intervention are achieved. Simultaneously, a real-time monitoring and circuit breaker mechanism is established, defining monitoring thresholds for key performance indicators. Once an abnormal indicator is detected or the execution effect deviates from expectations, a circuit breaker operation is immediately triggered, suspending strategy execution and activating the alternative plan.

[0262] Finally, a complete optimization decision-making solution is output, which includes the following core components: clearly specifying the primary strategy and its detailed execution parameter configuration; providing a complete expected cost and benefit analysis report, covering key indicators such as resource consumption and performance improvement; accompanying a comprehensive risk assessment result and a list of targeted countermeasures; establishing a precise execution timeline and milestone nodes; and defining a key monitoring indicator system for tracking the strategy's execution effect. This complete output solution ensures that resource optimization operations are carried out in an orderly manner within a controlled environment, ultimately achieving a safe and effective improvement in cluster resource utilization efficiency.

[0263] S500 executes the selected optimization strategy to change the physical layout of GPU resources.

[0264] In this embodiment of the invention, the execution includes dynamically migrating tasks that support checkpoints and / or scaling resources for tasks that support elastic scaling.

[0265] For tasks that support checkpoints, a dynamic migration technique is used, specifically including:

[0266] (1) Control the checkpoint operation of the task to be migrated and save the task status to shared storage.

[0267] A checkpoint trigger command is sent to the task to be migrated. After receiving the command, the task serializes its complete runtime task state (including model parameters, optimizer state, learning rate scheduler state, and random number generator state) and saves it to the cluster shared storage system.

[0268] (2) Start a new task instance on the target node and resume execution from the checkpoint.

[0269] A new task instance is initialized on a pre-selected target node. This task instance loads the saved checkpoint file from shared storage and precisely resumes task execution from the point of interruption. The entire process utilizes RDMA network technology to achieve high-speed data transmission, minimizing task downtime and ensuring an unaffected user experience.

[0270] For tasks that support elastic scaling, a resource scaling approach is adopted, which specifically includes dynamically adjusting the number of GPUs used by the task through the elastic interface of the deep learning framework.

[0271] By invoking the elastic control interface of deep learning frameworks (such as PyTorch Elastic), the scale of GPU resources used by tasks can be dynamically adjusted. Specific operations include: submitting resource adjustment requests to the task management framework, coordinating the synchronous updating of task parallelism parameters across all participating nodes, and reallocating the computation graph and data flow according to the new resource scale. This ensures the integrity and consistency of the training state during resource adjustment, achieving smooth scaling of computing resources.

[0272] During strategy execution, a temporary locking mechanism is implemented for computing resources involved in the reconstruction operation. A distributed lock service is used to exclusively occupy the target resource, preventing race conditions during resource state transitions. Simultaneously, transactional operations are employed to ensure the atomicity of resource reconstruction, guaranteeing that the operation either succeeds completely or is completely rolled back, avoiding intermediate states. A consistency protocol ensures the state synchronization of all participating nodes, ultimately achieving the integrity of task states and data consistency during resource reconstruction, ensuring service continuity and reliability.

[0273] (Example 2)

[0274] This embodiment provides a GPU resource optimization system based on dynamic prediction and topology heatmaps, including:

[0275] The heatmap construction module is used to construct server heatmaps based on historical task characteristics and the number of node topology connection edges.

[0276] The resource demand prediction and risk identification module is connected to the heat map construction module. It is used to predict the future integrated GPU resource demand based on the task queue to be scheduled and historical task data to generate a resource demand heat map. Based on the resource demand heat map and the server heat map, it quantifies and identifies the risk of GPU resource fragmentation through comparative analysis.

[0277] The strategy decision engine, connected to the resource demand prediction and risk identification module, is used to select the optimal optimization strategy from the preset strategy library in response to the quantified GPU resource fragmentation risk.

[0278] A resource reconfiguration executor, connected to the policy decision engine, is used to execute the selected optimization policy to change the physical layout of GPU resources.

[0279] The GPU resource optimization system based on dynamic prediction and topology heatmaps provided in this invention corresponds completely to the aforementioned method embodiments, forming a unified technical system of apparatus and method. The heatmap construction module in the system specifically implements step S100 in the method embodiments, constructing a server heatmap by processing historical task data and topology information; the resource demand prediction and risk identification module implements steps S200 and S300 in the method embodiments, completing the entire process from demand prediction to risk quantification; the strategy decision engine is dedicated to executing step S400 in the method embodiments, making intelligent strategy decisions based on multi-objective optimization; and the resource reconstruction executor corresponds to step S500 in the method embodiments, responsible for transforming the optimization strategy into actual resource reconstruction operations. This correspondence between the system and the method ensures the consistency of the technical solution under different implementation forms. The various modules in the system work collaboratively to completely reproduce the prediction-decision-execution technology closed loop described in the method embodiments, providing a unified solution for GPU cluster resource optimization.

[0280] Example: Implementation in a kilocalorie-level AI training cluster

[0281] (I) System Deployment and Integration Configuration

[0282] The GPU resource optimization system provided in this embodiment of the invention is deployed and implemented using a custom Kubernetes scheduler and controller. The deployment environment is a thousand-card-level AI training cluster, specifically configured as follows: the cluster contains 1024 NVIDIA A100 80GB GPUs, interconnected via a high-speed InfiniBand network, divided into 128 compute nodes, each equipped with 8 GPU cards. The workload mode is hybrid, including large-scale distributed training tasks requiring 32, 64, or even 128 cards, as well as small to medium-sized inference and model fine-tuning tasks requiring a single card or a small number of GPUs. The system achieves deep integration with the cluster management system by extending the Kubernetes scheduling framework, and the monitoring component collects task execution data and resource status information in real time.

[0283] (II) Implementation of core workflows

[0284] 1. Forecasting and Risk Identification Stage

[0285] The system's task queue analyzer continuously monitors the scheduling queue and identifies three key tasks awaiting scheduling: Job_A is a large-scale language model pre-training task requiring 16 consecutive A100 GPUs, expected to arrive in the scheduling window in 30 minutes; Job_B is a multimodal model training task requiring 8 consecutive A100 GPUs, already in the ready queue; and Job_C is a small-batch inference task requiring only 2 GPUs. The fragmentation risk identification module, through resource simulation analysis, discovers that although the current cluster statistics show 20 idle GPUs, these resources are scattered across 8 different physical nodes, forming resource fragmentation. The system predicts that when Job_A arrives in 30 minutes, it will be forced to queue due to the inability to find 16 consecutive GPUs. Based on this, the scenario is identified as a "high-risk fragmentation event," and the decision engine is immediately triggered to intervene.

[0286] 2. Intelligent decision-making and solution planning

[0287] Upon receiving a risk alert, the strategy decision engine initiates a multi-objective optimization analysis process. The engine identifies that Node-X currently has 4 idle GPUs and 4 GPUs occupied by the low-priority training task Job_Low, while Node-Y possesses ample scattered GPU resources. Using a cost-benefit model, the engine calculates that migrating Job_Low from Node-X to Node-Y, although incurring approximately 45 seconds of task interruption and network transmission overhead, allows Node-X to consolidate 8 consecutive GPU cards, immediately meeting the scheduling requirements of Job_B. Furthermore, the resources released after Job_B's completion can be merged with other idle resources in the cluster, preparing the necessary 16 consecutive GPU cards for the upcoming Job_A. Quantitative evaluation shows that the overall benefit of the migration operation significantly outweighs the cost, and the system generates a specific migration execution plan accordingly.

[0288] 3. Non-destructive execution and state recovery

[0289] After receiving the approved migration plan, the resource refactoring executor initiates a lossless migration operation for Job_Low. The executor first sends a checkpoint save signal to the running Job_Low task. This PyTorch training task serializes and saves its complete runtime state, including model parameters, optimizer state, learning rate scheduler, etc., to the shared storage system; this process takes approximately 15 seconds. Subsequently, the executor starts a new task instance on the target node Node-Y, restores the task state from the checkpoint file, and Job_Low resumes its training process from where it was interrupted. The entire migration operation keeps the task interruption time within 45 seconds, with minimal performance loss. After the migration is complete, Node-X successfully frees up 8 consecutive GPU resources, and the scheduler immediately schedules Job_B from the queue to run on this node, effectively avoiding the dilemma of resource idleness and task queuing coexisting.

[0290] (III) Quantitative evaluation of implementation effectiveness

[0291] During a 30-day continuous operation in a production environment, the system's performance was quantitatively evaluated using comparative experiments. Compared to a baseline cluster using a traditional scheduler, the cluster deploying this invention achieved significant improvements in key performance indicators: the average queue waiting time for large-scale training tasks requiring 16 or more GPUs was reduced from 4.5 hours to 2.1 hours, a decrease of 53%; the overall average GPU utilization of the cluster increased from 68% to 86%, significantly improving resource utilization efficiency; and the task scheduling failure rate due to resource fragmentation decreased from 15% to below 3%, greatly enhancing system reliability. Particularly noteworthy is that the resource reorganization operation based on checkpoint-based dynamic migration technology resulted in an average performance loss of less than 1% for the migrated tasks, far superior to the performance loss caused by task termination and restart in traditional solutions, achieving user-unnoticed resource optimization.

[0292] This invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being configured to perform the method described in this invention.

[0293] This invention also provides a computer-readable storage medium storing computer-executable instructions for performing the methods described in this invention.

[0294] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0295] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A GPU resource optimization method based on dynamic prediction and topology heatmaps, characterized in that, The method includes the following steps: S100, Construct a server heatmap, wherein the heatmap value is calculated based on a weighted average of node feature parameters in multiple dimensions; wherein the multiple feature parameters include at least the node load index obtained by aggregating task features and the topological connectivity centrality of the nodes. S200 predicts future integrated GPU resource requirements based on the queue of tasks to be scheduled and historical task data, in order to generate a resource demand heatmap. S300, based on the resource demand heat map and the server heat map, quantitatively identify the risk of GPU resource fragmentation through comparative analysis; S400, in response to the risk of fragmentation of quantized GPU resources, selects the optimal optimization strategy from a pre-built strategy library; S500, execute the selected optimization strategy to change the physical layout of GPU resources; the execution includes dynamically migrating tasks that support checkpointing, and / or scaling resources for tasks that support elastic scaling; S300, specifically includes: S310, compare the resource demand heat map with the future resource distribution map simulated based on the current resource status, and identify server nodes whose idle resource specifications cannot meet the requirements of any integrated task predicted in the resource demand heat map in the future resource distribution, as candidate nodes; the future resource distribution map is generated by integrating the current instantaneous idle resources with the simulated predicted release resources to calculate the resource distribution status at the future target time point; S320, for each candidate node, calculate the estimated load value of the candidate node in the future resource distribution map, compare the estimated load value with the dynamic reference value calculated based on the server heat map, and generate a quantified fragmentation risk score based on the comparison result; S330, mark the candidate nodes whose fragmentation risk scores are higher than a preset threshold as nodes to be optimized; The dynamic reference value is calculated using any of the following methods: Method A: Based on the heat values ​​of all nodes in the server heatmap, calculate the difference between the highest and lowest heat values ​​of all nodes, and divide the difference by the total number of nodes to obtain the average resource value as a dynamic reference value; Method B: Calculate the fluctuation of thermal values ​​at all nodes. If the fluctuation is less than a preset threshold, the average thermal value is used as the dynamic reference value. If the fluctuation is greater than or equal to the preset threshold, the average thermal value after removing the highest and lowest values ​​is used as the dynamic reference value.

2. The method according to claim 1, characterized in that, The heat values ​​of the server heatmap are obtained through the following steps: Obtain task path information and task characteristics from historical task datasets. The task characteristics include at least task priority, resource requirement specifications, actual runtime, and resource consumption intensity. Based on the task path information, each task is associated with the server nodes it passes through during its execution, and for a preset statistical time window, multiple feature parameters corresponding to each server node are calculated. The feature parameters of the multiple dimensions are weighted and normalized to generate thermal values ​​in the range of 0-100.

3. The method according to claim 2, characterized in that, The method for determining the weights of each parameter in the weighted normalization process includes the following steps: S1, cluster the historical task dataset according to its task characteristics to form multiple task clusters; S2, For the main parameter whose weight is to be determined, fix the weight value of the main parameter, and treat the weights of all other parameters as a whole, and randomly allocate them under preset constraints to form a weight combination. S3, input the data of the multiple task clusters into the heatmap calculation model in sequence, and apply the current weight combination to generate the corresponding heatmap sequence; S4, quantify the overall fluctuation of the heatmap sequence, repeat S2-S3 multiple times to statistically analyze the heatmap fluctuation distribution characteristics corresponding to the current weight value of the main parameter; S5, adjust the weight value of the main parameter, repeat S2-S4, and determine the range of main parameter weight values ​​that stabilizes the fluctuation of the heat map within a preset acceptable range; S6, take the other parameters as the main parameters in turn, and repeat S2-S5 until the weight range of all parameters is determined; S7. Within the determined range of parameter weights, with the goal of maximizing the stability of the heatmap, the optimal weight combination that satisfies the conditions is solved by a constraint optimization algorithm.

4. The method according to claim 1, characterized in that, In S400, the optimal optimization strategy is selected from a pre-set strategy library based on a multi-objective optimization function; The optimization objectives of the multi-objective optimization function include the performance overhead of the optimization strategy and the increase in system throughput due to the successful scheduling of future large tasks.

5. The method according to claim 1, characterized in that, The execution includes dynamically migrating tasks that support checkpointing and / or scaling resources for tasks that support elastic scaling.

6. The method according to claim 5, characterized in that, The dynamic migration of tasks that support checkpoints specifically includes: Control the checkpoint operation triggered by the task to be migrated, and save the task status to shared storage; Start a new task instance on the target node and resume execution from the checkpoint; The resource scaling for tasks that support elastic scaling specifically includes: dynamically adjusting the number of GPUs used by the task through the elastic interface of the deep learning framework.

7. A GPU resource optimization system based on dynamic prediction and topology heatmaps, characterized in that, The system for performing the method according to any one of claims 1 to 6, the system comprising: The heatmap construction module is used to construct server heatmaps based on historical task characteristics and the number of node topology connection edges. The resource demand prediction and risk identification module is connected to the heat map construction module. It is used to predict the future integrated GPU resource demand based on the task queue to be scheduled and historical task data to generate a resource demand heat map. Based on the resource demand heat map and the server heat map, it quantifies and identifies the risk of GPU resource fragmentation through comparative analysis. The strategy decision engine, connected to the resource demand prediction and risk identification module, is used to select the optimal optimization strategy from the preset strategy library in response to the quantified GPU resource fragmentation risk. A resource reconfiguration executor, connected to the policy decision engine, is used to execute the selected optimization policy to change the physical layout of GPU resources.

Images