Gpu load balancing method and system applied to server cluster

Abstract

The embodiment of the application provides a GPU load balancing method and system applied to a server cluster, relates to the technical field of computers, and first models the mapping relationship between each GPU load fingerprint (covering operation unit load, display memory occupation, and load change rate data) and task characteristics (containing task data volume, calculation complexity, and data dependency attribute), generates a load task mapping data set; a dynamic load compensation pool is constructed based on the data set, contains a GPU node group adapted to load characteristics, a task migration link parameter, and a load sharing rule; task data is split according to the rule and is transmitted to a corresponding node group for execution, and task migration execution data is generated; the compensation pool parameter and the rule are calibrated based on the execution data, and a calibration result is generated; the result is fed back to the mapping data set, the configuration is updated to form a cyclic adaptation process, server cluster GPU load balancing is realized, and the calculation efficiency and the resource utilization rate are improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a GPU load balancing method and system applied to server clusters. Background Technology

[0002] With the widespread adoption of server clusters, graphics processing units (GPUs), with their powerful parallel computing capabilities, have become a core component for handling complex computational tasks. Multiple GPUs working together can significantly improve the overall computing performance of a server cluster, meeting the demands of application scenarios with extremely high computational resource requirements, such as artificial intelligence training and large-scale data simulation.

[0003] However, in actual operation, the load on GPUs in a server cluster is often difficult to balance. On one hand, different tasks have different characteristics, such as varying data sizes, computational complexities, and data dependencies. These factors lead to significant differences in GPU resource consumption across tasks. On the other hand, the hardware characteristics of each GPU also differ. Differences in hardware parameters such as the performance of computing units and memory capacity result in varying performance and load conditions when processing the same task. Most existing load balancing methods attempt to achieve load balancing from a single dimension, such as simple task allocation or static GPU resource partitioning. They lack in-depth analysis of the complex mapping relationship between GPU load fingerprints and task characteristics, and cannot dynamically adjust task allocation and load distribution strategies according to actual conditions. This makes it difficult to truly achieve load balancing across GPUs in a server cluster in practical applications, affecting overall computational efficiency and resource utilization. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a GPU load balancing method and system for server clusters.

[0005] According to a first aspect of this application, a GPU load balancing method for server clusters is provided, the method comprising: The modelers establish a mapping relationship between the GPU load fingerprints and task features within the server cluster, generating a load-task mapping dataset. The GPU load fingerprints include computing unit load data, memory usage data, and load change rate data, while the task features include task data volume, computational complexity, and data dependency attributes. A dynamic load compensation pool is constructed based on a load task mapping dataset. The dynamic load compensation pool includes GPU node groups adapted to load characteristics, task migration link parameters, and load sharing rules. The GPU node groups are composed of GPUs with complementary load fingerprints. Task data is split according to the load sharing rules of the dynamic load compensation pool, and transmitted to the corresponding GPU node group for execution through task migration link parameters to generate task migration execution data. The task migration execution data includes the task segments undertaken by each GPU node, execution progress, and load change data. Based on the task migration execution data, the link parameters and load sharing rules of the dynamic load compensation pool are calibrated to generate parameter calibration results, which include link transmission parameter correction data and load sharing ratio adjustment data. The parameter calibration results are fed back to the load task mapping dataset, the mapping relationship and dynamic load compensation pool configuration are updated, and the above steps are repeated to form a cyclic adaptation process to achieve GPU load balancing of the server cluster.

[0006] According to a second aspect of this application, a GPU load balancing system for server clusters is provided. The GPU load balancing system for server clusters includes a processor and a readable storage medium. The readable storage medium stores a program that, when executed by the processor, implements the aforementioned GPU load balancing method for server clusters.

[0007] Based on any of the above aspects, a load-task mapping dataset is generated by modeling the mapping relationship between the load fingerprints of each GPU and task characteristics within the server cluster. A dynamic load balancing pool is constructed based on this dataset, containing GPU node groups adapted to load characteristics, task migration link parameters, and load sharing rules. Node groups composed of GPUs with complementary load fingerprints can fully leverage the advantages of each GPU, achieving load complementarity and balance. Task data is split according to the load sharing rules and transmitted to the corresponding GPU node groups for execution via the task migration link parameters. The generated task migration execution data records in detail the task acceptance status, execution progress, and load changes of each GPU node. Based on the task migration execution data, the link parameters and load sharing rules of the dynamic load balancing pool are calibrated. The generated parameter calibration results can reflect changes in the actual situation in a timely manner. The parameter calibration results are fed back to the load-task mapping dataset and relevant configurations are updated, forming a cyclical adaptation process. This allows for continuous and dynamic load balancing of GPUs in the server cluster according to constantly changing task requirements and GPU load conditions, effectively improving overall computing efficiency and resource utilization, and fully leveraging the computing potential of the server cluster. Attached Figure Description

[0008] Figure 1 A flowchart illustrating the GPU load balancing method for server clusters provided in an embodiment of this application is shown. Figure 2 This paper illustrates a schematic diagram of the component structure of a GPU load balancing system for server clusters provided in an embodiment of this application. Detailed Implementation

[0009] Figure 1 A flowchart illustrating the GPU load balancing method for server clusters provided in this application embodiment is shown, and the detailed steps are described below.

[0010] Step S110: Model the mapping relationship between the GPU load fingerprint and task features within the server cluster, and generate a load task mapping dataset. The GPU load fingerprint includes computing unit load data, memory usage data, and load change rate data. The task features include task data volume, computational complexity, and data dependency attributes.

[0011] This embodiment takes a distributed scientific computing server cluster as an example. The cluster contains multiple GPU nodes used to execute various scientific computing tasks, such as meteorological simulation and molecular dynamics simulation. To achieve efficient utilization and load balancing of GPU resources within the cluster, it is necessary to first establish an accurate mapping relationship between GPU load fingerprints and task characteristics.

[0012] Step S111: Collect real-time running data of each GPU in the server cluster. The real-time running data includes computing unit load data, memory usage data, and load change rate data. The data is categorized and organized according to GPU device identifier to generate a GPU running dataset.

[0013] A monitoring service deployed on the cluster management node continuously collects real-time operational data from each GPU at fixed time intervals. Computational unit load data is represented as a percentage of GPU core utilization, reflecting the workload of the GPU's computational units per unit of time. Memory usage data includes the currently used memory space and memory read / write bandwidth utilization. Load change rate data is calculated as the ratio of the difference between computational unit load data and memory usage data at two consecutive collection points to the time interval. The collected data is categorized according to GPU device identifiers, with each GPU device identifier corresponding to a separate set of operational data. This data is arranged chronologically to form a GPU operational dataset. For example, for a GPU with device identifier GPU001, its operational dataset contains computational unit load values, memory usage values, and load change rate values ​​corresponding to multiple timestamps.

[0014] Step S112: Extract all task items from the set of tasks to be executed, record the task data volume, computational complexity, and data dependency attributes of each task item, classify them according to data dependency attributes, and generate a task feature dataset.

[0015] A set of tasks to be executed is extracted from the cluster's task scheduling system. This set contains multiple different scientific computing tasks. For each task, the task data volume is determined based on the total size of the input data files to be processed; the computational complexity is related to the type of mathematical operations involved, the number of operations, and the number of data processing iterations; data dependency attributes describe the data relationship between this task and other tasks, such as whether it needs the output results of other tasks as input, or whether it provides input data for other tasks. Based on different data dependency attributes, task items are classified into categories such as no-dependency tasks, unidirectionally dependent tasks, and multidirectionally dependent tasks. A record is created for each task item, containing information such as task identifier, task data volume, computational complexity, and data dependency attributes, and organized according to the data dependency attribute categories to generate a task feature dataset.

[0016] Step S113: Extract features from the GPU runtime dataset, classify the features by computing unit load data, memory usage data, and load change rate data, generate load fingerprint data for each GPU, and associate each load fingerprint data with a unique GPU device identifier.

[0017] Feature extraction is performed on the GPU runtime dataset using a sliding time window method. For compute unit load data, statistical features such as average, maximum, minimum, and variance are calculated within each sliding window. Similarly, for memory usage data, features such as average, peak usage, and average bandwidth utilization are extracted within the sliding window. For load change rate data, features such as average change rate, maximum change rate, and stability index of the change trend are calculated within the sliding window. These extracted features are then categorized according to the types of compute unit load data, memory usage data, and load change rate data, and combined to form corresponding GPU load fingerprint data. Each load fingerprint is associated with a unique GPU device identifier for subsequent mapping and identification.

[0018] Step S114: Classify the task feature dataset by splitting the feature dimensions according to the task data volume, computational complexity, and data dependency attributes, and generate classified task feature data. Each task feature data is associated with a unique task identifier.

[0019] For each task item in the task feature dataset, its task data volume, computational complexity, and data dependency attributes are decomposed into independent feature dimensions. The task data volume dimension is divided into several sub-intervals based on the range of data size; the computational complexity dimension is divided into different levels based on the amount of computation; and the data dependency attribute dimension is encoded and represented according to the complexity of the dependency relationship. Through the above multi-dimensional decomposition and classification, the feature data of each task item is transformed into a structured feature vector, generating classified task feature data. Each classified task feature data is associated with a unique task identifier to ensure task traceability.

[0020] Step S115: Set the mapping rules between load fingerprints and task features. The mapping rules specify the task feature adaptation range corresponding to different load fingerprints and define the task feature dimensions and parameter ranges that each load fingerprint can undertake.

[0021] Based on historical task execution records and GPU performance test results, mapping rules between load fingerprints and task features were established. These rules clarify the range of task features that different types of load fingerprints can adapt to. For example, for GPU load fingerprints with high computational unit load characteristics, the adapted task features should be in a higher parameter range in terms of computational complexity; for load fingerprints with low memory usage characteristics, the adapted task data volume should be in a smaller parameter range. The mapping rules are described in detail in document form, including the identifier of each load fingerprint type, the parameter range range of each dimension of the corresponding task features, and the adaptation priority.

[0022] Step S116: Associate the load fingerprint data of each GPU with the classified task feature data according to the mapping rules, label the adaptation dimension and parameter range corresponding to each mapping relationship, and generate initial mapping data.

[0023] Based on the established mapping rules, the workload fingerprint data and categorized task feature data of all GPUs are traversed. For the workload fingerprint data of each GPU, the task feature data that matches it in the mapping rules is searched. The association is determined by comparing whether the parameters of each dimension of the task feature fall within the parameter range corresponding to the workload fingerprint. When all the dimension parameters of the task feature data meet the adaptation range of the workload fingerprint, the association between the two is established. The adaptation dimension and the specific parameter range corresponding to the mapping relationship are marked in the association record. For example, the workload fingerprint of GPU001 is associated with the feature data of task T001, and the adaptation dimensions include computational complexity (parameter range [C1, C2]) and task data volume (parameter range [D1, D2]), generating the initial mapping data.

[0024] Step S117: Add the historical load change data of each GPU and the historical execution data of similar tasks as new data records to the initial mapping data to form the expanded mapping data.

[0025] Historical load change data for each GPU over a period of time is collected, including load fingerprint changes across different time periods. Simultaneously, execution data for historical tasks similar to the current task are collected, including task characteristics, corresponding GPU load fingerprints, and execution results. This historical data is added as new records to the initial mapping data, enriching its sample size and diversity. For example, load change data for GPU001 handling similar tasks over the past three months and execution data for historical task T002 (similar to the current task T001) are added to the initial mapping data to form expanded mapping data.

[0026] Step S118: Index the expanded mapping data according to the GPU device identifier and task identifier, and record the collection timestamp and feature extraction algorithm identifier of each mapping data. Then, perform deduplication on the expanded mapping data, filter low-weight feature parameters based on the preset feature weight threshold, generate deduplicated and filtered mapping data, and generate the final load task mapping dataset to determine the correspondence, adaptation dimension and parameter range between each GPU load fingerprint and task feature.

[0027] A composite index based on GPU device identifiers and task identifiers is established for the expanded mapping data to enable quick querying and retrieval of mapping relationships between specific GPUs and tasks. Each mapping data record includes its acquisition timestamp and the identifier of the feature extraction algorithm used, which distinguishes different versions or types of feature extraction methods. The expanded mapping data undergoes deduplication, removing duplicate mapping relationship records and retaining only unique and valid records. Based on preset feature weight thresholds, each feature parameter in the mapping data is evaluated, filtering out low-weight feature parameters whose weights are below the threshold, as these have negligible impact on the mapping relationship. After deduplication and filtering, the final workload task mapping dataset is generated, which defines the correspondence between each GPU workload fingerprint and task features, the adaptation dimension, and the specific parameter range.

[0028] Step S120: Construct a dynamic load compensation pool based on the load task mapping dataset. The dynamic load compensation pool includes GPU node groups adapted to load characteristics, task migration link parameters, and load sharing rules. The GPU node groups are composed of GPUs with complementary load fingerprints.

[0029] A dynamic load balancing pool is constructed using information from the load task mapping dataset. This pool combines GPUs with complementary load fingerprints into node groups to achieve efficient processing of different task characteristics, and formulates corresponding task migration link parameters and load sharing rules.

[0030] Step S121: Extract GPU load fingerprint data, task feature data and mapping relationship from the load task mapping dataset, and filter GPUs with complementary load fingerprints. Complementary load fingerprints are reflected in the complementarity of parameters such as computing unit load data, memory usage data and load change rate data.

[0031] The workload fingerprint data of all GPUs, task feature data of each task, and the mapping relationships between them are extracted from the workload task mapping dataset. The workload fingerprint data of each GPU is analyzed to determine whether the workload fingerprints of the GPUs are complementary. Complementary workload fingerprints are specifically manifested in the complementarity of parameters in compute unit load data, memory usage data, and load change rate data. For example, one GPU may have a low compute unit load characteristic and a high memory usage characteristic, while another GPU may have a high compute unit load characteristic and a low memory usage characteristic; these two GPUs exhibit parameter complementarity in terms of compute units and memory. Complementary GPUs are selected by calculating the complementarity index between the load fingerprint feature vectors of each GPU. When the complementarity index reaches a set threshold, the workload fingerprints of two GPUs are considered complementary.

[0032] Step S122: Divide the GPUs with complementary load fingerprints into multiple GPU node groups. Associate each GPU node group with the corresponding task feature adaptation range, label the load fingerprint parameters and device identifiers of each GPU in the node group, and generate GPU node group configuration data.

[0033] The selected GPUs with complementary load fingerprints are divided into multiple GPU node groups according to a certain number and combination. The number of GPUs in each node group is determined based on the cluster size and task processing requirements, and typically includes multiple GPUs with different complementary characteristics. A corresponding task feature adaptation range is determined for each GPU node group. This range is based on a comprehensive analysis of the load fingerprint parameters of each GPU within the node group, i.e., the range of task feature parameters that the node group can efficiently process. The load fingerprint parameters of each GPU within each node group are annotated in the GPU node group configuration data, such as the average computing unit load, peak memory usage, etc., as well as the device identifier of each GPU. For example, node group G1 contains GPU001 and GPU002, and its adapted task feature range is medium-to-high computational complexity and medium data volume. The configuration data records the load fingerprint parameters and device identifiers of GPU001 and GPU002, generating the GPU node group configuration data.

[0034] Step S123: Plan task migration link parameters. Based on the network connection topology of the GPU node group in the server cluster, set the data transmission path, transmission rate, and data compression format, configure the primary transmission path and backup transmission path, and generate link parameter configuration data.

[0035] Step S1231: Retrieve the network connection topology of the server cluster, extract the network connection relationships, bandwidth allocation data, and transmission delay records of each GPU in the GPU node group and between node groups, and generate a network topology dataset.

[0036] In a distributed scientific computing server cluster scenario, the network connection topology is stored in the configuration database of the cluster management node. This topology is obtained by calling the database interface, from which direct connections between GPUs within a GPU node group are extracted, such as connection paths via PCIe switches. Connections between node groups via Ethernet or InfiniBand networks are also extracted, including network devices such as switches and routers traversed along the connection. Simultaneously, bandwidth allocation data for each connection link (the maximum allocated transmission bandwidth share) and historical transmission delay records (the statistical values ​​of data transmission delay over a past period) are extracted. The extracted network connection relationships, bandwidth allocation data, and transmission delay records are organized by link identifiers to generate a network topology dataset. This dataset is stored in a linked list structure, where each linked list node represents a link and contains information such as device identifiers at both ends of the link, bandwidth allocation values, and a list of delay records.

[0037] Step S1232: Based on the network topology dataset, plan multiple alternative data transmission paths. Each alternative path is associated with corresponding transmission delay, bandwidth utilization, and number of connected nodes to form a set of alternative paths.

[0038] Based on the network topology dataset, Dijkstra's algorithm is used to plan multiple alternative data transmission paths from the task source node to the target GPU node group. During path planning, transmission delay and bandwidth utilization are used as weighting factors, and paths are selected by calculating the combined weight of different paths. After each alternative path is generated, its corresponding transmission delay (the estimated transmission time from the origin to the destination), bandwidth utilization (the proportion of bandwidth currently used by the path to the total allocated bandwidth), and number of connected nodes (the number of network devices traversed by the path) are recorded. This information, along with the specific node sequence of the path, forms a set of alternative paths. This set is organized as an array, where each element represents an alternative path and its associated parameters.

[0039] Step S1233: Test the transmission stability of each alternative path, collect transmission delay fluctuation data, data packet loss rate, and bandwidth fluctuation range of each path within a preset time period, and generate path transmission performance data.

[0040] Transmission stability testing is performed on each path in the candidate path set. Test data packets are continuously sent to each path for a preset test duration, set based on the average execution cycle of the cluster tasks. During the test, transmission latency fluctuation data (the difference between the maximum and minimum transmission latency during the test period), packet loss rate (the ratio of lost test data packets to the total number of sent data packets), and bandwidth fluctuation range (the difference between the maximum and minimum actual bandwidth during the test period) are collected for each path. These collected parameters are categorized and organized according to path identifiers to generate path transmission performance data. This data is stored in a dictionary structure, with the path identifier as the key and a list of parameters containing transmission latency fluctuation data, packet loss rate, and bandwidth fluctuation range as values.

[0041] Step S1234: Select the alternative path with the smallest transmission delay fluctuation and the most stable bandwidth utilization as the main transmission path, and mark the connection nodes, transmission bandwidth limit, and transmission delay baseline value of the main transmission path.

[0042] The primary transmission path is selected from the set of candidate paths. First, the transmission delay fluctuation data of each candidate path are compared, and the path with the smallest transmission delay fluctuation is selected. When multiple paths have the same transmission delay fluctuation, their bandwidth utilization stability (i.e., bandwidth fluctuation range) is compared, and the path with the smallest bandwidth fluctuation range is selected as the primary transmission path. After determining the primary transmission path, its connecting nodes are labeled, i.e., the identifier sequence of all network devices and GPU nodes traversed by the path; the transmission bandwidth limit is the maximum allowed transmission rate of the path; and the transmission delay baseline value is the average transmission delay of the path during the test period, serving as a reference benchmark for subsequent path status monitoring.

[0043] Step S1235: Based on the path transmission performance data, sort the paths by transmission delay and data packet loss rate, and select the two alternative paths that rank immediately after the main transmission path as backup transmission paths.

[0044] For the candidate paths other than the primary transmission path, a comprehensive ranking is performed based on path transmission performance data. The ranking indicators include transmission delay and packet loss rate, with transmission delay weighted at 0.6 and packet loss rate weighted at 0.4. A comprehensive score is calculated for each path using the formula: Comprehensive Score = (Transmission Delay / Maximum Transmission Delay) × 0.6 + (Packet Loss Rate / Maximum Packet Loss Rate) × 0.4, where maximum transmission delay and maximum packet loss rate are the maximum values ​​among all candidate paths. The paths are then sorted from lowest to highest comprehensive score, and the first and second ranked paths are selected as backup transmission paths, denoted as the first backup path and the second backup path, respectively.

[0045] Step S1236: Set the transmission rate of the main transmission path. The transmission rate is determined based on the upper limit of the path bandwidth, the amount of data in the task segment, and the timing requirements of task execution. Reserve some bandwidth for load information synchronization.

[0046] The transmission rate setting for the main transmission path needs to comprehensively consider the path bandwidth limit, the data volume of task segments, and the timing requirements of task execution. First, determine the maximum allowable transmission time for each task segment based on the timing requirements; the task segment must be transmitted within this time to avoid affecting subsequent execution steps. Then, calculate the required minimum transmission rate based on the data volume of the task segments and the maximum allowable transmission time. Compare this minimum transmission rate with the path bandwidth limit, and take the smaller value as the initial transmission rate. Finally, reserve a portion of the bandwidth from the initial transmission rate for load information synchronization. The reserved bandwidth percentage is determined based on the average data volume of the load information and the synchronization frequency, typically 15%-20% of the initial transmission rate. The final transmission rate is the initial transmission rate minus the rate corresponding to the reserved bandwidth.

[0047] Step S1237: Configure the data compression format and select the corresponding compression format according to the type of task fragment data.

[0048] The types of task fragment data include text data, binary data, and image data. For text data, the LZ77 compression format is selected, which has high compression efficiency for repeating character sequences. For binary data, such as model parameter files, the DEFLATE compression format is selected, which combines LZ77 and Huffman coding, offering a moderate compression ratio and fast decompression speed. For image data, such as intermediate visualization results in scientific computing, the JPEG compression format is selected, as its compression ratio can be adjusted according to image quality requirements. During configuration, the file header identifier of the task fragment data is read, the data type is determined based on the identifier, and then the corresponding compression format is automatically matched and recorded for subsequent decompression operations.

[0049] Step S1238: Set up a path switching mechanism. When the transmission delay of the primary transmission path exceeds the baseline value setting range or the data packet loss rate reaches the set threshold, automatically switch to the backup transmission path and record the switching process.

[0050] The path switching mechanism includes triggering conditions and a switching procedure. Triggering conditions are set as follows: when the real-time transmission delay of the primary transmission path exceeds a set range of the transmission delay baseline value (±10% of the baseline value); or when the real-time data packet loss rate reaches a set threshold, which is typically 1% based on data integrity requirements. When either triggering condition is met, the switching procedure is initiated: first, data transmission on the primary transmission path is paused, and the current transmission progress (i.e., the proportion of transmitted data to the total data volume) is recorded; then, the backup transmission path is started, resuming data transmission from the paused progress; simultaneously, the timestamp of the switching process, the triggering condition type (delay exceeding the standard or packet loss rate exceeding the standard), and the path identifiers before and after the switch are recorded.

[0051] Step S1239: Label the task segment type and transmission priority corresponding to each transmission path, determine the path allocation rules for different task segments, and generate a path allocation table.

[0052] Each transmission path is labeled according to the task segment type and transmission priority. Task segment types include computationally intensive, data-intensive, and high real-time requirement; transmission priorities are divided into high, medium, and low levels. For computationally intensive task segments with high real-time requirements, they are assigned to the primary transmission path or the first backup path; for data-intensive task segments with low real-time requirements, they are assigned to the second backup path. The correspondence between task segment type, transmission priority, and path identifier is defined in the form of rules, such as "IF Task type = computationally intensive AND priority = high THEN path = primary path". These rules are compiled into a path allocation table, which is stored in a hash table structure, with the key being (task type, priority) and the value being a list of path identifiers.

[0053] Step S12310: Integrate the configuration of the primary transmission path and backup transmission path, transmission rate, data compression format, path switching mechanism and path allocation table to generate complete link parameter configuration data.

[0054] The configuration information for the primary and backup transmission paths, including path node sequences and transmission delay baselines; the set transmission rate; the selected data compression format; the defined path switching mechanism, including triggering conditions and switching procedures; and the generated path allocation table are integrated. During integration, this information is organized into structured data, such as XML, categorized by data type. This structured data includes sub-nodes for path configuration, rate configuration, compression configuration, switching mechanism, and allocation table. The generated link parameter configuration data is stored in the configuration file of the cluster management node, allowing the task scheduling module to read it in real time.

[0055] Step S124: Construct load sharing rules. The load sharing rules are based on the load fingerprint parameters and task feature adaptation range of each GPU in the GPU node group, and define the proportion of task segments undertaken by each GPU, data volume allocation standards, and execution order planning.

[0056] Based on the load fingerprint parameters and task characteristic adaptation range of each GPU within the GPU node group, load sharing rules are constructed. These rules clarify the proportion of task segments each GPU should handle within the node group; for example, GPUs with higher computational unit load capacity can be allocated a higher proportion of compute-intensive task segments. Data allocation standards are determined based on the memory capacity of each GPU and the memory usage characteristics in the load fingerprint, ensuring that the amount of data allocated to each GPU is within its acceptable range. Execution order planning considers task dependencies and the load change rate of each GPU, rationally arranging the execution order of task segments on each GPU to avoid excessive load fluctuations. For example, for a large task, according to the load sharing rules, 70% of the compute-intensive task segments are allocated to GPU001, and 30% of the memory-intensive task segments are allocated to GPU002. GPU001 is planned to start execution first, and GPU002 will start after GPU001 has executed for a period of time.

[0057] Step S125: Configure the status monitoring parameters of the dynamic load compensation pool. The status monitoring parameters include the load change monitoring cycle of each GPU in the GPU node group, the task migration link transmission status monitoring indicators, and the task execution progress monitoring nodes.

[0058] Configure status monitoring parameters for the dynamic load balancing pool to monitor its operational status in real time. The load change monitoring cycle for each GPU within the GPU node group is set based on the GPU's load change rate and task execution duration. A shorter monitoring cycle allows for more timely detection of load changes but increases system overhead, requiring a balance between timeliness and overhead. Task migration link transmission status monitoring metrics include transmission latency, data packet loss rate, and bandwidth utilization, which are used to evaluate link transmission performance. Task execution progress monitoring nodes are set at key time points or data processing stages during task execution to track task progress, such as setting monitoring nodes when the task is 30%, 50%, and 80% complete.

[0059] Step S126: Set the load sharing rule adjustment trigger conditions. The trigger conditions are set based on the load change rate data of each GPU in the GPU node group and the task migration link transmission status monitoring indicators, which are used to trigger the dynamic adjustment of the load sharing ratio.

[0060] Based on the load change rate data of each GPU within the GPU node group and the task migration link transmission status monitoring indicators, trigger conditions for adjusting load sharing rules are set. When the load change rate of a GPU exceeds the set upper limit, it indicates that the GPU's load is increasing too rapidly and may be overloaded, triggering an adjustment to the load sharing ratio. When the transmission latency of the task migration link is consistently higher than the set threshold, or the data packet loss rate exceeds a certain limit, it indicates that the current load sharing may be causing excessive pressure on the link transmission, also triggering an adjustment. For example, when the load change rate of GPU001 is greater than the set value V1 for two consecutive monitoring periods, or when the transmission latency of the main transmission path is greater than the threshold T1, the load sharing rule adjustment is triggered.

[0061] Step S127: Integrate GPU node group configuration data, link parameter configuration data, load sharing rules, status monitoring parameters, and adjustment trigger conditions to form a preliminary dynamic load balancing pool configuration; in the preliminary dynamic load balancing pool configuration, add the definition of task data interaction methods between GPU node groups and load information synchronization frequency parameters; verify the preliminary dynamic load balancing pool configuration based on preset node group load balancing thresholds and task feature adaptation thresholds; if the thresholds are not met, adjust the number of GPUs in the GPU node group or the task migration link parameters until the configuration meets the threshold requirements, generate the final dynamic load balancing pool, and determine the GPU node group composition, task migration link parameters, load sharing rules, and status monitoring mechanism.

[0062] The initial dynamic load balancing pool configuration is formed by integrating GPU node group configuration data, link parameter configuration data, load sharing rules, status monitoring parameters, and adjustment trigger conditions. This initial configuration includes definitions of task data interaction methods between GPU node groups, such as using message queues or direct memory access, and a load information synchronization frequency parameter, i.e., the time interval at which each node group reports its own load information to the load balancing pool management module. The initial configuration is then validated based on preset node group load balancing thresholds and task feature adaptation thresholds. The node group load balancing threshold measures the load balance among GPUs within a node group, while the task feature adaptation threshold assesses the node group's ability to adapt to task features. If the validation results do not meet the threshold requirements, the number of GPUs within the GPU node group is adjusted, such as by adding or removing GPUs, or task migration link parameters are adjusted, such as changing the transmission path or adjusting the transmission rate. After multiple adjustments and validations until the configuration meets the threshold requirements, the final dynamic load balancing pool is generated, clearly defining the composition of the GPU node group, task migration link parameters, load sharing rules, and status monitoring mechanisms.

[0063] Step S130: Split task data according to the load sharing rules of the dynamic load compensation pool, and transmit it to the corresponding GPU node group for execution through task migration link parameters to generate task migration execution data. The task migration execution data includes the task segments undertaken by each GPU node, execution progress, and load change data.

[0064] According to the load sharing rules in the dynamic load balancing pool, the received task data is split into multiple task fragments suitable for processing by different GPU node groups. The task fragments are then transmitted to the corresponding GPU node groups for execution using preset task migration link parameters, and relevant task migration execution data is recorded during the execution process.

[0065] Step S131: Extract the load sharing rules, task migration link parameters and GPU node group configuration data from the dynamic load compensation pool, and determine the task splitting criteria, transmission path and the GPU node group to be assigned.

[0066] Load sharing rules are extracted from the dynamic load balancing pool. These rules specify how to split tasks based on task characteristics and the load of GPU node groups. Task migration link parameters provide information such as data transmission paths and rates. GPU node group configuration data includes the composition of each node group and the range of task characteristics it can adapt to. Combining this information, the criteria for task splitting are determined, such as splitting according to the proportion of task data volume or computational complexity. A suitable transmission path is selected, i.e., the path used to transmit task fragments to the target GPU node group is determined based on the link parameter configuration data. Finally, the GPU node group that receives the task fragments is identified, i.e., the most suitable node group is selected based on the task characteristics and the adaptability range of the node group.

[0067] Step S132: Split the task data according to the load sharing rules, split the task into segments based on the task data volume and computational complexity, label the dependencies, execution priorities and data interaction requirements of each task segment, and generate a task segment dataset.

[0068] Based on the established task splitting criteria, the task data is split according to the task data volume and computational complexity. The task data is divided into multiple task fragments according to a certain size or computational load, with the size and computational load of each fragment matching the load capacity of the GPUs within the GPU node group. For each task fragment, its dependencies on other task fragments are labeled, indicating which task fragments need to complete before this fragment's execution and which task fragments need to start after it. Execution priorities are also labeled, with higher-priority task fragments executed first. Data interaction requirements are also labeled, such as what data the fragment needs to obtain from other fragments or node groups during execution, or what data it needs to provide to other fragments or node groups. These labeled task fragments are then organized to generate a task fragment dataset.

[0069] Step S133: Classify and package the task fragment dataset, divide the packaging batches according to execution priority, associate each packaging batch with the corresponding GPU node group, compress the data using the compression format in the link parameter configuration data, and generate packaged task fragment data.

[0070] The task fragment dataset is categorized and packaged according to execution priority. Task fragments with the same or similar execution priorities are grouped into the same batch for orderly transmission and execution. Each batch is associated with a corresponding GPU node group, specifying which node group the batch of task fragments should be transmitted to. Based on the data compression format specified in the link parameter configuration data, the task fragment data in each batch is compressed to reduce bandwidth consumption during transmission. After compression, packaged task fragment data is generated, with each data packet containing compressed task fragment data, fragment identifier, dependencies, execution priority, and other information.

[0071] Step S134: Transmit the packaged task fragment data to the corresponding GPU node group through the main transmission path set by the task migration link parameters, and simultaneously start transmission status monitoring to record transmission time and data integrity parameters.

[0072] Following the main transmission path defined in the task migration link parameters, the packaged task fragment data is transmitted from the task source node to the corresponding GPU node group. Simultaneously with the start of transmission, a transmission status monitoring mechanism is activated to monitor the data transmission process in real time. Transmission time is recorded—the total time spent from the start of data transmission to its completion. Data integrity parameters are also recorded; the consistency of data before and after transmission is verified using checksums or hash values ​​to ensure no data loss or corruption occurs during transmission. This transmission status data will be used to evaluate link transmission performance and for subsequent parameter calibration.

[0073] Step S135: If a transmission abnormality occurs on the main transmission path, switch to the backup transmission path to continue transmission, record the transmission path switching process, the location of the abnormality and the method of handling the abnormality, and supplement it to the transmission status data.

[0074] During data transmission, the transmission status of the primary transmission path is continuously monitored. If a transmission anomaly is detected on the primary transmission path, such as transmission delay exceeding a set threshold, excessively high packet loss rate, or transmission connection interruption, the system automatically triggers a path switching mechanism to switch data transmission to a backup transmission path. During the switching process, the system records in detail the time of the transmission path switch, the specific location of the anomaly (e.g., which network switch or transmission link is experiencing the problem), and the anomaly handling methods adopted, such as re-establishing the connection or retransmitting lost data blocks. This information is added to the transmission status data to comprehensively reflect the process and status of data transmission.

[0075] Step S136: After receiving the task fragment data, each GPU in the GPU node group loads the corresponding task fragment according to the load sharing rules, configures the running parameters adapted to the task characteristics, and starts the task fragment execution process.

[0076] Step S1361: Each GPU in the GPU node group receives the packaged task fragment data, decompresses the data in reverse according to the preset compression format, and restores the original data, dependencies, and execution priority parameters of the task fragment.

[0077] Each GPU within the GPU node group receives packaged task fragment data via a network interface. This task fragment data is transmitted in the form of data packets, each containing a header and a payload. The header stores information such as compression format identifiers and data length. Upon receiving a data packet, the GPU calls the corresponding decompression algorithm based on the compression format identifier in the header. For example, if an LZ77 compression format identifier is received, the LZ77 decompression function is called to decompress the received payload data in reverse. After decompression, the original data of the task fragment (i.e., the input data requiring computation) is extracted from the decompressed data; dependencies (information on which other task fragments the task fragment must wait for to complete before execution); and execution priority parameters (indicating the execution order of the task fragment within the node group). This restored information is stored at a designated address in the GPU's global memory for further processing.

[0078] Step S1362: Extract the dependency data of the task fragment, sort out the dependency relationship between the task fragment and other GPUs in the node group that undertake the task fragment, establish a data interaction relationship table, and determine the content and timing of the interaction data.

[0079] Dependency data is extracted from the restored task fragment information. This dependency data is stored in a directed graph structure, where nodes represent task fragments and directed edges represent dependencies. By traversing this directed graph, the dependency relationships between the current task fragment and other task fragments undertaken by other GPUs within the node group are identified. For example, if the current task fragment is a downstream task, it depends on the output data of upstream task fragments A and B; or if the current task fragment is an upstream task, its output data is required by downstream task fragments C and D. Based on the identified dependency relationships, a data interaction relationship table is established. This table includes the interaction object identifier, i.e., the device identifier of the associated other GPUs; the interaction data content, i.e., the names and data types of the data fields to be transmitted; and the interaction timing, i.e., at what stage the current task fragment is executing and when is data sent or received, such as sending intermediate results when 30% of the execution is complete, or receiving dependent data before execution begins.

[0080] Step S1363: Load the corresponding task fragments according to the load sharing rules, store the task fragment data in the preset video memory area, allocate independent computing unit resources for task fragment execution, and allocate independent video memory address space and computing unit thread groups for the task fragments.

[0081] Based on the load sharing rules in the dynamic load balancing pool, the task segment that the GPU should load at the moment is determined. The raw data of this task segment is read from global memory and stored in a pre-defined video memory region. This pre-defined video memory region is a contiguous block of memory pre-allocated within the GPU's video memory, associated with global memory through a memory mapping mechanism. Simultaneously, independent computing unit resources are allocated to this task segment; that is, a specific number of streaming multiprocessors are allocated from the GPU's stream processor array as dedicated computing units. An independent video memory address space is allocated to this task segment, using virtual memory address mapping and isolated from the video memory address spaces of other task segments. An independent computing unit thread group is allocated, containing multiple thread blocks. The number of threads in each thread block is determined based on the computational granularity of the task segment. The thread group establishes a mapping relationship with the task data processing unit through thread indices.

[0082] Step S1364: Based on the computational complexity and data volume parameters in the task characteristics, configure the running frequency, video memory read / write rate, and data caching strategy of the computing unit to adapt to the execution requirements of the task segment.

[0083] Based on the computational complexity parameter of the task segment (reflecting the number of floating-point operations), and the data volume parameter (the total number of bytes of the original task data), configure the operating frequency of the computing unit. For task segments with high computational complexity, set the operating frequency of the computing unit to the highest frequency supported by the GPU; for task segments with large data volumes, appropriately reduce the operating frequency to avoid overheating. Configure the memory read / write rate by adjusting the operating mode of the memory controller, such as enabling burst transfer mode, to improve the read / write rate of large data blocks. Configure the data caching strategy: for frequently accessed data during computation, such as convolution kernel parameters, store them in the L1 cache; for data with moderate access frequency, store them in the L2 cache; for data accessed only once, read directly from memory without caching.

[0084] Step S1365: Load the execution context data of the task fragment, wherein the execution context data includes the starting execution node pointer, intermediate data storage address, and error handler entry address of the task fragment.

[0085] Execution context data consists of environmental parameters for task fragment execution, stored in the context management module of the task scheduling system. The GPU loads the execution context data of the current task fragment from the context management module via remote procedure calls. This execution context data includes a starting execution node pointer (the address of the entry function for the task fragment's computation flow), intermediate data storage addresses (the memory addresses used to store intermediate results generated during computation), and an error handler entry address (the address of the error handling function to jump to when an error occurs during task execution, such as memory access violation). The loaded execution context data is stored in the GPU's dedicated registers for access by the computation unit during execution.

[0086] Step S1366: Establish a data interaction channel with other GPUs in the node group, and synchronize the execution status data of the task segments based on the transmission rate and interaction frequency set by the task migration link parameters.

[0087] Based on the interaction object identifiers recorded in the data interaction association table, a data interaction channel is established with other GPUs within the node group through a point-to-point communication interface between GPUs. This data interaction channel uses a message-passing-based communication protocol, such as MPI. The send and receive buffer sizes of the channel are configured according to the transmission rate set in the task migration link parameters to ensure that the data transmission rate does not exceed the link bandwidth limit. A timer is set according to the interaction frequency, i.e., the number of interactions per unit time, to periodically trigger the synchronization of execution status data. Execution status data includes information such as the execution progress of the current task segment, the amount of computation completed, and the amount of video memory used. This data is sent to the associated GPUs through the interaction channel in a fixed message format, and status data sent by the associated GPUs is received.

[0088] Step S1367: Start the task segment execution process, execute the task segments according to the preset execution order, and synchronously collect the computing unit load data, video memory usage data, and load change rate data during the execution process, and feed them back to the status monitoring node in real time.

[0089] The execution context pointer is invoked to initiate the execution flow of the task segment. During execution, computational operations are performed sequentially according to the preset execution order, i.e., the instruction sequence within the task segment. Simultaneously, GPU performance counters are used to collect computational unit load data (utilization of each streaming multiprocessor), memory usage data (the amount of memory used by the current task segment), and load change rate data (the ratio of the difference in computational unit load data between two adjacent collection cycles to the number of collection cycles). The collected data is packaged with timestamps and fed back in real-time to the cluster's status monitoring node via a network interface. This status monitoring node is a dedicated server within the cluster responsible for summarizing and analyzing the operational status data of each GPU.

[0090] Step S1368: If execution stalls due to insufficient resources occur during the execution of a task segment, the resource scheduling mechanism within the node group is triggered to request temporary computing unit resources and video memory resources.

[0091] During task segment execution, the usage of computing unit resources and video memory resources is monitored in real time. When computing unit resource utilization reaches 100% and there are unexecuted threads waiting for allocation, or video memory usage reaches a preset limit (usually 90% of the total video memory), causing task execution to stall, the node group's intra-node resource scheduling mechanism is triggered. This intra-node resource scheduling mechanism sends resource request messages to the resource manager within the node group to request temporary computing unit resources, i.e., borrowing idle streaming multiprocessors from other GPUs with lower loads within the node group; and to request temporary video memory resources, i.e., using the shared video memory pool resources within the node group. The resource request message contains information such as the type and quantity of resources requested, and the duration of use.

[0092] Step S1369: When the task segment reaches the data interaction node, synchronize data to the associated GPU according to the data interaction association table, and continue to execute the subsequent steps after receiving the interaction data fed back by the associated GPU.

[0093] The execution flow of a task segment includes pre-defined data interaction nodes, which are identified by special markers in the instruction sequence of the task segment. When execution reaches a data interaction node, the current computation operation is paused. Based on the interaction data content in the data interaction association table, the data to be sent is read from the intermediate data storage address and sent to the associated GPU through the established data interaction channel. Simultaneously, the execution blocks and waits to receive interaction data sent by the associated GPU, storing the received data in the designated intermediate data storage address. After all interaction data has been sent and received, the blocking is lifted, and the computation steps following the data interaction node continue.

[0094] Step S13610: After the task segment is completed, store the final execution result, record the execution time, total resource consumption, number of data interactions, generate a task segment execution report, and feed it back to the task migration execution data set.

[0095] When a task segment reaches the end marker of the instruction sequence, execution is considered complete. The final execution result is read from the registers of the arithmetic unit into the result storage area of ​​the video memory and transmitted to the cluster's shared storage system via the network interface. Simultaneously, the execution duration of the task segment (the time interval from start to finish), total resource usage (including the total time occupied by the arithmetic unit and the total space occupied by the video memory), and number of data interactions (the total number of data interactions with other GPUs during execution) are recorded. This information is compiled into a task segment execution report in JSON format, containing fields such as task identifier, execution duration, resource usage, and number of interactions. This task segment execution report is added to the task migration execution dataset, which is a table in a distributed database used to store execution information for all task segments.

[0096] Step S137: Collect real-time data during the execution of each GPU task segment, record the task segment identifier, execution progress, computing unit load data, memory usage data, and load change rate data to form GPU execution data.

[0097] During the execution of task segments by the GPU, relevant data is collected in real time through the GPU's built-in performance monitoring interface. The identifier of each task segment undertaken by the GPU is recorded to trace the source and ownership of the task segments; the execution progress is determined by calculating the proportion of completed computations to the total computation; and data on compute unit load, memory usage, and load change rate are continuously acquired through the monitoring interface, reflecting the GPU's resource usage and load change trends during execution. The collected data is then organized chronologically and by GPU identifier to form GPU execution data.

[0098] Step S138: Integrate task fragment splitting information, transmission status data, and GPU execution data, classify and organize them according to GPU node group identifier and task identifier, mark the execution time period and parameter change trajectory of each data, add task data interaction content records and interaction timestamps between GPUs in the node group to the task fragment execution data, generate the final task migration execution data, and determine the task fragments, execution progress, load change data and transmission status undertaken by each GPU node.

[0099] The task fragment splitting information, including the number of fragments, the size and characteristics of each fragment, transmission status data such as transmission path, transmission time, and data integrity, and GPU execution data such as execution progress and load parameters, are integrated. The integrated data is then categorized and organized according to GPU node group identifiers and task identifiers to ensure centralized management of data within the same node group and for the same task. Each data entry is labeled with its corresponding execution period, i.e., the time range of data collection, and parameter change trajectory, such as the change in computing unit load over time. Furthermore, the task fragment execution data includes records of task data interaction between GPUs within the node group, such as the name and amount of interacting data, and the timestamp of the interaction. Through these processes, the final task migration execution data is generated, which clearly identifies the task fragments undertaken by each GPU node, their execution progress, load change data, and transmission status.

[0100] Step S140: Based on the task migration execution data, calibrate the link parameters and load sharing rules of the dynamic load compensation pool, and generate parameter calibration results. The parameter calibration results include link transmission parameter correction data and load sharing ratio adjustment data.

[0101] By using task migration execution data, the link parameters and load sharing rules of the dynamic load balancing pool are analyzed and calibrated to identify and correct any problems, and to generate parameter calibration results.

[0102] Step S141: Compare the transmission time and data integrity parameters in the task migration execution data with the preset transmission performance threshold to identify parameters that exceed the threshold; compare the load parameters in the GPU execution data with the preset load target range to identify parameters that exceed the target range.

[0103] The system includes preset transmission performance thresholds, such as maximum allowable transmission time and minimum data integrity requirements; and preset load target ranges, which define the reasonable range within which parameters such as GPU compute unit load and memory usage should be when the GPU is performing a task normally. The transmission time in the task migration execution data is compared to the preset maximum transmission time threshold; if the transmission time exceeds this threshold, the transmission time parameter exceeds the threshold. Similarly, data integrity parameters are compared to the preset minimum data integrity threshold; if the data integrity parameter is below this threshold, the parameter exceeds the threshold. Likewise, the system compares the compute unit load data, memory usage data, and other load parameters in the GPU execution data with the preset load target ranges to identify parameters that exceed the range.

[0104] Step S142: Evaluate the transmission efficiency of the task migration link parameters. Based on the transmission time, data integrity parameters, and path switching records in the transmission status data, identify link segments with mismatched transmission rates and abnormal path transmission delays.

[0105] Based on information such as transmission time, data integrity parameters, and path switching records in the transmission status data, the transmission efficiency of the task migration link parameters is evaluated. Excessive transmission time may indicate an unreasonable current transmission rate setting or a bottleneck in the transmission path; low data integrity parameters suggest insufficient link transmission reliability; frequent path switching records indicate poor stability of the main transmission path. Through comprehensive analysis of the above data, situations where the transmission rate does not match the actual network bandwidth are identified, such as an excessively high transmission rate setting leading to network congestion, or an excessively low setting failing to fully utilize bandwidth; abnormally high latency links are also identified, indicating abnormally high latency in certain specific network connection sections.

[0106] Step S143: Correct the task migration link parameters, adjust the transmission rate of abnormal link segments, replace path segments whose transmission parameters fluctuate beyond the set range, configure dual-path redundant transmission mode, and generate link transmission parameter correction data.

[0107] For identified abnormal link segments, the task migration link parameters are corrected. For cases of mismatched transmission rates, the transmission rate of the abnormal link segment is adjusted to match the actual network bandwidth and transmission requirements. For path segments with transmission parameter fluctuations exceeding the set range, such as paths with excessively large latency fluctuations, they are replaced with more stable path segments. Simultaneously, to improve link reliability, a dual-path redundancy transmission mode is configured. This means that critical task data is transmitted simultaneously through two independent transmission paths; if one path fails, the other path can ensure normal data transmission. The corrected link parameters are then integrated to generate corrected link transmission parameters data.

[0108] Step S144: Analyze the adaptability of the load sharing rules. Based on the computing unit load data, memory usage data, and load change rate data in the GPU execution data, identify the load parameter deviations of each GPU after undertaking the task segment.

[0109] Step S1441: Extract the computing unit load data, video memory usage data, and load change rate data from the GPU execution data, classify and organize them according to GPU device identifier and task segment identifier, and generate a single GPU load time series dataset.

[0110] The GPU execution data is filtered to extract compute unit load data, which is a time series, with each data point containing a timestamp and the corresponding load percentage; memory usage data, also a time series, includes a timestamp and the corresponding memory usage; and load change rate data includes a timestamp and the corresponding rate value. The data is then categorized firstly by GPU device identifier, grouping data belonging to the same GPU together; and secondly by task segment identifier, separating data from different task segments on the same GPU. After categorization, a single-GPU load time series dataset is generated for each task segment of each GPU. This dataset uses the timestamp as the key and compute unit load, memory usage, and load change rate as values, and is stored in a time series database, such as InfluxDB.

[0111] Step S1442: Extract data of the stable phase of task segment execution from the single GPU load time series dataset. The stable phase data is the data of the intermediate period from the start of execution to the end of execution, excluding the instantaneous load at the start and the instantaneous load at the end.

[0112] The single-GPU load time-series dataset is divided into three phases: startup, stable, and termination. The startup phase consists of the first N time units after execution begins, where N is determined by the GPU's hardware initialization time. The termination phase consists of the last M time units before execution ends, where M is determined by the average time spent on data storage and result transmission. The intermediate period is the stable phase. All data points from the stable phase are extracted from the time-series dataset, excluding the startup instantaneous load data from the startup phase and the termination instantaneous load data from the termination phase. For example, if the total execution time of a task segment is T time units, the startup phase is 0-t1, the termination phase is t2-T, and the stable phase is t1-t2. Data points within this period are extracted.

[0113] Step S1443: Retrieve the GPU load stability parameter range set by the dynamic load compensation pool, and compare the differences between the computing unit load data, memory usage data, and load change rate data during the stable phase and the set range.

[0114] Retrieve the GPU load stability parameter range from the configuration data of the dynamic load compensation pool. This GPU load stability parameter range is the reasonable range set for each load parameter. For example, the operation unit load stability range is [a1, b1], the video memory occupancy stability range is [a2, b2], and the load change rate stability range is [a3, b3]. Compare each data point in the operation unit load data in the stable stage with [a1, b1] to determine whether it is within the range; similarly, compare the video memory occupancy data with [a2, b2], and the load change rate data with [a3, b3]. Calculate the degree of exceeding the range for each parameter, that is, the difference between the exceeding value and the range boundary. For example, if the operation unit load value is c1, if c1 > b1, the exceeding degree is c1 - b1; if c1 < a1, the exceeding degree is a1 - c1.

[0115] Step S1444: Mark the load parameters that exceed the set range, record the exceeding time period, the exceeding amplitude, and the corresponding task segment identifier, associate the data volume, calculation complexity characteristics, and load sharing ratio data of the task segment, and determine whether the load parameter exceeding the set range is related to the task feature estimation deviation or improper sharing ratio.

[0116] Mark the load parameters that exceed the set range, and record its exceeding time period, that is, the start and end timestamps when the parameter exceeds the range; the exceeding amplitude, that is, the exceeding degree calculated in step S1443; and the corresponding task segment identifier. Associate the task feature data of this task segment through the task segment identifier, including the data volume and calculation complexity characteristics; and the load sharing ratio data assigned to this task segment in the load sharing rule. Analyze the relevance between the task feature data and the load parameter exceeding: if the actual data volume of the task is greater than the estimated data volume, it may cause the video memory occupancy to exceed the range; if the actual value of the calculation complexity is higher than the estimate, it may cause the operation unit load to exceed the range. Analyze the sharing ratio data: if the proportion of the task segment承接 by this GPU is higher than its load capacity ratio, it may cause the load parameter to exceed the range.

[0117] Step S1445: Associate the corresponding load sharing ratio data, calculate the difference between the actual data volume and proportion of the task segment承接 and the preset value, and determine whether the load parameter deviation is due to an unreasonable sharing ratio.

[0118] Obtain the preset load sharing ratio for the GPU from the load sharing rules; this is the proportion of task segment data that the GPU should handle relative to the total data volume of the node group. Obtain the actual task segment data volume handled by the GPU from the task migration execution data. Calculate the difference between the actual data volume and the preset data volume (actual data volume - preset data volume); calculate the difference between the actual ratio and the preset ratio (actual ratio - preset ratio). If the absolute value of the difference exceeds the set allowable error range, such as 5%, it is preliminarily determined that the load parameter deviation may stem from an unreasonable load sharing ratio. For example, if the actual ratio is 10% higher than the preset ratio, and the computing unit load exceeds the range, it can be concluded that the load sharing ratio is too high, leading to excessive load.

[0119] Step S1446: Combine the computational complexity and data volume parameters in the task feature data to analyze the difference between the actual load requirement and the estimated load requirement of the task segment, and determine whether the load parameter deviation is due to the deviation in the task feature estimation.

[0120] The estimated computational complexity and estimated data volume of the task segment are obtained from the task feature data; the actual computational complexity is deduced from the GPU execution data, i.e., the actual number of floating-point operations calculated based on the arithmetic unit load data and execution time; the actual data volume is the actual amount of data processed during the execution of the task segment. The difference rate between the actual computational complexity and the estimated computational complexity is calculated as (actual - estimated) / estimated; the difference rate between the actual data volume and the estimated data volume is also calculated. If the absolute value of the difference rate is greater than a set threshold, such as 10%, it is determined that the load parameter deviation may originate from the task feature estimation deviation. For example, if the actual data volume difference rate is 20% and the video memory usage exceeds the range, it can be concluded that the data volume estimation is too low, resulting in insufficient video memory.

[0121] Step S1447: Track the correlation of load parameter deviations of each GPU within the node group, identify situations where multiple GPUs exhibit the same type of deviation simultaneously, and determine whether it stems from an overall adaptability issue with the load sharing rules.

[0122] Collect load parameter deviation data for all GPUs within the node group and categorize them by deviation type, such as excessive compute unit load, excessive memory usage, and unstable load change rate. Analyze the frequency of similar deviations within the node group. If more than 50% of the total number of GPUs in the node group simultaneously exhibit excessive compute unit load, it is determined that the issue may stem from an overall adaptability problem with the load sharing rules. For example, if the compute unit load of most GPUs in the node group exceeds the specified range, and these GPUs are handling high-computational-complexity tasks, it indicates an overall imbalance in the allocation of high-computational-complexity tasks, failing to adequately consider the overall computing power of the node group.

[0123] Step S1448: Record the load parameter deviations according to the deviation type, and mark the GPU identifier, task segment identifier, triggering factor and scope of influence corresponding to the deviation to form a deviation classification table.

[0124] Load parameter deviations are categorized by deviation type, such as compute unit load deviation, memory usage deviation, and load change rate deviation. For each type of deviation, the corresponding GPU identifier (GPU device number where the deviation occurred), task segment identifier (task segment ID causing the deviation), triggering factor (unreasonable allocation ratio, task feature prediction deviation, or overall rule problem determined in steps S1444-S1447), and impact range (the degree of impact of the deviation on task execution progress, such as slight delay, severe delay, or execution failure) are recorded. This information is compiled into a deviation classification table, which is a table in a relational database and includes fields such as deviation type, GPU identifier, task identifier, triggering factor, and impact range.

[0125] Step S1449: Statistically analyze the frequency and impact of different types of deviations, determine the core parameters that need to be adjusted in the load sharing rules, generate load parameter deviation analysis data, and determine the details, triggering factors, and correlations of load parameter deviations for each GPU.

[0126] Statistical analysis is performed on the data in the deviation classification table. The frequency of occurrence of different types of deviations is counted, i.e., the number of records for each deviation type in the dataset. A weighted score for the degree of impact is calculated: 1 point for minor delay, 3 points for severe delay, and 5 points for execution failure. The total score for each deviation type is then calculated. Priorities are determined based on frequency and total score. The load sharing rule parameters corresponding to higher priority deviation types are the core parameters that need adjustment. For example, if the computing unit load deviation has the highest frequency and the highest total score, then the core parameter is the weight for allocating computational complexity. The statistical results, core parameters, deviation details for each GPU (such as the magnitude and duration of the deviation), triggering factors, and correlations (such as the correlation with deviations from other GPUs) are compiled into load parameter deviation analysis data. This load parameter deviation analysis data is stored in XML format and serves as the basis for adjusting the load sharing rules.

[0127] Step S145: Adjust the load sharing ratio. For GPUs whose load change rate exceeds the set range, reduce the proportion of task segments and data volume they undertake, and allocate the corresponding task segments to GPUs with stable load parameters within the node group, generating load sharing ratio adjustment data.

[0128] For GPUs whose load change rate exceeds the set range (i.e., GPUs with excessive load fluctuations), adjust the proportion of task segments and the amount of data they handle. Reduce the proportion of task segments handled by this GPU to decrease its data processing volume and alleviate load fluctuations. Distribute the reduced task segments from this GPU to other GPUs within the node group with stable load parameters. These GPUs' compute unit load, memory usage, and load change rate are all within the preset target range and change relatively smoothly. Through these adjustments, the load of each GPU within the node group becomes more balanced and stable, generating load sharing ratio adjustment data and recording the adjusted task segment proportion and data amount for each GPU.

[0129] Step S146: Evaluate the coordination of task execution progress. Based on the execution progress data and data interaction records of each GPU, identify progress deviations caused by asynchronous execution of task segments and data interaction delays.

[0130] The coordination of task execution progress is assessed based on the execution progress data and data interaction records of each GPU. Execution progress data reflects the completion status of task segments on each GPU; data interaction records document the time and content of data transfers between GPUs. If the execution progress of some GPUs lags significantly behind other GPUs, or if severe delays occur during data interaction, causing waiting GPUs to be unable to continue execution in a timely manner, a progress deviation will occur. By analyzing this data, situations where task segments are not executed in sync can be identified, such as a segment's execution progress being much lower than other related segments, and progress deviations caused by data interaction delays.

[0131] Step S147: Optimize the execution order planning in the load sharing rules, adjust the execution timing of asynchronous task segments, reserve data interaction buffer periods in advance, and generate execution order optimization data.

[0132] For identified task execution progress deviations, optimize the execution order planning in the load sharing rules. Adjust the execution timing of asynchronous task segments; for example, schedule slower task segments earlier or adjust their start and end times to coordinate their progress with other related task segments. For task segments involving data interaction, reserve a data interaction buffer period in advance; that is, allocate sufficient time for data transmission and reception in the task segment execution plan to avoid execution stalls due to delayed data arrival. Record the optimized execution order planning to generate execution order optimization data.

[0133] Step S148: Integrate link transmission parameter correction data, load sharing ratio adjustment data, and execution order optimization data; supplement calibration basis, calibration range, and execution priority; form preliminary parameter calibration results; sort the preliminary parameter calibration results by dynamic load compensation pool identifier; label the associated GPU node group, task identifier, and parameter comparison before and after calibration for each calibration content; generate final parameter calibration results; and determine the details of link transmission parameter correction, load sharing ratio adjustment values, and execution order optimization direction.

[0134] The data on link transmission parameter correction, load sharing ratio adjustment, and execution order optimization are integrated. Each correction and adjustment data point is supplemented with calibration basis, explaining the reasons and grounds for calibration, such as which transmission time data or load parameter deviations were used for correction; the calibration scope is clearly defined, i.e., which GPU node groups, task types, or transmission links the correction applies to; execution priorities are set, assigning higher priority to calibration items with greater impact to ensure their priority application. The preliminary parameter calibration results are sorted according to the identifier of the dynamic load compensation pool, allowing the calibration results of each compensation pool to be displayed centrally. Each calibration item is labeled with its corresponding associated GPU node group, i.e., the node group involved in the calibration item; the corresponding task identifier is labeled, indicating which tasks the calibration was performed on; a comparison of parameters before and after calibration is provided, such as changes in transmission rate values ​​before and after adjustment, and the distribution of load sharing ratios before and after adjustment. Through these processes, the final parameter calibration results are generated, clarifying the details of link transmission parameter correction, the adjusted values ​​of load sharing ratios, and the direction of execution order optimization.

[0135] Step S150: Feed back the parameter calibration results to the load task mapping dataset, update the mapping relationship and dynamic load compensation pool configuration, repeat the above steps to form a cyclic adaptation process, and realize GPU load balancing of the server cluster.

[0136] The generated parameter calibration results are fed back to the load task mapping dataset to update the mapping relationships and the configuration of the dynamic load compensation pool. By continuously repeating the above modeling, building, splitting execution and calibration steps, a continuously optimized cyclic adaptation process is formed, which ultimately achieves load balancing of the server cluster GPUs.

[0137] Step S151: Feed back the link transmission parameter correction data, load sharing ratio adjustment data, and execution order optimization data from the parameter calibration results to the load task mapping dataset construction stage, replacing the original link parameter planning, load sharing logic, and timing planning rules.

[0138] The link transmission parameter correction data, load sharing ratio adjustment data, and execution order optimization data from the parameter calibration results are fed back into the load task mapping dataset construction stage. In this stage, the original link parameter planning is replaced with new link transmission parameter correction data to reflect optimized transmission paths and rates; the original load sharing logic is updated with load sharing ratio adjustment data to make task allocation more balanced; and the original timing planning rules are replaced with execution order optimization data to improve the coordination of task execution.

[0139] Step S152: Update the task migration link association information in the load task mapping dataset based on the link transmission parameter correction data, supplement the transmission characteristic data of the new link parameters, and correct the mapping adaptation range between GPU load fingerprint and task characteristics.

[0140] Based on the corrected link transmission parameters, the task migration link association information recorded in the load task mapping dataset is updated, such as the identification of the transmission path and the set value of the transmission rate. Simultaneously, transmission characteristic data of the new link parameters is supplemented, such as the corrected transmission latency and bandwidth utilization. This data can more accurately reflect the actual transmission performance of the link. Based on the new link transmission characteristics, the mapping adaptation range between the GPU load fingerprint and task characteristics is corrected. For example, for task characteristics requiring a large amount of data transmission, the adapted GPU load fingerprint should take into account the impact of data transmission on the GPU load, and the parameter ranges of computing unit load and memory usage should be appropriately adjusted.

[0141] Step S153: Adjust the GPU load capacity assessment logic in the load task mapping dataset by combining the load sharing ratio adjustment data, update the task feature adaptation range of each GPU, and optimize the mapping relationship between load fingerprint and task features.

[0142] By incorporating load sharing ratio adjustment data, the logic used to evaluate GPU load capacity in the load task mapping dataset is adjusted. For example, for GPUs with a decreased load sharing ratio, their load capacity evaluation results may need to be adjusted downwards to reflect changes in their task processing capabilities; conversely, for GPUs with an increased load sharing ratio, their load capacity evaluation results may need to be adjusted upwards. Based on the adjusted load capacity evaluation logic, the task feature adaptation range for each GPU is updated, expanding or narrowing the range of task feature parameters that it can efficiently process. Through these adjustments, the mapping relationship between load fingerprints and task features is optimized, making it more consistent with actual load conditions and task requirements.

[0143] Step S154: Optimize the data to update the task execution timing association information in the load task mapping dataset according to the execution order, and adjust the execution priority parameter in the task features.

[0144] Based on the execution order, the data is optimized by updating the task execution timing information recorded in the load task mapping dataset, such as the start and end times of task segments and their sequential relationship with other task segments. Simultaneously, based on the optimized execution order, the execution priority parameters in the task features are adjusted to match the task execution priority with the optimized timing plan. For example, for task segments that need to be executed earlier, their execution priority parameters are increased to ensure they are processed first during task scheduling.

[0145] Step S155: Reconstruct the dynamic load balancing pool based on the updated load task mapping dataset, adjust the GPU node group composition, task migration link parameters, load sharing rules and status monitoring mechanism, and update the configuration of the dynamic load balancing pool using the calibrated link transmission parameters, load sharing ratio and execution order planning.

[0146] Based on the updated load task mapping dataset, the dynamic load balancing pool is reconstructed. The load fingerprint complementarity of each GPU is reassessed, and the composition of GPU node groups is adjusted, potentially adding or removing some GPUs to optimize node group performance. Task migration link parameter configurations, including transmission paths, transmission rates, and data compression formats, are updated according to calibrated link transmission parameters. Load sharing rules are updated using adjusted load sharing ratios and execution order planning. Parameters of the status monitoring mechanism, such as monitoring cycles and threshold settings, are optimized based on new load conditions and task characteristics. Through these adjustments, the configuration of the dynamic load balancing pool is updated with calibrated parameters, enabling it to better adapt to the current task load and cluster status.

[0147] Step S156: Apply the reconstructed dynamic load balancing pool to the new round of task splitting and migration, repeat the task splitting, transmission, execution and data collection process, and generate new task migration execution data.

[0148] The reconstructed dynamic load balancing pool is applied to the new round of task processing. Newly received task data is split according to the new load sharing rules, and task fragments are transmitted to the corresponding GPU node groups for execution using updated task migration link parameters. During execution, relevant task migration execution data is collected, including the task fragments undertaken by each GPU node, execution progress, and load change data. By repeating these processes, new task migration execution data is generated.

[0149] Step S157: Based on the new task migration execution data, execute the parameter calibration process again to generate a new round of parameter calibration results and continuously optimize the load task mapping dataset and dynamic load compensation pool configuration.

[0150] Using the newly generated task migration execution data, the parameter calibration process is executed again. Following the previously described comparison, evaluation, correction, and optimization steps, the new task migration execution data is analyzed to identify potential new issues in the link parameters and load sharing rules, generating a new round of parameter calibration results. These new calibration results are then fed back into the load task mapping dataset and dynamic load balancing pool configuration for continuous optimization and adjustment.

[0151] Step S158: Record the data update content, link adjustment details, task migration execution results and calibration basis in each cycle adaptation process to form a cycle adaptation record.

[0152] During each adaptation cycle, detailed records are kept of the updates to various data, such as which mapping relationships in the load task mapping dataset were modified, and which parameters in the dynamic load balancing pool configuration were adjusted; specific details of link adjustments are recorded, such as changes in transmission paths and adjustments to transmission rates; task migration execution results are recorded, such as changes in task execution time and load balancing metrics; and the basis for each calibration is recorded, such as the calibration decisions made based on which monitoring data or evaluation results.

[0153] Step S159: During the cyclic adaptation process, real-time load fingerprint data and task feature data of each GPU are collected synchronously to track load changes and task execution patterns, and the load fingerprint extraction logic and task feature classification criteria are iteratively optimized.

[0154] For example, step S1591: Extract GPU load data, task feature data, load task mapping dataset and parameter calibration results from the multi-round cyclic adaptation records, and summarize them to form an optimization analysis dataset.

[0155] GPU load data from multiple rounds of iterative adaptation is extracted from the records. This includes load fingerprint data for all GPUs in each round; task feature data, including feature parameters of the task segments processed in each round; load-task mapping datasets, including the updated mapping relationships in each round; and parameter calibration results, including link correction data and amortization adjustment data in each round. The above data is then aggregated by round, with each round's data serving as a sample for the optimization analysis dataset. During data aggregation, time alignment is performed on data from different rounds to ensure consistency in the time dimension; the data format is standardized, unifying field names and data types. The generated optimization analysis dataset is stored in a data warehouse using a star schema design. The fact table contains round information, and the dimension tables contain GPU load, task features, mapping datasets, calibration results, etc.

[0156] Step S1592: Analyze and optimize the load balancing stable cases in the dataset. From the data records of the load balancing stable cases, statistically analyze the concentrated distribution range of GPU load fingerprint parameters, the frequency of occurrence of task feature classification, and the successful matching pattern of mapping relationship.

[0157] Stable load balancing cases were selected from the optimization analysis dataset. These cases consisted of rounds where the load parameters of each GPU within the node group were within the target range, and task execution was latency-free. Statistical analysis was performed on the GPU load fingerprint parameters in these cases, calculating the arithmetic mean and standard deviation of each fingerprint parameter. The mean ± 2 standard deviations were used as the central distribution interval. The frequency of task feature classifications was statistically analyzed, representing the proportion of each task feature classification appearing in stable cases out of the total number of occurrences. Successful matching patterns were analyzed, representing the probability of a specific combination of load fingerprint and task feature appearing in stable cases. Combinations with probabilities higher than a set threshold were considered successful matching patterns. For example, a combination where the computational unit load parameter in the load fingerprint is in the range [a, b] and the task feature has low computational complexity, with a successful matching probability of 80%, was considered a successful pattern.

[0158] Step S1593: Optimize the load fingerprint extraction logic based on effective features, adjust the extraction cycle of computing unit load data, video memory usage data, and load change rate data, supplement the time dimension features of load data, and improve fingerprint recognition.

[0159] Based on the concentrated distribution range of load fingerprint parameters in stable cases, effective features that contribute significantly to load balancing are identified, i.e., parameters with narrow distribution ranges and high discriminative power. For these effective features, their extraction periods are adjusted. For example, the extraction period for computing unit load data is shortened from T to T / 2 to improve temporal resolution; the extraction period for load change rate data is extended from 2T to 3T to smooth short-term fluctuations. Simultaneously, temporal dimensions of the load data are supplemented, such as adding intraday time-period features (morning, afternoon, night) and intraweekly time-period features (weekdays, weekends). These temporal features are combined with the original load parameters to form new load fingerprint vectors, thereby improving fingerprint discriminative power, i.e., increasing the Euclidean distance between fingerprint vectors of different load states.

[0160] Step S1594: Optimize the task feature classification criteria, refine the classification dimensions of data dependency attributes, further split according to data interaction frequency and dependency level, and supplement the task execution time feature.

[0161] Refine the original classification criteria for data dependency attributes. The original criteria may only be divided into no dependency, one-way dependency, and two-way dependency. Now, it is split according to the data interaction frequency, such as low-frequency interaction (interaction times < 5), medium-frequency interaction (5 ≤ interaction times < 20), and high-frequency interaction (interaction times ≥ 20); and split according to the dependency level, such as primary dependency (direct dependency), secondary dependency (dependency of dependency), and multi-level dependency (dependency level ≥ 3). At the same time, supplement the task execution duration feature, that is, the estimated execution duration of the task segment, and divide it into short duration (< t1), medium duration (t1 ≤ duration < t2), and long duration (≥ t2). Through these refinements and supplements, the task feature classification is made more accurate, which can better match the GPU load fingerprint.

[0162] Step S1595: Adjust the mapping rules between the load fingerprint and task features, expand the adaptation range of the effective mapping relationship, and correct the mapping interval that caused load parameter deviation before.

[0163] Adjust the mapping rules between the load fingerprint and task features according to the successful matching mode, and expand the adaptation range of the effective mapping relationship. That is, if the adaptation range of a certain successful mode is narrow, appropriately expand its parameter interval. For example, expand the adaptation interval of the computational complexity from [a, b] to [a - Δ, b + Δ], where Δ is the expansion value determined according to the stable case statistics. For the mapping interval that caused load parameter deviation before, such as the task feature interval corresponding to a certain load fingerprint is too wide, resulting in some tasks exceeding the GPU capacity, make corrections, reduce its adaptation interval or adjust the interval boundary. For example, in the original rule, "the interval for the load fingerprint F to adapt to the task feature T is [c, d]". After deviation analysis, it is found that the upper limit d of this interval is too high, and it is corrected to [c, d - Δ'].

[0164] Step S1596: Apply the optimized load fingerprint extraction logic, task feature classification criteria, and mapping rules to the construction of the load task mapping dataset to generate a new load task mapping dataset.

[0165] Use the optimized load fingerprint extraction logic to re-extract the features from the GPU operation dataset to generate new load fingerprint data; use the optimized task feature classification criteria to re-classify the task feature dataset to generate new task feature data; use the adjusted mapping rules to re-establish the association between the load fingerprint and task features. Construct a new load task mapping dataset according to the process of steps S110 - S118. This load task mapping dataset contains the optimized mapping relationship, adaptation dimensions, and parameter intervals. During the construction process, adopt the same data storage format and index structure as the original dataset to facilitate the smooth upgrade of the system.

[0166] Step S1597: Reconstruct the dynamic load compensation pool based on the new load task mapping dataset, execute the task splitting, migration, execution and parameter calibration process, and collect running data to verify the optimization effect.

[0167] Based on the new load task mapping dataset, the dynamic load balancing pool is reconstructed according to steps S120-S148, including re-selecting GPU node groups, planning link parameters, and formulating load sharing rules. The reconstructed load balancing pool is applied to actual task processing, executing task splitting, migration, and execution processes, and calibrating parameters. Throughout the process, runtime data is collected, including the load parameters of each GPU, task execution time, and link transmission performance, and compared with historical data before optimization to verify the optimization effect. For example, changes in node group load balancing, average task execution time, and link transmission latency are compared before and after optimization.

[0168] Step S1598: Compare the frequency of load parameter deviations, task migration transmission delays, and overall task execution time in the loop adaptation records before and after optimization.

[0169] Extract the load parameter deviation frequency (the number of times the load parameter exceeds the range per unit time) from the loop adaptation records across multiple rounds before optimization; task migration transmission latency (the average latency of a task segment being transmitted from the source node to the target GPU); and overall task execution time (the total time from start to finish of the entire task). Compare the above data with the corresponding data from the same number of rounds after optimization to calculate the improvement rate. For example, the load parameter deviation frequency reduction rate = (frequency before optimization - frequency after optimization) / frequency before optimization; the transmission latency reduction rate and execution time reduction rate are calculated similarly. If the improvement rate reaches the set target value, such as a deviation frequency reduction rate ≥ 20%, the optimization is considered effective.

[0170] Step S1599: To address the existing shortcomings, and in conjunction with the deviation records in the optimization analysis dataset, further fine-tune the load fingerprint extraction parameters, task feature classification thresholds, and mapping rule details. Continuously incorporate new iterative adaptation data, repeat the optimization analysis, parameter adjustment, and effect verification steps, and form an iterative optimization mechanism for the load fingerprint extraction logic and task feature classification standards.

[0171] Based on the optimization effect verification results, if some indicators fail to reach the target value, such as a transmission latency reduction rate of only 10%, then the shortcomings are analyzed. Combining the deviation records in the optimization analysis dataset, the root cause of the problem is located, such as improper setting of the classification threshold for a certain type of task feature leading to mapping deviation. Further fine-tuning of load fingerprint extraction parameters is then performed, such as adjusting the sliding window size; fine-tuning of task feature classification thresholds, such as adjusting the boundary between short and medium durations of data; and fine-tuning of mapping rule details, such as adjusting the adaptation weights of a certain type of fingerprint and feature. New iterative adaptation data is continuously added to the optimization analysis dataset, and steps S1591-S1598 are repeated to form an iterative optimization mechanism. This iterative optimization mechanism is triggered by a timed task, such as performing an optimization analysis and parameter adjustment after every 10 rounds of iterative adaptation.

[0172] Step S1510: Repeat the above cyclic adaptation process and continuously update the load task mapping dataset and dynamic load compensation pool configuration based on the real-time load data collected in the cyclic adaptation process.

[0173] The aforementioned iterative adaptation process is repeated continuously: based on the current load task mapping dataset and dynamic load balancing pool configuration, tasks are processed; task migration execution data is collected; parameters are calibrated; the dataset and configuration are updated; and then applied to new task processing. Throughout this process, the load task mapping dataset and dynamic load balancing pool configuration are dynamically adjusted based on the real-time load data continuously collected during the iterative adaptation process. This ensures that the GPU load of the server cluster remains balanced, thereby improving the overall cluster's operating efficiency and task processing capacity.

[0174] Furthermore, Figure 2 A schematic diagram of the hardware structure of a GPU load balancing system 100 applied to a server cluster for implementing the methods provided in the embodiments of this application is shown. Figure 2 As shown, the GPU load balancing system 100 applied to a server cluster may include at least one processor 102 (the processor 102 may be, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, a transmission device 106 for communication functions, and a controller 108. Those skilled in the art will understand that... Figure 2 The structure shown is for illustrative purposes only and does not limit the structure of the GPU load balancing system 100 applied to a server cluster. For example, the GPU load balancing system 100 applied to a server cluster may also include more than Figure 2 The more or fewer components shown, or having the same Figure 2 The different configurations shown.

[0175] The memory 104 can be used to store software programs and modules of application software, such as the program instructions corresponding to the method embodiments described above in this application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-described GPU load balancing method applied to a server cluster. The transmission device 106 is used to acquire or send data via a network.

[0176] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

Claims

1. A GPU load balancing method applied to server clusters, characterized in that, The method includes: The modelers establish a mapping relationship between the GPU load fingerprints and task features within the server cluster, generating a load-task mapping dataset. The GPU load fingerprints include computing unit load data, memory usage data, and load change rate data, while the task features include task data volume, computational complexity, and data dependency attributes. A dynamic load compensation pool is constructed based on a load task mapping dataset. The dynamic load compensation pool includes GPU node groups adapted to load characteristics, task migration link parameters, and load sharing rules. The GPU node groups are composed of GPUs with complementary load fingerprints. Task data is split according to the load sharing rules of the dynamic load compensation pool, and transmitted to the corresponding GPU node group for execution through task migration link parameters to generate task migration execution data. The task migration execution data includes the task segments undertaken by each GPU node, execution progress, and load change data. Based on the task migration execution data, the link parameters and load sharing rules of the dynamic load compensation pool are calibrated to generate parameter calibration results, which include link transmission parameter correction data and load sharing ratio adjustment data. The parameter calibration results are fed back to the load task mapping dataset, the mapping relationship and dynamic load compensation pool configuration are updated, and the above steps are repeated to form a cyclic adaptation process to achieve GPU load balancing of the server cluster.

2. The GPU load balancing method for server clusters according to claim 1, characterized in that, The modeling server cluster establishes a mapping relationship between the GPU load fingerprints and task features, generating a load-task mapping dataset, including: Collect real-time running data of each GPU in the server cluster. The real-time running data includes computing unit load data, video memory usage data, and load change rate data. The data is categorized and organized according to GPU device identification to generate a GPU running dataset. Extract all task items from the set of tasks to be executed, record the task data volume, computational complexity, and data dependency attributes of each task item, classify them according to data dependency attributes, and generate a task feature dataset; Feature extraction is performed on the GPU runtime dataset. The feature types are divided according to the computing unit load data, memory usage data, and load change rate data. Load fingerprint data corresponding to each GPU is generated, and each load fingerprint data is associated with a unique GPU device identifier. The task feature dataset is classified by splitting the feature dimensions according to the task data volume, computational complexity, and data dependency attributes, generating classified task feature data, and each task feature data is associated with a unique task identifier. Define the mapping rules between load fingerprints and task features. The mapping rules specify the task feature adaptation range corresponding to different load fingerprints and define the task feature dimensions and parameter ranges that each load fingerprint can handle. According to the mapping rules, the load fingerprint data of each GPU is associated with the classified task feature data, and the adaptation dimension and parameter range corresponding to each mapping relationship are labeled to generate the initial mapping data. The historical load change data of each GPU and the historical execution data of similar tasks are added as new data records to the initial mapping data to form the expanded mapping data; The expanded mapping data is indexed by GPU device identifier and task identifier. After recording the collection timestamp and feature extraction algorithm identifier of each mapping data, the expanded mapping data is deduplicated. Low-weight feature parameters are filtered based on preset feature weight thresholds to generate deduplicated and filtered mapping data. The final load task mapping dataset is generated to determine the correspondence between each GPU load fingerprint and task features, the adaptation dimension, and the parameter range.

3. The GPU load balancing method for server clusters according to claim 1, characterized in that, The construction of a dynamic load balancing pool based on the load task mapping dataset includes: Extract GPU load fingerprint data, task feature data and mapping relationship from the load task mapping dataset, and filter GPUs with complementary load fingerprints. Complementary load fingerprints are reflected in the complementary parameters of computing unit load data, memory usage data and load change rate data. The GPUs with complementary load fingerprints are divided into multiple GPU node groups. Each GPU node group is associated with a corresponding task feature adaptation range. The load fingerprint parameters and device identifiers of each GPU in the node group are labeled to generate GPU node group configuration data. Plan task migration link parameters, based on the network connection topology of GPU node groups in the server cluster, set data transmission path, transmission rate, data compression format, configure primary transmission path and backup transmission path, and generate link parameter configuration data; The load sharing rules are constructed based on the load fingerprint parameters and task feature adaptation range of each GPU in the GPU node group, defining the proportion of task segments undertaken by each GPU, the data volume allocation standard, and the execution order planning. Configure the status monitoring parameters for the dynamic load balancing pool. The status monitoring parameters include the load change monitoring cycle of each GPU in the GPU node group, the task migration link transmission status monitoring indicators, and the task execution progress monitoring nodes. Set the load sharing rule adjustment trigger conditions. The trigger conditions are based on the load change rate data of each GPU in the GPU node group and the task migration link transmission status monitoring indicators, which are used to trigger the dynamic adjustment of the load sharing ratio. Integrate GPU node group configuration data, link parameter configuration data, load sharing rules, status monitoring parameters and adjustment trigger conditions to form a preliminary dynamic load compensation pool configuration. In the initial dynamic load balancing pool configuration, the task data interaction method between GPU node groups and the load information synchronization frequency parameter are added. Based on the preset node group load balancing threshold and task feature adaptation threshold, the initial dynamic load compensation pool configuration is verified. If the threshold is not reached, the number of GPUs in the GPU node group or the task migration link parameters are adjusted until the configuration meets the threshold requirements, generating the final dynamic load compensation pool and determining the GPU node group composition, task migration link parameters, load sharing rules and status monitoring mechanism.

4. The GPU load balancing method for server clusters according to claim 1, characterized in that, The task data is split according to the load sharing rules of the dynamic load balancing pool, and transmitted to the corresponding GPU node group for execution through task migration link parameters, generating task migration execution data, including: Extract the load sharing rules, task migration link parameters and GPU node group configuration data from the dynamic load compensation pool to determine the task splitting criteria, transmission path and the GPU node group to be assigned. The task data is split according to the load sharing rules, and the task segments are divided based on the task data volume and computational complexity. The dependencies, execution priorities and data interaction requirements of each task segment are labeled to generate a task segment dataset. The task fragment dataset is classified and packaged, and the packaging batches are divided according to the execution priority. Each packaging batch is associated with a corresponding GPU node group. The data is compressed using the compression format in the link parameter configuration data to generate packaged task fragment data. By using the main transmission path set by the task migration link parameters, the packaged task fragment data is transmitted to the corresponding GPU node group, and transmission status monitoring is started simultaneously to record transmission time and data integrity parameters. If a transmission abnormality occurs on the primary transmission path, switch to the backup transmission path to continue transmission. Record the transmission path switching process, the location of the abnormality and the handling method, and add it to the transmission status data. After receiving the task fragment data, each GPU in the GPU node group loads the corresponding task fragment according to the load sharing rules, configures the running parameters adapted to the task characteristics, and starts the task fragment execution process. Real-time data is collected during the execution of each GPU task segment, and the task segment identifier, execution progress, computing unit load data, video memory usage data, and load change rate data are recorded to form GPU execution data. Integrate task fragment splitting information, transmission status data, and GPU execution data, classify and organize them according to GPU node group identifier and task identifier, and mark the execution time period and parameter change trajectory of each data item; In the task fragment execution data, add records of task data interaction content and interaction timestamps between GPUs within the node group to generate the final task migration execution data, and determine the task fragments undertaken by each GPU node, execution progress, load change data and transmission status.

5. The GPU load balancing method for server clusters according to claim 1, characterized in that, The link parameters and load sharing rules of the dynamic load balancing pool are calibrated based on task migration execution data to generate parameter calibration results, including: Compare the transmission time and data integrity parameters in the task migration execution data with the preset transmission performance thresholds to identify parameters that exceed the thresholds; compare the load parameters in the GPU execution data with the preset load target range to identify parameters that exceed the target range. Evaluate the transmission efficiency of task migration link parameters, and identify link segments with mismatched transmission rates and abnormal path transmission delays based on transmission time, data integrity parameters, and path switching records in the transmission status data. Correct the task migration link parameters, adjust the transmission rate of abnormal link segments, replace path segments whose transmission parameters fluctuate beyond the set range, configure dual-path redundant transmission mode, and generate link transmission parameter correction data. The adaptability of the load sharing rules is analyzed, and the deviation of the load parameters of each GPU after undertaking the task segment is identified based on the computing unit load data, memory usage data, and load change rate data in the GPU execution data. Adjust the load sharing ratio. For GPUs whose load change rate exceeds the set range, reduce the proportion of task segments and data volume they undertake, and allocate the corresponding task segments to GPUs with stable load parameters within the node group to generate load sharing ratio adjustment data. Assess the coordination of task execution progress by identifying progress deviations caused by asynchronous execution of task segments and data interaction delays based on the execution progress data and data interaction records of each GPU. Optimize the execution order planning in the load sharing rules, adjust the execution timing of asynchronous task segments, reserve data interaction buffer periods in advance, and generate execution order optimization data; Integrate link transmission parameter correction data, load sharing ratio adjustment data, and execution order optimization data, supplement calibration basis, calibration range, and execution priority, and form preliminary parameter calibration results; The preliminary parameter calibration results are sorted by the dynamic load balancing pool identifier. The associated GPU node group, task identifier, and parameter comparison before and after calibration are marked for each calibration content. The final parameter calibration results are generated to determine the details of link transmission parameter correction, load sharing ratio adjustment values, and execution order optimization directions.

6. The GPU load balancing method for server clusters according to claim 1, characterized in that, The step of feeding back the parameter calibration results to the load task mapping dataset, updating the mapping relationship and dynamic load compensation pool configuration, and repeating the above steps to form a cyclical adaptation process to achieve GPU load balancing of the server cluster includes: The link transmission parameter correction data, load sharing ratio adjustment data, and execution order optimization data from the parameter calibration results are fed back to the load task mapping dataset construction stage to replace the original link parameter planning, load sharing logic, and timing planning rules. Based on the link transmission parameter correction data, update the task migration link association information in the load task mapping dataset, supplement the transmission characteristic data of the new link parameters, and correct the mapping adaptation range between GPU load fingerprint and task characteristics. The GPU load capacity assessment logic in the load task mapping dataset is adjusted by combining the load sharing ratio adjustment data, updating the task feature adaptation range of each GPU, and optimizing the mapping relationship between load fingerprint and task features. Based on the execution order, optimize the update of task execution timing association information in the load task mapping dataset and adjust the execution priority parameter in the task features; The dynamic load balancing pool is reconstructed based on the updated load task mapping dataset. The composition of GPU node groups, task migration link parameters, load sharing rules and status monitoring mechanisms are adjusted. The configuration of the dynamic load balancing pool is updated using calibrated link transmission parameters, load sharing ratio and execution order planning. The reconstructed dynamic load balancing pool is applied to a new round of task splitting and migration, repeatedly executing the task splitting, transmission, execution and data collection process to generate new task migration execution data. Based on the new task migration execution data, the parameter calibration process is executed again to generate a new round of parameter calibration results, continuously optimizing the load task mapping dataset and dynamic load compensation pool configuration; Record the data updates, link adjustment details, task migration execution results, and calibration basis for each adaptation cycle to form an adaptation cycle record; During the cyclic adaptation process, real-time load fingerprint data and task feature data of each GPU are collected synchronously to track load changes and task execution patterns, and the load fingerprint extraction logic and task feature classification criteria are iteratively optimized. Repeat the above iterative adaptation process, and continuously update the load task mapping dataset and dynamic load compensation pool configuration based on the real-time load data collected during the iterative adaptation process.

7. The GPU load balancing method for server clusters according to claim 3, characterized in that, The planned task migration link parameters, based on the network connection topology of the GPU node group within the server cluster, set the data transmission path, transmission rate, and data compression format, configure the primary and backup transmission paths, and generate link parameter configuration data, including: Retrieve the network connection topology of the server cluster, extract the network connection relationships, bandwidth allocation data, and transmission delay records of each GPU and node group within the GPU node group, and generate a network topology dataset. Based on the network topology dataset, multiple alternative data transmission paths are planned. Each alternative path is associated with corresponding transmission delay, bandwidth utilization, and number of connected nodes to form a set of alternative paths. Test the transmission stability of each alternative path, collect transmission delay fluctuation data, data packet loss rate, and bandwidth fluctuation range for each path within a preset time period, and generate path transmission performance data; Select the alternative path with the smallest transmission delay fluctuation and the most stable bandwidth utilization as the main transmission path, and mark the connection nodes, transmission bandwidth limit, and transmission delay baseline value of the main transmission path. Based on the path transmission performance data, the two alternative paths that rank immediately after the main transmission path are selected as backup transmission paths, and sorted by transmission delay and data packet loss rate. Set the transmission rate of the main transmission path. The transmission rate is determined based on the upper limit of the path bandwidth, the amount of data in the task segment, and the timing requirements of task execution. Reserve some bandwidth for load information synchronization. Configure the data compression format and select the appropriate compression format based on the type of task fragment data; Set up a path switching mechanism so that when the transmission delay of the primary transmission path exceeds the baseline value setting range or the data packet loss rate reaches the set threshold, it automatically switches to the backup transmission path and records the switching process. Label the task segment type and transmission priority corresponding to each transmission path, determine the path allocation rules for different task segments, and generate a path allocation table; The system integrates the configuration of the primary and backup transmission paths, transmission rates, data compression formats, path switching mechanisms, and path allocation tables to generate complete link parameter configuration data.

8. The GPU load balancing method for server clusters according to claim 4, characterized in that, After receiving task fragment data, each GPU in the GPU node group loads the corresponding task fragment according to the load sharing rules, configures the running parameters adapted to the task characteristics, and starts the task fragment execution process, including: Each GPU in the GPU node group receives the packaged task fragment data, decompresses the data in reverse according to the preset compression format, and restores the original data, dependencies, and execution priority parameters of the task fragment. Extract the dependency data of the task fragment, sort out the dependency relationship between the task fragment and other GPUs in the node group that undertake the task fragment, establish a data interaction relationship table, and determine the content and timing of the interaction data; Load the corresponding task segments according to the load sharing rules, store the task segment data in the preset video memory area, allocate independent computing unit resources for task segment execution, and allocate independent video memory address space and computing unit thread group to the task segments. Based on the computational complexity and data volume parameters in the task characteristics, configure the running frequency, video memory read / write speed, and data caching strategy of the computing unit to adapt to the execution requirements of the task segment. Load the execution context data of the task fragment, wherein the execution context data includes the starting execution node pointer, intermediate data storage address and error handler entry address of the task fragment; Establish data interaction channels with other GPUs within the node group, and synchronize the execution status data of task segments based on the transmission rate and interaction frequency set by the task migration link parameters; Initiate the task segment execution process, execute the task segments according to the preset execution order, and synchronously collect the computing unit load data, video memory usage data, and load change rate data during the execution process, and feed them back to the status monitoring node in real time; If execution stalls due to insufficient resources during task segment execution, the resource scheduling mechanism within the node group is triggered to request temporary computing unit resources and video memory resources. When a task segment reaches a data interaction node, it synchronizes data to the associated GPU according to the data interaction association table, and continues to execute subsequent steps after receiving the interaction data fed back by the associated GPU. After the task segment is completed, the final execution result is stored, the execution time, total resource consumption, and number of data interactions are recorded, a task segment execution report is generated, and the results are fed back to the task migration execution data set.

9. The GPU load balancing method for server clusters according to claim 5, characterized in that, The analysis of the load sharing rules' adaptability, based on the computational unit load data, memory usage data, and load change rate data in the GPU execution data, identifies the load parameter deviations of each GPU after undertaking a task segment, including: Extract the computation unit load data, memory usage data, and load change rate data from the GPU execution data, classify and organize them according to GPU device identifier and task segment identifier, and generate a single GPU load time series dataset. Data from the stable phase of task segment execution is extracted from the single GPU load time series dataset. The stable phase data is the data in the middle period from the start of execution to the end of execution, excluding the instantaneous load at the start and the instantaneous load at the end. Retrieve the GPU load stability parameter range set by the dynamic load compensation pool, and compare the differences between the computing unit load data, memory usage data, and load change rate data during the stable phase and the set range. Mark load parameters that exceed the set range, record the time period, magnitude of the exceedance and the corresponding task segment identifier, associate the data volume, computational complexity characteristics and load sharing ratio data of the task segments, and determine whether the load parameters exceeding the set range are related to the deviation of task characteristic prediction or improper sharing ratio. Associate the corresponding load sharing ratio data, calculate the difference between the actual amount and ratio of task segment data received and the preset value, and determine whether the load parameter deviation is due to an unreasonable sharing ratio. By combining the computational complexity and data volume parameters in the task feature data, we can analyze the difference between the actual load demand and the estimated load demand of the task segment and determine whether the load parameter deviation is due to the deviation in the task feature estimation. Track the correlation of load parameter deviations among GPUs within a node group, identify situations where multiple GPUs exhibit the same type of deviation simultaneously, and determine whether it stems from an overall adaptability issue with the load sharing rules. Record the load parameter deviations by type, label the GPU identifier, task segment identifier, triggering factor and scope of impact of the deviation, and form a deviation classification table. The frequency and impact of different types of deviations are statistically analyzed to determine the core parameters that need to be adjusted in the load sharing rules, generate load parameter deviation analysis data, and determine the details, triggering factors, and correlations of load parameter deviations for each GPU.

10. A GPU load balancing system for server clusters, characterized in that, It includes a processor and a readable storage medium storing a program that, when executed by the processor, implements the GPU load balancing method for server clusters as described in any one of claims 1-9.

Images