A computing power and power collaborative cross-domain computing power dynamic scheduling method, system and device

Abstract

The application relates to a computing power and power collaborative cross-domain computing power dynamic scheduling method, system and device, and belongs to the technical field of computer room computing power scheduling. The method first collects four types of multi-dimensional data, including a target task characteristic vector, a computer room position and a link state, a computing power machine hardware configuration and an operating state, and a computer room power supply state; then, based on the power supply data and in combination with wind and light power generation power prediction, the reliability index of power supply of each computer room is calculated; subsequently, candidate node sets are screened according to the power supply reliability and network quality indexes, and a cross-domain communication topology is adaptively selected according to the link quality; then, taking transmission time delay, scheduling cost, computing power utilization rate and service level agreement as multi-optimization targets, the load distribution weight of each node is calculated through reinforcement learning and a greedy decision algorithm; finally, the weight distribution model is pipelined and resource configuration instructions are issued. The computing power and power collaborative scheduling is realized, the cross-domain dispersed computing power resources are effectively invigorated, and the overall utilization rate of computing power is improved.

Description

Technical Field

[0001] This application relates to the technical field of data center computing power scheduling, and in particular to a cross-domain dynamic computing power scheduling method, system and equipment that coordinates computing power and power. Background Technology

[0002] As the number of parameters in generative AI large models continues to evolve from billions to tens or hundreds of billions, model training, fine-tuning, and online inference place extremely high demands on the scale of computing power clusters, the performance of GPU computing power machines, and the network transmission latency between data centers. The supply capacity of single-region computing power clusters is gradually becoming unable to match the computing power demand of the explosive growth of large model businesses. Cross-domain multi-data center computing power collaborative scheduling has become an important development direction for tapping the potential of computing power resources and breaking the supply bottleneck.

[0003] However, existing multi-datacenter computing power scheduling platforms still have significant technical shortcomings: the GPUs, CPUs, memory, video memory, and interconnection links of the computing machines in existing data centers generally adopt factory static fixed configurations, and computing resources cannot be dynamically and elastically adjusted according to task characteristics. This can easily lead to a supply gap of insufficient computing resources when carrying large model tasks, and can also easily cause a large amount of resources to be wasted due to small-scale tasks monopolizing high-specification computing machines, directly resulting in the overall utilization rate of the computing cluster remaining at a low level for a long time. Summary of the Invention

[0004] In order to improve the overall utilization rate of cross-domain computing resources and fully utilize the distributed computing resources across domains, this application provides a method, system and device for dynamic scheduling of cross-domain computing power in a computing power-power coordinated manner.

[0005] Firstly, this application provides a cross-domain dynamic scheduling method for computing power in conjunction with power supply, employing the following technical solution: A cross-domain dynamic scheduling method for computing power coordination includes: Collect multi-dimensional data, including: feature vectors of the target task parsed from the task platform; data center location information and link status information collected through the data center distance perception mechanism; hardware configuration and operating status information of each computing machine collected through the computing machine status perception mechanism; and power supply status information of each data center collected through the power monitoring interface. Based on the power supply status information, the real-time total power consumption, renewable power ratio, grid input power fluctuation parameters, and energy storage charge status of each computer room are obtained. The future wind and solar power generation of each computer room is predicted by the power supply prediction model. The power supply reliability index of each computer room is calculated based on the prediction results and the energy storage charge status. Based on the power supply reliability index and the link bandwidth, packet loss rate and latency jitter in the link status information, a set of candidate nodes that meet the power supply and network quality conditions is selected from all available computer rooms. The cross-domain communication topology is adaptively selected based on the comprehensive network quality score of each node in the candidate node set. Data compression is adaptively performed on the nodes participating in cross-domain communication based on the network and power supply status. With minimum transmission latency, lowest scheduling cost, highest computing power utilization, and satisfaction of service level agreement indicators as multiple optimization objectives, the load allocation weight of each candidate node is calculated based on the feature vector of the target task, the currently enabled cross-domain communication topology, and the status of each node, and the allocation relationship of the pipeline stage is determined based on the multidimensional power weighting and deviation correction algorithm, combined with the feature vector of the target task, the currently enabled cross-domain communication topology, and the status of each node. The model pipeline stages are allocated to the corresponding candidate nodes according to the load allocation weight ratio, and a configuration instruction is sent to the target data center to instruct the target data center to perform resource scheduling configuration on the target computing machine.

[0006] By adopting the above technical solutions and integrating a multi-dimensional data acquisition mechanism that combines task characteristics, data center geography and link status, computing machine operating status, and power supply status, a comprehensive status awareness foundation covering computing, network, and energy is established. This fundamentally avoids resource mismatch and power supply instability problems that can easily be caused by single-dimensional decision-making, ensuring the overall rationality of scheduling decisions. Through multi-dimensional quantitative analysis of data center real-time power consumption, renewable electricity ratio, grid fluctuation parameters, and energy storage status, combined with wind and solar power generation prediction models to calculate the power supply reliability index, the uncertainty of power output on the power side is transformed into quantifiable and constrained scheduling indicators. This not only guides computing tasks to be scheduled in green electricity-rich areas based on the proportion of renewable electricity, effectively reducing carbon emissions and electricity costs of data centers, and improving the level of new energy consumption, thus meeting the green development needs of computing infrastructure under the dual-carbon goals, but also avoids the risk of power outages caused by wind and solar power output fluctuations and unstable grid input power through the linkage assessment of energy storage charge status and power generation prediction, improving the power supply reliability and continuous stability of computing task operation. Candidate node sets are selected based on dual constraints of power supply reliability and network quality. The cross-domain communication topology is adaptively adapted based on the comprehensive network quality score of each node. This approach eliminates nodes with insufficient power supply or substandard network quality from the scheduling source, reducing the probability of task scheduling failures and operational interruptions. Furthermore, it dynamically optimizes cross-domain communication paths based on real-time link status, effectively reducing latency, packet loss, and jitter in cross-domain data transmission. This provides stable and efficient communication support for pipeline-type distributed computing tasks, further ensuring the operational efficiency of cross-domain computing power collaboration. In the load allocation phase, a reinforcement learning-integrated greedy decision-making algorithm framework is employed. With multiple optimization objectives—minimum transmission latency, lowest scheduling cost, highest computing power utilization, and compliance with service level agreements—the algorithm combines task characteristics and node status to solve for the load allocation weights of each candidate node. This enables rapid output of the optimal load allocation scheme in complex and ever-changing cross-domain computing scenarios, effectively balancing the computing power load of each node and avoiding the coexistence of local node overload and resource idleness, thereby improving the overall utilization rate of cross-domain computing resources. Overall, it not only fully utilizes the distributed computing resources across domains and improves the precision and comprehensive benefits of computing power scheduling, but also achieves the dual value of green and low-carbon operation and power supply security through the synergistic optimization of computing power and electricity.

[0007] Optionally, the steps for collecting multi-dimensional data include: The target task is obtained through the task platform interface. The task type of the target task is automatically parsed as pre-training, fine-tuning or online inference. The model parameter quantity, parallel computing method, service level agreement latency index, concurrent throughput requirement and hardware resource requirement specification are extracted and the structured output is the feature vector of the target task. By collecting latitude and longitude coordinates, detecting routing links, and monitoring network round-trip latency, the latitude and longitude coordinates, link bandwidth, packet loss rate, and latency jitter parameters of each data center can be obtained in real time. Through the computing power machine status awareness channel, the GPU model, video memory capacity, number of CPU cores, memory size, interconnection protocol type, load utilization and power consumption parameters of each computing power machine in each data center can be obtained in real time. By accessing the data center power distribution system through the smart grid interface, the system can obtain the real-time total power consumption of the cabinet, the instantaneous proportion of renewable power, the short-time standard deviation of grid input power, and the state of charge of the local energy storage system, and encapsulate them into power supply status information.

[0008] Optionally, the steps for selecting a candidate node set from all available data centers specifically include: Set a power supply condition judgment function and a network quality condition judgment function. The power supply condition judgment function outputs a first value when the power supply reliability index is greater than or equal to a first index threshold and the short-time standard deviation of the grid input power obtained from the power supply status information is less than or equal to a fluctuation tolerance threshold; otherwise, it outputs a second value. The network quality condition judgment function outputs a first value when the link bandwidth in the link status information is greater than or equal to the minimum bandwidth threshold, the packet loss rate is less than or equal to the maximum packet loss rate threshold, and the latency jitter is less than or equal to the maximum jitter threshold; otherwise, it outputs a second value. Only when both the power supply condition determination function and the network quality condition determination function output the first value will the corresponding computer room be included in the candidate node set for this round of scheduling.

[0009] Optionally, the step of adaptively selecting the cross-domain communication topology based on the comprehensive network quality score of each node in the candidate node set specifically includes: The network quality comprehensive score is calculated based on the ratio of the available bandwidth to the reference bandwidth, the deviation of the one-way transmission delay from the reference delay, and the deviation of the delay jitter from the reference jitter. When the overall network quality score is greater than or equal to the high-quality link threshold, a fully connected pipeline topology is enabled among the candidate node set. When the overall network quality score is less than the high-quality link threshold but greater than or equal to the low-quality link threshold, a star topology is enabled among the candidate node set. When the overall network quality score is less than the low-quality link threshold, the candidate node set is prohibited from participating in cross-domain communication cooperation.

[0010] By adopting the above technical solution, the hierarchical adaptive topology switching mechanism can dynamically adjust the communication architecture according to the real-time fluctuations of the cross-domain link status, always matching the optimal transmission mode under the current network conditions. This not only fully taps the performance value of high-quality links, but also effectively avoids the operational risks caused by weak links, improves the scenario adaptability and operational robustness of the cross-domain computing power scheduling system in complex network environments, and further ensures the stable achievement of service level agreement indicators.

[0011] Optionally, the steps for nodes participating in cross-domain communication to adaptively perform data compression based on network and power supply status include: The compression factor is dynamically calculated based on the power supply reliability index and network quality comprehensive score of each candidate node, and sparsification and quantization compression are performed on the tensor to be transmitted based on the compression factor; wherein, when the target task is a pre-training or fine-tuning task, the tensor to be transmitted is a gradient tensor; when the target task is an online inference task, the tensor to be transmitted is an intermediate activation tensor for cross-domain transmission. Wherein, the compression factor = base compression rate + first weighting term + second weighting term, the first weighting term = power sensitivity coefficient * (1 - difference of power supply reliability index), and the second weighting term = network sensitivity coefficient * (1 - difference of network quality comprehensive score).

[0012] By adopting the above technical solutions, the dynamic compression mechanism has taken over and refined the constraints of cross-domain computing power scheduling at the data transmission level. Through refined adaptive compression adjustment, it has effectively reduced the time cost and resource consumption of cross-domain communication, improved the operational robustness of the scheduling system in complex network environments and power fluctuation scenarios, and achieved a dynamic balance between compression efficiency and calculation accuracy. This has comprehensively enhanced the overall operational efficiency and scenario adaptability of the computing power-power collaborative scheduling system.

[0013] Optionally, the step of calculating the load allocation weight for each candidate node specifically includes: Establish a mapping relationship between distance, transmission delay, and data transmission cost; The complete computation graph is divided into pipeline stages by layer, with the number of nodes equal to the number of nodes in the candidate node set, and the computational load ratio of each pipeline stage is calculated. The preliminary weights are calculated based on the first power of the remaining computing power capacity of each candidate node, the second power of the power supply reliability index, and the third power of the network quality comprehensive score. By combining the feature vector of the target task, the currently enabled cross-domain communication topology, and the distance-transmission delay-data transmission cost mapping relationship, the initial weights are weighted and corrected to obtain the load allocation weights; wherein, the distance-transmission delay-data transmission cost mapping relationship serves as the quantitative mapping basis between the transmission delay optimization objective and the scheduling cost optimization objective; The actual allocation ratio of each candidate node is obtained by dividing the load allocation weight of each candidate node by the sum of the load allocation weights of all candidate nodes.

[0014] By adopting the above technical solutions, accurate load allocation under multiple constraints is achieved through multi-dimensional power calculation and weighted correction. The executability of the scheduling scheme is ensured by closely following the task form of the model pipeline. At the same time, it has both policy flexibility and scenario adaptability. It can continuously output the optimal load allocation scheme in complex and ever-changing cross-domain computing power environment, and comprehensively enhance the decision-making scientificity and operational efficiency of the computing power-electricity collaborative scheduling system.

[0015] Optionally, the step of issuing a configuration command to the target data center to instruct the target data center to perform resource scheduling configuration for the target computing power machine specifically includes: The model pipeline stages are allocated to the corresponding candidate nodes according to the load allocation weight ratio, and configuration instructions are sent to the target data center to instruct the target data center to perform GPU virtualization partitioning, elastic adjustment of video memory and memory resources, switching of interconnect communication protocols, and allocation of CPU core resources on the target computing machine.

[0016] Optionally, the scheduling method further includes: The status of each active node is continuously monitored. When any active node meets the preset migration trigger condition, the corresponding active migration process is triggered. The migration trigger condition includes at least one of the following: the power supply reliability index is lower than the first danger threshold, the network quality comprehensive score is lower than the second danger threshold, or the available video memory capacity is lower than the capacity threshold. The specific steps to trigger the corresponding proactive migration process include: An interrupt command is sent to the target node that triggers the backtracking condition, and the target node stops executing new forward computation; if the target task is a pre-training or fine-tuning task, the non-critical optimizer state is discarded, and only the gradient core snapshot composed of the elements whose absolute value is within a preset high proportion in the gradient tensor is retained; if the target task is an online inference task, the context state of the current incomplete inference request is extracted to form an inference state snapshot. The gradient core snapshot or the inference state snapshot is transmitted back to the main scheduling domain with the highest priority via the residual network channel. After confirming receipt of the gradient core snapshot or the inference state snapshot, an acknowledgment signal is sent to the target node. The target node clears the GPU memory and temporary cache of the current training task, resets its own state to standby idle state, marks it as a standby node in the global resource list, and moves it into the next round of candidate queue of the candidate node set.

[0017] By adopting the above technical solutions, the proactive migration mechanism, together with the front-end node selection, load allocation, and dynamic compression mechanisms, forms a complete scheduling closed loop. This not only significantly reduces the failure rate of cross-domain distributed tasks through proactive risk management and ensures the stable achievement of service level agreement indicators, but also balances fault recovery efficiency and resource utilization benefits through lightweight core data retention and rapid resource recycling design. This comprehensively enhances the adaptability and robustness of the scheduling system in complex cross-domain scenarios such as power fluctuations, network instability, and dynamic load changes, providing a solid fault-tolerant guarantee for the long-term stable operation of cross-domain model pipeline tasks.

[0018] Secondly, this application provides a cross-domain dynamic computing power scheduling system that coordinates computing power and power supply, employing the following technical solution: A cross-domain dynamic computing power scheduling system that coordinates computing power and power supply includes: The data acquisition module is configured to: acquire multi-dimensional data, including: the feature vector of the target task parsed from the task platform; the location information and link status information of the computer room acquired through the computer room distance perception mechanism; the hardware configuration and operating status information of each computing machine acquired through the computing machine status perception mechanism; and the power supply status information of each computer room acquired through the power monitoring interface. The power prediction and analysis module is configured to: obtain the real-time total power consumption, renewable power ratio, grid input power fluctuation parameters and energy storage charge status of each computer room based on the power supply status information; predict the future wind and solar power generation of each computer room through the power supply prediction model; and calculate the power supply reliability index of each computer room based on the prediction results and the energy storage charge status. The node selection and topology construction module is configured to: select a set of candidate nodes that meet the power supply and network quality conditions from all available computer rooms based on the power supply reliability index and the link bandwidth, packet loss rate and latency jitter in the link status information; adaptively select the cross-domain communication topology based on the comprehensive network quality score of each node in the candidate node set; and adaptively perform data compression for nodes participating in cross-domain communication based on network and power supply status. The scheduling decision module is configured to: take the minimum transmission latency, the lowest scheduling cost, the highest computing power utilization, and the service level agreement indicators as multiple optimization objectives, combine the feature vector of the target task, the currently enabled cross-domain communication topology, and the status of each node, calculate the load allocation weight of each candidate node based on the multi-dimensional power weighting and deviation correction algorithm, and determine the allocation relationship of the pipeline stage based on the greedy matching strategy. The task allocation and configuration distribution module is configured to: allocate the model pipeline stages to the corresponding candidate nodes according to the proportion of the load allocation weight, and issue configuration instructions to the target data center to instruct the target data center to perform resource scheduling configuration on the target computing machine.

[0019] Thirdly, this application provides a computer device that adopts the following technical solution: A computer device includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement a cross-domain dynamic scheduling method for computing power and power coordination as described in the first aspect.

[0020] In summary, this application includes at least one of the following beneficial technical effects: A cross-domain scheduling system that integrates computing power, power, and network across multiple dimensions is constructed. Through full-domain status awareness and quantitative assessment of power supply reliability, the system achieves precise matching between computing tasks and green electricity supply, effectively improving the capacity for renewable energy absorption, reducing carbon emissions and operating costs of data centers. At the same time, by combining network quality constraints and adaptive communication topology switching, the system avoids the risk of task interruption caused by power fluctuations and link degradation, and simultaneously ensures the green and low-carbon attributes and operational stability of cross-domain computing power collaboration.

[0021] A complete scheduling closed loop of refined load allocation and proactive fault-tolerant migration is formed. Based on multi-objective optimization algorithms, a balanced allocation of cross-domain computing resources is achieved, taking into account multiple benefits such as transmission latency, scheduling cost, computing power utilization and service level agreement. At the same time, through real-time monitoring of node status and a lightweight core data proactive migration mechanism, node operation risks are mitigated in advance, task failure rate is reduced, and computing power resource turnover efficiency and overall system robustness are improved. Attached Figure Description

[0022] Figure 1 This is a first flowchart of an embodiment of the method of this application; Figure 2 This is a second flowchart of an embodiment of the method of this application; Figure 3 This is the third flowchart of an embodiment of the method of this application. Detailed Implementation

[0023] To make the purpose, technical solution, and advantages of this application clearer, the following description is provided in conjunction with the appendix. Figures 1-3 The present application will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the application.

[0024] The first embodiment of this application discloses a cross-domain dynamic scheduling method for computing power coordination. (Refer to...) Figure 1 The cross-domain computing power dynamic scheduling method includes S110-S150: S110 collects multi-dimensional data, including: feature vectors of the target task parsed from the task platform; data center location information and link status information collected through the data center distance perception mechanism; hardware configuration and operating status information of each computing machine collected through the computing machine status perception mechanism; and power supply status information of each data center collected through the power monitoring interface. S120 obtains the real-time total power consumption, renewable power ratio, grid input power fluctuation parameters and energy storage charge status of each computer room based on power supply status information, and predicts the future wind and solar power generation of each computer room through the power supply prediction model. Based on the prediction results and energy storage charge status, it calculates the power supply reliability index of each computer room. S130: Based on the power supply reliability index and the link bandwidth, packet loss rate and latency jitter in the link status information, a set of candidate nodes that meet the power supply and network quality conditions is selected from all available computer rooms. The cross-domain communication topology is adaptively selected based on the comprehensive network quality score of each node in the candidate node set. Data compression is adaptively performed on the nodes participating in cross-domain communication based on the network and power supply status. S140 takes minimum transmission latency, minimum scheduling cost, maximum computing power utilization, and service level agreement indicators as multiple optimization objectives. It combines the feature vector of the target task, the currently enabled cross-domain communication topology, and the status of each node. It calculates the load distribution weight of each candidate node based on the multi-dimensional power weighting and deviation correction algorithm, and determines the allocation relationship of the pipeline stage based on the greedy matching strategy. S150 allocates the model pipeline stages to the corresponding candidate nodes according to the load allocation weight ratio, and issues configuration instructions to the target data center to instruct the target data center to perform resource scheduling configuration on the target computing machine.

[0025] S110, the specific steps for collecting multi-dimensional data include: The target task is obtained through the task platform interface. The task type of the target task is automatically parsed as pre-training, fine-tuning or online inference. The model parameter quantity, parallel computing method, service level agreement latency index, concurrent throughput requirement and hardware resource requirement specification are extracted and the structured output is the feature vector of the target task. By collecting latitude and longitude coordinates, detecting routing links, and monitoring network round-trip latency, the latitude and longitude coordinates, link bandwidth, packet loss rate, and latency jitter parameters of each data center can be obtained in real time. Through the computing power machine status awareness channel, the GPU model, video memory capacity, number of CPU cores, memory size, interconnection protocol type, load utilization and power consumption parameters of each computing power machine in each data center can be obtained in real time. By accessing the data center power distribution system through the smart grid interface, the system can obtain real-time power distribution data at the rack level and summarize it by data center dimension. This allows the system to obtain the real-time total power consumption, instantaneous proportion of renewable power, short-time standard deviation of grid input power, and state of charge of the local energy storage system for each data center, which is then encapsulated as power supply status information.

[0026] Specifically, for step S110, a unified task access gateway exposes a task platform interface compatible with the OpenAPI specification, receiving JSON-formatted task description files submitted by users. The backend uses a rule engine combined with a predefined task metadata schema to automatically parse the data. First, it reads the `type` field from the task description, matching it with three enumerated values: pre-training, fine-tuning, and online inference. If the field is missing, a lightweight keyword matching algorithm performs semantic recognition on the task description text. For example, matching keywords like "full parameter training" and "base pre-training" classifies it as a pre-training task; matching keywords like "LoRA fine-tuning" and "full parameter fine-tuning" classifies it as a fine-tuning task; and matching keywords like "online service" and "inference interface deployment" classifies it as an online inference task. Then, it sequentially extracts the model parameter fields and converts them to billion-parameter units, reading... The `parallel_mode` field identifies data parallelism, tensor parallelism, pipeline parallelism, or hybrid parallelism modes. The end-to-end latency metric is parsed from the `sla` field and standardized to milliseconds. The `throughput` field is extracted to obtain the concurrent throughput requirement and standardized to requests per second. The `hardware_spec` field is read to parse hardware resource specifications such as the minimum GPU model, minimum memory limit, and minimum CPU core count. Finally, all extracted discrete features are one-hot encoded, and continuous features are normalized (logarithmic normalization is used for model parameters to compress numerical magnitude, and max-min normalization is used for other continuous features such as the reciprocal of latency and throughput). The features are then concatenated to form a fixed-dimensional feature vector. For example, for a LoRA fine-tuning task with 7B parameters, an SLA latency requirement of 200ms, a concurrent throughput of 50QPS, a hardware requirement of a single 80GB GPU, and a data parallel mode, the final generated feature vector can contain 16 dimensions of features, including one-hot encoding of the task type, log-normalized parameter count, parallel mode encoding, normalized reciprocal of latency, normalized throughput value, and hardware requirement encoding. These features can be directly used as the input vector for subsequent scheduling decisions.

[0027] After structuring the information on the task side, the next step is to collect basic information about the data centers. This can be achieved through a distributed probing architecture, deploying a dedicated network probing agent in each computing power data center. This agent has a built-in high-precision BeiDou positioning module, which can obtain the latitude and longitude coordinates of the data center's location with an accuracy of up to five decimal places, corresponding to meter-level positioning accuracy. At the same time, the agent periodically sends ICMP probe packets and TCP bandwidth probe streams to the probe nodes of all other data centers, using a bidirectional active probing mechanism. It performs a full-link probe every 5 seconds, obtaining the available bandwidth of the links between different data centers and unifying it in Gbps units. It also calculates the packet loss rate within one minute and the standard deviation of the round-trip latency, i.e., the latency jitter parameter, unifying it in milliseconds units. In addition, the agent also uses the traceroute tool to probe the number of hops and the traversed operator nodes of cross-domain links. The collected raw data is reported to the network status time-series database of the global scheduling center through the in-band management channel. The database is implemented using InfluxDB, with a secondary index built by data center pairs and timestamps. The scheduling module reads the sliding average of the latest probe cycle as the current link status information each time. For example, the detection results from the Beijing data center to the Shanghai data center show an available bandwidth of 10Gbps, a packet loss rate of 0.02%, an average round-trip latency of 25ms, and a latency jitter of 1.2ms. This data will be stored in a structured manner and updated in real time.

[0028] Beyond data collection at the data center level, fine-grained awareness of the operational status of individual computing machines is also required. This can be achieved through node-level lightweight agents. A data collection agent is deployed on each computing server, and multi-dimensional data collection is achieved through the server's out-of-band IPMI management interface and hardware / software interfaces such as the NVIDIA Management Library and CPU performance counters within the system. Specifically, GPU model and memory capacity are read via the NVML interface, with memory units uniformly in GB. The number of CPU cores and memory size are read through the operating system's sysfs file system, with memory units uniformly in GB. The interconnection protocol type is identified by reading the configuration information of the network card and access switch, such as NVLink 4, PCIe 4.0, RoCE v2, and InfiniBand. For HDR and similar types, load utilization is measured using four metrics: GPU utilization, video memory utilization, CPU utilization, and memory utilization, all expressed as percentages. Power consumption is measured in watts via the IPMI interface, with a sampling frequency of once every 10 seconds. The collected data is reported to the local resource management node in the data center via the data center's internal control VLAN. The local node then aggregates the data and synchronizes it to the global scheduling center. The scheduling center performs a moving average on the data from three consecutive sampling periods to eliminate errors caused by instantaneous fluctuations. For example, for a server equipped with eight A100 80GB GPUs, the collected data shows an average GPU utilization of 45%, video memory utilization of 62%, CPU utilization of 30%, total power consumption of 3200 watts, and interconnect protocols of NVLink 4 and RoCE v2. This data is updated in real time in the global resource view for subsequent assessment of remaining computing capacity and load allocation calculations.

[0029] Power supply status is the core basis for cross-domain green dispatch, which can be achieved through industrial power communication protocol integration. The global dispatch center establishes encrypted connections with the power distribution automation systems of each data center through a power communication gateway conforming to the IEC 61850 standard, collecting cabinet-level power distribution data every 5 seconds. Real-time total power consumption is read by smart power meters on the distribution cabinets, with the unit uniformly set to kilowatts. The instantaneous proportion of renewable power is obtained through the new energy grid-connected metering device of the power distribution system, calculated as real-time wind and solar power output divided by the total input power of the data center, expressed as a percentage. The short-time standard deviation of grid input power is calculated by taking the standard deviation of grid input power sampling points within the most recent minute, with the unit set to kilowatts, and is used to characterize grid power. The fluctuation level is assessed by obtaining the state of charge (SBC) of the local energy storage system through data reported by the energy storage converter. This SBC, expressed as a percentage, reflects the remaining capacity of the energy storage batteries. All collected power data undergoes data cleaning and format standardization at the edge gateway before being uploaded to the power status database of the dispatch center via a dedicated power communication channel. The database employs a time-series storage structure, establishing a secondary index based on the data center ID and timestamp. Outlier removal is performed to eliminate jumps caused by instrument sampling errors. The final data is encapsulated into a power supply status information structure containing four core fields. For example, power status data from a western photovoltaic power station's supporting data center shows a total cabinet power consumption of 1200 kW, a renewable energy instantaneous percentage of 85%, a short-time standard deviation of grid input power of 18 kW, and an energy storage SBC of 72%.

[0030] For step S120, the power supply reliability index calculation can be achieved by combining a time-series prediction model based on the collected power status data. First, four basic indicators are extracted from the power supply status information: real-time total power consumption of each equipment room, the proportion of renewable energy, grid input power fluctuation parameters, and energy storage charge status. Then, a pre-trained power supply prediction model is used to predict the wind and solar power generation of each equipment room for the next hour, minute by minute. This prediction model uses an attention-enhanced LSTM time-series network architecture. Input features include historical 24-hour wind and solar power output data, local weather station-provided light intensity and wind speed prediction data, and historical grid fluctuation data. The output is a 60-minute prediction sequence of photovoltaic and wind power generation, both in kilowatts. The model is trained using three months of historical wind and solar data and meteorological data. After obtaining the prediction results, the current energy storage charge status and the real-time data of the equipment rooms are combined. Total power consumption is used to calculate the power supply reliability index, specifically using a three-dimensional weighted summation method. Preset weights are assigned to renewable energy self-sufficiency rate, grid stability, and energy storage backup capability (e.g., the weights can be configured in the backend to be 0.4, 0.3, and 0.3). Each dimension is first normalized to the range of 0 to 1. Renewable energy self-sufficiency rate is the percentage of time in the next hour when the predicted renewable energy generation can cover the power consumption of the data center. Grid stability is 1 minus the normalized standard deviation of grid input power (normalization specifically involves dividing the current short-time standard deviation of grid input power by a preset grid power standard deviation limit threshold, which is set according to the total power distribution capacity of the data center; a ratio greater than 1 is truncated to 1). Energy storage backup capability is the ratio of the current energy storage state of charge to its rated capacity. The weighted summation of these three factors yields a power supply reliability index ranging from 0 to 1, with higher values ​​indicating stronger power supply stability. For example, if a photovoltaic (PV) supporting equipment room can cover 75% of the time for PV power generation in the next hour, the grid fluctuation normalization value is 0.05, and the energy storage state of charge is 70%, then the power supply reliability index is calculated as 0.75*0.4+(1-0.05)*0.3+0.7*0.3=0.3+0.285+0.21=0.795, which is at a relatively high level of power supply stability.

[0031] In S130, the specific steps for selecting the candidate node set from all available data centers include: Define a power supply condition judgment function and a network quality condition judgment function. The power supply condition judgment function outputs a first value when the power supply reliability index is greater than or equal to a first index threshold and the short-time standard deviation of the grid input power obtained from the power supply status information is less than or equal to a fluctuation tolerance threshold; otherwise, it outputs a second value. The network quality condition judgment function outputs a first value when the link bandwidth in the link status information is greater than or equal to a minimum bandwidth threshold, the packet loss rate is less than or equal to a maximum packet loss rate threshold, and the delay jitter is less than or equal to a maximum jitter threshold; otherwise, it outputs a second value. Only when both the power supply condition judgment function and the network quality condition judgment function output the first value will the corresponding computer room be included in the candidate node set for this round of scheduling.

[0032] Specifically, for step S130, the candidate node set selection is achieved through a double Boolean decision function. The dispatch center pre-configures power supply-related thresholds and network quality-related thresholds. For the power supply dimension, considering that the power supply reliability index reflects the macro-level overall power supply trend for the next hour, to prevent equipment downtime caused by sudden and drastic power fluctuations in the power grid, a first index threshold (e.g., 0.6) and a fluctuation tolerance threshold (e.g., 30 kW) are set. For the network dimension, to exclude nodes on the verge of network outage or severely degraded, a minimum bandwidth threshold, a maximum packet loss rate threshold, and a maximum jitter threshold are set.

[0033] At the start of each round of scheduling, the scheduling center traverses all online computer rooms globally. The power supply condition judgment function of the computer room outputs logical truth when the power supply reliability index is greater than or equal to the first index threshold and the short-time standard deviation of the grid input power is less than or equal to the fluctuation tolerance threshold. The network quality condition judgment function outputs logical truth when the link bandwidth is greater than or equal to the minimum bandwidth threshold, the packet loss rate is less than or equal to the maximum packet loss rate threshold, and the latency jitter is less than or equal to the maximum jitter threshold.

[0034] Only data centers whose two decision functions both output logically true (i.e., the first value) are included in the candidate node set for this round of scheduling; if either condition is not met, the data center is directly excluded. For example, in a certain round of scheduling, the Zhangbei data center had a power supply reliability index of 0.82, a short-time standard deviation of grid input power of 12 kW, and all link indicators met the network thresholds, so it was successfully included in the candidate set; while a data center in Guangdong had a power supply reliability index of 0.53, a grid standard deviation of 45 kW, and an excessive packet loss rate, failing to meet both conditions, and therefore was not included in the candidate node set.

[0035] This dual screening mechanism eliminates nodes with high power supply risks and severely degraded networks at the source, ensuring the basic operational stability of scheduling tasks. Subsequent network quality comprehensive score evaluation and topology adaptive selection are both performed within this set of candidate nodes that have passed the basic admission screening.

[0036] In S130, the steps for adaptively selecting the cross-domain communication topology based on the comprehensive network quality score of each node in the candidate node set specifically include: The overall network quality score is calculated based on the ratio of the available bandwidth to the reference bandwidth, the deviation of the one-way transmission delay from the reference delay, and the deviation of the delay jitter from the reference jitter. When the overall network quality score is greater than or equal to the high-quality link threshold, a fully connected pipeline topology is enabled among the candidate node sets. When the overall network quality score is less than the high-quality link threshold but greater than or equal to the low-quality link threshold, a star topology is enabled among the candidate node sets. When the overall network quality score is less than the low-quality link threshold, the candidate node set is prohibited from participating in cross-domain communication cooperation.

[0037] Specifically, after node selection, the network quality assessment and topology selection phase begins, achieved through a multi-indicator weighted scoring system. First, reference baseline values ​​are set for three network indicators: available bandwidth, one-way transmission latency, and latency jitter. The reference bandwidth is set to 10Gbps, the reference latency to 50 milliseconds, and the reference jitter to 5 milliseconds. Then, normalized scores for each indicator are calculated. The available bandwidth score is calculated as the ratio of the current available bandwidth to the reference bandwidth, with an upper limit of 1 (meaning that exceeding the reference bandwidth results in a full score). The one-way transmission latency score is calculated... The latency jitter score is calculated as 1 minus the absolute value of the difference between the current one-way latency and the reference latency, divided by the reference latency. When the difference is negative, i.e., the latency is lower than the reference value, a full score of 1 is directly obtained. The latency jitter score is calculated as 1 minus the ratio of the current jitter value to the reference jitter value. When the jitter value is less than the reference value, a full score of 1 is obtained. Then, weights are assigned to the three indicators, such as configuring a bandwidth weight of 0.4, a latency weight of 0.4, and a jitter weight of 0.2. The three normalized scores are multiplied by their corresponding weights and then summed to obtain the comprehensive network quality score, which ranges from 0 to 1. The higher the score, the better the link quality.

[0038] After calculating the comprehensive link score between all node pairs in the candidate node set, the lowest link score in the candidate node set is used as the overall network quality score of the candidate node set. This score is then compared with preset high-quality link thresholds and low-quality link thresholds, where the high-quality link threshold can be set to 0.8 and the low-quality link threshold to 0.5. When the overall comprehensive score is greater than or equal to 0.8, it is determined to be a high-quality cross-domain link. In this case, a fully connected pipeline topology is enabled between all nodes in the candidate node set, allowing each node to directly transmit tensors (such as gradients, activation values, or KV caches) to be transmitted with all other nodes, maximizing pipeline parallel efficiency. When the overall comprehensive score is between 0.5 and 0.8, it is determined to be a medium-quality link. In this case, a star topology is enabled, selecting the node with the best network quality as the central node. All other nodes only interact with the central node, reducing the number of cross-node communications and mitigating performance losses caused by insufficient link quality. When the overall comprehensive score is less than 0.5, it is determined to be a low-quality link, prohibiting nodes in this candidate node set from participating in cross-domain communication collaboration; they can only independently execute local single-machine tasks. For example, the network quality score between the Beijing and Shanghai data centers is 0.85, which is considered a high-quality link. Therefore, the nodes in both locations adopt a fully connected pipeline topology. The score between Beijing and a remote data center in western China is 0.62, which is considered a medium-quality link. Therefore, a star topology with the Beijing node as the center is adopted.

[0039] In S130, the steps for nodes participating in cross-domain communication to adaptively perform data compression based on network and power supply status include: The compression factor is dynamically calculated based on the power supply reliability index and network quality comprehensive score of each candidate node, and sparsification and quantization compression are performed on the tensor to be transmitted based on the compression factor; where, when the target task is a pre-training or fine-tuning task, the tensor to be transmitted is a gradient tensor; when the target task is an online inference task, the tensor to be transmitted is an intermediate activation tensor for cross-domain transmission. Wherein, compression factor = base compression rate + first weighting term + second weighting term, first weighting term = power sensitivity coefficient * (1 - difference in power supply reliability index), second weighting term = network sensitivity coefficient * (1 - difference in network quality comprehensive score).

[0040] Specifically, to adapt to different network and power supply conditions, gradient compression achieves adaptive adjustment through a dynamic compression factor. First, a base compression ratio of 0.1 is preset, representing a 10% sparsity compression of the gradient tensor by default. Simultaneously, two hyperparameters are set: a power sensitivity coefficient and a network sensitivity coefficient. Both range from 0 to 0.4, representing the influence weights of power supply and network conditions on compression strength, respectively. These are typically set to 0.3 and 0.4 by default, reflecting the greater impact of network conditions on communication compression. Then, two weighted terms are calculated. The first weighted term is the power sensitivity coefficient multiplied by 1 and the current node's power supply reliability. The difference in the power supply reliability index increases with lower power supply reliability, resulting in higher compression intensity and reduced computation and transmission power consumption of nodes, adapting to unstable power supply conditions. The second weighting term is the difference between the network sensitivity coefficient multiplied by 1 and the comprehensive network quality score of the corresponding link. The worse the network quality, the larger the weighting term. This reduces data transmission volume through higher compression ratio, alleviating link bandwidth and latency pressure. The final compression factor is obtained by adding the base compression ratio and the two weighting terms. The theoretical range of the compression factor is 0.1 to 0.8. Calculation results outside this range are automatically truncated to the boundary value.

[0041] After obtaining the compression factor, a two-step operation of sparsification and quantization compression is performed on the tensor to be transmitted based on this value. The first step, sparsification, retains the element with the largest absolute value in the tensor according to the proportion corresponding to the compression factor, setting the remaining elements to zero. The second step performs 8-bit integer quantization on the retained non-zero elements, compressing the original 32-bit floating-point data into 8-bit integers, further reducing the data volume. If the current task is online inference, since the intermediate activation tensor has a significant impact on the model's output accuracy, the sparsification step is skipped, and only 8-bit integer quantization is performed based on the compression factor. For example, when a node is performing a fine-tuning task, the power supply reliability index... With a coefficient of 0.7, a comprehensive link network quality score of 0.6, a basic compression rate of 0.1, a power sensitivity coefficient of 0.3, and a network sensitivity coefficient of 0.4, the first weighting term is 0.3*(1-0.7)=0.09, the second weighting term is 0.4*(1-0.6)=0.16, and the compression factor is 0.1+0.09+0.16=0.35. This means that a 35% sparsity is applied to the gradient tensor, retaining the top 65% of the gradient elements by absolute value, and then 8-bit quantization is performed. The final data volume can be reduced to about 16.25% of the original, effectively reducing the amount of data transmitted across domains and the power consumption of node computing.

[0042] Reference Figure 2 In S140, the steps for calculating the load allocation weights of each candidate node specifically include S210-S250: S210, establish the distance-transmission delay-data transmission cost mapping relationship; S220 divides the complete computation graph into pipeline stages with the same number of nodes as the candidate node set, and calculates the computational load ratio of each pipeline stage. S230, calculate the preliminary weights based on the first power of the remaining computing power capacity of each candidate node, the second power of the power supply reliability index, and the third power of the network quality comprehensive score; S240, combining the feature vector of the target task, the currently enabled cross-domain communication topology, and the distance-transmission delay-data transmission cost mapping relationship, the initial weights are weighted and corrected to obtain the load allocation weights; among them, the distance-transmission delay-data transmission cost mapping relationship serves as the quantitative mapping basis between the transmission delay optimization objective and the scheduling cost optimization objective; S250: Divide the load allocation weight of each candidate node by the sum of the load allocation weights of all candidate nodes to obtain the actual allocation ratio of each candidate node.

[0043] Specifically, for step S140, in order to quantify the scheduling latency and cost targets, the mapping relationship can be established through geographic calculation and data fitting. First, based on the latitude and longitude coordinates of each data center, the spherical distance between any two data centers is calculated using the Haversine formula, with the unit being kilometers. Then, actual one-way transmission latency data of the link at different distances, as well as the unit bandwidth leasing cost of operator leased lines and the unit data transmission cost of data traffic billed, are collected, with the unit being yuan / GB. A mapping function between distance, transmission latency, and data transmission cost is established through polynomial fitting. Specifically, firstly, distance is used as the independent variable and transmission delay as the first dependent variable. A linear mapping function of delay with respect to distance is fitted. Since the speed of light in optical fiber is approximately 200,000 kilometers per second, the delay and distance are basically linearly related. The processing delay of fixed equipment is also included as an intercept term. For example, one-way delay (milliseconds) = distance (km) / 200 + 5, where 5 milliseconds is the processing delay of the two ends. Then, distance and transmission delay are used together as independent variables, and data transmission cost is used as the second dependent variable. A mapping function of cost with respect to distance and delay is fitted. Generally, the greater the distance, the higher the latency requirement, and the higher the cost per unit time. The higher the transmission cost, the better. For example, the unit GB transmission cost of ordinary leased lines is positively correlated with distance, while the cost of low-latency leased lines increases exponentially. Finally, the two fitted functions are encapsulated into a mapping table and an interpolation calculation interface. When scheduling, by inputting the data center numbers of the two nodes, the corresponding transmission latency and unit data transmission cost can be quickly queried or calculated. For example, the spherical distance from Beijing to Shanghai is about 1080 kilometers. Through the mapping relationship, the one-way transmission latency is calculated to be about 10.4 milliseconds, and the unit GB data transmission cost is about 0.8 yuan. This mapping relationship will serve as a quantitative basis for optimizing latency and cost targets during subsequent load allocation.

[0044] In the task load partitioning phase, computation graph segmentation and pipeline load calculation can be implemented based on a deep learning compiler. First, the complete deep learning computation graph corresponding to the target task is loaded. The computation graph contains all operators, input and output tensors, and dependencies of the model. Then, based on the number of nodes N in the candidate node set, the computation graph is segmented into N consecutive pipeline stages according to the network layer structure of the model. The segmentation principle is to balance the total number of operators and computational load in each stage as much as possible, while ensuring that the amount of interaction data between stages is minimized and avoiding the transfer of excessive activation tensors across stages. After segmentation, for each pipeline stage, the floating-point operation volume of each stage is calculated using an operator performance analysis tool, i.e., an estimation model based on FLOPs and hardware computing power, in TFLOPs. Then, the floating-point operation volume of a single stage is divided by the total floating-point operation volume of all stages to obtain the computational load ratio of each pipeline stage. The sum of the load ratios of all stages is 1. For example, for a large language model with an 80-layer Transformer structure, when the number of candidate nodes is 4, each 20 layers are divided into a pipeline stage. By calculating the FLOPs of each stage, the computational load ratios of the four stages are 0.23, 0.27, 0.26, and 0.24, respectively, which are basically balanced. This load ratio will serve as a reference benchmark for subsequent node load allocation to ensure that the computational tasks allocated to each node match its computing power capacity.

[0045] After load balancing, the initial weight calculation is achieved through multi-dimensional power-law weighting. First, the remaining computing power capacity of each candidate node is calculated. This remaining capacity is obtained by multiplying the node's theoretical hardware computing power by the current idle resource ratio. The theoretical GPU computing power is read from a predefined computing power parameter table based on the GPU model, with the unit uniformly set to TFLOPS (FP32). The idle resource ratio is 1 minus the GPU utilization percentage. For example, an A100 GPU has a theoretical FP32 computing power of 19.5 TFLOPS, and the current GPU utilization is 40%, then the remaining computing power of a single card is 19.5 * 0.6 = 11.7 TFLOPS. TFLOPS, for multi-GPU nodes, accumulates the remaining computing power of all GPUs to obtain the total remaining computing power capacity of the entire machine. Then, three power-law parameters are set: the first power, the second power, and the third power, corresponding to the weight sensitivity of remaining computing power, power supply reliability, and network quality, respectively. For example, the first power is set to 1, the second power to 0.5, and the third power to 0.8, reflecting that remaining computing power is the core allocation basis, while power supply and network are adjustment factors. The initial weight calculation formula is: the initial weight equals the first power of the remaining computing power capacity multiplied by the second power of the power supply reliability index multiplied by the third power of the network quality comprehensive score. Through power-law calculation, the influence of different indicators on the weight can be adjusted; the higher the power, the more significant the influence of that indicator. For example, if a candidate node has a remaining computing power capacity of 100... With TFLOPS, a power reliability index of 0.8, and a network quality score of 0.9, the initial weights are calculated as follows: first power of 1, second power of 0.5, and third power of 0.8. The initial weights are 100^1 × 0.8^0.5 × 0.9^0.8 ≈ 100 × 0.894 × 0.919 ≈ 82.16. These initial weights comprehensively reflect the node's computing power supply capacity, power supply stability, and network transmission capacity, serving as the basis for load allocation.

[0046] To ensure weights align with the personalized needs of tasks, weight adjustments are made through a task feature-cost mapping. First, key features such as task type, latency sensitivity, and cost sensitivity are extracted from the target task's feature vector. Latency sensitivity is calculated from the SLA latency metric; the stricter the SLA, the higher the latency sensitivity. Cost sensitivity is calculated from the task's budget metric; the tighter the budget, the higher the cost sensitivity. Then, for each candidate node, transmission latency and unit data transmission cost are calculated using the distance-transmission latency-data transmission cost mapping relationship. If the currently adaptively selected cross-domain communication topology is a star topology, the one-way transmission latency and unit data transmission cost between the node and the central scheduling node are calculated as evaluation values. If the current topology is a fully connected pipeline topology, the average one-way transmission latency and average unit data transmission cost between the node and all other nodes in the candidate node set are calculated as evaluation values. Finally, latency deviation and cost deviation are calculated separately. Latency deviation is the ratio of the current node's evaluated transmission latency to the ideal minimum latency of the entire network, and cost deviation is the ratio of the current node's evaluated transmission cost to the ideal minimum cost of the entire network.

[0047] Based on this, two correction coefficients are constructed: a latency correction coefficient and a cost correction coefficient. The latency correction coefficient is equal to 1 minus the difference between latency sensitivity multiplied by latency deviation minus 1, and the cost correction coefficient is equal to 1 minus the difference between cost sensitivity multiplied by cost deviation minus 1. The values ​​of both correction coefficients are constrained within a preset range (e.g., between 0.5 and 1.5) to avoid over-correction. Finally, the initial weight of each node is multiplied by the latency correction coefficient and the cost correction coefficient in turn to obtain the final load allocation weight. At the same time, combined with the hardware requirements of the task, if the node hardware configuration just meets the minimum requirements of the task, it is multiplied by a preset base coefficient (e.g., 1.0); if it exceeds the requirements by a large margin, it is multiplied by a preset redundancy coefficient (e.g., 1.1) to further optimize the rationality of the weights. For example, for a high latency-sensitive online inference task, if a node has an initial weight of 80, a latency deviation of 1.2, a latency sensitivity of 0.8, a cost sensitivity of 0.3, and a cost deviation of 1.1, then the latency correction coefficient is 1 - 0.8 * (1.2 - 1) = 0.84, the cost correction coefficient is 1 - 0.3 * (1.1 - 1) = 0.97, and the final load allocation weight is 80 × 0.84 × 0.97 ≈ 65.18. This correction mechanism can quantify the task's performance requirements and scheduling cost objectives into the weight calculation, making the load allocation more closely match the actual needs of the task.

[0048] After obtaining the corrected weights, the actual allocation ratio is calculated through normalization. After obtaining the load allocation weights of all candidate nodes, all weights are summed to obtain the total weight value. Then, the load allocation weight of each node is divided by the total weight to obtain the actual load allocation ratio of that node. The sum of the allocation ratios of all nodes is 1. This ratio represents the proportion of the computational load that each node needs to bear to the total task load. For example, if the load allocation weights of four candidate nodes are 60, 80, 70, and 90, and the total weight is 300, then the actual allocation ratios of the four nodes are 0.2, 0.267, 0.233, and 0.3, respectively. The scheduling system will match this ratio with the previously divided pipeline stage load and allocate the pipeline stage with the corresponding amount of computation to each node to ensure that the load allocation ratio matches the overall capacity of the node.

[0049] In S150, the specific steps of issuing configuration instructions to the target data center to instruct it to perform resource scheduling configuration for the target computing power machine include: The model pipeline stages are assigned to the corresponding candidate nodes according to the load allocation weight ratio, and configuration instructions are issued to the target data center to instruct the target data center to perform GPU virtualization partitioning, elastic adjustment of video memory and memory resources, switching of interconnect communication protocols, and allocation of CPU core resources on the target computing machine.

[0050] Specifically, for step S150, the final task distribution and resource allocation are completed through management channels and virtualization technology. Based on the calculated actual allocation ratio of each node, combined with the previously divided pipeline stages and their load ratios, a greedy matching algorithm is used to allocate pipeline stages to corresponding candidate nodes. The allocation principle is to prioritize assigning high-load stages to high-weight nodes, ensuring that the deviation between the total load ratio allocated to each node and its actual allocation ratio is controlled within a preset tolerance range (e.g., 5%). After allocation, the global scheduling center issues configuration instructions to the target data center through the data center management interface. The instructions include the allocated pipeline stage number, model slice parameters, etc. Upon receiving instructions regarding the number of computing instances and resource configuration requirements, the local control node in the target data center executes a series of elastic resource configuration operations on the designated target computing machines: First, it performs hard isolation or soft partitioning of GPU resources using Multi-Instance GPU (MIG) technology or container-level time sharding technology, allocating a corresponding number of computing instances and GPU memory capacity according to task requirements to achieve fine-grained reuse of GPU resources; second, it adjusts the resource quotas for GPU memory and RAM, elastically adjusting the available memory size through memory over-allocation and dynamic reservation mechanisms to meet the GPU memory and RAM requirements of the task; subsequently, it switches the interconnection communication protocol according to the cross-node topology type, such as enabling RoCE in a fully connected topology. The RoCEv2 protocol optimizes multi-node communication, switching to TCP in star topologies to reduce protocol overhead. Finally, CPU affinity is used to allocate a specified number of CPU cores to tasks and bind them to corresponding NUMA nodes, reducing performance loss caused by cross-NUMA access. For example, after allocating 20% ​​of the computing load to a node, an instruction is issued to use MIG technology to split an isolated computing instance from two physical cards on the node's 8-card A100 server and allocate it to the task, allocating 160GB of video memory, 64 CPU cores, and enabling the RoCEv2 interconnect protocol. After the resource configuration is completed, the node can start the corresponding pipeline computing task.

[0051] Furthermore, cross-domain dynamic computing power scheduling methods also include: The status of each active node is continuously monitored. When any active node meets the preset migration trigger condition, the corresponding active migration process is triggered. The migration trigger conditions include at least one of the following: the power supply reliability index is lower than the first danger threshold, the network quality comprehensive score is lower than the second danger threshold, or the available video memory capacity is lower than the capacity threshold.

[0052] Reference Figure 3 The specific steps that trigger the corresponding active migration process include S310-S330: S310, send an interrupt command to the target node that triggered the backtracking condition, and the target node stops executing new forward computation; if the target task is a pre-training or fine-tuning task, discard the non-critical optimizer state and retain only the gradient core snapshot composed of the elements whose absolute value is within a preset high proportion in the gradient tensor; if the target task is an online inference task, extract the context state of the current incomplete inference request to form an inference state snapshot. S320, the gradient core snapshot or inference state snapshot is sent back to the main scheduling domain with the highest priority through the residual network channel; S330, after confirming that it has received the gradient core snapshot or inference state snapshot, sends an acknowledgment signal to the target node. The target node clears the GPU memory and temporary cache of the current task, resets its own state to standby idle state, marks it as a standby node in the global resource list, and moves it into the next round of the candidate node set's candidate queue.

[0053] Specifically, regarding the abnormal migration mechanism during task execution, status monitoring and triggering are implemented through high-frequency inspection logic. During task execution, the status monitoring module of the global scheduling center continuously collects the running status data of all active computing nodes at a frequency of once per second, including core indicators such as power supply reliability index, network quality comprehensive score, and available video memory capacity. The real-time values ​​are compared with the preset migration trigger thresholds. As long as any indicator meets the corresponding triggering condition, the active migration process of that node is immediately triggered.

[0054] The preset migration trigger conditions include three core risk scenarios: The first is power supply risk, where the node's power supply reliability index is lower than the first danger threshold, which is usually set to 0.3, indicating that the power supply stability is at a high risk and there is a risk of power outage at any time. The second is network risk, where the node's network quality score is lower than the second danger threshold, which is usually set to 0.2, indicating that the link can no longer support normal cross-domain data transmission and that communication packet loss and latency have exceeded the task's tolerance range. The third is resource risk, where the node's available video memory capacity is lower than the capacity threshold, which is set according to the task's minimum video memory requirements, for example, 1.2 times the video memory required by the task, indicating that the remaining video memory is insufficient to ensure the normal operation of the task. For example, if a running node experiences a sudden drop in photovoltaic power generation due to cloud cover, causing the power supply reliability index to drop to 0.25, which is lower than the first danger threshold of 0.3, the monitoring module will immediately trigger the node's active migration process after recognizing that this condition is met, to avoid the loss of task training data due to power outage.

[0055] After the scheduling center triggers the migration process, it first sends a task interruption command to the target node through a high-priority control signaling channel. Upon receiving the interruption command, the target node's computing process immediately stops receiving new forward computing tasks, terminates the currently executing iteration or inference step, and does not start new computing iterations.

[0056] Subsequently, differentiated state preservation is performed based on the task type: If the current task is pre-training or fine-tuning, the optimizer state is filtered, discarding non-critical optimizer state data, such as the second moment of the momentum matrix and historical statistics of the adaptive learning rate, retaining only the critical state such as the first-order momentum necessary for the optimizer to run. Simultaneously, a core snapshot is extracted from the gradient tensor generated in the current iteration. Based on a preset high-order ratio, typically the top 10% of elements by absolute value, the elements with the largest absolute values ​​in the gradient tensor are selected, retaining only their values ​​and corresponding index positions to form the gradient core snapshot; all other gradient elements are discarded. If the current task is online inference, there is no need to process the optimizer and gradients; the target node directly extracts the key-value pairs of the currently incomplete inference batch. The cache tensor or request session context constitutes an inference state snapshot. For example, if a 7B model triggers a backhaul during the fine-tuning phase, its gradient tensor is originally about 28GB in size. After extracting the first 10% of the core elements, not only 2.8GB of gradient values ​​(using 32-bit floating-point numbers) needs to be retained, but also the corresponding 2.8GB of index position data (using 32-bit integers to record the position of the first 10% of elements in the original tensor by absolute value). The total size of the gradient core snapshot is about 5.6GB, which greatly reduces the bandwidth pressure and time consumption of the backhaul compared to the original 28GB.

[0057] After a snapshot is generated, the high-priority backhaul phase begins. Once the target node generates a gradient core snapshot or inference state snapshot, it is immediately marked as the highest-priority transmission task. The scheduling center reserves all available bandwidth in the remaining network channels for this transmission task, suspends all non-critical background data transmission, and allocates all available bandwidth resources to the snapshot backhaul task. During transmission, a breakpoint resumption and forward error correction coding mechanism are used to ensure the reliability and transmission rate of data transmission even under poor link quality, avoiding retransmission delays due to packet loss. The target address of the transmission is the central storage node of the main scheduling domain. During the backhaul process, the target node will report the transmission progress to the scheduling center in real time to ensure that the scheduling center can grasp the backhaul status. For example, if a node triggers a backhaul while performing a 7B model fine-tuning task, the remaining available bandwidth is 1Gbps (i.e., 125MB / s), and transmitting a 5.6GB gradient core snapshot can be completed in about 45 seconds; if a backhaul is triggered while performing an online inference task, assuming the current concurrent request for the KV Cache inference state snapshot size is 1.5GB, it can be completed in about 12 seconds under the same bandwidth, ensuring that the inference context is quickly preserved before the node fails.

[0058] Finally, node reset and state update are implemented through the resource management module. When the central storage node of the main scheduling domain fully receives the returned snapshot (gradient core snapshot or inference state snapshot) and confirms that the data is complete and error-free through checksum verification, it immediately sends an acknowledgment signal back to the target node. After receiving the acknowledgment signal, the target node first performs a video memory and memory cleanup operation, calls the GPU driver interface to clear all GPU video memory occupied by the current task, and deletes the task's temporary cache file, intermediate activation values, non-critical optimizer states, or key-value pairs in memory. All task-related data, including cache data, is cached, and all occupied hardware resources are released. The node's running state is then reset to standby / idle, all task-related computation processes are stopped, and the node's computing resources are reintegrated into the global idle resource pool. Finally, the node's status label is updated in the global resource list from "active computing node" to "standby node," and it is moved to the next round of the candidate node set's candidate queue, awaiting re-participation in the next round of scheduling. After all operations are completed, the target node reports a migration completion signal to the scheduling center, ending the entire active migration process. The scheduling center then initiates the task rescheduling process, redistributing the migrated computing load to other normal candidate nodes and instructing the new nodes to load the received snapshot data to restore the training iteration or inference context, ensuring the overall continuity and data integrity of the task.

[0059] Based on the above method embodiments, the second embodiment of this application discloses a cross-domain dynamic scheduling system for computing power and power coordination. The cross-domain dynamic scheduling system for computing power and power coordination of this application embodiment can implement any of the above-mentioned methods for cross-domain dynamic scheduling of computing power and power coordination, and the specific working process of each module in the cross-domain dynamic scheduling system for computing power and power coordination can refer to the corresponding process in the above method embodiments.

[0060] For ease of understanding, an example is as follows: A cross-domain dynamic computing power scheduling system that coordinates computing power and power supply includes: The data acquisition module is configured to collect multi-dimensional data, including: the feature vector of the target task parsed from the task platform; the location information and link status information of the data center collected through the data center distance perception mechanism; the hardware configuration and operating status information of each computing machine collected through the computing machine status perception mechanism; and the power supply status information of each data center collected through the power monitoring interface. The power forecasting and analysis module is configured to: obtain the real-time total power consumption, renewable power ratio, grid input power fluctuation parameters and energy storage charge status of each computer room based on the power supply status information; predict the future wind and solar power generation of each computer room through the power supply forecasting model; and calculate the power supply reliability index of each computer room based on the forecast results and energy storage charge status. The node selection and topology construction module is configured to: select a set of candidate nodes that meet the power supply and network quality conditions from all available computer rooms based on the power supply reliability index and the link bandwidth, packet loss rate and latency jitter in the link status information; adaptively select the cross-domain communication topology based on the comprehensive network quality score of each node in the candidate node set; and adaptively perform data compression for nodes participating in cross-domain communication based on network and power supply status. The scheduling decision module is configured to: take the minimum transmission latency, the lowest scheduling cost, the highest computing power utilization, and the service level agreement indicators as multiple optimization objectives, combine the feature vector of the target task, the currently enabled cross-domain communication topology, and the status of each node, calculate the load distribution weight of each candidate node based on the multi-dimensional power weighting and deviation correction algorithm, and determine the allocation relationship of the pipeline stage based on the greedy matching strategy. The task allocation and configuration distribution module is configured to allocate the model pipeline stages to the corresponding candidate nodes according to the load allocation weight ratio, and issue configuration instructions to the target data center to instruct the target data center to perform resource scheduling configuration on the target computing machine.

[0061] The third embodiment of this application provides a computer device, which may include a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to realize a cross-domain dynamic scheduling method for computing power and power coordination.

[0062] The memory can communicate with the processor via a communication bus, which can be an address bus, a data bus, a control bus, etc.

[0063] Additionally, the memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device.

[0064] Furthermore, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0065] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0066] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is only one example of a series of equivalent or similar features.

Claims

1. A cross-domain dynamic scheduling method for computing power in collaboration with power supply, characterized in that, include: Collect multi-dimensional data, including: feature vectors of the target task parsed from the task platform; data center location information and link status information collected through the data center distance perception mechanism; hardware configuration and operating status information of each computing machine collected through the computing machine status perception mechanism; and power supply status information of each data center collected through the power monitoring interface. Based on the power supply status information, the real-time total power consumption, renewable power ratio, grid input power fluctuation parameters, and energy storage charge status of each computer room are obtained. The future wind and solar power generation of each computer room is predicted by the power supply prediction model. The power supply reliability index of each computer room is calculated based on the prediction results and the energy storage charge status. Based on the power supply reliability index and the link bandwidth, packet loss rate and latency jitter in the link status information, a set of candidate nodes that meet the power supply and network quality conditions is selected from all available computer rooms. The cross-domain communication topology is adaptively selected based on the comprehensive network quality score of each node in the candidate node set. Data compression is adaptively performed on the nodes participating in cross-domain communication based on the network and power supply status. With minimum transmission latency, lowest scheduling cost, highest computing power utilization, and satisfaction of service level agreement indicators as multiple optimization objectives, the load allocation weight of each candidate node is calculated based on the feature vector of the target task, the currently enabled cross-domain communication topology, and the status of each node, and the allocation relationship of the pipeline stage is determined based on the multidimensional power weighting and deviation correction algorithm, combined with the feature vector of the target task, the currently enabled cross-domain communication topology, and the status of each node. The model pipeline stages are allocated to the corresponding candidate nodes according to the load allocation weight ratio, and a configuration instruction is sent to the target data center to instruct the target data center to perform resource scheduling configuration on the target computing machine.

2. The cross-domain dynamic scheduling method for computing power and power coordination according to claim 1, characterized in that, The specific steps for collecting multi-dimensional data include: The target task is obtained through the task platform interface. The task type of the target task is automatically parsed as pre-training, fine-tuning or online inference. The model parameter quantity, parallel computing method, service level agreement latency index, concurrent throughput requirement and hardware resource requirement specification are extracted and the structured output is the feature vector of the target task. By collecting latitude and longitude coordinates, detecting routing links, and monitoring network round-trip latency, the latitude and longitude coordinates, link bandwidth, packet loss rate, and latency jitter parameters of each data center can be obtained in real time. Through the computing power machine status awareness channel, the GPU model, video memory capacity, number of CPU cores, memory size, interconnection protocol type, load utilization and power consumption parameters of each computing power machine in each data center can be obtained in real time. By accessing the data center power distribution system through the smart grid interface, the system can obtain the real-time total power consumption of the cabinet, the instantaneous proportion of renewable power, the short-time standard deviation of grid input power, and the state of charge of the local energy storage system, and encapsulate them into a power supply state vector.

3. The cross-domain dynamic scheduling method for computing power and power coordination according to claim 1, characterized in that, The specific steps for selecting a candidate node set from all available data centers include: Set a power supply condition judgment function and a network quality condition judgment function. The power supply condition judgment function outputs a first value when the power supply reliability index is greater than or equal to a first index threshold and the short-time standard deviation of the grid input power obtained from the power supply status information is less than or equal to a fluctuation tolerance threshold; otherwise, it outputs a second value. The network quality condition judgment function outputs a first value when the link bandwidth in the link status information is greater than or equal to the minimum bandwidth threshold, the packet loss rate is less than or equal to the maximum packet loss rate threshold, and the latency jitter is less than or equal to the maximum jitter threshold; otherwise, it outputs a second value. Only when both the power supply condition determination function and the network quality condition determination function output the first value will the corresponding computer room be included in the candidate node set for this round of scheduling.

4. The cross-domain dynamic scheduling method for computing power and power coordination according to claim 3, characterized in that, The steps for adaptively selecting a cross-domain communication topology based on the comprehensive network quality score of each node in the candidate node set specifically include: The network quality comprehensive score is calculated based on the ratio of the available bandwidth to the reference bandwidth, the deviation of the one-way transmission delay from the reference delay, and the deviation of the delay jitter from the reference jitter. When the overall network quality score is greater than or equal to the high-quality link threshold, a fully connected pipeline topology is enabled among the candidate node set. When the overall network quality score is less than the high-quality link threshold but greater than or equal to the low-quality link threshold, a star topology is enabled among the candidate node set. When the overall network quality score is less than the low-quality link threshold, the candidate node set is prohibited from participating in cross-domain communication cooperation.

5. The cross-domain dynamic scheduling method for computing power and power coordination according to claim 3, characterized in that, The steps for nodes participating in cross-domain communication to adaptively perform data compression based on network and power supply status include: The compression factor is dynamically calculated based on the power supply reliability index and network quality comprehensive score of each candidate node, and sparsification and quantization compression are performed on the tensor to be transmitted based on the compression factor; wherein, when the target task is a pre-training or fine-tuning task, the tensor to be transmitted is a gradient tensor; when the target task is an online inference task, the tensor to be transmitted is an intermediate activation tensor for cross-domain transmission. Wherein, the compression factor = base compression rate + first weighting term + second weighting term, the first weighting term = power sensitivity coefficient * (1 - difference of power supply reliability index), and the second weighting term = network sensitivity coefficient * (1 - difference of network quality comprehensive score).

6. The cross-domain dynamic scheduling method for computing power and power coordination according to claim 3, characterized in that, The specific steps for calculating the load allocation weights of each candidate node include: Establish a mapping relationship between distance, transmission delay, and data transmission cost; The complete computation graph is divided into pipeline stages by layer, with the number of nodes equal to the number of nodes in the candidate node set, and the computational load ratio of each pipeline stage is calculated. The preliminary weights are calculated based on the first power of the remaining computing power capacity of each candidate node, the second power of the power supply reliability index, and the third power of the network quality comprehensive score. By combining the feature vector of the target task, the currently enabled cross-domain communication topology, and the distance-transmission delay-data transmission cost mapping relationship, the initial weights are weighted and corrected to obtain the load allocation weights; wherein, the distance-transmission delay-data transmission cost mapping relationship serves as the quantitative mapping basis between the transmission delay optimization objective and the scheduling cost optimization objective; The actual allocation ratio of each candidate node is obtained by dividing the load allocation weight of each candidate node by the sum of the load allocation weights of all candidate nodes.

7. The cross-domain dynamic scheduling method for computing power and power coordination according to claim 1, characterized in that, The steps of issuing configuration instructions to the target data center to instruct the target data center to perform resource scheduling configuration for the target computing power machine specifically include: The model pipeline stages are allocated to the corresponding candidate nodes according to the load allocation weight ratio, and configuration instructions are sent to the target data center to instruct the target data center to perform GPU virtualization partitioning, elastic adjustment of video memory and memory resources, switching of interconnect communication protocols, and allocation of CPU core resources on the target computing machine.

8. The cross-domain dynamic scheduling method for computing power and power coordination according to claim 1, characterized in that, The scheduling method further includes: The status of each active node is continuously monitored. When any active node meets the preset migration trigger condition, the corresponding active migration process is triggered. The migration trigger condition includes at least one of the following: the power supply reliability index is lower than the first danger threshold, the network quality comprehensive score is lower than the second danger threshold, or the available video memory capacity is lower than the capacity threshold. The specific steps to trigger the corresponding proactive migration process include: An interrupt command is sent to the target node that triggers the backtracking condition, and the target node stops executing new forward computation; if the target task is a pre-training or fine-tuning task, the non-critical optimizer state is discarded, and only the gradient core snapshot composed of the elements whose absolute value is within a preset high proportion in the gradient tensor is retained; if the target task is an online inference task, the context state of the current incomplete inference request is extracted to form an inference state snapshot. The gradient core snapshot or the inference state snapshot is transmitted back to the main scheduling domain with the highest priority via the residual network channel. After confirming receipt of the gradient core snapshot or the inference state snapshot, an acknowledgment signal is sent to the target node. The target node clears the GPU memory and temporary cache of the current training task, resets its own state to standby idle state, marks it as a standby node in the global resource list, and moves it into the next round of candidate queue of the candidate node set.

9. A cross-domain dynamic scheduling system for computing power and power coordination, characterized in that, The method for cross-domain dynamic scheduling of computing power in collaboration with power supply as described in any one of claims 1 to 8 includes: The data acquisition module is configured to: acquire multi-dimensional data, including: the feature vector of the target task parsed from the task platform; the location information and link status information of the computer room acquired through the computer room distance perception mechanism; the hardware configuration and operating status information of each computing machine acquired through the computing machine status perception mechanism; and the power supply status information of each computer room acquired through the power monitoring interface. The power prediction and analysis module is configured to: obtain the real-time total power consumption, renewable power ratio, grid input power fluctuation parameters and energy storage charge status of each computer room based on the power supply status information; predict the future wind and solar power generation of each computer room through the power supply prediction model; and calculate the power supply reliability index of each computer room based on the prediction results and the energy storage charge status. The node selection and topology construction module is configured to: select a set of candidate nodes that meet the power supply and network quality conditions from all available computer rooms based on the power supply reliability index and the link bandwidth, packet loss rate and latency jitter in the link status information; adaptively select the cross-domain communication topology based on the comprehensive network quality score of each node in the candidate node set; and adaptively perform data compression for nodes participating in cross-domain communication based on network and power supply status. The scheduling decision module is configured to: take the minimum transmission latency, the lowest scheduling cost, the highest computing power utilization, and the service level agreement indicators as multiple optimization objectives, combine the feature vector of the target task, the currently enabled cross-domain communication topology, and the status of each node, calculate the load allocation weight of each candidate node based on the multi-dimensional power weighting and deviation correction algorithm, and determine the allocation relationship of the pipeline stage based on the greedy matching strategy. The task allocation and configuration distribution module is configured to: allocate the model pipeline stages to the corresponding candidate nodes according to the proportion of the load allocation weight, and issue configuration instructions to the target data center to instruct the target data center to perform resource scheduling configuration on the target computing machine.

10. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the cross-domain dynamic scheduling method for computing power and power coordination as described in any one of claims 1 to 8.

Images