Distributed computing power scheduling methods and systems

Abstract

This application relates to the field of computing power scheduling technology, and provides a distributed computing power scheduling method and system. By constructing an integrated architecture of data perception, load prediction and intelligent scheduling, it breaks through the limitations of traditional computing power scheduling systems that rely on a single indicator or static strategy, and builds an adaptive and closed-loop optimized intelligent scheduling system. The scheduling system acquires multi-source data in real time and fuses it into a perception vector to accurately characterize the system's operating status, and uses hierarchical load prediction to accurately predict load changes. Combined with dual scheduling paths and execution feedback optimization, it can efficiently allocate resources in different scenarios. This scheduling system effectively improves the resource utilization of distributed systems, reduces energy consumption costs, reduces task execution latency, and enhances system stability and fault tolerance, providing an innovative computing power scheduling solution for large-scale distributed computing scenarios.

Description

Technical Field

[0001] This application relates to the field of computing power scheduling technology, and in particular to distributed computing power scheduling methods and systems. Background Technology

[0002] With the rapid development of technologies such as cloud computing, big data and artificial intelligence, distributed scheduling systems have become the core infrastructure supporting large-scale data processing and complex computing tasks. In the field of distributed computing power scheduling, existing technologies mainly revolve around resource allocation, load balancing and task scheduling. Early distributed scheduling systems mostly adopted static allocation strategies, pre-setting resource allocation rules and directly allocating fixed computing power resources according to task requirements.

[0003] As system scale and task complexity increase, dynamic scheduling algorithms have emerged. For example, queue-based scheduling algorithms schedule tasks based on priority and resource availability, while heuristic algorithms iteratively optimize to find near-optimal solutions. In recent years, machine learning-based scheduling methods have been applied, using historical data to train models that predict load trends to aid resource allocation. Simultaneously, some scheduling systems introduce edge computing nodes, decentralizing some scheduling decisions to the network edge and reducing pressure on central nodes. However, most of these approaches fail to address how to predict load using multi-dimensional data to achieve differentiated computing power scheduling in complex application scenarios. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this application provides a distributed computing power scheduling method and system.

[0005] In the first aspect, this application provides a distributed computing power scheduling system, which includes: real-time acquisition of hardware signals, network packets and task sets, processing by edge nodes to obtain resource status, network quality and task status respectively, and fusing them into a perceptual vector based on a spatiotemporal graph convolutional network;

[0006] Based on the perception vector, a dynamic time window is configured, and hierarchical load forecasting is performed according to the dynamic time window to generate a load forecasting matrix. The load forecasting matrix includes cell identifiers, load forecast values ​​and forecast confidence levels, and the forecast deviation rate is monitored to trigger the re-execution of hierarchical load forecasting.

[0007] The load prediction matrix is ​​extracted and combined with resource status, network quality, and task status to make a comprehensive judgment to execute a dual scheduling mechanism. The dual scheduling mechanism includes a pre-migration path and a temporary scheduling path. When executing the pre-migration path, the migration task and target unit are selected based on the resource value of each unit, and a scheduling instruction is generated. When executing the temporary scheduling path, a scheduling instruction is generated based on a multi-objective optimization algorithm. The scheduling instruction includes the migration task, source unit, target unit, and resource allocation. The execution results of the scheduling instruction are monitored synchronously to adjust the hierarchical load prediction.

[0008] As an optional implementation, the fusion logic of the perception vectors includes:

[0009] Using resource status, network quality, and task status as nodes, the weights in the spatial dimension are determined based on the resource affinity between nodes, and temporal connections are established for the same node to determine its weight in the time dimension, thus forming a spatiotemporal graph.

[0010] Spatial and temporal convolutions are performed on the spatiotemporal graph through a spatiotemporal graph convolutional network to obtain spatial and temporal features, and feature weights for spatial and temporal features are dynamically assigned based on a self-attention mechanism.

[0011] Based on the feature weights, spatial and temporal features are processed through a hierarchical fusion method to obtain the perception vector.

[0012] As an optional implementation, the re-execution logic for the tiered load prediction includes:

[0013] Real-time monitoring of the load prediction value and the actual load prediction deviation rate of each cell in the load prediction matrix; configuration of deviation threshold; comparison of prediction deviation rate and deviation threshold to trigger re-execution of tiered load prediction.

[0014] When the re-execution of tiered load forecasting is triggered, the influencing factors and types of deviations that cause the forecasting bias are determined through feature importance analysis. Based on the scope of the influencing factors, the forecasting level of tiered load forecasting is located, and the re-execution level is determined.

[0015] The adjustment method for the re-execution level is determined based on the deviation type, and the adjustment effect is monitored to determine whether the re-execution of the hierarchical load prediction is triggered again.

[0016] As an optional implementation, the processing sub-logic of the first-level prediction includes extracting and aggregating the perception vectors in the data buffer, predicting the load trend within the corresponding dynamic time window through a spatiotemporal graph neural network, and generating an initial load prediction value.

[0017] The second-level prediction processing logic includes receiving the initial load prediction value, predicting the load trend within the corresponding dynamic time window through the Long Short-Term Memory network based on the resource status of the perception vector in the data buffer, so as to correct the initial load prediction value.

[0018] The third-level prediction processing logic includes receiving the corrected load prediction value, and obtaining the load prediction value through reinforcement learning based on the task state of the perception vector in the data buffer.

[0019] As an optional implementation, the sub-logic for generating the load prediction matrix includes:

[0020] Receive sensing vectors, map the sensing vectors to unit identifiers according to their physical location and functional attributes, and classify and store the sensing vectors in the data cache area according to the attributes of the unit identifiers.

[0021] Extract the perception vector from the data cache to determine historical load volatility, and adjust the length of the time window based on the historical load volatility to configure a dynamic time window;

[0022] Tiered load forecasting is performed based on a dynamic time window. Tiered load forecasting includes first-level forecasting, second-level forecasting, and third-level forecasting to obtain load forecast values.

[0023] The evaluation dimensions are determined to be data integrity, result consistency and prediction stability. The prediction confidence is calculated for the load prediction value of each unit, and the evaluation weights of the evaluation dimensions are dynamically adjusted according to the historical prediction error to correct the prediction confidence. A load prediction matrix including unit identifier, load prediction value and prediction confidence is generated.

[0024] As an optional implementation, the execution logic of the pre-migration path includes:

[0025] The static value of each unit is determined based on hardware performance, energy efficiency, and reliability; the dynamic value of each unit is determined based on load forecast and resource imbalance; and the resource value of each unit is determined by combining the static and dynamic values.

[0026] Configure a value threshold, compare the resource value with the value threshold to filter source units, extract the task set in the source unit, and select migration tasks according to the task urgency.

[0027] Combining the resource value and load prediction of the target unit, the Hungarian algorithm is used to match the migration task with the target unit to obtain the matching result;

[0028] Based on the matching results, scheduling instructions are generated, including migration tasks, source units, target units, and resource allocation. The execution results of the scheduling instructions are executed and monitored. The execution results include task execution progress and data transmission volume. The execution results are compared with the target results to determine the migration success rate. Based on the migration success rate, the hierarchical load forecast is adjusted.

[0029] As an optional implementation, the execution logic of the temporary scheduling path includes:

[0030] The task completion time, resource consumption, and system stability are defined as optimization objectives. The objective weights of the optimization objectives are adjusted according to the task type. Task constraints and resource constraints are established by combining the load prediction matrix and resource status to form a multi-objective optimization function.

[0031] Solving the multi-objective optimization function yields a set of scheduling paths. Based on task urgency and resource imbalance, a temporary scheduling path is selected from the set of scheduling paths to generate scheduling instructions.

[0032] The system executes and monitors the execution results of scheduling instructions, compares the execution results with the target results to determine the achievement rate of the optimization objectives, adjusts the target weights of the multi-objective optimization function based on the achievement rate of the optimization objectives, and assists in adjusting the hierarchical load forecast.

[0033] As an optional implementation, the execution judgment sub-logic of the dual scheduling mechanism includes:

[0034] Extract the prediction confidence of the load prediction matrix, determine the resource imbalance of each unit based on the resource status, determine the network stability of each unit based on the network quality, and determine the task urgency of each unit based on the task status.

[0035] Configure scheduling thresholds and compare them with prediction confidence, resource imbalance, network stability, and task urgency to determine the dual scheduling mechanism executed by each unit.

[0036] Monitor the execution effect of the dual scheduling mechanism to dynamically adjust the scheduling threshold.

[0037] As an optional implementation, the resource status includes resource utilization, memory pressure, and IO wait queue; the network quality includes bandwidth fluctuation, transmission latency, and packet loss rate; and the task status includes computational density and data dependencies.

[0038] Secondly, this application provides a distributed computing power scheduling method, which includes: acquiring hardware signals, network packets and task sets in real time, obtaining resource status, network quality and task status through edge node processing, and fusing them into a perceptual vector based on a spatiotemporal graph convolutional network;

[0039] Based on the perception vector, a dynamic time window is configured, and hierarchical load forecasting is performed according to the dynamic time window to generate a load forecasting matrix. The forecast deviation rate is monitored to trigger the re-execution of hierarchical load forecasting.

[0040] The load prediction matrix is ​​extracted and combined with resource status, network quality and task status to make a comprehensive judgment to execute a dual scheduling mechanism, which includes a pre-migration path and a temporary scheduling path.

[0041] When executing the pre-migration path, the migration task and target unit are selected based on the resource value of each unit, and scheduling instructions are generated. The execution results of the scheduling instructions are monitored to adjust the tiered load forecast.

[0042] When executing temporary scheduling paths, scheduling instructions are generated based on a multi-objective optimization algorithm, and the execution results of the scheduling instructions are monitored to adjust the hierarchical load forecast.

[0043] Compared with existing technologies, the beneficial effects of this application are as follows: By constructing an integrated architecture of data perception, load prediction, and intelligent scheduling, it breaks through the limitations of traditional computing power scheduling systems that rely on single indicators or static strategies, and constructs an adaptive and closed-loop optimized intelligent scheduling system. The scheduling system acquires multi-source data in real time and fuses it into a perception vector to accurately characterize the system's operating status. It also uses hierarchical load prediction to accurately predict load changes. Combined with dual scheduling paths and execution feedback optimization, it can efficiently allocate resources in different scenarios. This scheduling system effectively improves the resource utilization of distributed systems, reduces energy consumption costs, reduces task execution latency, and enhances system stability and fault tolerance, providing an innovative computing power scheduling solution for large-scale distributed computing scenarios.

[0044] By acquiring hardware signals, network packets, and task sets in real time, the system solves the problems of incomplete data acquisition and poor real-time performance in traditional systems. After processing by edge nodes to obtain resource status, network quality, and task status, these are fused into a perception vector based on a spatiotemporal graph convolutional network. This fully explores the spatiotemporal correlations between data, transforming scattered heterogeneous data into structured and computable feature representations. This not only provides accurate basic data for subsequent load prediction but also enables rapid response to changes in system status. Compared to traditional systems that rely on only a single data dimension, this system can improve the accuracy of resource status perception and provide a more reliable basis for computing power scheduling.

[0045] Based on the dynamic time window configuration of the perception vector, it can adapt to the fluctuation characteristics of system load and avoid the prediction deviation caused by the fixed window. The hierarchical load prediction combines spatiotemporal graph neural network, long short-term memory network and reinforcement learning. Compared with the single prediction model, the prediction accuracy is significantly improved. The prediction deviation rate is monitored and the re-execution mechanism is triggered to realize the targeted optimization of hierarchical load prediction. It can quickly adapt to system changes, reduce the accumulation of prediction errors, and the final generated load prediction matrix provides a risk assessment basis for scheduling decisions, helping the scheduling system to make reasonable scheduling decisions in different scenarios.

[0046] By extracting the load prediction matrix and combining it with multi-dimensional information, a dual scheduling mechanism is implemented. The pre-migration path performs task migration based on the resource value of the unit, which can optimize resource distribution in advance, reduce the risk of resource imbalance during the system load stabilization period, and improve the long-term resource utilization. The temporary scheduling path is based on a multi-objective optimization algorithm to deal with sudden loads, which can quickly adjust the computing power allocation in emergency situations to ensure the execution of critical tasks. The dynamic selection of the dual scheduling path and the execution result feedback mechanism realize the closed-loop optimization of the scheduling strategy. The scheduling threshold and hierarchical load prediction are dynamically adjusted according to the real-time status of the system and the execution effect, which enables the scheduling system to have self-learning and self-adaptive capabilities, significantly enhancing the flexibility and reliability of the distributed system. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0048] Figure 1 This is a system flowchart of the distributed computing power scheduling system provided in the embodiments of this application;

[0049] Figure 2 This is a sub-logic diagram for generating the load prediction matrix of the distributed computing power scheduling system provided in the embodiments of this application;

[0050] Figure 3 The execution logic diagram of the pre-migration path of the distributed computing power scheduling system provided in the embodiments of this application;

[0051] Figure 4 This is a flowchart illustrating the distributed computing power scheduling method provided in an embodiment of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of this application more apparent and understandable, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0053] Example 1

[0054] like Figure 1 The diagram shows a system flowchart of a distributed computing power scheduling system provided in this application embodiment. The system includes a feature perception module, a load prediction module, and a computing power scheduling module.

[0055] The feature perception module is used to acquire hardware signals, network packets and task sets in real time. It obtains resource status, network quality and task status through edge node processing, and fuses them into perception vectors based on spatiotemporal graph convolutional networks.

[0056] Hardware signals from the CPU and GPU are obtained through hardware performance counters. Network packets are captured by deploying eBPF probes. Task sets are obtained and timestamps are added to the hardware signals, network packets, and task sets. Then, the hardware signals, network packets, and task sets are aligned using a sliding window algorithm to ensure synchronized data acquisition and transmission to edge nodes.

[0057] Distributed system computing power scheduling requires real-time monitoring of hardware resource status. Traditional polling methods cannot meet the accuracy requirements. It is necessary to obtain micro-architecture level data through the underlying hardware interface and deploy hardware performance counters on edge nodes to obtain hardware signals from the CPU and GPU, directly obtaining underlying hardware data, avoiding middleware parsing delays, and providing real-time basis for resource status extraction. At the same time, traditional network packet capture tools have high performance overhead and are difficult to adapt to high-throughput scenarios. Therefore, eBPF probes are deployed to capture network packets. Simultaneously, task sets are obtained, and timestamps are added to hardware signals, network packets, and task sets to ensure data time consistency. This enables the acquisition of multi-source data, avoiding the delays and data loss caused by traditional polling methods. The synchronously acquired data provides the basis for subsequent sliding window alignment, ensuring the correlation of data in the time dimension.

[0058] Different types of data have different acquisition frequencies and transmission delays, requiring alignment to eliminate time discrepancies. The window size is dynamically adjusted based on data type and system load. A timestamp-based sliding window algorithm merges hardware signals, network packets, and task sets within the same window into multi-source data. Data exceeding the window range is pruned or padded; preferably, nearest neighbor interpolation is used to fill missing values. The hash value of the multi-source data is calculated to verify data integrity; if verification fails, data is re-requested. This ensures strict alignment of multi-source data in the time dimension, improving the accuracy of subsequent feature extraction. The aligned data is then transmitted to edge nodes for caching and processing, providing standardized input for relevant feature extraction.

[0059] Resource status includes resource utilization, memory pressure, and I / O wait queue; network quality includes bandwidth volatility, transmission latency, and packet loss rate; and task status includes computational density and data dependencies.

[0060] The raw multi-source data needs to be transformed into quantifiable feature indicators. Data cached in edge nodes is obtained, and by processing this data at the edge nodes, resource status, network quality, and task status are derived. Resource status includes resource utilization, memory pressure, and I / O wait queue. Resource utilization is calculated as the ratio of current usage to total resources, directly reflecting hardware resource consumption. Memory pressure is calculated as (used memory - cache) / total memory × 100%, eliminating cache interference in the evaluation. The I / O wait queue represents the number of processes currently waiting for I / O operations, measuring storage performance bottlenecks. Network quality includes bandwidth fluctuation rate, ... Transmission latency and packet loss rate are used to determine network traffic stability. Bandwidth fluctuation rate = (current bandwidth - historical average bandwidth) / historical average bandwidth × 100%. Transmission latency is calculated as the difference between response time and request time to evaluate network transmission efficiency. Packet loss rate is determined by the ratio of lost data packets to total data packets sent to locate network fault points. Task status includes computation density and data dependency. Computation density is the ratio of CPU cycles to instructions to distinguish the computational intensity of a task. Data dependency is the product of the number of dependent tasks and the average data transfer volume to quantify task coupling.

[0061] The original data contains missing values ​​and outliers, requiring processing to improve data quality. The extracted resource status, network quality, and task status are processed for missing values, outlier correction, and data standardization. For a small number of missing data points, linear interpolation is used, while for a large number of missing data points, they are marked as invalid and re-acquired. Minimum-maximum normalization is used to map feature values ​​from different ranges to the [0,1] interval, eliminating dimensional differences. This improves data reliability and consistency, avoiding interference from outliers in subsequent analysis. The cleaned and standardized data serves as input to the spatiotemporal graph convolutional network, enhancing the stability and accuracy of network training.

[0062] Specifically, the fusion logic of the perceptual vectors includes:

[0063] Using resource status, network quality, and task status as nodes, the weights in the spatial dimension are determined based on the resource affinity between nodes, and temporal connections are established for the same node to determine its weight in the time dimension, thus forming a spatiotemporal graph.

[0064] Spatial and temporal convolutions are performed on the spatiotemporal graph through a spatiotemporal graph convolutional network to obtain spatial and temporal features, and feature weights for spatial and temporal features are dynamically assigned based on a self-attention mechanism.

[0065] Based on the feature weights, spatial and temporal features are processed through a hierarchical fusion method to obtain the perception vector.

[0066] In distributed systems, hardware signals, network quality, and task status exhibit complex spatiotemporal relationships. Structured modeling is necessary to transform discrete features into a computable graph structure, providing a foundation for subsequent feature extraction. Each feature value of resource status, network quality, and task status is treated as a node, and edge weights are determined through a bidirectional evaluation mechanism. Firstly, resource affinity between nodes is calculated based on historical data; for example, if long-term monitoring reveals stable network bandwidth requirements for a certain type of task, the weight of the edge between the corresponding task status node and network quality node is fixed. Secondly, edge weights are dynamically adjusted based on real-time data. If network congestion occurs, the weight of the edge between affected nodes is immediately reduced, highlighting the changes in feature relationships under emergency conditions. Simultaneously, a multi-layered time edge structure is constructed. Short-term time edges connect nodes at adjacent time points to capture immediate changes; medium-term time edges connect nodes at intervals to reflect trend changes; and long-term time edges connect key time nodes to grasp the overall trend. Preferably, short-term time edges are 1-5 minutes, medium-term time edges are 10-30 minutes, and long-term time edges are 1-2 hours. The connection strategy for each layer of time edges is dynamically adjusted according to the degree of data fluctuation; when data fluctuations are severe, the weight of short-term time edges is increased.

[0067] The graph pruning technique periodically removes edges with low correlation and redundant nodes. If the weight of an edge between a node and other nodes is consistently less than a weight threshold, the node and its related edges are removed from the spatiotemporal graph, reducing computational complexity. The constructed spatiotemporal graph can more accurately and dynamically reflect the spatiotemporal correlation of the system state. Compared with a simple graph structure, it can capture more hidden relationships, improving the adaptability of the spatiotemporal graph convolutional network to complex scenarios. The high-quality spatiotemporal graph provides excellent input for the spatiotemporal graph convolutional network, making subsequent feature extraction more targeted and accurate, and reducing invalid computation.

[0068] The original features in the spatiotemporal graph need to be further extracted and fused using deep learning algorithms to uncover the deeper value of the data and generate more representative feature vectors. Multiple sets of spatial convolutional layers with different kernel sizes are designed. Small convolutional kernels focus on local features of nodes and capture the close relationship between adjacent nodes, while large convolutional kernels focus on global features of nodes and analyze feature collaborations in a wider range. The results of multi-scale convolution are computed in parallel and then integrated using a weighted fusion method to obtain spatial features containing different granularities. In the temporal convolution stage, a gating mechanism is used to selectively memorize and update time series information. For sudden system anomalies, such as sudden network packet loss, changes can be quickly captured and the expression of relevant features can be enhanced to obtain temporal features, thus avoiding the smoothing of important time series information.

[0069] The self-attention mechanism includes content-based attention and location-based attention. Content-based attention assigns feature weights based on the similarity of node features. If two task state nodes have similar computational densities and close data dependencies, their weights in feature fusion are increased. Location-based attention considers the topological position of nodes in the spatiotemporal graph, assigning higher weights to nodes closer to core nodes or critical paths to highlight their key impact on the system state. Then, through a hierarchical fusion approach, different types of features at the same time point are first fused in the spatial dimension to obtain preliminary fused features. Then, the preliminary fused features are fused again in the temporal dimension to finally generate a perception vector containing comprehensive spatiotemporal information. During the fusion process, the feature weights of spatial and temporal features are dynamically adjusted according to different application scenarios. Through multi-scale convolution, enhanced temporal processing, and self-attention mechanisms, the comprehensiveness and accuracy of feature extraction are significantly improved. The generated perception vector can better reflect the essence of the system state, providing strong support for subsequent decision-making and directly determining the prediction accuracy of the load prediction module and the decision quality of the computing power scheduling module.

[0070] The load forecasting module is used to configure a dynamic time window based on the perception vector, perform hierarchical load forecasting according to the dynamic time window to generate a load forecasting matrix, which includes cell identifiers, load forecast values ​​and forecast confidence levels, and monitors the forecast deviation rate to trigger the re-execution of hierarchical load forecasting.

[0071] Furthermore, the processing sub-logic of the first-level prediction includes extracting and aggregating the perception vectors in the data cache, predicting the load trend within the corresponding dynamic time window through a spatiotemporal graph neural network, and generating an initial load prediction value.

[0072] During the operation of the distributed system, the feature perception module continuously acquires and processes hardware signals, network packets and task sets. The generated perception vectors are distributed and stored in various edge nodes. The first-level prediction first needs to aggregate these perception vectors distributed in different physical locations and functional domains. The scheduling system will construct a two-layer aggregation tree structure based on physical location and functional attributes. The upper-layer nodes dynamically allocate weights according to the historical load volatility of the lower-layer nodes and calculate the weighted average.

[0073] After data aggregation is completed, the scheduling system maps each aggregated perception vector to a graph node through a spatiotemporal graph neural network. The weights of the edges between nodes are calculated in real time based on the historical intensity of collaborative changes. That is, the weights of the edges between nodes that frequently experience load peaks are increased. To capture system state changes at different scales, parallel multi-scale convolutional kernels are designed. Small-sized convolutional kernels focus on capturing local fluctuation features, while large-sized convolutional kernels analyze global trends. At the same time, based on the spatiotemporal attention mechanism, key nodes and time points are automatically focused. That is, during peak business periods, the feature extraction weights of core computing nodes are enhanced, thereby generating initial load prediction values.

[0074] Furthermore, the second-level prediction processing logic includes receiving the initial load prediction value, predicting the load trend within the corresponding dynamic time window through a long short-term memory network based on the resource status of the perception vector in the data buffer, so as to correct the initial load prediction value.

[0075] After the first-level prediction generates the initial load prediction value, the second-level prediction first performs in-depth mining of resource status. The scheduling system will use the random forest algorithm to evaluate the impact of resource indicators such as resource utilization, memory pressure and IO wait queue on load changes, and construct feature importance ranking. For key resource indicators, not only are the original values ​​extracted, but also lag features and differential features are constructed. Preferably, the resource utilization sequence of the past 5 minutes and the change in adjacent time points are extracted. In order to capture the synergistic effect between resources, resource indicators with similar change patterns are clustered to form feature groups such as storage-related and computing-related features.

[0076] After feature extraction, temporal prediction correction is performed through a multi-branch long short-term memory network. Different branches focus on processing different types of resource features, i.e., independent GPU branches handle deep learning task loads. Each long short-term memory network unit uses an attention gating mechanism to quickly forget outdated historical patterns when a change in system architecture or a switch in workload type is detected. In order to retain the effective information in the initial prediction, a residual correction strategy is adopted, which uses the output of the long short-term memory network as a correction amount to be superimposed on the initial load prediction value to correct the initial load prediction value.

[0077] Furthermore, the third-level prediction processing logic includes receiving the corrected load prediction value and obtaining the load prediction value through reinforcement learning based on the task state of the perception vector in the data buffer.

[0078] Based on the obtained corrected load prediction values, the third-level prediction transforms the task state characteristics into a computable vector representation. The scheduling system first constructs a task dependency graph, where nodes represent task instances and edges represent data flow and dependency strength. The graph structure is then transformed into a low-dimensional vector through a graph embedding algorithm. At the same time, dynamic indicators such as task execution progress and resource consumption rate are extracted to form a complete task state representation.

[0079] The state space of reinforcement learning is the task state, and the action space is the load prediction and adjustment. A multi-dimensional reward function is designed that includes prediction accuracy, resource balance and response speed. A hierarchical policy network is used to achieve long-term policy learning and short-term rapid adjustment. The upper network learns a general prediction policy based on historical data, and the lower network dynamically adjusts local prediction parameters based on real-time feedback. The two network layers work together to optimize through a parameter sharing mechanism to improve the response capability of reinforcement learning to sudden load changes, thereby obtaining the load prediction value.

[0080] Furthermore, such as Figure 2 As shown, the sub-logic for generating the load prediction matrix includes:

[0081] Receive sensing vectors, map the sensing vectors to unit identifiers according to their physical location and functional attributes, and classify and store the sensing vectors in the data cache area according to the attributes of the unit identifiers.

[0082] Extract the perception vector from the data cache to determine historical load volatility, and adjust the length of the time window based on the historical load volatility to configure a dynamic time window;

[0083] Tiered load forecasting is performed based on a dynamic time window. Tiered load forecasting includes first-level forecasting, second-level forecasting, and third-level forecasting to obtain load forecast values.

[0084] The evaluation dimensions are determined to be data integrity, result consistency and prediction stability. The prediction confidence is calculated for the load prediction value of each unit, and the evaluation weights of the evaluation dimensions are dynamically adjusted according to the historical prediction error to correct the prediction confidence. A load prediction matrix including unit identifier, load prediction value and prediction confidence is generated.

[0085] The raw sensing vectors are scattered and lack structured identifiers, making them difficult to use directly for analysis. A unified management system needs to be established through mapping and classification. The scheduling system constructs a two-layer mapping mechanism. Physical location mapping uses a geo-hashing algorithm to encode the physical coordinates of data centers, racks, and nodes as unique identifiers. Functional attribute mapping generates functional labels based on attributes such as task type and hardware type. Task types include AI computing and data storage, and hardware types include CPU clusters and GPU arrays. The two types of identifiers are combined to form a composite unit identifier, in the form of "DC01-Rack05-CPU-DeepLearning". When storing, a content-based caching strategy is adopted, storing frequently accessed sensing vectors in the high-speed cache of edge nodes, while storing low-frequency data in a distributed file system. At the same time, a multi-level index structure is established. The main index is classified by unit identifier, and the secondary index records metadata such as data update time and associated task ID. This achieves efficient organization and rapid location of sensing vectors, reduces data management complexity, improves subsequent analysis efficiency, and provides a complete data source for historical load volatility calculation, ensuring the accuracy of time window configuration.

[0086] The scheduling system experiences significant variations in load frequency, making it impossible to balance prediction accuracy and computational resource consumption with a fixed time window. Dynamic adaptation is necessary. This approach utilizes multi-scale volatility analysis, simultaneously calculating historical load volatility at 1-minute, 5-minute, and 15-minute granularities. A sliding window is used to calculate the standard deviation of historical load at each time granularity, yielding the historical load volatility. A trend factor is determined based on the deviation of the current load volatility from the mean of historical load volatility. The time window length is adjusted by combining historical load volatility and the trend factor. Preferably, if the 15-minute historical load volatility exceeds a volatility threshold and the trend factor is positive, it is considered a sudden load event, and the time window is shortened to 1 minute. If the 15-minute historical load volatility remains below the volatility threshold, the window is extended to 30 minutes. Furthermore, the weight of volatility at each time granularity can be dynamically adjusted according to different load scenarios to ensure the time window length aligns with actual needs. This approach accurately matches the scheduling system's load change rhythm, improving prediction timeliness in high-volatility scenarios and reducing computational overhead in low-volatility scenarios. The optimized time window provides adapted data slices for tiered load prediction, enhancing the performance of each level of prediction.

[0087] Single predictions are insufficient to take into account the overall trend of the system, resource dynamics, and task characteristics. Therefore, a layered and progressive approach is needed to improve prediction accuracy. In the first-level prediction, a dynamic spatiotemporal graph neural network is constructed. The weights of the edges between nodes are dynamically updated based on real-time resource affinity. When two nodes frequently collaborate to process the same task, the weights of their edges are automatically enhanced. In addition, features at different scales are extracted in parallel through a multi-head attention mechanism. Small convolutional kernels capture local node changes, while large convolutional kernels analyze the global load trend and output the initial load prediction value.

[0088] In the second-level prediction, potential correlations are mined through feature cross-analysis of resource status, that is, CPU utilization and memory pressure are jointly analyzed to identify resource bottlenecks. A long short-term memory network combined with a gating mechanism is used to selectively memorize historical resource status and correct the initial load prediction value. In the third-level prediction, task dependencies are modeled as a task dependency graph. When a new task is detected, the task priority and resource demand prediction are quickly updated through reinforcement learning, and the final load prediction value is output. The three levels of prediction work together to refine the prediction from global to local, effectively capturing the complex dynamic changes of the scheduling system. The high-precision load prediction value provides a reliable basis for the calculation of prediction confidence.

[0089] Single load forecasts lack reliability assessment, requiring multi-dimensional quantification of forecast confidence and dynamic optimization of the assessment system based on actual performance. Data integrity is assessed by calculating the missing rate of the perception vector and the proportion of outliers, while consistency is compared with the differences in the three-level forecast results. For forecast stability, the fluctuation range of historical forecasts is analyzed. An error tracing mechanism is established: if historical errors are mainly caused by missing data, the assessment weight of data integrity is automatically increased; if errors are caused by fluctuations in the hierarchical forecasts, the weight of forecast stability is increased. The combination of assessment weights is iteratively optimized through a genetic algorithm to minimize historical forecast errors.

[0090] For low-confidence predictions, a hybrid correction strategy is triggered. If data is missing, the feature perception module is automatically requested to collect supplementary data. If the deviation in the hierarchical prediction is due to errors, the hierarchical load prediction re-execution process is invoked. Finally, a three-dimensional matrix including unit identifier, load prediction value, and prediction confidence is generated. This quantifies the reliability of the prediction, helps the computing power scheduling module identify high-risk predictions, and continuously improves the prediction quality based on the dynamic optimization evaluation system. The load prediction matrix provides the computing power scheduling module with comprehensive decision-making basis and supports the accurate execution of the dual scheduling mechanism.

[0091] Specifically, the re-execution logic for tiered load forecasting includes:

[0092] Real-time monitoring of the load prediction value and the actual load prediction deviation rate of each cell in the load prediction matrix; configuration of deviation threshold; comparison of prediction deviation rate and deviation threshold to trigger re-execution of tiered load prediction.

[0093] When the re-execution of tiered load forecasting is triggered, the influencing factors and types of deviations that cause the forecasting bias are determined through feature importance analysis. Based on the scope of the influencing factors, the forecasting level of tiered load forecasting is located, and the re-execution level is determined.

[0094] The adjustment method for the re-execution level is determined based on the deviation type, and the adjustment effect is monitored to determine whether the re-execution of the hierarchical load prediction is triggered again.

[0095] Real-time monitoring of prediction deviations prevents erroneous predictions from impacting computing power scheduling decisions. Dynamic monitoring using multiple thresholds is employed, setting emergency, warning, and alert thresholds. Preferably, the emergency threshold is a prediction deviation rate greater than 30%, the warning threshold is a prediction deviation rate within the range of (15%, 30%), and the alert threshold is a prediction deviation rate within the range of [5%, 15%). These thresholds are dynamically adjusted based on the scheduling system's load characteristics; that is, thresholds are relaxed during high-load periods to avoid frequent false triggers, while thresholds are tightened during low-load periods to increase sensitivity. When the prediction deviation rate exceeds the emergency threshold, a re-execution of the tiered load prediction is triggered. This accurately identifies actual prediction failure scenarios, balances the timeliness of correction with system stability, and promptly triggers re-execution requests to initiate deviation location and correction processes, preventing error accumulation.

[0096] Different types of biases require targeted correction, and accurately locating the source of the problem can avoid blind adjustments. By analyzing SHAP values, the contribution of each input feature to the prediction bias is quantified. Based on the contribution to the prediction bias, the influencing factors and bias types that cause the bias are determined. If the SHAP value shows that network latency has the highest contribution, it is judged as a network-type bias, and a mapping relationship between bias type and prediction level is established. Resource-type biases correspond to the second level of prediction correction, network-type biases trigger the first level of spatiotemporal graph parameter adjustment, and task-type biases initiate the third level of reinforcement learning strategy optimization. At the same time, based on the cross-validation mechanism, the accuracy of the positioning results is verified by changing key feature values ​​to avoid misjudgment by a single method. This achieves accurate positioning and hierarchical classification of biases, narrows the correction range, improves correction efficiency, and provides clear correction levels to guide subsequent targeted adjustments, reducing invalid calculations.

[0097] To ensure the effectiveness of the correction strategy, a closed-loop optimization is formed, avoiding over-adjustment or under-correction. For network-related biases, the weights of edges in the spatiotemporal graph are recalculated to enhance the connection strength of relevant network nodes, and the structure of the spatiotemporal graph is reconstructed based on the TCP retransmission rate. For resource-related biases, different resource allocation strategies are simulated to evaluate the factors affecting prediction errors, and the network parameters of the Long Short-Term Memory network are automatically adjusted. When the bias is task-related, the reward function of reinforcement learning is updated to increase sensitivity to changes in task priority, and historical experience from similar tasks is transferred to the current prediction reinforcement learning network through transfer learning. The effect of each adjustment is monitored; if the error does not decrease or even increases after adjustment, a rollback mechanism is triggered to restore the system to its pre-adjustment state, and other correction strategies are tried. By continuously monitoring changes in prediction error and prediction confidence, it is determined whether the re-execution process needs to be restarted. This achieves robust optimization of hierarchical load prediction, improving accuracy while ensuring system stability, forming a closed loop of continuous improvement. The optimized hierarchical load prediction generates a more accurate load prediction matrix, supporting the computing power scheduling module to make more reliable decisions.

[0098] The computing power scheduling module is used to extract the load prediction matrix and combine it with resource status, network quality and task status to make a comprehensive judgment to execute a dual scheduling mechanism. The dual scheduling mechanism includes a pre-migration path and a temporary scheduling path. When executing the pre-migration path, the migration task and target unit are selected based on the resource value of each unit, and a scheduling instruction is generated. When executing the temporary scheduling path, a scheduling instruction is generated based on a multi-objective optimization algorithm. The scheduling instruction includes the migration task, source unit, target unit and resource allocation. The execution results of the scheduling instruction are monitored synchronously to adjust the hierarchical load prediction.

[0099] Furthermore, the execution judgment sub-logic of the dual scheduling mechanism includes:

[0100] Extract the prediction confidence of the load prediction matrix, determine the resource imbalance of each unit based on the resource status, determine the network stability of each unit based on the network quality, and determine the task urgency of each unit based on the task status.

[0101] Configure scheduling thresholds and compare them with prediction confidence, resource imbalance, network stability and task urgency to determine the dual scheduling mechanism executed by each unit. The dual scheduling mechanism includes pre-migration path and temporary scheduling path.

[0102] Monitor the execution effect of the dual scheduling mechanism to dynamically adjust the scheduling threshold.

[0103] Distributed system scheduling decisions require comprehensive multi-dimensional information. A single indicator cannot fully reflect the system status, necessitating the construction of a quantitative evaluation system to provide a basis for scheduling path selection. The scheduling system scans the load prediction matrix in real time, extracting the prediction confidence of each unit. For resource imbalance calculation, CPU utilization, memory pressure, and IO wait queue are used as the basis, and resource imbalance is assessed by combining standard deviation and coefficient of variation. When the standard deviation of a unit's resource utilization is consistently higher than that of other units, and memory pressure and IO wait queue values ​​fluctuate drastically, the unit is considered to have high resource imbalance. In terms of network stability assessment, indicators such as bandwidth volatility, transmission latency, and packet loss rate are mapped to a 0-1 range, and network stability is generated through weighted fusion. Task urgency is determined based on the computational density and data dependencies in the task status, combined with task priority and deadline. Tasks with high real-time requirements and strong data dependencies automatically receive higher urgency scores. This establishes a comprehensive and quantitative system status evaluation system, avoiding the one-sidedness of scheduling decisions, and providing standardized multi-dimensional indicators with accurate input for scheduling threshold configuration and path selection.

[0104] Fixed scheduling thresholds are insufficient to adapt to dynamic system changes and need to be flexibly adjusted based on system operating conditions to ensure that scheduling decisions are neither excessively frequent nor fail to respond promptly to demands. The scheduling system employs an adaptive threshold adjustment strategy. The prediction confidence threshold is dynamically adjusted based on historical prediction errors; if recent low-confidence predictions trigger multiple erroneous schedulings, the prediction confidence threshold will be increased. The resource imbalance threshold is correlated with the system load level; it is relaxed during high-load periods and tightened during low-load periods. Network stability thresholds are set with multiple warning lines, including a first threshold and a second threshold. When network stability falls below the first threshold, a temporary scheduling path is triggered. When network stability is between the first and second thresholds, other indicators need to be considered for comprehensive judgment. The threshold for task urgency changes dynamically based on the proportion of urgent tasks in the task queue. Preferably, the proportion of urgent tasks is greater than 60%, which lowers the trigger threshold for temporary scheduling. During the comparison process, boundary cases are handled by a fuzzy decision algorithm. When the prediction confidence is slightly higher than the prediction confidence threshold, but both resource imbalance and task urgency are at a high level, a temporary scheduling path is still triggered. The configuration of dynamic scheduling thresholds improves the adaptability and accuracy of scheduling decisions. Fuzzy decision-making solves the path selection problem in complex scenarios, and clear path selection results drive the generation and execution of subsequent scheduling instructions.

[0105] To ensure the continued effectiveness of the scheduling strategy, feedback evaluation of the execution results is necessary to form a closed-loop optimization mechanism. The scheduling system establishes a multi-dimensional effect monitoring system. After the pre-migration path is executed, indicators such as resource imbalance, task completion time, and network latency are compared before and after the migration. If the improvement in resource imbalance does not meet expectations, the trigger threshold for resource imbalance is lowered. After the temporary scheduling path is executed, the task completion rate and system stability fluctuations are monitored. If the completion rate of urgent tasks is less than the completion threshold, the judgment criteria for task urgency are adjusted. The threshold combination is automatically optimized using a reinforcement learning algorithm with historical scheduling results as a reward signal. If the scheduling success rate under a certain threshold combination continuously improves, the weight of that combination is increased; otherwise, it is adjusted. Through real-time feedback and intelligent learning, the scheduling strategy is continuously optimized to improve the system's adaptability and overall performance. The optimized threshold parameters provide a more accurate judgment benchmark for subsequent scheduling decisions.

[0106] Specifically, such as Figure 3 As shown, the execution logic of the pre-migration path includes:

[0107] The static value of each unit is determined based on hardware performance, energy efficiency, and reliability; the dynamic value of each unit is determined based on load forecast and resource imbalance; and the resource value of each unit is determined by combining the static and dynamic values.

[0108] Configure a value threshold, compare the resource value with the value threshold to filter source units, extract the task set in the source unit, and select migration tasks according to the task urgency.

[0109] Combining the resource value and load prediction of the target unit, the Hungarian algorithm is used to match the migration task with the target unit to obtain the matching result;

[0110] Based on the matching results, scheduling instructions are generated, including migration tasks, source units, target units, and resource allocation. The execution results of the scheduling instructions are executed and monitored. The execution results include task execution progress and data transmission volume. The execution results are compared with the target results to determine the migration success rate. Based on the migration success rate, the hierarchical load forecast is adjusted.

[0111] Traditional static resource assessment cannot reflect real-time load changes. A comprehensive evaluation combining static attributes and dynamic status is necessary to provide a scientific basis for task migration. Static value is determined based on hardware performance, energy efficiency, and reliability. Hardware performance is assessed using parameters such as CPU core count, clock speed, and memory bandwidth. Energy efficiency is quantified by the energy consumption ratio per unit of computing power. Reliability is evaluated using historical hardware failure records and health monitoring data. Dynamic value is determined based on load prediction and resource imbalance. Units with higher load predictions and more severe resource imbalances have lower dynamic values. Preferably, if a unit's load prediction consistently exceeds 80% and its resource imbalance is high, its dynamic value decreases significantly. The final resource value is calculated using a weighted fusion method. This comprehensive evaluation system, integrating static and dynamic factors, more accurately reflects the true resource value of units, ensuring the rationality of migration decisions. Resource value ranking provides quantitative support for source unit selection and task selection.

[0112] Prioritize migrating suitable tasks from low-value units to balance migration benefits and costs, avoiding excessive migration that could impact system stability. The scheduling system dynamically configures value thresholds based on historical resource value distribution. Preferably, the bottom 20% of units by value are selected as candidate source units. During the selection process, the current load of the unit is considered, prioritizing source units with a load higher than 75%. After extracting the task sets of the source units, a hierarchical selection strategy is used to first exclude tasks with high urgency to ensure critical business continuity. Secondly, task groups with strong data dependencies are prioritized for overall migration. Finally, tasks with high migration cost-effectiveness are selected based on task computation density and resource consumption. In other words, computationally intensive and non-urgent tasks are prioritized for inclusion in the migration list. This intelligent selection of source units and migration tasks reduces migration costs, improves resource optimization efficiency, and minimizes the impact on core business.

[0113] A single objective cannot meet the needs of complex scenarios, necessitating the introduction of multi-dimensional evaluation metrics to optimize the matching process. A comprehensive scoring function is constructed, incorporating the resource value and load prediction of the target unit. Target units with high resource value and low predicted load receive higher matching scores. The Hungarian algorithm, based on the Pareto optimality concept, allows for the balancing of different metrics during the matching process, prioritizing target units with significantly higher resource value between load prediction and resource value. Furthermore, for task groups with strong data dependencies, matching constraints on the same or adjacent units are added to reduce cross-node data transmission overhead. This multi-objective optimization matching strategy enhances the overall efficiency of the migration scheme, reduces the negative impact of migration on system performance, and provides detailed solutions for generating scheduling instructions based on accurate matching results.

[0114] To ensure a controllable migration process, the prediction and scheduling strategies are optimized through feedback of execution results, forming a closed-loop optimization. Based on the matching results, the scheduling system generates detailed scheduling instructions, including the migration task sequence, source unit, target unit, and resource allocation. During execution, the execution results of the scheduling instructions are monitored in real time, i.e., task execution progress and data transmission volume are monitored. If task execution delays exceed expectations or data transmission volume suddenly increases, dynamic adjustments are triggered, i.e., some non-critical task migrations are suspended to prioritize urgent tasks. After the migration is completed, the migration success rate is calculated by comparing the task completion time and resource balance with the target results. If the migration success rate is lower than the set success standard, the reasons are analyzed and feedback is given to the hierarchical load forecasting, triggering adjustments to the hierarchical load forecasting and optimizing subsequent scheduling strategies. Through the closed-loop execution and feedback mechanism, the effectiveness of the migration is ensured, the coordination between prediction and scheduling is improved, the overall system performance is optimized, and the optimized hierarchical load forecasting and scheduling strategies improve the accuracy and efficiency of future resource scheduling.

[0115] Specifically, the execution logic of the temporary scheduling path includes:

[0116] The task completion time, resource consumption, and system stability are defined as optimization objectives. The objective weights of the optimization objectives are adjusted according to the task type. Task constraints and resource constraints are established by combining the load prediction matrix and resource status to form a multi-objective optimization function.

[0117] Solving the multi-objective optimization function yields a set of scheduling paths. Based on task urgency and resource imbalance, a temporary scheduling path is selected from the set of scheduling paths to generate scheduling instructions.

[0118] The system executes and monitors the execution results of scheduling instructions, compares the execution results with the target results to determine the achievement rate of the optimization objectives, adjusts the target weights of the multi-objective optimization function based on the achievement rate of the optimization objectives, and assists in adjusting the hierarchical load forecast.

[0119] In emergency scenarios, multiple objectives need to be balanced. Traditional single-objective optimization cannot meet complex needs, necessitating the construction of a flexible multi-objective optimization system. The scheduling system presets three optimization objectives: task completion time, resource consumption, and system stability. It dynamically adjusts the weights for different task types. When establishing constraints, task constraints include task deadlines, data dependencies, and lower limits on resource requirements. Resource constraints are based on load prediction matrices and real-time resource status, setting upper limits on CPU, memory, and network bandwidth usage for target units. A multi-objective optimization function is constructed through a weighted summation method. A penalty function is introduced to handle constraint violations, i.e., when a task exceeds its deadline or resources are over-allocated, the objective function value is significantly reduced. The flexible multi-objective optimization function adapts to different scenario requirements, and the constraints ensure the feasibility of the scheduling scheme and improve resource utilization efficiency. The multi-objective optimization function provides a mathematical model basis for the generation of temporary scheduling paths.

[0120] In emergency scenarios, high-quality scheduling solutions need to be generated rapidly. Traditional precise algorithms are too time-consuming, so heuristic algorithms and intelligent selection strategies need to be combined. A non-dominated sorting genetic algorithm is used to solve the multi-objective optimization function. Through fast non-dominated sorting and congestion calculation, a Pareto optimal solution set, i.e., a set of scheduling paths, is generated. When selecting temporary scheduling paths, a decision matrix is ​​constructed, using task urgency, resource imbalance, and path comprehensive score as decision indicators. For urgent tasks, the path with the shortest completion time is prioritized. When the resource imbalance of the scheduling system is severe, the path that can balance resources to the greatest extent is selected. Simultaneously, the indicator weights are adjusted based on historical scheduling results. If the success rate of prioritizing paths based on task completion time is high for a certain type of task, the weight of that indicator is increased. Finally, scheduling instructions are generated based on the decision results. By rapidly generating multi-objective optimized temporary scheduling paths, the intelligent selection strategy ensures that the solution fits the actual needs, improves emergency response capabilities, and the generated scheduling instructions directly guide resource scheduling execution.

[0121] To continuously optimize the scheduling strategy, feedback evaluation of execution results is necessary to dynamically adjust the optimization function and hierarchical load prediction. During the execution of scheduling instructions, task execution progress, resource usage, and system stability indicators are acquired in real time. After execution, the target results such as task completion time, total resource consumption, and system stability are compared to calculate the achievement rate of each optimization objective. If the achievement rate of the task completion time target is low, the target weight of that optimization objective is increased. If resource consumption does not meet expectations, resource constraint parameters are adjusted, and the execution results are fed back to the load prediction module. If the actual load deviates significantly from the predicted value, the prediction model is re-executed to update the prediction parameters. Simultaneously, an online learning algorithm is used to continuously optimize the weight allocation and constraint settings of the multi-objective optimization function based on historical execution data. This closed-loop feedback mechanism enables continuous optimization of the scheduling strategy, improves prediction accuracy, and enhances the overall system performance and adaptability. The optimized multi-objective function and hierarchical load prediction provide more accurate support for subsequent scheduling decisions.

[0122] Example 2

[0123] like Figure 4 The diagram shows a flowchart of a distributed computing power scheduling method provided in this application embodiment. The method includes:

[0124] The system acquires hardware signals, network packets, and task sets in real time. It processes these signals through edge nodes to obtain resource status, network quality, and task status, and then fuses them into a perceptual vector based on a spatiotemporal graph convolutional network.

[0125] Based on the perception vector, a dynamic time window is configured, and hierarchical load forecasting is performed according to the dynamic time window to generate a load forecasting matrix. The forecast deviation rate is monitored to trigger the re-execution of hierarchical load forecasting.

[0126] The load prediction matrix is ​​extracted and combined with resource status, network quality and task status to make a comprehensive judgment to execute a dual scheduling mechanism, which includes a pre-migration path and a temporary scheduling path.

[0127] When executing the pre-migration path, the migration task and target unit are selected based on the resource value of each unit, and scheduling instructions are generated. The execution results of the scheduling instructions are monitored to adjust the tiered load forecast.

[0128] When executing temporary scheduling paths, scheduling instructions are generated based on a multi-objective optimization algorithm, and the execution results of the scheduling instructions are monitored to adjust the hierarchical load forecast.

[0129] Since the principle of the method in this application embodiment is similar to that of the system described in this application embodiment, the implementation of the method is the same as that of the system, and the repeated parts will not be described again.

Claims

1. A distributed computing power scheduling system, characterized in that, include: Hardware signals, network packets, and task sets are acquired in real time. Resource status, network quality, and task status are obtained through edge node processing, and then fused into a perception vector based on a spatiotemporal graph convolutional network. Based on the perception vector, a dynamic time window is configured, and hierarchical load forecasting is performed according to the dynamic time window to generate a load forecasting matrix. The load forecasting matrix includes cell identifiers, load forecast values ​​and forecast confidence levels, and the forecast deviation rate is monitored to trigger the re-execution of hierarchical load forecasting. The load prediction matrix is ​​extracted and combined with resource status, network quality and task status to make a comprehensive judgment to execute a dual scheduling mechanism. The dual scheduling mechanism includes a pre-migration path and a temporary scheduling path. When executing the pre-migration path, the migration task and target unit are selected based on the resource value of each unit, and a scheduling instruction is generated. When executing the temporary scheduling path, a scheduling instruction is generated based on a multi-objective optimization algorithm. The scheduling instruction includes the migration task, source unit, target unit and resource allocation. The execution results of the scheduling instruction are monitored synchronously to adjust the hierarchical load prediction. The execution logic of the pre-migration path includes: The static value of each unit is determined based on hardware performance, energy efficiency, and reliability; the dynamic value of each unit is determined based on load forecast and resource imbalance; and the resource value of each unit is determined by combining the static and dynamic values. Configure a value threshold, compare the resource value with the value threshold to filter source units, extract the task set in the source unit, and select migration tasks according to the task urgency. Combining the resource value and load prediction of the target unit, the Hungarian algorithm is used to match the migration task with the target unit to obtain the matching result; Based on the matching results, a scheduling instruction is generated that includes migration tasks, source units, target units, and resource allocation. The execution results of the scheduling instructions are executed and monitored. The execution results include task execution progress and data transmission volume. The execution results are compared with the target results to determine the migration success rate. The tiered load forecast is adjusted based on the migration success rate. The execution logic of the temporary scheduling path includes: The task completion time, resource consumption, and system stability are defined as optimization objectives. The objective weights of the optimization objectives are adjusted according to the task type. Task constraints and resource constraints are established by combining the load prediction matrix and resource status to form a multi-objective optimization function. Solving the multi-objective optimization function yields a set of scheduling paths. Based on task urgency and resource imbalance, a temporary scheduling path is selected from the set of scheduling paths to generate scheduling instructions. The system executes and monitors the execution results of scheduling instructions, compares the execution results with the target results to determine the achievement rate of the optimization objectives, adjusts the target weights of the multi-objective optimization function based on the achievement rate of the optimization objectives, and assists in adjusting the hierarchical load forecast.

2. The distributed computing power scheduling system as described in claim 1, characterized in that, The fusion logic of the perception vectors includes: Using resource status, network quality, and task status as nodes, the weights in the spatial dimension are determined based on the resource affinity between nodes, and temporal connections are established for the same node to determine its weight in the time dimension, thus forming a spatiotemporal graph. Spatial and temporal convolutions are performed on the spatiotemporal graph through a spatiotemporal graph convolutional network to obtain spatial and temporal features, and feature weights for spatial and temporal features are dynamically assigned based on a self-attention mechanism. Based on the feature weights, spatial and temporal features are processed through a hierarchical fusion method to obtain the perception vector.

3. The distributed computing power scheduling system as described in claim 2, characterized in that, The re-execution logic for the tiered load forecast includes: Real-time monitoring of the load prediction value and the actual load prediction deviation rate of each cell in the load prediction matrix; configuration of deviation threshold; comparison of prediction deviation rate and deviation threshold to trigger re-execution of tiered load prediction. When the re-execution of tiered load forecasting is triggered, the influencing factors and types of deviations that cause the forecasting bias are determined through feature importance analysis. Based on the scope of the influencing factors, the forecasting level of tiered load forecasting is located, and the re-execution level is determined. The adjustment method for the re-execution level is determined based on the deviation type, and the adjustment effect is monitored to determine whether the re-execution of the hierarchical load prediction is triggered again.

4. The distributed computing power scheduling system as described in claim 3, characterized in that, The first-level prediction processing logic includes extracting and aggregating the perception vectors in the data cache, predicting the load trend within the corresponding dynamic time window through a spatiotemporal graph neural network, and generating the initial load prediction value. The second-level prediction processing logic includes receiving the initial load prediction value, predicting the load trend within the corresponding dynamic time window through the Long Short-Term Memory network based on the resource status of the perception vector in the data buffer, so as to correct the initial load prediction value. The third-level prediction processing logic includes receiving the corrected load prediction value, obtaining the load prediction value through reinforcement learning based on the task state of the perception vector in the data buffer.

5. The distributed computing power scheduling system as described in claim 4, characterized in that, The sub-logic for generating the load prediction matrix includes: Receive sensing vectors, map the sensing vectors to unit identifiers according to their physical location and functional attributes, and classify and store the sensing vectors in the data cache area according to the attributes of the unit identifiers. Extract the perception vector from the data cache to determine historical load volatility, and adjust the length of the time window based on the historical load volatility to configure a dynamic time window; Tiered load forecasting is performed based on a dynamic time window. Tiered load forecasting includes first-level forecasting, second-level forecasting, and third-level forecasting to obtain load forecast values. The evaluation dimensions are determined to be data integrity, result consistency and prediction stability. The prediction confidence is calculated for the load prediction value of each unit, and the evaluation weights of the evaluation dimensions are dynamically adjusted according to the historical prediction error to correct the prediction confidence. A load prediction matrix including unit identifier, load prediction value and prediction confidence is generated.

6. The distributed computing power scheduling system as described in claim 5, characterized in that, The execution judgment sub-logic of the dual scheduling mechanism includes: Extract the prediction confidence of the load prediction matrix, determine the resource imbalance of each unit based on the resource status, determine the network stability of each unit based on the network quality, and determine the task urgency of each unit based on the task status. Configure scheduling thresholds and compare them with prediction confidence, resource imbalance, network stability, and task urgency to determine the dual scheduling mechanism executed by each unit. Monitor the execution effect of the dual scheduling mechanism to dynamically adjust the scheduling threshold.

7. The distributed computing power scheduling system as described in claim 6, characterized in that, The resource status includes resource utilization, memory pressure, and IO wait queue; the network quality includes bandwidth volatility, transmission latency, and packet loss rate; and the task status includes computational density and data dependencies.

8. A distributed computing power scheduling method, implemented based on the distributed computing power scheduling system according to any one of claims 1-7, characterized in that, include: Hardware signals, network packets, and task sets are acquired in real time. Resource status, network quality, and task status are obtained through edge node processing, and then fused into a perception vector based on a spatiotemporal graph convolutional network. Based on the perception vector, a dynamic time window is configured, and hierarchical load forecasting is performed according to the dynamic time window to generate a load forecasting matrix. The forecast deviation rate is monitored to trigger the re-execution of hierarchical load forecasting. The load prediction matrix is ​​extracted and combined with resource status, network quality and task status to make a comprehensive judgment to execute a dual scheduling mechanism, which includes a pre-migration path and a temporary scheduling path. When executing the pre-migration path, the migration task and target unit are selected based on the resource value of each unit, and scheduling instructions are generated. The execution results of the scheduling instructions are monitored to adjust the tiered load forecast. When executing temporary scheduling paths, scheduling instructions are generated based on a multi-objective optimization algorithm, and the execution results of the scheduling instructions are monitored to adjust the hierarchical load forecast.

Images