Distributed task execution time prediction method based on heterogeneous graph neural network

Abstract

The invention discloses a distributed task execution time prediction method based on a heterogeneous graph neural network, and the method comprises the steps: obtaining multi-source information, constructing a heterogeneous graph which comprises various types of nodes and various relationships, and employing a relationship perception feature learning mechanism and a type perception neighborhood sampling mode, different relationships and importance of neighbor nodes are distinguished, thereby generating high-order representations of tasks and resources. On the basis, the total execution time is decomposed into two parts of queuing waiting time and actual execution time, and the final execution time is predicted and synthesized based on the resource dynamic state and the task-resource interaction characteristics. Monte Carlo Dropout is further utilized to obtain uncertainty estimation, confidence levels are divided according to prediction variance, a prediction strategy is adaptively adjusted, and the robustness of the model under the condition of high load or insufficient data is improved. According to the method, the execution time prediction precision and reliability in the distributed scheduling environment can be remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of distributed task scheduling technology, specifically a method for predicting the execution time of distributed tasks based on heterogeneous graph neural networks. Background Technology

[0002] Task execution time prediction is a key technology in distributed task scheduling, directly impacting the accuracy of scheduling decisions and overall system performance. In the current computing environment, multiple interconnected clusters form the core infrastructure supporting efficient scheduling of large-scale workloads. These facilities integrate dispersed computing and storage resources into a unified virtual resource pool, enabling flexible allocation and efficient utilization of resources. Its core value lies in its ability to dynamically allocate resources and schedule tasks based on workload demands, thereby improving the overall system's resource utilization, processing power, and reliability. However, existing execution time prediction methods face numerous challenges when dealing with distributed environments.

[0003] Traditional prediction methods primarily rely on linear regression, statistical averaging, or rule-driven models. These methods treat tasks, resources, and performance records as independent samples, failing to effectively utilize the structured relationships between them. Although graph neural networks (GNNs) can model graph-structured data, standard GNNs assume the homogeneity of nodes and edges, making it impossible to express the multi-type relationships and heterogeneous semantics between task-resource-performance records.

[0004] Existing methods still face the following problems when dealing with task execution time prediction in distributed environments:

[0005] 1. Insufficient handling of resource heterogeneity: Distributed environments contain various heterogeneous resource types such as CPU, GPU, memory, storage, and network bandwidth, with significant differences in hardware configurations across different clusters. Traditional methods employ a unified resource abstraction model, which fails to accurately capture the impact of different hardware configurations on task execution performance, resulting in large prediction errors.

[0006] 2. Lack of Spatiotemporal Dynamic Modeling: Resource availability changes dynamically over time, and the operating time windows of different clusters vary significantly. Empirical studies show that although research facilities and edge sites contribute approximately 60% of the total capacity, they only operate for 12-14 hours per day, with resource utilization fluctuating dramatically at different times. Existing methods lack the ability to model the time dimension and cannot adapt to this dynamic resource availability pattern.

[0007] 3. Insufficient modeling of task complexity: Modern distributed tasks typically consist of multiple operators, forming a complex directed acyclic graph structure. Different operators have vastly different resource requirements, and tasks exhibit data and execution dependencies. Traditional methods struggle to model these complex structured relationships and heterogeneous requirements.

[0008] 4. Lack of Quantification of Prediction Uncertainty: Existing methods typically only provide point estimates and cannot quantify the reliability of the prediction. In resource-constrained or high-risk scenarios, the lack of uncertainty assessment can lead scheduling systems to make suboptimal or even erroneous decisions. Summary of the Invention

[0009] In view of the above-mentioned problems, the present invention is proposed.

[0010] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a distributed task execution time prediction method based on heterogeneous graph neural networks, comprising:

[0011] Obtain multi-source data of tasks, resources, performance records and resource parameters in the scheduling system, construct task nodes, resource nodes, performance record nodes and parameter nodes respectively, and establish a multi-relationship heterogeneous graph based on task execution relationship, resource consumption relationship and parameter configuration relationship;

[0012] The heterogeneous graph is subjected to relation-aware feature learning, which includes three parts: relation-aware aggregation, multi-layer feature propagation and attention mechanism enhancement. Combined with the heterogeneous neighborhood sampling strategy, high-order representations of task nodes and resource nodes are generated.

[0013] The results of the feature learning are input into the multilayer perceptron model to predict the queuing time and the actual execution time, and then synthesize the total execution time.

[0014] Uncertainty estimation is performed using Monte Carlo Dropout, and time forecasting adjustment strategies are implemented based on the confidence level of the forecast variance.

[0015] Online updates are performed based on newly arrived real execution records to achieve adaptive optimization of the model.

[0016] As a preferred embodiment of the distributed task execution time prediction method based on heterogeneous graph neural network described in this invention, the multi-relationship heterogeneous graph includes: constructing task nodes according to the structure and resource requirements of the scheduling task itself; constructing resource nodes according to the hardware capabilities and current state of computing resources; using the static configuration items of resources as parameter nodes; and generating performance record nodes based on the task execution records in the resource nodes.

[0017] Constructing heterogeneous graphs Where G represents a heterogeneous graph, V represents a set of nodes, E represents a set of edges, T represents a set of types, and R represents a set of relations;

[0018] In set V, Represents a set of task nodes; Represents a set of resource nodes; Represents the set of performance record nodes. This represents a set of parameter nodes; the type set T={job,resource,performance,parameter}, where job represents the task type, resource represents the resource type, performance represents the performance record type, and parameter represents the parameter type.

[0019] Edge set E represents the interaction relationships between nodes, and relation set R defines semantic types, including: task execution relationships, in the form of a triple: (job, executes-on, performance), indicating that the source node type is job, the relation type is executes-on, and the target node type is performance; resource consumption relationships, in the form of a triple: (performance, consumes, resource), indicating that the source node type is performance, the relation type is consumes, and the target node type is resource; parameter configuration relationships, in the form of a triple: (resource, has-param, parameter), indicating that the source node type is resource, the relation type is has-param, and the target node type is parameter.

[0020] As a preferred embodiment of the distributed task execution time prediction method based on heterogeneous graph neural networks described in this invention, wherein: after the construction of nodes and edges is completed, an index structure is established for each node type and relation type;

[0021] For different relation types Construct the corresponding adjacency matrix ,in This represents an adjacency matrix of relation type r, recording the neighbor connections under relation r.

[0022] As a preferred embodiment of the distributed task execution time prediction method based on heterogeneous graph neural networks described in this invention, the relationship-aware aggregation includes setting independent feature transformation matrices and aggregation rules for each relationship type, so that the model can distinguish the feature differences brought about by task execution relationships, resource consumption relationships and other heterogeneous semantic relationships during feature propagation.

[0023] When updating the node representation, the model performs relation-distinguishing aggregation on the features of neighbor nodes from different relation types, and performs semantic transformation based on the feature transformation matrix corresponding to each relation; and integrates the aggregated results of each relation, and generates the current layer representation of the node through the activation function;

[0024] The multi-layer feature propagation includes deep feature propagation using a multi-layer stacked structure: the initial layer propagates the original feature vector of node v. The concatenation operation CONCAT is performed with the neighbor feature aggregation result AGG, and then passed through the weight matrix W of layer 0. (0) By transforming and applying the activation function σ, we obtain the hidden representation of node v in layer 1. The formula is: ;

[0025] in, Represents the original characteristics of neighbor node u, neighbor node N(v) represents the set of neighbors of node v;

[0026] The intermediate layer iteratively updates the node representation to capture multi-hop neighborhood information: node v at the th hop... +1 layer representation By the Layer weight matrix Acting on node v at the th Layer representation Its neighbor node u in the th Layer representation The aggregation results are concatenated and then processed through an activation function to obtain the following formula:

[0027]

[0028] The final layer generates an interaction representation between task j and resource res: ;

[0029] Where j represents the task index, res represents the resource index, (L) represents the final layer L, and INTERACT is the interaction function.

[0030] As a preferred embodiment of the distributed task execution time prediction method based on heterogeneous graph neural networks described in this invention, the heterogeneous neighborhood sampling strategy includes: for any node, under the condition of setting the total number of samples, calculating the sampling weight of each relation type according to the proportion of the number of neighbor nodes of different relation types; distributing the total number of samples to different relation types according to the sampling weights, and performing uniform random sampling in the neighbor set of each relation type to obtain the corresponding number of neighbor nodes;

[0031] The enhanced attention mechanism consists of relation-level attention and node-level attention; for each relation type r, the model learns an attention coefficient. ,in This characterizes the degree to which the relationship contributes to the overall aggregation process;

[0032] Introduce node-level attention weights for each relation type. ,in Used to characterize the importance of node u relative to the current node v under relation r;

[0033] Based on a two-layer attention mechanism, node v at the th... The layer representation vector is finally calculated as follows:

[0034]

[0035] in, This represents the result of node v being updated at the current layer. This represents the feature transformation matrix corresponding to relation r. Represents a non-linear activation function. This represents the set of neighbors connected to node v under relation r.

[0036] As a preferred embodiment of the distributed task execution time prediction method based on heterogeneous graph neural networks described in this invention, the input of the multilayer perceptron model is: the final representation of the task node, the final representation of the resource node, and the interaction representation between the task and the resource.

[0037] This represents the final embedding of task j. This indicates the final embedding of the resource `res`. If we represent the task-resource interaction embedding, then the prediction model is represented as: ;

[0038] in, It is a multilayer perceptron structure containing multiple hidden layers. Dropout is added after each layer to reduce the risk of overfitting. The final output layer is responsible for generating the execution time prediction value of the corresponding task on the target resource.

[0039] The total execution time is decomposed into two components: queuing time and actual execution time. The two components are predicted separately: the queuing time is predicted based on the dynamic operating status of the resource side and is achieved through a queuing time prediction sub-model; the actual execution time is predicted by using a multilayer perceptron model to learn the influence of task characteristics and resource capabilities on runtime. Finally, the two parts are integrated to obtain the total time prediction.

[0040] As a preferred embodiment of the distributed task execution time prediction method based on heterogeneous graph neural networks described in this invention, the time prediction adjustment strategy includes: maintaining Dropout activation during the inference phase, performing multiple forward propagations on the same task-resource pair to obtain multiple sets of prediction values, using the mean of the prediction values ​​as the final prediction mean μ, and using the variance calculated from the standard deviation of the prediction values ​​as the uncertainty index.

[0041] Based on the magnitude of the variance, the prediction results are divided into high, medium, and low confidence levels: high-confidence predictions directly use the prediction mean μ as the execution time estimate; medium-confidence predictions use a conservative estimate. ,in This represents a conservative estimate of the execution time; low-confidence triggers exploratory scheduling, which obtains real execution data by performing experiments on multiple candidate resources to update model parameters.

[0042] A distributed task execution time prediction system based on heterogeneous graph neural network using any of the methods described in this invention, wherein: a construction unit acquires multi-source data of tasks, resources, performance records and resource parameters in the scheduling system, constructs task nodes, resource nodes, performance record nodes and parameter nodes respectively, and establishes a multi-relationship heterogeneous graph based on task execution relationship, resource consumption relationship and parameter configuration relationship;

[0043] The learning unit performs relation-aware feature learning on the heterogeneous graph, including three parts: relation-aware aggregation, multi-layer feature propagation, and attention mechanism enhancement. It also combines a heterogeneous neighborhood sampling strategy to generate high-order representations of task nodes and resource nodes.

[0044] The computing unit inputs the results of the feature learning into the multilayer perceptron model to predict the queuing time and the actual execution time, and then synthesizes the total execution time.

[0045] The adjustment unit performs uncertainty estimation via Monte Carlo Dropout and executes a time-based forecast adjustment strategy based on the confidence level of the forecast variance.

[0046] The update unit performs online updates based on newly arrived real execution records, enabling adaptive optimization of the model.

[0047] A computer device includes: a memory and a processor; the memory stores a computer program, wherein: when the processor executes the computer program, it implements the steps of the method described in any one of the present invention.

[0048] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of the present invention.

[0049] The beneficial effects of this invention are as follows: By constructing a multi-relationship heterogeneous graph containing task nodes, resource nodes, performance record nodes, and parameter nodes, this invention expresses the task requirements, resource capabilities, historical execution behavior, and hardware configuration of a distributed scheduling system in a unified and structured manner, achieving accurate modeling of task-resource interaction characteristics. Utilizing a relationship-aware feature learning mechanism, feature transformation and aggregation rules are set for different relationship types. Combined with type-aware neighborhood sampling and a two-layer attention mechanism, the model can distinguish between various semantic relationships such as task execution, resource consumption, and parameter configuration, thereby effectively improving the expressive power of node representation.

[0050] By decomposing the total execution time into queuing time and actual execution time, this invention improves the accuracy and interpretability of time estimation by predicting based on dynamic features such as resource utilization and queue length, as well as task-resource interaction embedding. Simultaneously, Monte Carlo Dropout is introduced to achieve uncertainty estimation, and confidence levels are categorized based on prediction variance. This supports high-confidence direct prediction, medium-confidence conservative prediction, and low-confidence exploratory scheduling, making the prediction process more robust to load fluctuations and insufficient data. Combined with an online update mechanism based on real execution records, this invention can continuously adapt to dynamic system changes, significantly improving the accuracy, robustness, and overall efficiency of execution time prediction and distributed scheduling systems. Attached Figure Description

[0051] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 The first embodiment of the present invention provides an overall flowchart of a distributed task execution time prediction method based on heterogeneous graph neural networks;

[0053] Figure 2 A schematic diagram of the heterogeneous graph structure in a distributed task execution time prediction method based on a heterogeneous graph neural network provided in the first embodiment of the present invention;

[0054] Figure 3 A comparison of the prediction performance of a distributed task execution time prediction method based on heterogeneous graph neural networks provided in the first embodiment of the present invention—MSE and MAPE;

[0055] Figure 4 A comparison of the prediction performance of a distributed task execution time prediction method based on heterogeneous graph neural networks provided in the first embodiment of the present invention—MAE and R² scores;

[0056] Figure 5 A comparison of the prediction accuracy of a distributed task execution time prediction method based on heterogeneous graph neural networks under different workloads, provided in the second embodiment of the present invention;

[0057] Figure 6 A comparison of the cross-cluster generalization ability of a distributed task execution time prediction method based on heterogeneous graph neural networks provided in the second embodiment of the present invention. Detailed Implementation

[0058] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0059] Example 1, referring to Figures 1-4 As an embodiment of the present invention, a distributed task execution time prediction method based on heterogeneous graph neural networks is provided, comprising:

[0060] S1: Obtain multi-source data of tasks, resources, performance records and resource parameters in the scheduling system, construct task nodes, resource nodes, performance record nodes and parameter nodes respectively, and establish a multi-relationship heterogeneous graph based on task execution relationship, resource consumption relationship and parameter configuration relationship.

[0061] This stage primarily transforms the multi-source information from the original scheduling system into a structured form suitable for graph neural network computation. Unlike traditional methods that simply concatenate all data into feature vectors, this invention explicitly distinguishes node types and their relationships, enabling the model to directly capture and utilize the structured associations between key entities such as tasks, resources, and performance records.

[0062] Specifically, this stage includes: defining node types, defining relationship types, extracting node features, and constructing a multi-relationship heterogeneous graph structure accordingly.

[0063] First, construct the heterogeneous graph. Where G represents a heterogeneous graph, V represents a set of nodes (Vertices), E represents a set of edges (Edges), T represents a set of types (Types), and R represents a set of relations (Relations). This represents a set of task nodes (where job represents the index of the "task" type). Represents a set of resource nodes (where resource represents the index of the "resource" type); This represents the set of performance record nodes (where performance represents the index of the "performance record" type). This represents the set of parameter nodes (where parameter represents the subscript of the "parameter" type). The type set T={job,resource,performance,parameter} (where job,resource,performance,parameter are set elements, with the same meaning as the node type subscript) explicitly encodes node heterogeneity.

[0064] Establish a multi-relation edge set: Edge set E represents the interaction relationships between nodes, and relation set R defines the semantic types. Specifically, it includes: task execution relations (job, executes-on, performance) (this is a triplet relation, indicating that the source node type is job, the relation type is executes-on, and the target node type is performance), resource consumption relations (performance, consumes, resource) (indicating that the source node type is performance, the relation type is consumes, and the target node type is resource), and parameter configuration relations (resource, has-param, parameter) (indicating that the source node type is resource, the relation type is has-param, and the target node type is parameter). This multi-relation structure can fully express the complex dependencies between tasks, resources, and performance.

[0065] Node feature extraction: For task nodes, the feature vector includes resource requirements (number of CPU cores, number of GPUs, memory size, storage capacity), task type identifier, DAG structure features (number of operators, dependency depth, parallelism), and data features (input data size, intermediate result size). For resource nodes, the feature vector includes hardware configuration (CPU model, GPU model, memory capacity, network bandwidth), geographic location coordinates, time features (current time slot, availability window), and utilization status. For performance recording nodes, features include execution time decomposition (queue time, execution time), resource consumption, network overhead, and timestamps.

[0066] Build a global heterogeneous graph index and distribute it to the prediction module: After building the nodes and edges, establish an index structure for each node type and relation type. For different relation types... Construct the corresponding adjacency matrix Here, A represents the adjacency matrix, and r represents the index of the relation type, i.e., a specific relation type r belongs to R, recording the neighbor connections under that relation. By establishing this heterogeneous graph structure, efficient querying and feature propagation of historical scheduling data can be performed, which is crucial for accurately predicting task execution time.

[0067] S2: Perform relation-aware feature learning on the heterogeneous graph, including three parts: relation-aware aggregation, multi-layer feature propagation, and attention mechanism enhancement, and combine the heterogeneous neighborhood sampling strategy to generate high-order representations of task nodes and resource nodes.

[0068] The core task of this stage is to conduct multi-level, relationship-aware representation learning on heterogeneous graphs, enabling the model to fully capture high-order interaction features between tasks, resources, and historical records. Unlike the assumption of homogeneous node and edge types in traditional GNNs, specialized aggregation and propagation mechanisms are designed for different relationship types.

[0069] The high-order representation learning of task-resources is accomplished through multi-layer graph convolution, mainly including three parts: relation-aware aggregation, multi-layer feature propagation, and attention mechanism enhancement.

[0070] Design of a Relationship-Aware Aggregation Mechanism: Addressing the issue of diverse node and edge types in heterogeneous graphs, a relationship-aware feature aggregation mechanism is proposed. Its core idea is that each relationship type uses an independent feature transformation method and aggregation rule, enabling the model to distinguish the importance and semantic differences between different relationships such as "task → resource" and "resource → performance record." When updating the representation of a node, the model processes features passed from neighbors of different relationship types separately. For example, information from neighbors connected through "task dependency relationships" and information from neighbors connected through "resource consumption relationships" should be interpreted semantically differently, thus using different transformation matrices and aggregation functions. Finally, the aggregation results of each relationship are unified and processed through an activation function to obtain the node representation of the current layer. This approach allows the model to distinguish the importance of different relationships at the structural level, thereby more accurately capturing the interaction features between tasks, resources, and historical records.

[0071] Multi-layer graph convolutional architecture: Employs a multi-layer stacked structure for deep feature propagation. The initial layer takes the original feature vector of node v. (in The hidden representation of node v in layer 1 is obtained by concatenating the feature (represented by the feature) with the aggregation result of the neighbor features (AGG, represented by the aggregation function) using the CONCAT operation, and then transforming it through the weight matrix W(0) of layer 0 (W represents the weight matrix, and (0) represents the parameters of layer 0) and the activation function σ. The formula is: ,in These are the original features of the neighbor node u. N(v) represents the set of neighbors of node v (N represents Neighbors). The intermediate layer iteratively updates the node representation to capture multi-hop neighborhood information: node v at the th... +1 layer representation By the Layer weight matrix Acting on node v at the th Layer representation Its neighbor node u in the th Layer representation The aggregation results are concatenated and then processed through an activation function to obtain the final product, as shown in the formula: The final layer generates an interaction representation between task j and resource res. (where j is the task index, res is the resource index, (L) represents the final L-th layer, and INTERACT is the interaction function). The interaction function can take the form of dot product, concatenation, or bilinear.

[0072] Heterogeneous Neighborhood Sampling Strategy: To improve computational efficiency while maintaining expressive power, type-aware neighborhood sampling is employed. For each node, given the total number of samples, for each relation type, an importance weight is calculated based on the proportion of its neighbors to the total number of neighbors. Subsequently, the number of samples for each relation type is allocated proportionally according to this weight, and a corresponding number of nodes are uniformly and randomly sampled from the neighbor set of that relation type. This sampling strategy ensures that the neighbors of each relation type are fully represented according to their importance during the sampling process.

[0073] To more effectively differentiate the impact of different neighbor nodes on feature aggregation, a two-layer attention mechanism consisting of relation-level attention and node-level attention is employed in the model. This mechanism first introduces attention weights at the relation level to measure the importance of different relation types to the final task. Specifically, for each relation type r, the model learns an attention coefficient. ,in This characterizes the contribution of the relationship to the overall aggregation process. For example, for some prediction tasks, the "historical performance association" between nodes may have a higher influence, while in other tasks, the "resource allocation association" is more critical to the outcome; the model can automatically adjust for these differences. At a further node level, neighboring nodes under the same relationship type do not necessarily have the same value. To achieve finer-grained differentiation within a relationship, node-level attention weights are introduced for each relationship type. ,in This is used to characterize the importance of node u relative to the current node v in relation r. Through this mechanism, neighboring nodes with higher information value receive greater weight, while the interference from noise or irrelevant connections is weakened. Based on this two-layer attention mechanism, node v in the... The layer representation vector is finally calculated as follows:

[0074]

[0075] in, This represents the updated representation of node v at the current layer. This represents the feature transformation matrix corresponding to relation r. It is a non-linear activation function. This represents the set of neighbors connected to node v under relation r. Through this adaptive weighted aggregation process, the model can simultaneously identify "which relations are more important" and "which neighbors under the same relation are more worthy of attention," thereby capturing the most valuable structural information for the task and significantly improving the accuracy and robustness of feature representation.

[0076] In this embodiment, the relation-level attention coefficient This is obtained by applying a learnable attention scoring function to the aggregated information corresponding to relation type r. Specifically, the model sets an independent trainable attention parameter vector a for each relation r. r and based on a r Intermediate features z in relation aggregation r Linear combination calculation of attention score e r Then, softmax normalization is performed on the scoring results of all relations to obtain the relation-level attention coefficients. During model training, a r It participates in forward computation and receives backpropagation gradients from the execution-time prediction loss function, thus continuously updating during training. It can automatically reflect the importance of each relationship type.

[0077] In other alternative implementations, relation-level attention coefficients can also be obtained in the following ways: directly outputting the weights of different relations through a gating network; determining relation weights by calculating the similarity or matching degree between relation features; or automatically generating corresponding weights based on the contribution of different relations in historical training. These methods can all distinguish the importance of relations and constitute equivalent solutions to this embodiment.

[0078] In this embodiment, node-level attention weights are used to distinguish the importance of different neighbor nodes under the same relation type. The model inputs the features of the current node and one of its neighbors into a trainable scoring function, and judges the importance of the neighbor by the score. Subsequently, the scores of all neighbors under the same relation are normalized to obtain the final node attention weights. During training, the model continuously updates the scoring function through error backpropagation, giving greater weights to neighbors with higher information value, while automatically reducing the weights of unimportant or noisy neighbors.

[0079] In other implementations, node-level attention weights can also be generated by one of the following methods: using a similarity-based scoring method, such as determining the weight based on the degree of similarity between node features; using a multi-head attention approach to generate multiple attention branches and then merging them; or automatically setting weights based on the historical influence or performance of neighboring nodes.

[0080] S3: Input the results of the feature learning into the multilayer perceptron model to predict the queuing time and the actual execution time, and synthesize the total execution time.

[0081] After completing high-order feature learning on the heterogeneous graph, this stage aims to map the learned task, resource, and task-resource interaction representations to the final execution time predictions. To more accurately reflect the real distributed task execution process, this invention utilizes a prediction framework consisting of a multilayer perceptron predictor and an execution time decomposition model, and employs a multi-task joint optimization strategy to improve prediction stability and convergence. The prediction performance of this invention is compared with several baseline methods on the MSE and MAPE metrics. Figure 3 Provided. For example... Figure 4 As shown, this method also demonstrates significant advantages under both MAE and R² metrics. R² represents the proportion of the actual data variation that the model can explain.

[0082] Multilayer Perceptron Predictor: In this invention, a multilayer perceptron structure is first used to convert the final embedding vector into an actual execution time prediction result. The model input includes three types of information: the final representation of task nodes, the final representation of resource nodes, and the interaction representation between tasks and resources. These three together constitute the key factors affecting execution time.

[0083] Specifically, if using This represents the final embedding of task j. This indicates the final embedding of the resource `res`. If we represent the task-resource interaction embedding, then the prediction model can be formally represented as:

[0084]

[0085] in It uses a multilayer perceptron (MLP) architecture. This MLP contains multiple hidden layers, with Dropout added after each layer to reduce the risk of overfitting. The final output layer is responsible for generating the predicted execution time of the corresponding task on the target resource.

[0086] Execution Time Decomposition Model: The total execution time is decomposed into two components: queuing time and actual execution time. The two components are predicted separately: the queuing time prediction is based on the dynamic operating status of the resources and is achieved through a queuing time prediction sub-model; the actual execution time prediction is obtained by using a multilayer perceptron model to learn the influence of task characteristics and resource capabilities on runtime. Finally, the two parts are integrated to obtain the total time prediction.

[0087] In this embodiment, the queuing time prediction sub-model is implemented using a neural network-based regression model. This model takes the dynamic operating status of the target resource as input, including queuing-related features such as real-time resource utilization, queue length, task arrival rate, and average waiting time. The model uses a multi-layer fully connected network to perform a non-linear mapping on these features, learning the relationship between resource load changes and queuing time, thereby outputting a predicted value for the queuing time. This sub-model is optimized along with the entire system's training process, enabling the prediction results to gradually approximate actual queuing behavior as the operating data changes.

[0088] In other alternative implementations, the queuing time prediction sub-model can also be implemented using non-neural network alternatives, such as analytical computational models based on queuing theory, which estimate waiting time using resource arrival rates and service rates; time series prediction methods can also be used, constructing trend models based on historical resource load changes to predict future queuing conditions; tree models or statistical regression models can also be used, learning the correspondence between dynamic load and queuing time through feature splitting or probability distribution fitting. All of these alternative methods can achieve queuing time estimation and have the same technical effect as this embodiment.

[0089] Multi-task learning objective: Define a joint loss function to simultaneously optimize two prediction objectives: the total loss consists of the prediction error of queuing time, the prediction error of execution time, and an L2 regularization term, where the regularization term is controlled by an adjustable regularization coefficient.

[0090] An early stopping mechanism is adopted: training is terminated early when the validation set loss does not improve for several consecutive rounds.

[0091] S4: Estimate uncertainty using Monte Carlo Dropout and implement time forecasting adjustment strategies based on the confidence level of the forecast variance.

[0092] Given the dynamic and uncontrollable nature of distributed computing environments, the reliability of prediction results is equally crucial. Therefore, this method not only outputs the predicted values ​​but also provides the corresponding quantification of uncertainty. The Monte Carlo Dropout method is used to quantify prediction uncertainty, and the prediction strategy is dynamically adjusted based on the uncertainty level, thereby improving the robustness and adaptability of scheduling decisions.

[0093] Monte Carlo Dropout Uncertainty Estimation: Its core idea is to enable Dropout multiple times during the inference phase, causing slight differences in the model's predictions with each forward propagation, thus generating a series of predictions. After model training is complete, for a given task-resource pair, the Dropout mechanism from the training phase is kept active, and multiple forward propagations are performed to obtain a set of predictions. The average of these predictions is then calculated as the final prediction mean μ, and its standard deviation is calculated; the variance is then calculated as follows: The variance reflects the uncertainty of the model regarding the prediction result. The larger the variance, the more uncertain the model is about the prediction, thus requiring a more robust strategy in scheduling decisions.

[0094] Confidence Interval Division: To enable the scheduling system to adopt different response strategies based on uncertainty, the prediction results are divided into multiple confidence levels (such as "high, medium, and low") according to their variance. Samples with smaller variances indicate that the model is more confident in its predictions, while larger variances mean that the results may have a greater risk of bias. For example: dividing the prediction results into different confidence levels based on the prediction variance: when At that time, high confidence was maintained. At that time, the confidence level was medium to high. At a moderate confidence level, when The timeframe represents a low confidence level. Different confidence intervals represent different ranges of error risk, thus supporting the selection of subsequent adaptive prediction strategies.

[0095] Adjusting forecasting behavior based on uncertainty: High-confidence forecasts directly use the forecast mean μ as an execution time estimate, suitable for routine scheduling decisions; medium-confidence forecasts use a conservative estimate. (in The coefficient 1.5 represents the conservatively estimated execution time, used to add a safety margin to the mean. This strategy is suitable for resource-constrained scenarios. Low-confidence predictions trigger exploratory scheduling, running small-scale trials on multiple candidate resources and updating the model after collecting actual execution data. This multi-level strategy can reduce the risk of scheduling failure while ensuring prediction efficiency, making the system more robust in the face of uncertainty.

[0096] S5: Online updates are performed based on newly arrived real execution records to achieve adaptive optimization of the model.

[0097] A continuous learning mechanism is established to adapt to environmental changes. When a new task execution record arrives, the system calculates the error between the predicted and actual values. If the error exceeds a preset threshold, the new record is added to the training buffer. Once the buffer has accumulated a sufficient number of samples, the system performs a small-batch parameter update process. Simultaneously, the system tracks prediction performance metrics in real time, and automatically triggers model retraining when the average prediction accuracy within the sliding window falls below a set threshold. Through this mechanism, the model can continuously adjust itself as the system's operating state dynamically changes, maintaining sensitivity and adaptability to changes in resource status.

[0098] On the other hand, this embodiment also provides a distributed task execution time prediction system based on heterogeneous graph neural networks, which includes:

[0099] The construction unit acquires multi-source data on tasks, resources, performance records, and resource parameters from the scheduling system, and constructs task nodes, resource nodes, performance record nodes, and parameter nodes respectively. Based on task execution relationships, resource consumption relationships, and parameter configuration relationships, a multi-relationship heterogeneous graph is established.

[0100] The learning unit performs relation-aware feature learning on the heterogeneous graph, including three parts: relation-aware aggregation, multi-layer feature propagation, and attention mechanism enhancement. It also combines a heterogeneous neighborhood sampling strategy to generate high-order representations of task nodes and resource nodes.

[0101] The computing unit inputs the results of the feature learning into the multilayer perceptron model to predict the queuing time and the actual execution time, and then synthesizes the total execution time.

[0102] The adjustment unit performs uncertainty estimation via Monte Carlo Dropout and executes a time-based forecast adjustment strategy based on the confidence level of the forecast variance.

[0103] The update unit performs online updates based on newly arrived real execution records, enabling adaptive optimization of the model.

[0104] If the above functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0105] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0106] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0107] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0108] Example 2, refer to Figure 5 and Figure 6 As an embodiment of the present invention, a distributed task execution time prediction method based on heterogeneous graph neural networks is provided. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0109] One limitation of prediction algorithms based on the unified resource model is their inability to distinguish the differentiated impact of different resource types on task performance, leading to larger prediction errors in heterogeneous environments. To address this issue, this invention incorporates multiple types of relationships between tasks, resources, and performance records, constructing a heterogeneous graph structure. Through relationship-aware graph convolution, a high-order representation of the task-resource relationship is learned, achieving accurate predictions for combinations of different hardware configurations and task types. To ensure the quality of the prediction results, not only are multiple relationship indexes established in the graph structure to express the heterogeneous semantics between nodes, but also uncertainty quantification is provided through an execution time decomposition model and Monte Carlo Dropout, thereby eliminating the dependence on the unified resource abstraction and enhancing adaptability to heterogeneous environments (e.g., ...). Figure 6 As shown in the figure, it ensures the reliability of the prediction results and provides a more accurate basis for decision-making in the scheduling system.

[0110] Experimental results demonstrate that this invention achieves excellent prediction performance across various workloads. Compared to baseline methods such as linear regression, random forest, and standard graph neural networks, the prediction error of this invention is significantly reduced, and the goodness of fit reaches a high level. Figure 5 The prediction accuracy under different workloads is demonstrated, validating the model's generalizability. It also exhibits high accuracy in resource type matching, effectively avoiding performance degradation caused by resource type mismatch. Experimental results fully demonstrate the effectiveness and reliability of this invention.

[0111] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A distributed task execution time prediction method based on heterogeneous graph neural networks, characterized in that, include: Obtain multi-source data of tasks, resources, performance records and resource parameters in the scheduling system, construct task nodes, resource nodes, performance record nodes and parameter nodes respectively, and establish a multi-relationship heterogeneous graph based on task execution relationship, resource consumption relationship and parameter configuration relationship; The heterogeneous graph is subjected to relation-aware feature learning, which includes three parts: relation-aware aggregation, multi-layer feature propagation and attention mechanism enhancement. Combined with the heterogeneous neighborhood sampling strategy, high-order representations of task nodes and resource nodes are generated. The results of the feature learning are input into the multilayer perceptron model to predict the queuing time and the actual execution time, and then synthesize the total execution time. Uncertainty estimation is performed using Monte Carlo Dropout, and time forecasting adjustment strategies are implemented based on the confidence level of the forecast variance. Online updates are performed based on newly arrived real execution records to achieve adaptive optimization of the model.

2. The distributed task execution time prediction method based on heterogeneous graph neural networks as described in claim 1, characterized in that: The multi-relationship heterogeneous graph includes constructing task nodes based on the structure and resource requirements of the scheduling task itself; Construct resource nodes based on the hardware capabilities and current state of computing resources; The static configuration items of the resources are used as parameter nodes, and performance record nodes are generated based on the task execution records in the resource nodes. Constructing heterogeneous graphs Where G represents a heterogeneous graph, V represents a set of nodes, E represents a set of edges, T represents a set of types, and R represents a set of relations; In set V, Represents a set of task nodes; Represents a set of resource nodes; Represents the set of performance record nodes. This represents a set of parameter nodes; the type set T={job,resource,performance,parameter}, where job represents the task type, resource represents the resource type, performance represents the performance record type, and parameter represents the parameter type. Edge set E represents the interaction relationships between nodes, and relation set R defines semantic types, including: task execution relationships, in the form of a triple: (job, executes-on, performance), indicating that the source node type is job, the relation type is executes-on, and the target node type is performance; resource consumption relationships, in the form of a triple: (performance, consumes, resource), indicating that the source node type is performance, the relation type is consumes, and the target node type is resource; parameter configuration relationships, in the form of a triple: (resource, has-param, parameter), indicating that the source node type is resource, the relation type is has-param, and the target node type is parameter.

3. The distributed task execution time prediction method based on heterogeneous graph neural networks as described in claim 2, characterized in that: After completing the construction of nodes and edges, an index structure is established for each node type and relationship type; For different relation types Construct the corresponding adjacency matrix ,in This represents an adjacency matrix of relation type r, recording the neighbor connections under relation r.

4. The distributed task execution time prediction method based on heterogeneous graph neural networks as described in claim 3, characterized in that: The relation-aware aggregation includes setting independent feature transformation matrices and aggregation rules for each relation type, so that the model can distinguish feature differences brought about by task execution relations, resource consumption relations and other heterogeneous semantic relations during the feature propagation process; When updating the node representation, the model performs relation-distinguishing aggregation on the features of neighbor nodes from different relation types, and performs semantic transformation based on the feature transformation matrix corresponding to each relation; and integrates the aggregated results of each relation, and generates the current layer representation of the node through the activation function; The multi-layer feature propagation includes deep feature propagation using a multi-layer stacked structure: the initial layer propagates the original feature vector of node v. The concatenation operation CONCAT is performed with the neighbor feature aggregation result AGG, and then passed through the weight matrix W of layer 0. (0) By transforming and applying the activation function σ, we obtain the hidden representation of node v in layer 1. The formula is: ; in, Represents the original characteristics of neighbor node u, neighbor node N(v) represents the set of neighbors of node v; The intermediate layer iteratively updates the node representation to capture multi-hop neighborhood information: node v at the th hop... +1 layer representation By the Layer weight matrix Acting on node v at the th Layer representation Its neighbor node u in the th Layer representation The aggregation results are concatenated and then processed through an activation function to obtain the following formula: The final layer generates an interaction representation between task j and resource res: ; Where j represents the task index, res represents the resource index, (L) represents the final layer L, and INTERACT is the interaction function.

5. The distributed task execution time prediction method based on heterogeneous graph neural networks as described in claim 4, characterized in that: The heterogeneous neighborhood sampling strategy includes, for any node, calculating the sampling weight of each relation type based on the proportion of neighboring nodes of different relation types, under the condition of setting the total number of samples; The total number of samples is allocated to different relation types according to the sampling weight, and uniform random sampling is performed in the neighbor set of each relation type to obtain the corresponding number of neighbor nodes; The enhanced attention mechanism consists of relation-level attention and node-level attention; for each relation type r, the model learns an attention coefficient. ,in This characterizes the degree to which the relationship contributes to the overall aggregation process; Introduce node-level attention weights for each relation type. ,in Used to characterize the importance of node u relative to the current node v under relation r; Based on a two-layer attention mechanism, node v at the th The layer representation vector is finally calculated as follows: in, This represents the result of node v being updated at the current layer. This represents the feature transformation matrix corresponding to relation r. Represents a non-linear activation function. This represents the set of neighbors connected to node v under relation r.

6. The distributed task execution time prediction method based on heterogeneous graph neural networks as described in claim 5, characterized in that: The inputs to the multilayer perceptron model are: the final representation of the task node, the final representation of the resource node, and the interaction representation between the task and the resource. This represents the final embedding of task j. This indicates the final embedding of the resource `res`. If we represent the task-resource interaction embedding, then the prediction model is represented as: ; in, It is a multilayer perceptron structure containing multiple hidden layers. Dropout is added after each layer to reduce the risk of overfitting. The final output layer is responsible for generating the execution time prediction value of the corresponding task on the target resource. The total execution time is decomposed into two components: queuing time and actual execution time. The two components are predicted separately: the queuing time is predicted based on the dynamic operating status of the resource side and is achieved through a queuing time prediction sub-model; the actual execution time is predicted by using a multilayer perceptron model to learn the influence of task characteristics and resource capabilities on runtime. Finally, the two parts are integrated to obtain the total time prediction.

7. The distributed task execution time prediction method based on heterogeneous graph neural networks as described in claim 6, characterized in that: The time prediction adjustment strategy includes keeping Dropout active during the inference phase, performing multiple forward propagations on the same task-resource pair to obtain multiple sets of prediction values, using the mean of the prediction values ​​as the final prediction mean μ, and using the variance calculated from the standard deviation of the prediction values ​​as the uncertainty index. Based on the magnitude of the variance, the prediction results are divided into high, medium, and low confidence levels: high-confidence predictions directly use the prediction mean μ as the execution time estimate; medium-confidence predictions use a conservative estimate. ,in This represents a conservative estimate of the execution time; low-confidence triggers exploratory scheduling, which obtains real execution data by performing experiments on multiple candidate resources to update model parameters.

8. A distributed task execution time prediction system based on heterogeneous graph neural networks, employing the method described in any one of claims 1-7, characterized in that: The construction unit acquires multi-source data of tasks, resources, performance records and resource parameters in the scheduling system, and constructs task nodes, resource nodes, performance record nodes and parameter nodes respectively. It also establishes a multi-relationship heterogeneous graph based on task execution relationships, resource consumption relationships and parameter configuration relationships. The learning unit performs relation-aware feature learning on the heterogeneous graph, including three parts: relation-aware aggregation, multi-layer feature propagation, and attention mechanism enhancement. It also combines a heterogeneous neighborhood sampling strategy to generate high-order representations of task nodes and resource nodes. The computing unit inputs the results of the feature learning into the multilayer perceptron model to predict the queuing time and the actual execution time, and then synthesizes the total execution time. The adjustment unit performs uncertainty estimation via Monte Carlo Dropout and executes a time-based forecast adjustment strategy based on the confidence level of the forecast variance. The update unit performs online updates based on newly arrived real execution records, enabling adaptive optimization of the model.

9. A computer device, comprising: A memory and a processor; the memory stores a computer program, characterized in that: when the processor executes the computer program, it implements the steps of the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-7.

Images