Distributed computing power dynamic scheduling method and device, electronic equipment and storage medium

Abstract

The application provides a distributed computing power dynamic scheduling method and device, electronic equipment and storage medium, and belongs to the technical field of distributed computing. The method comprises the following steps: generating target feature data set based on distributed node computing power resource data and task running state data, and then obtaining node load prediction result; determining task priority by combining the prediction result, task computing power demand and running state, and dividing task node group according to the prediction result, computing power resource and node configuration data. The target execution node of the task is selected based on the task priority, node group allocation result and node load data. The target node load and task execution delay data are monitored in real time during task execution, and when the load exceeds the threshold value or the delay exceeds the standard, the cluster computing power configuration is adjusted based on the node configuration data and cluster idle resource data, and the related tasks on the target execution node are migrated according to the task priority. The application can improve the scheduling accuracy of complex computing tasks.

Description

Technical Field

[0001] This application belongs to the field of distributed computing technology, and more specifically, relates to a method and apparatus for dynamic scheduling of distributed computing power, electronic equipment, and storage media. Background Technology

[0002] With the rapid iteration of technologies such as cloud computing, artificial intelligence, and big data processing, global demand for computing power has exploded. Distributed computing systems, with their core advantages of resource sharing and elastic scaling, have become the core infrastructure supporting various complex computing tasks. Currently, while existing distributed computing scheduling technologies have made some progress, core problems remain: scheduling decisions lack comprehensiveness and foresight. Existing solutions rely solely on limited-dimensional real-time data for resource allocation and adjustment, resulting in low adaptability of scheduling strategies to actual scenarios. This leads to a chain reaction of problems such as insufficient resource utilization, fluctuating task execution latency, and poor system stability, making it difficult to meet the ever-increasing demands for precise and efficient scheduling from complex computing tasks. Summary of the Invention

[0003] The purpose of this application is to provide a method, device, electronic equipment, and storage medium for dynamic scheduling of distributed computing power, so as to improve the scheduling accuracy of complex computing tasks.

[0004] A first aspect of this application provides a method for dynamic scheduling of distributed computing power, including: The target feature dataset is obtained based on the computing resource data and task running status data of distributed nodes, and the node load prediction result is obtained based on the target feature dataset; the distributed node consists of multiple nodes; Task priorities are determined based on node load prediction results, task computing power requirement data, and task running status data; node group allocation results for tasks are determined based on node load prediction results, computing power resource data, and node configuration data. The target execution node for each task is determined based on task priority, node group allocation results, and node load data; the target execution node is used to call node hardware resources to execute the task according to the task's computing power requirements data. During task execution, acquire target node load and task execution delay data for the target execution node; if the target node load exceeds the load threshold and / or the task execution delay data exceeds the delay range, adjust the cluster computing power configuration based on node configuration data and cluster idle resource data, and migrate tasks on the target execution node based on task priority.

[0005] A second aspect of this application provides a distributed computing power dynamic scheduling device, comprising: The load prediction module is used to obtain a target feature dataset based on the computing resource data and task running status data of distributed nodes, and to obtain the node load prediction results based on the target feature dataset; the distributed nodes consist of multiple nodes; The node group allocation module is used to determine task priority based on node load prediction results, task computing power requirement data, and task running status data; and to determine the node group allocation results for tasks based on node load prediction results, computing power resource data, and node configuration data. The node allocation module is used to determine the target execution node for each task based on task priority, the node group allocation result of the task, and node load data; the target execution node of the task is used to call the node hardware resources to execute the task according to the task computing power requirement data; The dynamic scheduling module is used to obtain the target node load and task execution delay data during task execution. If the target node load exceeds the load threshold and / or the task execution delay data exceeds the delay range, the cluster computing power configuration is adjusted based on the node configuration data and cluster idle resource data, and the tasks on the target execution node are migrated based on task priority.

[0006] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the above-described distributed computing power dynamic scheduling method.

[0007] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described distributed computing power dynamic scheduling method.

[0008] The beneficial effects of the distributed computing power dynamic scheduling method and apparatus, electronic device, and storage medium provided in this application embodiment are as follows: This application embodiment constructs a feature set by integrating the computing power resource data and task running status data of distributed nodes, accurately predicts the future load of nodes, and then combines the load prediction results, task computing power requirements, node configuration and other multi-dimensional key information to determine task priority and node group allocation in a coordinated manner. This completely gets rid of the dependence of existing solutions on limited real-time data, and allows scheduling decisions to be based on current actual needs and predict future node status. It fundamentally improves the adaptability of scheduling strategies to actual scenarios and resolves the core contradiction caused by one-sided decision-making in existing technologies.

[0009] This application embodiment utilizes multi-dimensional data-driven decision-making to accurately match tasks to suitable node groups and target execution nodes, avoiding the dual waste of resource idleness and node overload. Simultaneously, differentiated scheduling based on task priority ensures that high-demand and high-urgency tasks are executed first, effectively reducing the fluctuation range of task execution latency. During task execution, this application embodiment monitors node load and task latency in real time. When anomalies occur, it dynamically adjusts computing power configuration based on cluster idle resources and scientifically migrates tasks based on priority, avoiding resource waste and task interruptions caused by blind expansion or random migration, thus ensuring the continuous and stable operation of the system. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A flowchart illustrating a distributed computing power dynamic scheduling method provided in an embodiment of this application; Figure 2 This is a structural block diagram of a distributed computing power dynamic scheduling device provided in an embodiment of this application; Figure 3 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0012] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0013] It is understood that in the embodiments of this application, data such as user information are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with relevant laws, regulations and standards.

[0014] It should be noted that the terms "first," "second," etc., used in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in sequences other than those illustrated or described herein.

[0015] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a distributed computing power dynamic scheduling method provided in an embodiment of this application. The method can be executed by an electronic device, and specifically, the method may include S101 to S104.

[0016] S101: Obtain the target feature dataset based on the computing power resource data and task running status data of distributed nodes, and obtain the node load prediction result based on the target feature dataset.

[0017] In this embodiment, the target feature dataset is obtained based on the computing resource data and task running status data of distributed nodes, including: Temporal features are extracted from the computing resource data of distributed nodes to obtain a subset of temporal features; The correlation feature analysis and extraction are performed on the computing resource data and task running status data of distributed nodes to obtain a subset of correlation features; An initial feature dataset is obtained based on a subset of temporal features and a subset of related features. Feature filtering is then performed on the initial feature dataset to obtain the target feature dataset.

[0018] In this embodiment, the node load prediction result is obtained based on the target feature dataset, including: The target feature dataset is divided into time-series feature-related data and node-related feature-related data. Based on time-series characteristic data, the time-series load prediction component is obtained through an LSTM model; Based on node association feature-related data, the associated load prediction component is obtained through the GCN model; The time-series load prediction component and the associated load prediction component are weighted and fused to obtain the node load prediction result.

[0019] In this embodiment, computing power resource data refers to the occupancy and availability status data of distributed node hardware resources, such as CPU utilization, GPU computing power utilization, and memory utilization. Task execution status data refers to relevant status information during task execution, such as task type, execution progress, and queue length. Target feature dataset refers to the effective set of features that can be used for load prediction after extraction, analysis, and filtering. Time-series feature extraction refers to extracting the regular features of data changes over time, such as the trend of CPU utilization changes of nodes every 5 seconds extracted according to a time series. Time-series feature subset refers to a set containing only time-related features, such as time-series data of node memory utilization within 1 hour. Correlation feature analysis and extraction refers to mining the mutual influence features between node computing power data and task status data, and between nodes, such as the correlation between GPU-intensive tasks and node GPU utilization. Correlation feature subset refers to a set containing only correlation-related features, such as the correlation between the frequency of data interaction between nodes and the load. Initial feature dataset refers to the original feature set obtained by merging the time-series feature subset and the correlation feature subset.

[0020] Feature filtering refers to the process of removing redundant and invalid features and retaining key and effective features. Temporal feature-related data refers to feature data in the target feature dataset that reflects changes over time. Node association feature-related data refers to feature data in the target feature dataset that reflects the relationships between nodes and tasks, and between nodes. Long Short-Term Memory (LSTM) models are neural network models adept at capturing long-term temporal dependencies and are used for temporal load prediction. Temporal dependency modeling and prediction constructs a time-dimensional dependency model to predict future load changes in nodes. Temporal load prediction component: Load prediction results based on temporal features output by the LSTM model. Graph Convolutional Network (GCN) models are neural network models adept at mining graph structure relationships and are used for association load prediction. Node association modeling and prediction refers to constructing a node association graph model to predict load changes in nodes affected by associations. Association load prediction component: Load prediction results based on association features output by the GCN model. Weighted fusion refers to the process of merging two types of prediction components according to preset weights to obtain the final result. Node load prediction result: The predicted value of the future load state of the node after fusion.

[0021] Considering that distributed node load is influenced by both temporal variation patterns and inter-node relationships, a single feature or model is insufficient for accurate prediction. Therefore, this embodiment extracts temporal features and correlation features separately, allowing for a focus on core influencing factors from different dimensions and avoiding feature interference. The LSTM model is well-suited for capturing long dependencies in time-series data, while the GCN model aligns with the network topology correlation characteristics of distributed nodes. Feature selection in this embodiment reduces redundancy overhead, and weighted fusion combines the advantages of both models, overcoming the limitations of a single model. Ultimately, this improves the accuracy and reliability of node load prediction, providing high-quality data support for subsequent scheduling decisions.

[0022] For example, a data acquisition agent deployed across distributed nodes collects computing resource data and task execution status data in real time at a frequency of 1-5 seconds per acquisition. The computing resource data includes CPU utilization, memory usage, GPU computing power utilization, network bandwidth utilization, and disk I / O rate; the task execution status data includes task type, execution progress, queue length, and computing power requirements, ensuring the integrity and real-time nature of the collected data.

[0023] This embodiment can sort the collected computing resource data by timestamp to construct a single-dimensional time series for each node. This embodiment can employ a sliding window method, setting the window size to 60 collection points and the step size to 10 collection points, extracting features such as the mean, variance, peak value, and trend slope of the data within each window to form a subset of time-series features for each node. For example, this embodiment can extract the mean and peak values ​​of each window within the past hour for the CPU utilization data of a certain node using a sliding window, forming a subset of CPU time-series features for that node.

[0024] This embodiment can analyze the correlation between computing resource data and task running status data of the same node, and identify strong correlation features through correlation analysis, such as the positive correlation between GPU-intensive tasks and node GPU utilization, and the correlation between IO-intensive tasks and disk IO rate. This embodiment can construct a node association graph with distributed nodes as vertices and network connections and data interaction relationships between nodes as edges, extract features such as data interaction frequency and bandwidth usage correlation between nodes, and integrate them to form a subset of association features.

[0025] This embodiment can merge the temporal feature subset and associated feature subset of each node according to the node identifier to form the initial feature dataset of each node. This embodiment can use the variance threshold method to remove invalid features with too small variance, such as constant value features; this embodiment can use the mutual information method to calculate the correlation between each feature and the load prediction target, and retain features with mutual information values ​​higher than a preset threshold; this embodiment can use the variance inflation factor method to eliminate multicollinearity features, and finally obtain the target feature dataset.

[0026] This embodiment can split the target feature dataset into time-series feature-related data and node-related feature-related data based on feature type. Time-series feature-related data includes all features that reflect changes in the time dimension, such as the time series of CPU utilization and memory usage of each node in different time periods; node-related feature-related data includes all features that reflect relationships, such as the relationship between task type and GPU utilization, and the frequency of data interaction between nodes.

[0027] The time-series characteristic data is the time-series monitoring data of distributed node computing resources. Specifically, it can include computing resource indicators such as CPU utilization (percentage), GPU computing power utilization (percentage), memory occupancy (percentage), and disk IO rate (MB / s) collected for each node every second. It is organized according to the "node-time step" dimension. For example, the time-series characteristic data of node A is: [CPU utilization (t-99), GPU utilization (t-99), memory occupancy (t-99), IO rate (t-99), ..., CPU utilization (t), GPU utilization (t), memory occupancy (t), IO rate (t)], where t is the current time and t-99 is the time 99 seconds ago. This data represents the real-time change status of multi-dimensional computing resources of a single node within 100 consecutive seconds.

[0028] After receiving the aforementioned time-series feature-related data, the input layer first performs normalization processing, mapping different units of computing power indicators to the 0-1 range. For example, 50% CPU utilization is mapped to 0.5, and 100MB / s IO rate (maximum value 200MB / s) is mapped to 0.5. This is used to eliminate the differences in the units of indicators such as CPU, GPU, and IO, and to prevent a certain indicator from dominating model learning due to its large numerical range. Subsequently, the normalized data is reshaped into a three-dimensional time-series vector of [number of nodes, time step (100), feature dimension (4)]. The "number of nodes" corresponds to all computing power nodes in the distributed cluster, the "time step 100" corresponds to a continuous 100-second acquisition window, and the "feature dimension 4" corresponds to the four computing power indicators of CPU, GPU, memory, and IO. This vector represents the multi-dimensional computing power resource status sequence of each node in the cluster within a fixed time window, which is the basis for the model to learn the time-series rules.

[0029] The LSTM model consists of an input layer, three hidden layers, and an output layer. The input layer receives temporal feature data and converts it into a fixed-length temporal vector. Each hidden layer contains 128 neurons and uses the ReLU activation function. The temporal vector is processed step-by-step through gating mechanisms (input gate, forget gate, and output gate), as follows: The input gate is used to calculate the weight value (0-1) of the time series vector at the current time step (e.g., time t), filter effective computing power information (e.g., effective data of sudden increase in GPU utilization at node t), and filter invalid noise (e.g., 0.1% fluctuation caused by acquisition error) in order to identify computing power status data that have a real impact on changes in node load. The forget gate is used to calculate the forgetting weight of the time series vector at historical time steps (such as t-50 time), discard meaningless historical data (such as random fluctuation data when the node is idle), and retain key historical patterns (such as the periodic characteristics of the daily increase in computing power load at 9 o'clock), so as to eliminate redundant historical information and focus on the time series patterns that have predictive value for future load. The output gate integrates the current information filtered by the input gate with the historical information retained by the forget gate, and outputs the feature representation of the time step. The three hidden layers are stacked one after another to capture long-term time-series dependencies at different time scales, such as the load increase trend at the 10-minute level and the periodic wave at the 1-hour level, so as to mine the changing trend and periodic characteristics of the node's computing power load over time, avoid gradient vanishing and ensure that long-term periodic patterns are not lost.

[0030] The output layer uses a linear activation function to map the feature representation output by the hidden layer into a single-dimensional value (0-1 interval). This value is the temporal load prediction component, which predicts the overall load value of the node in the next 10 minutes based solely on the temporal pattern of the node's own computing power resources (which can be reverse-normalized to restore the actual load percentage, such as 0.7 corresponding to 70% overall load). This reflects the future load level dominated by the node's own temporal computing power characteristics.

[0031] During training, the actual load data of historical nodes are used as labels (the actual comprehensive load percentage of the corresponding time period in the history of the node, i.e. the baseline value predicted by the model). The Adam optimization algorithm is used to adjust parameters such as gating weights and neuron biases round by round. The training is iterated for about 100 rounds until the loss function converges. After convergence, the model can stably output load prediction values ​​that conform to the temporal patterns of the nodes themselves.

[0032] The GCN model consists of an input layer, two hidden layers, and an output layer. The node association feature data refers to the computational power association features between distributed nodes. Specifically, it includes each node's computational power redundancy data (such as the number of remaining CPU cores and GPU computing power), task running status association features (such as the proportion of interaction between the tasks undertaken by the node and the tasks of other nodes), and the network bandwidth utilization rate between nodes (reflecting the tightness of data interaction). This node association feature data is used to quantify the computational power interaction relationship between a single node and other nodes in the cluster, as well as its ability to adapt to tasks. The adjacency matrix of the node association graph is an N×N matrix (N is the number of nodes), and the matrix element A_ij∈[0,1] represents the computational power interaction strength between node i and node j. For example, A_ij=0.8 indicates that the frequency of node i transmitting task data to node j accounts for 80% of the total transmission volume, which is used to quantify the tightness of the computational power association between nodes.

[0033] This embodiment can map node association feature data into a 3-dimensional node feature vector (quantified value of computing power association features of a single node), such as [CPU redundancy 0.6, task interaction ratio 0.7, bandwidth utilization 0.5]; and then combine it with the adjacency matrix to construct graph structure data, in the format of (node ​​feature matrix, adjacency matrix), that is, integrate the computing power association features of distributed nodes with the topological interaction relationship between nodes to form graph data that GCN can process, retaining the dual information of "node self-features + inter-node associations".

[0034] After receiving the graph structure data, the input layer standardizes the node feature matrix to eliminate numerical differences in the computational power correlation features between different nodes. For example, node A has a task interaction ratio of 0.9, while node B has 0.2. Standardization allows the model to learn the correlation features of different nodes fairly. At the same time, the adjacency matrix is ​​normalized to eliminate feature aggregation bias caused by differences in node degree, ensuring that the correlation features of each node are mined fairly.

[0035] There are two hidden layers, each with 64 neurons, using the Sigmoid activation function. The core processing involves feature aggregation of neighboring nodes, as detailed below: The first hidden layer is used to sum the feature vectors of all its neighboring nodes j for each node i by weighting the normalized adjacency matrix. For example, the neighboring nodes j1 and j2 of node i have weights of 0.7 and 0.3 respectively. Then, it is concatenated with the feature vector of node i itself and output as a fused feature after Sigmoid activation. This is to explore the influence of the computing power status of directly adjacent nodes on the load of node i. For example, if node j1 is saturated with load, migrating tasks to node i will cause the load of i to increase. The second hidden layer is used to repeat the "adjacency aggregation + activation" process on the fusion features output by the first layer in order to explore the potential impact of the load of node i on the indirectly adjacent nodes (such as the adjacent node k of j1) and capture the cluster-level computing power correlation pattern, such as the chain correlation of node k task surge → j1 load increase → i load increase.

[0036] The output layer uses a linear activation function to map the fusion features of the second hidden layer into a single-dimensional value (0-1 interval), which is the associated load prediction component. This achieves the prediction of a node's comprehensive load value for the next 10 minutes based solely on the computing power correlation features between nodes, reflecting the influence of other nodes in the cluster on that node's load.

[0037] During training, the actual load data of historical nodes are also used as labels. The adjacency aggregation weights and neuron parameters are adjusted through the backpropagation algorithm and iterated until the loss function converges. After convergence, the model can capture the influence of the potential computing power correlation between nodes on the load.

[0038] This embodiment effectively eliminates redundant information, reduces computational overhead, and improves feature quality by extracting and filtering temporal and correlation features separately. The LSTM model accurately captures the temporal dependencies of load, while the GCN model fully explores the correlation characteristics between nodes. After weighted fusion, the advantages of both types of features are taken into account, significantly improving the accuracy and reliability of node load prediction. This embodiment is adapted to the node and task characteristics of distributed systems, providing high-quality data support for subsequent task priority allocation and node group assignment, solving the problem of inaccurate predictions by single features or models, and ensuring the scientific and rational nature of scheduling decisions.

[0039] S102: Determine task priority based on node load prediction results, task computing power requirement data, and task running status data; determine the node group allocation result of the task based on node load prediction results, computing power resource data, and node configuration data.

[0040] In this embodiment, task priority is determined based on node load prediction results, task computing power requirement data, and task running status data, including: The task's real-time requirements and completion time limit are obtained based on the task's running status data; The scale of computing power demand is obtained based on the task's computing power demand data; The load trend characteristics of nodes are obtained based on the node load prediction results; The priority score of a task is calculated based on its real-time requirements, completion time, computing power requirements, and node load trend characteristics. Task priority is determined based on priority scores.

[0041] In this embodiment, the node group allocation result for the task is determined based on the node load prediction result, computing resource data, and node configuration data, including: The hardware capabilities of each node are classified based on the node configuration data to obtain the hardware capability level of each node. The allocation result of the first node group is determined based on the hardware capability level of each node; Redundancy analysis is performed on the computing power resource data to obtain the computing power redundancy data of each node; Determine the future load risk level of each node based on the node load prediction results; For each node, the node group load adaptation score data is determined based on the node's computing power redundancy data and future load risk level. The node group allocation result for the task is determined based on the first node group allocation result and the node group load adaptation score data of each node.

[0042] In this embodiment, task computing power requirement data refers to the quantitative information of hardware resources consumed by task execution. For example, a model training task requires 80 CPU cores, 32GB of memory, and 2 GPUs. Task real-time requirements refer to the required execution response speed of the task. For example, real-time data transmission tasks require millisecond-level response, while offline data analysis tasks have no strict real-time requirements. Task completion deadlines refer to the deadlines that tasks must complete. For example, a report generation task must be completed within 24 hours, while an emergency processing task must be completed within 1 hour. Computing power requirement scale refers to the size level of the task's computing power requirement. For example, a large-scale simulation task requires ultra-large-scale computing power, while a small file processing task requires small-scale computing power. Load trend characteristics refer to the future load change trend of the node. For example, the node load prediction result shows a continuous upward trend or a stable trend in the next hour. Priority score refers to a quantitative value representing the task execution priority calculated by comprehensively considering multiple factors. For example, an urgent task scores 95 points, while a normal task scores 60 points. Node configuration data refers to the inherent parameter information of the node hardware, such as the number of CPU cores, GPU model, memory capacity, and disk storage size.

[0043] Hardware capability grading refers to the hardware performance level classified according to node configuration data. For example, nodes are divided into high-end, mid-range, and low-end levels based on the number of CPU cores and GPU computing power. Hardware capability level refers to the grading result of node hardware performance. For example, a node configured with 64 CPU cores and 4 high-end GPUs has a high hardware capability level. First node group allocation result refers to the task-node group correspondence based only on the initial classification of hardware capability levels. For example, tasks requiring ultra-large-scale computing power are initially allocated to the high-end node group. Redundancy analysis refers to the analysis process of the remaining available computing power resources of a node. For example, redundancy is analyzed by the difference between currently occupied resources and total configured resources. Computing power redundancy data refers to the data of currently unoccupied and allocable computing power resources of a node. For example, a node has a total of 128 CPU cores, 48 ​​of which are currently occupied, and the computing power redundancy data is 80 CPU cores. Future load risk level refers to the degree of future overload risk of a node determined based on load prediction results. For example, a predicted load exceeding 80% is high risk, 50%-80% is medium risk, and below 50% is low risk. Node group load matching score data refers to the quantitative score of the matching task derived from the combination of node computing power redundancy and future load risk. For example, a node with sufficient redundancy and low risk scores 85 points; a node with insufficient redundancy and high risk scores 30 points. Node group allocation result refers to the final correspondence between the determined tasks and the matching node groups. For example, assigning high-end computing power demand tasks to the low-risk node set in the high-end node group.

[0044] Task prioritization and node group allocation are core prerequisites for distributed computing power scheduling, and their scientific validity directly determines the scheduling effect. Relying on a single factor to determine priority or node group can easily lead to scheduling imbalance. Therefore, this embodiment considers a comprehensive decision-making process that combines the task's own attributes (real-time requirements, completion deadlines, and computing power needs) with the node's status (load trends, computing resources, and hardware configuration) to ensure that priorities align with the urgency and needs of the tasks, and that node groups are adapted to node capabilities and load risks. This embodiment first performs preliminary grouping based on hardware capabilities, and then optimizes this by combining load adaptation scoring. This ensures a basic match between hardware and task requirements while mitigating the risk of future node overload. Multi-dimensional priority score calculation avoids misjudgments caused by a single factor, laying a reliable foundation for subsequent accurate scheduling.

[0045] For example, this embodiment can obtain task computing power requirement data from the task management system to clarify the resource specifications such as CPU, GPU, and memory required by the task. Based on the parameter configuration at the time of task submission, this embodiment can extract the task's real-time requirements and completion time limit, for example, labeling a task as a real-time response type with a completion time limit of 3 hours. This embodiment can classify computing power requirement scale levels based on the quantitative indicators of the task computing power requirement data. For example, it can define a CPU requirement ≥64 cores and a GPU requirement ≥4 as ultra-large scale, a CPU requirement 24-64 cores and a GPU requirement 1-2 as large scale, and a CPU requirement <24 cores and no GPU requirement as small to medium scale. This embodiment can analyze the load change trend of each node over a future preset time period (e.g., 1 hour) from the node load prediction results, classifying the load trend into three categories: continuously rising, stable, and continuously falling. For example, if the future load of node A increases from 50% to 75%, its trend characteristic is continuously rising.

[0046] This embodiment allows setting weights for task real-time requirements (30%), task completion time (30%), computing power demand (20%), and node load trend characteristic adaptability (20%). Each dimension can be quantitatively scored; for example, millisecond-level real-time requirements score 100 points, second-level requirements score 80 points; completion time ≤ 1 hour scores 100 points, 1-6 hours scores 80 points; ultra-large-scale computing power demand scores 100 points, large-scale demand scores 80 points; node load trend and task execution adaptability score 100 points, basic adaptability scores 70 points. The scores of each dimension are weighted and summed to obtain the task priority score. This embodiment can classify tasks according to their priority scores; for example, a score ≥ 90 points indicates Level 1 priority (highest), 70-89 points indicates Level 2 priority, 50-69 points indicates Level 3 priority, and < 50 points indicates Level 4 priority. The final priority of each task is determined based on its score.

[0047] This embodiment can acquire node configuration data for all distributed nodes, select three core indicators—CPU core count, GPU computing power, and memory capacity—and set tiered thresholds. For example, high-end nodes require CPU ≥ 64 cores, GPU computing power ≥ 50 TFLOPS, and memory ≥ 128 GB; mid-range nodes require CPU 32-64 cores, GPU computing power 20-50 TFLOPS, and memory 64-128 GB; and low-end nodes require CPU < 32 cores, GPU computing power < 20 TFLOPS, and memory < 64 GB. The hardware capability level of each node is determined based on the compliance of these indicators. This embodiment can initially allocate node groups based on the compatibility between the computing power requirements of the task and the hardware capability level. For example, ultra-large-scale and large-scale computing power demand tasks are allocated to the high-end node group, while small-to-medium-scale computing power demand tasks are allocated to the mid-to-low-end node group, forming the first node group allocation result.

[0048] This embodiment can acquire the computing power resource data of each node. By subtracting the currently occupied resources from the total configured resources, redundant resources in dimensions such as CPU, GPU, and memory are calculated. For example, node B has a total of 64 CPU cores, with 24 cores currently occupied, resulting in a CPU computing power redundancy of 40 cores; and a total of 128GB of memory, with 48GB currently occupied, resulting in a memory redundancy of 80GB. The computing power redundancy data of each node is then integrated. This embodiment can set load thresholds based on the node load prediction results. For example, a load ≥80% is considered high risk, 50%-80% is medium risk, and <50% is low risk. By comparing the predicted load value of each node, its future load risk level is determined.

[0049] This embodiment can set a weight of 60% for computing power redundancy data and 40% for future load risk level. Computing power redundancy is scored according to its sufficiency: 100 points for fully meeting the computing power requirements of the corresponding task, 80 points for basically meeting them, and 50 points for partially meeting them; 100 points for low risk, 60 points for medium risk, and 30 points for high risk. The weighted sum is then used to obtain the node group load adaptation score data for each node. This embodiment can combine the first node group allocation result with the selection of nodes within the initially allocated node groups whose node group load adaptation score data is higher than a preset threshold (e.g., 60 points), forming the final task-node group correspondence, i.e., the task's node group allocation result.

[0050] This embodiment determines task priorities by comprehensively considering task real-time performance, completion deadlines, computing power requirements, and node load trends, avoiding misjudgments due to a single factor and ensuring more comprehensive and accurate priority allocation. When determining node group allocation results, a hardware capability-based hierarchical approach is first established to ensure compatibility. Then, computing power redundancy and future load risk optimization are considered, balancing hardware compatibility and load security to avoid resource waste or node overload caused by task-node group mismatch. The overall process scientifically coordinates multi-dimensional key information about tasks and nodes, providing a reliable basis for subsequent target execution node allocation and effectively improving the rationality and efficiency of distributed computing power scheduling.

[0051] S103: Determine the target execution node for each task based on task priority, task node group allocation results, and node load data; the target execution node is used to call node hardware resources to execute the task according to the task computing power requirement data.

[0052] In this embodiment, the target execution node for each task is determined based on task priority, task node group allocation results, and node load data, including: The load level of each node is classified based on the node load data to obtain the node load level of each node. The candidate execution node pool is determined based on the node group allocation results of the task; Candidate nodes are selected from the pool of candidate execution nodes based on the node load level of each node. The target execution node for a task is selected from the candidate nodes based on task priority.

[0053] In this embodiment, node load data refers to the real-time occupancy status of the distributed nodes' current hardware resources, such as CPU real-time utilization of 70%, memory real-time utilization of 65%, and GPU real-time utilization of 60%. Load level grading refers to the process of classifying load occupancy levels based on node load data, such as classifying levels according to resource utilization ranges. Node load level refers to the result of load level grading, such as high load, medium load, and low load levels. The candidate execution node pool refers to the set of nodes suitable for the task determined based on the node group allocation results, such as the set of all high-end nodes corresponding to the node group allocation results for a certain task. Candidate nodes refer to the nodes remaining after load filtering from the candidate execution node pool, such as low-load level nodes in the candidate execution node pool. The target execution node refers to the node ultimately determined to undertake task execution, such as a low-load high-end node selected for a high-priority task.

[0054] Determining the target execution node is a crucial step in scheduling, requiring a balance between task compatibility and node operational stability. Directly selecting from all nodes can easily lead to inefficient matching or node overload. Therefore, this embodiment first locks down the candidate execution node pool based on node group allocation results, ensuring basic compatibility between nodes and tasks; then, it filters nodes according to load data, excluding high-load nodes to avoid overload risks; this embodiment also incorporates task priority allocation to ensure high-priority tasks receive high-quality node resources, achieving a balance between compatibility, efficiency, and stability.

[0055] For example, this embodiment can collect node load data from all distributed nodes and set load grading thresholds. For instance, a real-time resource utilization rate below 50% is considered a low load level, 50% to 80% is a medium load level, and above 80% is a high load level. By comparing the real-time load data of each node, the node load level corresponding to each node is determined, thus completing the load level grading. This embodiment can also obtain the node group allocation result of a task and include all nodes in the corresponding node group to form a candidate execution node pool. For example, if the node group allocation result of a task is a mid-range node group, then all nodes in the mid-range node group constitute the candidate execution node pool for that task.

[0056] This embodiment can traverse each node in the candidate execution node pool and query its load level. High-load nodes are removed, and low- and medium-load nodes are retained to form a candidate node set. For example, if there are 10 nodes in the candidate execution node pool, 3 high-load nodes are removed, leaving 7 candidate nodes. This embodiment can assign candidate nodes to each task in descending order of task priority. High-priority tasks are first selected from low-load nodes in the candidate node pool; if there are not enough low-load nodes, then medium-load nodes are selected; low-priority tasks are selected sequentially from the remaining candidate nodes, ultimately determining the target execution node for each task.

[0057] This embodiment locks the range of suitable nodes based on the node group allocation results, combines load grading to filter low-load nodes, and allocates target execution nodes according to priority. This not only ensures the basic compatibility between tasks and nodes, but also avoids node overload, ensuring that high-priority tasks get high-quality resources first, improving scheduling accuracy and execution efficiency, and ensuring stable system operation.

[0058] S104: During task execution, obtain the target node load and task execution delay data of the target execution node; if the target node load exceeds the load threshold and / or the task execution delay data exceeds the delay range, adjust the cluster computing power configuration based on the node configuration data and cluster idle resource data, and migrate the tasks on the target execution node based on the task priority.

[0059] In this embodiment, the cluster computing power configuration is adjusted based on node configuration data and cluster idle resource data, and tasks on the target execution node are migrated based on task priority, including: Determine the adjustable threshold for node resources based on node configuration data; Cluster scalable resource data is obtained based on cluster idle resource data; Based on the cluster's scalable resource data and the adjustable threshold of node resources, the computing power configuration is adjusted to obtain the cluster's computing power optimization configuration scheme. Based on task priority, the task data on the target execution node is filtered to obtain the target migration task list; Determine the target migration node list based on the cluster computing power optimization configuration scheme; Task migration is performed based on the target migration task list and the target migration node list.

[0060] In this embodiment, target node load refers to the real-time hardware resource occupancy status of the target execution node during task execution, such as a real-time CPU utilization of 85% and a real-time GPU utilization of 90%. Task execution latency data refers to the difference between the actual time taken from task startup to the current progress and the preset time, such as a preset execution time of 10 minutes and an actual execution time of 15 minutes. Load threshold refers to a pre-set upper limit standard for node load, such as a CPU utilization of 80% as the load threshold. Latency range refers to the allowed execution time range of the task, such as a latency range of 0-5 minutes for a certain task. Cluster idle resource data refers to the total unused computing resources of all nodes in the distributed cluster, such as 300 idle CPU cores and 500GB of memory in the cluster.

[0061] Node resource adjustability threshold refers to the upper limit of resources that a node can dynamically allocate, such as a CPU adjustability threshold of 30% of the total number of cores for a node. Cluster scalable resource data refers to the quantitative information on available resources that can be added or allocated in the cluster, such as the ability to temporarily expand by 200 CPU cores. Computing power configuration adjustment refers to the operation of reallocating cluster computing power resources. Cluster computing power optimization configuration scheme refers to the optimized cluster computing power allocation scheme, such as allocating idle resources to high-load node regions. Task data refers to configuration, progress, and other information related to task execution. Target migration task list refers to the set of tasks to be migrated, filtered by priority, such as tasks with priority level three and below on the target execution node. Target migration node list refers to the set of nodes adapted to the migration tasks in the cluster computing power optimization configuration scheme. Task migration refers to the process of transferring target migration tasks from the original node to the target migration node.

[0062] During task execution, node load and task latency are susceptible to dynamic factors. Failure to adjust in a timely manner can lead to node overload, task timeouts, and compromised system stability. Therefore, it is necessary to monitor key indicators in real time and dynamically optimize when anomalies are triggered. This embodiment first clarifies the adjustable boundaries of node resources and the idle resources of the cluster to ensure that computing power configuration adjustments are compliant and supported by resources. This embodiment selects migration tasks based on task priority to ensure that high-priority tasks are not affected. This embodiment combines optimization schemes to determine migration nodes, ensuring that tasks are adapted to the new nodes after migration. This embodiment balances resource utilization efficiency and task priority, avoiding single-node overload and task execution anomalies through dynamic adjustment and scientific migration, thus ensuring the overall stability of the cluster and the effectiveness of task execution.

[0063] For example, a monitoring agent deployed on the target execution node collects target node load and task execution latency data every 2 seconds and uploads it synchronously to the cluster scheduling center. The scheduling center compares the target node load with a preset load threshold and the task execution latency data with a preset latency range. If the target node load exceeds the load threshold or the task execution latency data exceeds the latency range, it determines that an adjustment mechanism needs to be triggered. The scheduling center calls the node configuration data and, combined with the node's hardware performance limit and the resources required for basic operation, sets an adjustable threshold for node resources. For example, if a node has a total of 128 CPU cores and requires 32 cores for basic operation, then the adjustable threshold for node resources is set to 96 cores, meaning that a maximum of 96 CPU cores can be allocated for load balancing.

[0064] The scheduling center aggregates the computing resource data and current occupancy data of all distributed nodes, calculates the idle resources of each node, and accumulates them to obtain the cluster's idle resource data. Combined with the cluster's elastic scaling capability, faulty nodes and reserved resources are removed, and resources that can be allocated immediately are selected to form the cluster's scalable resource data. For example, it might show that the cluster can be expanded to have 200 CPU cores and 300GB of memory. The scheduling center then combines this scalable resource data with the adjustable thresholds of node resources to plan resource allocation strategies. For example, 100 CPU cores and 150GB of memory from the cluster's scalable resources might be allocated to the node group where the target execution node resides, while simultaneously limiting the target execution node's resource occupancy to not exceed its adjustable threshold, thus forming a cluster computing power optimization configuration scheme that balances resource utilization and node security.

[0065] The scheduling center obtains the task priorities and task data of all tasks on the target execution node, sorts them from low to high priority, and prioritizes filtering low-priority tasks. For example, it filters out level 4 and level 3 priority tasks, excluding level 1 and level 2 high-priority tasks, forming a target migration task list to ensure the continuous and stable execution of high-priority tasks. Based on the cluster computing power optimization configuration scheme, it selects nodes from the cluster that meet the computing power requirements of the target migration task and whose current load is at a low to medium level. For example, for the small-to-medium scale computing power requirements of the target migration task, it selects nodes with compatible hardware capabilities and a load below 50%, forming a target migration node list, and initially matches suitable nodes for each migration task.

[0066] The scheduling center sends migration commands to the target execution node and the target migration node. The target execution node suspends the execution of the target migration task, packages the task data and execution progress information, and transmits it to the corresponding target migration node through the high-speed network within the cluster. After receiving the data, the target migration node resumes the task execution state based on the task data, allocates hardware resources according to the cluster's computing power optimization configuration scheme, and completes the entire task migration process. After the migration is completed, the scheduling center continuously monitors the load of the target execution node and the execution status of the migration task to ensure the adjustment effect.

[0067] This embodiment effectively avoids node overload and task timeouts by real-time monitoring of node load and task latency, triggering timely dynamic adjustments. A computing power optimization plan is formulated based on node configuration and cluster idle resources to ensure scientific and compliant resource allocation and improve cluster resource utilization. Migrating tasks are selected according to task priority to ensure that high-priority tasks are not affected, balancing scheduling fairness and efficiency. By accurately matching migration tasks with target nodes, the risk of task migration and the probability of interruption are reduced, significantly improving the operational stability and task execution reliability of the distributed computing power system, providing continuous and efficient computing power support for complex tasks.

[0068] Corresponding to the distributed computing power dynamic scheduling method in the above embodiment, Figure 2 This is a structural block diagram of a distributed computing power dynamic scheduling device provided in one embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. References Figure 2 The distributed computing power dynamic scheduling device 20 includes: a load prediction module 21, a node group allocation module 22, a node allocation module 23, and a dynamic scheduling module 24.

[0069] The load prediction module 21 is used to obtain a target feature dataset based on the computing power resource data and task running status data of the distributed nodes, and to obtain the node load prediction result based on the target feature dataset; the distributed nodes are multiple nodes; The node group allocation module 22 is used to determine task priority based on node load prediction results, task computing power requirement data, and task running status data; and to determine the node group allocation results of the task based on node load prediction results, computing power resource data, and node configuration data. The node allocation module 23 is used to determine the target execution node for each task based on task priority, the node group allocation result of the task, and node load data; the target execution node of the task is used to call node hardware resources to execute the task according to the task computing power requirement data. The dynamic scheduling module 24 is used to obtain the target node load and task execution delay data of the target execution node during task execution; if the target node load exceeds the load threshold and / or the task execution delay data exceeds the delay range, the cluster computing power configuration is adjusted based on the node configuration data and the cluster idle resource data, and the tasks on the target execution node are migrated based on the task priority.

[0070] In one embodiment of this application, when the load prediction module 21 obtains the target feature dataset based on the computing power resource data and task running status data of distributed nodes, it is specifically used for: Temporal features are extracted from the computing resource data of distributed nodes to obtain a subset of temporal features; The correlation feature analysis and extraction are performed on the computing resource data and task running status data of distributed nodes to obtain a subset of correlation features; An initial feature dataset is obtained based on a subset of temporal features and a subset of related features. Feature filtering is then performed on the initial feature dataset to obtain the target feature dataset.

[0071] In one embodiment of this application, when the load prediction module 21 obtains the node load prediction result based on the target feature dataset, it is specifically used for: The target feature dataset is divided into time-series feature-related data and node-related feature-related data. Based on time-series characteristic data, the time-series load prediction component is obtained through an LSTM model; Based on node association feature-related data, the associated load prediction component is obtained through the GCN model; The time-series load prediction component and the associated load prediction component are weighted and fused to obtain the node load prediction result.

[0072] In one embodiment of this application, when determining task priority based on node load prediction results, task computing power requirement data, and task running status data, the node group allocation module 22 is specifically used for: The task's real-time requirements and completion time limit are obtained based on the task's running status data; The scale of computing power demand is obtained based on the task's computing power demand data; The load trend characteristics of nodes are obtained based on the node load prediction results; The priority score of a task is calculated based on its real-time requirements, completion time, computing power requirements, and node load trend characteristics. Task priority is determined based on priority scores.

[0073] In one embodiment of this application, when determining the node group allocation result for a task based on node load prediction results, computing resource data, and node configuration data, the node group allocation module 22 is specifically used for: The hardware capabilities of each node are classified based on the node configuration data to obtain the hardware capability level of each node. The allocation result of the first node group is determined based on the hardware capability level of each node; Redundancy analysis is performed on the computing power resource data to obtain the computing power redundancy data of each node; Determine the future load risk level of each node based on the node load prediction results; For each node, the node group load adaptation score data is determined based on the node's computing power redundancy data and future load risk level. The node group allocation result for the task is determined based on the first node group allocation result and the node group load adaptation score data of each node.

[0074] In one embodiment of this application, when determining the target execution node for each task based on task priority, the node group allocation result of the task, and node load data, the node allocation module 23 is specifically used for: The load level of each node is classified based on the node load data to obtain the node load level of each node. The candidate execution node pool is determined based on the node group allocation results of the task; Candidate nodes are selected from the pool of candidate execution nodes based on the node load level of each node. The target execution node for a task is selected from the candidate nodes based on task priority.

[0075] In one embodiment of this application, when the dynamic scheduling module 24 adjusts the cluster computing power configuration based on node configuration data and cluster idle resource data, and performs task migration on the target execution node based on task priority, it is specifically used for: Determine the adjustable threshold for node resources based on node configuration data; Cluster scalable resource data is obtained based on cluster idle resource data; Based on the cluster's scalable resource data and the adjustable threshold of node resources, the computing power configuration is adjusted to obtain the cluster's computing power optimization configuration scheme. Based on task priority, the task data on the target execution node is filtered to obtain the target migration task list; Determine the target migration node list based on the cluster computing power optimization configuration scheme; Task migration is performed based on the target migration task list and the target migration node list.

[0076] See Figure 3 , Figure 3 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 3 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of the modules in the aforementioned device embodiments, for example... Figure 2The functions of the load prediction module 21, node group allocation module 22, node allocation module 23, and dynamic scheduling module 24 are shown.

[0077] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0078] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0079] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory. For example, the memory 304 may also store information about task data.

[0080] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation methods described in the embodiments of the distributed computing power dynamic scheduling method provided in the embodiments of this application, or they can execute the implementation methods of the electronic device 300 described in the embodiments of this application, which will not be repeated here.

[0081] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0082] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD) card, flash card, etc., equipped on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0083] Those skilled in the art will recognize that the modules / units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0084] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0085] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules, units, or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces or modules / units, or it may be an electrical, mechanical, or other form of connection.

[0086] The modules / units described as separate components may or may not be physically separate. Similarly, the components shown as modules / units may or may not be physical modules / units; they may be located in one place or distributed across multiple network modules / units. Some or all of the modules / units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0087] Furthermore, the functional modules / units in the various embodiments of this application can be integrated into one processing module / unit, or each module / unit can exist physically separately, or two or more modules / units can be integrated into one module / unit. The integrated modules / units described above can be implemented in hardware or in the form of software functional modules / units.

[0088] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for dynamic scheduling of distributed computing power, characterized in that, include: A target feature dataset is obtained based on the computing power resource data and task running status data of distributed nodes, and node load prediction results are obtained based on the target feature dataset. The distributed node consists of multiple nodes; Based on the task running status data, the task real-time requirements and task completion time limit are obtained; The scale of computing power demand is obtained based on the task's computing power demand data; The load trend characteristics of the nodes are obtained based on the node load prediction results; The priority score of the task is calculated based on the task's real-time requirements, the task's completion time limit, the computing power requirement, and the node's load trend characteristics. Task priority is determined based on the priority score; The hardware capabilities of each node are classified based on the node configuration data to obtain the hardware capability level of each node. The allocation result of the first node group is determined based on the hardware capability level of each node; Redundancy analysis is performed on the computing power resource data to obtain the computing power redundancy data of each node; Determine the future load risk level of each node based on the node load prediction results; For each node, the node group load adaptation score data is determined based on the node's computing power redundancy data and future load risk level. The node group allocation result of the task is determined based on the first node group allocation result and the node group load adaptation score data of each node. The target execution node for each task is determined based on the task priority, the node group allocation result of the task, and the node load data; the target execution node of the task is used to call the node hardware resources to execute the task according to the task computing power requirement data; During task execution, acquire target node load and task execution delay data for the target execution node; if the target node load exceeds the load threshold and / or the task execution delay data exceeds the delay range, determine the adjustable threshold for node resources based on the node configuration data; Cluster scalable resource data is obtained based on cluster idle resource data; Based on the cluster's scalable resource data and the node's adjustable resource threshold, the computing power configuration is adjusted to obtain a cluster computing power optimization configuration scheme; based on task priority, the task data on the target execution node is filtered to obtain a target migration task list; The target migration node list is determined based on the cluster computing power optimization configuration scheme. The task migration is performed based on the target migration task list and the target migration node list.

2. The distributed computing power dynamic scheduling method as described in claim 1, characterized in that, The target feature dataset is obtained from the distributed node-based computing resource data and task running status data, including: Temporal features are extracted from the computing resource data of distributed nodes to obtain a subset of temporal features; The correlation feature analysis and extraction are performed on the computing resource data and task running status data of distributed nodes to obtain a subset of correlation features; An initial feature dataset is obtained based on the temporal feature subset and the associated feature subset. Feature filtering is performed on the initial feature dataset to obtain the target feature dataset.

3. The distributed computing power dynamic scheduling method as described in claim 1, characterized in that, The process of obtaining node load prediction results based on the target feature dataset includes: The target feature dataset is divided into time-series feature-related data and node-related feature-related data; Based on the aforementioned time-series characteristic related data, the time-series load prediction component is obtained through an LSTM model; Based on the node association feature related data, the association load prediction component is obtained through the GCN model; The time-series load prediction component and the associated load prediction component are weighted and fused to obtain the node load prediction result.

4. The distributed computing power dynamic scheduling method as described in claim 1, characterized in that, The process of determining the target execution node for each task based on the task priority, the node group allocation result of the task, and the node load data includes: Based on the node load data, the load level of each node is classified to obtain the node load level of each node; The candidate execution node pool is determined based on the node group allocation results of the task; Based on the node load level of each node, nodes are filtered from the candidate execution node pool to obtain candidate nodes; The target execution node for a task is selected from the candidate nodes based on task priority.

5. A distributed computing power dynamic scheduling device using the distributed computing power dynamic scheduling method according to any one of claims 1-4, characterized in that, include: The load prediction module is used to obtain a target feature dataset based on the computing power resource data and task running status data of distributed nodes, and to obtain node load prediction results based on the target feature dataset. The distributed node consists of multiple nodes; The node group allocation module is used to determine task priority based on the node load prediction results, task computing power requirement data and task running status data. The node group allocation result for the task is determined based on the node load prediction result, the computing power resource data, and the node configuration data; The node allocation module is used to determine the target execution node for each task based on the task priority, the node group allocation result of the task, and the node load data; the target execution node of the task is used to call the node hardware resources to execute the task according to the task computing power requirement data; The dynamic scheduling module is used to acquire target node load and task execution delay data during task execution; if the target node load exceeds the load threshold and / or the task execution delay data exceeds the delay range, the adjustable threshold of node resources is determined based on the node configuration data. Cluster scalable resource data is obtained based on cluster idle resource data; Based on the cluster's scalable resource data and the node's adjustable resource threshold, the computing power configuration is adjusted to obtain a cluster computing power optimization configuration scheme; based on task priority, the task data on the target execution node is filtered to obtain a target migration task list; The target migration node list is determined based on the cluster computing power optimization configuration scheme. The task migration is performed based on the target migration task list and the target migration node list.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 4.

Images