Computing power network resource scheduling method

Abstract

The invention relates to a computing power network resource scheduling method. The method comprises the following steps: acquiring floating point operation performance parameters and operation states of node processors and energy index data of data centers where the node processors are located, and calculating to generate a node list; constructing a global resource pool based on the list, and generating a resource distribution table containing the total calculation power of the region; obtaining calculation requirements and time delay constraints of the task queue, extracting feature vectors in combination with the resource distribution table, and generating a resource utilization rate table; obtaining network link flow data, predicting a link congestion probability through a long short-term memory network, and generating a flow control strategy table; and finally, updating resource pool network constraints according to the resource utilization rate table and the flow control strategy table, and remapping tasks by taking node effective computing power as a weight to generate a scheduling execution scheme. According to the method, accurate quantitative evaluation of the computing power resources is realized, the matching precision of tasks and the computing power resources is effectively improved, the resource utilization rate of the computing power network can be remarkably improved, and the overall scheduling efficiency and stability of the system are enhanced.

Description

Technical Field

[0001] The present invention belongs to the field of computer science and technology, and in particular relates to a computing power network resource scheduling method. Background Art

[0002] With the rapid development of the digital economy, various compute-intensive tasks, such as VR, AR, and large-scale model training, have experienced explosive growth. These tasks are dispersed across diverse regions, and the demand for computing power and network resources is becoming increasingly complex and diverse. Computing networks, as a new type of information infrastructure for computing-network convergence, are centered on computing and based on the network. They enable on-demand allocation and flexible scheduling of computing, storage, and network resources across the cloud, edge, and end. However, computing networks encompass a vast array of resources, characterized by their dynamic, heterogeneous nature, and the rich variety of computing network services, resulting in a high degree of complexity. Furthermore, current computing networks often separate network scheduling from computing power scheduling, preventing a comprehensive computing network resource scheduling system and making it difficult to achieve a globally optimal solution for computing network resources. This severely impacts the overall operational efficiency of computing networks. Therefore, achieving multi-factor integrated scheduling of computing networks to improve resource utilization and service quality has become a major technical challenge facing the computing network field. Summary of the Invention

[0003] Based on this, it is necessary to provide a computing network resource scheduling method that can achieve accurate quantitative evaluation of computing resources, effectively improve the matching accuracy between tasks and computing resources, and enhance the overall scheduling efficiency and stability of the system to address the above technical problems.

[0004] In a first aspect, the present application provides a computing power network resource scheduling method, comprising:

[0005] Obtain the floating-point computing performance parameters, operating status, and energy index data of each node processor in the data center; calculate the data based on preset weights and standardized methods to generate a node list.

[0006] A global resource pool is built based on the node list, and an algorithm is used to divide each node into three types of computing power areas to generate a resource distribution table containing the total computing power of the area.

[0007] Obtain the computing requirements and latency constraints of the task queue and combine them with the resource distribution table, extract the feature vector, and use the deep reinforcement learning framework to build the task requirement-region computing power decision and generate the resource utilization table.

[0008] Obtain network link traffic data and use long short-term memory network to predict link congestion probability. If it exceeds the preset threshold, adjust the path selection to generate a flow control strategy table.

[0009] Update the resource pool network constraints according to the resource utilization table and flow control policy table, remap tasks based on the effective computing power of the nodes as weights, and generate a scheduling execution plan.

[0010] In one embodiment, data is calculated based on preset weights and a standardized method to generate a node list containing unified computing power values and resource utilization rates, including:

[0011] The floating-point operation peak performance parameter data is normalized according to the preset performance weight coefficient to obtain the standardized theoretical computing power value.

[0012] The running status data is dynamically attenuated according to the preset status weight coefficient to generate the actual computing power value after dynamic adjustment.

[0013] The theoretical computing power value and the actual computing power value are linearly superimposed to generate a node computing power matrix containing a unified computing power indicator.

[0014] Obtain the energy indicator data and computing power resource rate of the data center to which each node belongs, and calculate the comprehensive evaluation value of the data center; energy indicator data includes but is not limited to electricity price data, new energy utilization rate and green electricity utilization rate.

[0015] Nodes are screened based on the comprehensive evaluation value of the data center, and the screened nodes are combined with the node computing power matrix to generate a node list containing resource load levels and data center labels.

[0016] In one embodiment, a global resource pool is constructed based on the node list. An algorithm is used to divide each node into three computing power zones, and a resource distribution table containing the total computing power of each zone is generated, including:

[0017] Get the node computing power indicators in the node list; node computing power indicators include memory capacity and computing throughput.

[0018] Calculate the quantile threshold based on the node computing power indicator; the quantile threshold includes the memory capacity quantile and the computing throughput quantile.

[0019] Initialize the cluster centers of the K-means algorithm based on the quantile threshold; the cluster centers contain the initial values of the three computing power dimensions: high, medium, and low.

[0020] Iteratively cluster the node computing power indicators according to the cluster center to generate regional division results; the regional division results include the regional labels corresponding to each node.

[0021] Aggregate the total computing power of each region based on the regional tags to generate a resource distribution table containing the regional tags and the total computing power. The total computing power includes the total memory and the total computing power.

[0022] Among them, if the difference in total computing power between regions in the resource distribution table exceeds the preset threshold, the quantile threshold is readjusted and secondary clustering is triggered to obtain the secondary clustering result.

[0023] Based on the secondary clustering results, the regional labels and total computing power values of the resource distribution table are updated to obtain the final resource distribution table.

[0024] In one embodiment, the computing requirements and latency constraints of the task queue are obtained and combined with the resource distribution table, feature vectors are extracted, and a deep reinforcement learning model is used to construct a task requirement-region computing power decision, generating a resource utilization table, including:

[0025] Obtain the computing resource type and task priority of the task queue, combine the node computing power status and regional bandwidth capacity in the resource distribution table, extract the feature vector and generate the matching matrix between the task and the region.

[0026] If the variance of the eigenvector in the matching matrix exceeds the preset threshold, a deep network algorithm is used to dynamically adjust the task allocation ratio and generate a node mapping decision table with task requirements and regional computing power as the state space.

[0027] The resource utilization of each node is calculated according to the node mapping decision table. When it exceeds the preset range, low-priority tasks are migrated to low-load nodes to generate an optimized task distribution.

[0028] The dynamic weight allocation parameters are calculated based on the maximum allowable delay and historical delay fluctuation rate of task distribution, and input into the deep reinforcement learning model to generate the regional computing power decision table.

[0029] Based on the regional computing power decision table, task requirements and node resources are matched, and the load balancing coefficient is updated to generate a resource utilization table.

[0030] In one embodiment, the resource utilization of each node is calculated using the following formula:

[0031]

[0032] Among them, U i represents the resource utilization of node i, n represents the total number of resource types, r ij represents the current usage of resource type j on node i, c ij represents the total capacity of resource type j on node i, ω j represents the weight of resource type j, α j represents the sensitivity coefficient of resource type j, m represents the total number of tasks running on node i, and p k represents the priority of task k, t ik represents the current execution time of task k on node i, s ikrepresents the estimated total execution time of task k on node i, σ ik represents the historical execution time volatility of task k on node i, σ avg represents the cluster average volatility, b i represents the current network bandwidth usage of node i, B i represents the total network bandwidth capacity of node i, ρ i represents the network congestion probability of node i predicted based on the long short-term memory network.

[0033] In one embodiment, a long short-term memory network is used to predict the link congestion probability. If the probability exceeds a preset threshold, the path selection is adjusted to generate a flow control strategy table, including:

[0034] The initial link with the shortest physical path or the least number of hops is calculated based on the shortest path algorithm for the task source node and the target node in the network link traffic data.

[0035] The historical traffic sequence of the initial link is extracted and input into the long short-term memory network, which outputs the link congestion probability in the future time window.

[0036] If the link congestion probability exceeds a preset threshold, the link switching mechanism is triggered to select a suboptimal path from the alternative links and generate a flow control policy table of link path priorities and congestion prediction results.

[0037] In one embodiment, the resource pool network constraints are updated according to the resource utilization table and the flow control policy table, tasks are remapped with the effective computing power of the nodes as weights, and a scheduling execution plan is generated, including:

[0038] The node computing power parameters in the resource utilization table are weighted and integrated with the comprehensive evaluation value of the data center to obtain the node effective computing power weight.

[0039] A consistent hash ring is constructed based on the effective computing power weight of the nodes, and virtual nodes are allocated according to the weight ratio to quantify the node scheduling priority.

[0040] The link path priority and link congestion threshold are extracted from the flow control policy table to generate a network bandwidth matrix including the inter-node communication delay constraint.

[0041] The network bandwidth matrix is dynamically mapped and matched with the hash ring, and a task migration path table carrying load status and link constraints is generated by superposition.

[0042] According to the task migration path table, the virtual node distribution is optimized with the link congestion threshold as the boundary condition, and the scheduling execution plan including traffic balancing parameters is generated in combination with the data center priority.

[0043] In one embodiment, after generating the scheduling execution plan, the method further includes:

[0044] Obtain the execution delay rate and resource occupancy rate in the task feedback data after executing the scheduling execution plan.

[0045] The multi-dimensional scheduling feature vector is calculated based on the execution delay rate and resource occupancy rate.

[0046] The multi-dimensional scheduling feature vector is input into the weighted scoring model and the scheduling effect score is output.

[0047] Generate a dynamic optimization parameter set based on the scheduling effect score, and extract the reward function adjustment factor and computing power weight correction coefficient.

[0048] The weight distribution of the reinforcement learning model is updated according to the reward function adjustment factor to obtain the strategy evaluation matrix.

[0049] A scheduling decision tree for resource allocation is constructed based on the computing power weight correction coefficient and the strategy evaluation matrix.

[0050] Generate an iterative scheduling execution plan based on the scheduling decision tree.

[0051] In a second aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the above method when executing the computer program.

[0052] In a third aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the above method when executed by a processor.

[0053] The above-mentioned computing power network resource scheduling method, computer equipment and storage medium obtain the floating-point operation performance parameters, operating status and energy indicator data of each node processor and the data center where it is located, and calculate and generate a node list by preset weights combined with a standardized method; construct a global resource pool based on the list, use an algorithm to divide the nodes into three types of computing power areas, and generate a resource distribution table containing the total computing power of the area; obtain the computing requirements and delay constraints of the task queue, extract feature vectors in combination with the resource distribution table, use a deep reinforcement learning framework to construct a task requirement-regional computing power decision and generate a resource utilization table; obtain network link traffic data, predict the link congestion probability through a long short-term memory network, and if it exceeds a preset threshold, adjust the path selection to generate a flow control strategy table; finally, update the resource pool network constraints according to the resource utilization table and the flow control strategy table, remap tasks with the effective computing power of the node as the weight, and generate a scheduling execution plan. This method achieves accurate quantitative evaluation of computing power resources through multi-dimensional data collection and standardized processing; with the help of algorithm clustering and deep reinforcement learning strategies, it effectively improves the matching accuracy of tasks and computing power resources; utilizes network traffic prediction and dynamic control mechanisms to ensure the stability of data transmission; finally, it realizes task remapping based on multi-table collaborative updating and optimizes resource pool network constraints, which can significantly improve the utilization rate of computing power network resources, reduce task execution latency, enhance the overall scheduling efficiency and stability of the system, and provide a reliable solution for the efficient management of computing power network resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0055] Figure 1 A flowchart of a computing power network resource scheduling method provided by an embodiment of the present invention;

[0056] Figure 2 A flowchart of generating a node list by calculating data based on preset weights and a standardized method according to an embodiment of the present invention;

[0057] Figure 3 The embodiment of the present invention provides a flowchart for obtaining the computing requirements and delay constraints of the task queue combined with the resource distribution table, extracting feature vectors and using a deep reinforcement learning model to construct a task requirement-region computing power decision, and generating a resource utilization table. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0059] In one embodiment, Figure 1 As shown, the present application provides a computing power network resource scheduling method, which may include the following steps:

[0060] Step S101, obtain the floating-point computing performance parameters, operating status and energy index data of the data center of each node processor; calculate the data according to the preset weights and the standardized method to generate a node list.

[0061] Specifically, the floating-point performance parameters of each node processor (such as static indicators such as theoretical peak computing power and instruction cycles), operating status data (including real-time load information such as processor occupancy, memory occupancy, and temperature thresholds), and energy indicator data of the data center (such as electricity price data, new energy utilization rate, and green electricity utilization rate) are obtained. The multi-dimensional data is quantified using preset performance weight coefficients and status weight coefficients, combined with standardized calculation methods: First, the floating-point peak performance parameters are standardized using a normalization method to eliminate the magnitude deviation caused by hardware differences between different nodes and obtain the theoretical computing power value; second, the operating status data is corrected in real time based on a dynamic attenuation model to generate an actual computing power value that reflects the actual load capacity of the node; finally, energy indicator data and computing power resource rate (the ratio of the node's effective computing power to the total computing power of the data center) are introduced to construct a comprehensive evaluation model. Through weighted calculation, the data center nodes with the best cost-effectiveness ratio are selected. The processed computing power indicators are integrated with the data center label to generate a node list containing a unified computing power value, resource utilization rate, and comprehensive data center evaluation level.

[0062] Step S102: Build a global resource pool based on the node list, use an algorithm to divide each node into three types of computing power areas, and generate a resource distribution table containing the total computing power of the area.

[0063] Based on the generated node list, all node resources are integrated to build a global resource pool to achieve unified management of computing power network resources. A K-means algorithm based on quantile threshold initialization is used to analyze and process node computing power indicators (such as memory capacity and computing throughput). The cluster center is determined based on the calculated quantile threshold. Through iterative clustering, each node is divided into three types of computing power areas: high, medium, and low. The total computing power of each area is calculated (covering total memory, total computing power, etc.) to generate a resource distribution table, which clearly shows the distribution of computing power resources in different areas.

[0064] Step S103: Obtain the computing requirements and latency constraints of the task queue in combination with the resource distribution table, extract the feature vector, and use the deep reinforcement learning framework to construct the task requirement-region computing power decision to generate a resource utilization table.

[0065] Specifically, the computing resource type (such as CPU and GPU requirements) and task priority of the tasks in the task queue are obtained. Combined with the latency constraints, the information is correlated with the node computing power status, regional bandwidth capacity and other data in the resource distribution table for analysis, and the eigenvector is extracted to generate the matching matrix between the task and the region. If the variance of the eigenvector in the matching matrix exceeds the preset threshold, the deep reinforcement learning framework is used to construct the state space based on the task requirements and regional computing power, dynamically adjust the task allocation ratio, and generate a node mapping decision table. Based on this table, the resource utilization of each node is calculated. When the utilization exceeds the preset range, the low-priority tasks are migrated to form an optimized task distribution, which is finally input into the deep reinforcement learning model to generate a resource utilization table.

[0066] Step S104: Obtain network link traffic data, use a long short-term memory network to predict link congestion probability, and if it exceeds a preset threshold, adjust the path selection to generate a traffic control strategy table.

[0067] Furthermore, acquiring network link traffic data is fundamental to the entire process. This involves collecting data on traffic volume and flow direction for each link in the network over a specific time period. Next, a long short-term memory (LSTM) network is used to predict link congestion probabilities. Using this collected link traffic data as input, the LSTM network is trained and learned to capture temporal correlations and underlying patterns in link traffic data. Based on past and current traffic data, the probability of link congestion occurring within a specific timeframe is predicted. Finally, if the predicted link congestion probability exceeds a preset threshold, it indicates a high probability of future link congestion, negatively impacting network transmission performance. In this case, path selection is adjusted by replanning data transmission routes to avoid potentially congested links and select more accessible links for data transmission. After path adjustment, a flow control policy table is generated, detailing the predicted link congestion and the corresponding path adjustment policy. This table allows for effective control and management of network traffic during actual network operation, ensuring stable and efficient network operation.

[0068] Step S105: Update the resource pool network constraints according to the resource utilization table and the flow control strategy table, remap the tasks with the effective computing power of the nodes as weights, and generate a scheduling execution plan.

[0069] Specifically, we first extract parameters such as node load status and resource consumption rate from the resource utilization table, and combine them with the link bandwidth constraints and path priority matrix in the traffic control strategy table to construct a multidimensional optimization model that includes computing resources and network transmission constraints; based on the effective computing power of the node (that is, the actual available computing power after considering the processor performance, energy consumption cost and comprehensive evaluation value of the data center) as the weight coefficient, we remap the task queue through the weighted graph algorithm, and assign high-priority tasks to nodes with high effective computing power weight and low network transmission cost; during the mapping process, we dynamically adjust the network topology constraints of the resource pool to ensure that the task migration path meets the link congestion threshold and delay constraints; and finally generate a scheduling execution plan that includes task allocation plan, data transmission path and resource reservation strategy to achieve coordinated optimization configuration of computing resources and network resources.

[0070] The above-mentioned computing network resource scheduling method obtains the floating-point computing performance parameters, operating status, and energy index data of each node processor and the data center in which it is located, and calculates and generates a node list by combining preset weights with a standardized method; constructs a global resource pool based on the list, divides the nodes into three types of computing power regions using an algorithm, and generates a resource distribution table containing the total computing power of the region; obtains the computing requirements and delay constraints of the task queue, extracts feature vectors based on the resource distribution table, and uses a deep reinforcement learning framework to construct a task requirement-region computing power decision and generate a resource utilization table; obtains network link traffic data, predicts the link congestion probability through a long short-term memory network, and adjusts the path selection to generate a flow control strategy table if it exceeds a preset threshold; finally, updates the resource pool network constraints based on the resource utilization table and the flow control strategy table, remaps tasks using the node's effective computing power as weight, and generates a scheduling execution plan. This method realizes the precise quantitative evaluation of computing power resources, effectively improves the matching accuracy between tasks and computing power resources, ensures the stability of data transmission, and can significantly improve the utilization of computing network resources, reduce task execution delay, and enhance the overall scheduling efficiency and stability of the system.

[0071] In one embodiment, Figure 2 As shown, the data is calculated based on the preset weights and the standardization method to generate a node list containing unified computing power values and resource utilization rates, which may include the following steps:

[0072] Step S201 , normalizing the floating-point operation peak performance parameter data according to a preset performance weight coefficient to obtain a standardized theoretical computing power value.

[0073] Step S202: Perform dynamic attenuation calculation on the operating status data according to a preset status weight coefficient to generate a dynamically adjusted actual computing power value.

[0074] Preferably, the operating status data includes processor occupancy and memory occupancy.

[0075] Step S203: linearly superimpose the theoretical computing power value and the actual computing power value to generate a node computing power matrix containing a unified computing power index.

[0076] Step S204: Obtain energy index data and computing resource rate of the data center to which each node belongs, and calculate the comprehensive evaluation value of the data center; the energy index data includes but is not limited to electricity price data, new energy utilization rate, and green electricity utilization rate.

[0077] Step S205 , screening the nodes according to the comprehensive evaluation value of the data center, combining the screened nodes with the node computing power matrix, and generating a node list including resource load levels and data center labels.

[0078] First, the floating-point operation peak performance parameter data is normalized according to the preset performance weight coefficient to eliminate the magnitude difference of hardware performance of different nodes and obtain the standardized theoretical computing power value; secondly, for the operating status data including processor occupancy and memory occupancy, dynamic attenuation calculation is performed according to the preset state weight coefficient to generate the actual computing power value reflecting the real-time load of the node; then, the theoretical computing power value and the actual computing power value are linearly superimposed to construct a node computing power matrix containing a unified computing power indicator to achieve comprehensive quantification of the node computing power; then, the electricity price data of the data center to which each node belongs is obtained. Based on the data center's energy consumption, new energy utilization rate, green electricity utilization rate and computing power resource rate (that is, the proportion of the node's effective computing power in the total computing power of the data center), the comprehensive evaluation value of the data center is calculated through a multi-dimensional weighted formula: comprehensive evaluation value = computing power resource rate × α + electricity price × β + new energy utilization rate × γ + green electricity utilization rate × δ (α+β+γ+δ=1, each coefficient is dynamically adjusted according to the green and low-carbon goals); the cost-effectiveness of the data center is quantitatively evaluated; finally, the nodes are screened according to the comprehensive evaluation value of the data center, and only the nodes with qualified evaluation values are retained. The nodes are then combined with the node computing power matrix to generate a node list containing resource load levels and data center labels.

[0079] This embodiment achieves accurate evaluation of node computing power and efficient screening of data centers through multi-dimensional data processing and quantitative analysis: by normalizing the peak performance parameters of floating-point operations and dynamically attenuating the operating status data through preset weights, a node computing power matrix with unified computing power indicators is constructed to ensure the standardization and real-time nature of node computing power evaluation; electricity price data, new energy utilization rate, green electricity utilization rate and computing power resource rate are introduced to calculate the comprehensive evaluation value and screen the nodes, combining computing power capacity with cost-effectiveness to generate a node list containing resource load level and data center label, providing a standardized data foundation that takes into account both performance and cost for the subsequent resource pool construction, and improving the scientificity and economy of computing power network resource scheduling.

[0080] In one embodiment, a global resource pool is constructed based on a node list, and an algorithm is used to divide each node into three computing power regions to generate a resource distribution table containing the total computing power of each region. The following steps may be included:

[0081] Step S301, obtain the node computing power indicators in the node list; the node computing power indicators include memory capacity and computing throughput.

[0082] Step S302: Calculate quantile thresholds based on the node computing power indicator; the quantile thresholds include memory capacity quantiles and computing throughput quantiles.

[0083] Step S303: Initialize the cluster centers of the K-means algorithm based on the quantile threshold; the cluster centers include initial values of the three computing power dimensions of high, medium, and low.

[0084] Step S304: Iteratively cluster the node computing power indicators according to the cluster center to generate a regional division result; the regional division result includes a regional label corresponding to each node.

[0085] Step S305: Aggregate the total computing power of each region according to the region label to generate a resource distribution table including the region label and the total computing power; the total computing power includes the total memory and the total computing power.

[0086] Preferably, nodes within each region are categorized and aggregated based on the generated regional labels. For each region, the sum of the memory capacity and computing throughput of all nodes within it are calculated, representing the region's total memory and computing power. These two metrics together constitute the region's total computing power. Each region's regional label is then combined with the corresponding total computing power to generate a detailed resource distribution table.

[0087] Among them, if the difference in total computing power between regions in the resource distribution table exceeds the preset threshold, the quantile threshold is readjusted and secondary clustering is triggered to obtain the secondary clustering result.

[0088] Step S306: Update the region labels and total computing power values of the resource distribution table according to the secondary clustering results to obtain a final resource distribution table.

[0089] Specifically, node computing power indicators such as memory capacity and computing throughput are first extracted from the node list as the basic data for regional division. Based on these indicators, the memory capacity quantile and computing throughput quantile are calculated to obtain the quantile threshold, which is used to determine the initial cluster centers for the three computing power dimensions of high, medium, and low. Subsequently, the K-means algorithm is used to iteratively cluster the node computing power indicators based on the cluster centers, assigning each node a corresponding regional label to complete the regional division. Next, the total computing power values of each region, such as the total memory and total computing power, are aggregated according to the regional labels to generate a resource distribution table containing regional labels and total computing power values. If the difference in total computing power values between regions in the resource distribution table exceeds the preset threshold, the quantile threshold is readjusted, the clustering operation is performed again, and the regional labels and total computing power values of the resource distribution table are updated based on the secondary clustering results to obtain the final resource distribution table.

[0090] This embodiment uses the K-means algorithm initialized by quantile thresholds to perform regional division, combined with the dynamic adjustment mechanism of secondary clustering, which can effectively adapt to the diversity and complexity of node computing power indicators. Using the quantile threshold to determine the cluster center avoids the randomness of the initial cluster value and improves the accuracy and rationality of regional division; the secondary clustering mechanism ensures the balance of the total computing power of each region and avoids the excessive concentration or dispersion of computing power resources. The resource distribution table finally generated clearly presents the distribution pattern of resources in the computing power network, provides a scientific basis for the matching and scheduling of subsequent tasks and computing power resources, and helps to improve the utilization efficiency of computing power network resources and optimize overall resource allocation.

[0091] In one embodiment, Figure 3 As shown, obtaining the computing requirements and latency constraints of the task queue combined with the resource distribution table, extracting feature vectors, and using a deep reinforcement learning model to build a task requirement-region computing power decision to generate a resource utilization table can include the following steps:

[0092] Step S401: Obtain the computing resource type and task priority of the task queue, combine the node computing power status and regional bandwidth capacity in the resource distribution table, extract the feature vector and generate a matching matrix between the task and the region.

[0093] In step S402, if the variance of the eigenvector in the matching matrix exceeds a preset threshold, a deep network algorithm is used to dynamically adjust the task allocation ratio and generate a node mapping decision table using task requirements and regional computing power as the state space.

[0094] Step S403 , calculating the resource utilization of each node according to the node mapping decision table, and migrating low-priority tasks to low-load nodes when the utilization exceeds a preset range, thereby generating an optimized task distribution.

[0095] Step S404: Calculate dynamic weight allocation parameters based on the maximum allowable delay of task distribution and historical delay fluctuation rate, and input them into the deep reinforcement learning model to generate a regional computing power decision table.

[0096] Preferably, first obtain the maximum allowable delay D of the task distribution max and the historical lagged volatility σ D Using a specific calculation method, the two are combined to calculate the dynamic weight distribution parameter ω, the calculation formula is in represents the average allowed delay, Denotes the average latency volatility, and λ represents the adjustment coefficient. This formula quantifies the constraints and historical fluctuations of task latency into weight parameters. The calculated dynamic weight allocation parameter ω, along with other relevant feature parameters, is then input into the deep reinforcement learning model. Based on these input parameters, the model learns and makes decisions in a state space constructed from task requirements and regional computing power, and iteratively calculates and outputs a regional computing power decision table.

[0097] Step S405: Match task requirements with node resources based on the regional computing power decision table, update the load balancing coefficient and generate a resource utilization table.

[0098] First, the computing resource type and priority information of tasks in the task queue are obtained. Node computing power status and regional bandwidth capacity data are also extracted from the resource distribution table. A feature extraction method is used to generate a task-region matching matrix to quantify the degree of fit between tasks and the computing power of each region. If the variance of the eigenvectors in the matching matrix exceeds a preset threshold, this indicates significant fluctuation in the matching between tasks and regions. In this case, a deep network algorithm is used to construct a state space based on task requirements and regional computing power. Task allocation ratios are dynamically adjusted to form a node mapping decision table. Based on this decision table, the resource utilization of each node is calculated. When utilization exceeds a preset range, low-priority tasks are migrated to underloaded nodes, resulting in an optimized task distribution. Next, dynamic weight allocation parameters are calculated based on the maximum allowable latency and historical latency fluctuations of the task distribution. These parameters are then input into a deep reinforcement learning model to generate a regional computing power decision table. Finally, based on the regional computing power decision table, task requirements are accurately matched with node resources, the load balancing coefficient is updated, and a resource utilization table is generated, completing the full process of task scheduling optimization.

[0099] This embodiment achieves efficient matching and refined scheduling of tasks and computing resources. The construction of the matching matrix provides a quantitative basis for task allocation. The deep network algorithm and dynamic adjustment strategy can quickly respond to changes in task requirements and computing power status, effectively improving the rationality of task allocation. The task migration strategy based on resource utilization avoids the problem of node load imbalance and ensures the stable operation of the system. By combining the delay index to calculate the dynamic weight and using deep reinforcement learning to generate a decision table, task scheduling fully considers time sensitivity and optimizes resource allocation plans. The final generated resource utilization table provides data support for the continuous optimization of the computing power network scheduling strategy, significantly improving the efficiency of computing power resource utilization and task execution efficiency.

[0100] In one embodiment, the resource utilization of each node can be calculated using the following formula:

[0101]

[0102] Among them, U i represents the resource utilization of node i, n represents the total number of resource types, r ij represents the current usage of resource type j on node i, c ij represents the total capacity of resource type j on node i, ω j represents the weight of resource type j, α j represents the sensitivity coefficient of resource type j, m represents the total number of tasks running on node i, and p k represents the priority of task k, t ik represents the current execution time of task k on node i, s ik represents the estimated total execution time of task k on node i, σ ik represents the historical execution time volatility of task k on node i, σ avg represents the cluster average volatility, b i represents the current network bandwidth usage of node i, B i represents the total network bandwidth capacity of node i, ρ i represents the network congestion probability of node i predicted based on the long short-term memory network.

[0103] Preferably, the total number of resource types may include n resource types such as CPU, GPU, memory, storage, etc.

[0104] ω j The weight of resource type j is dynamically determined by the analytic hierarchy process (AHP) or entropy weight method, reflecting the contribution of different resources to the overall utilization.

[0105] α j The sensitivity coefficient of resource type j is dynamically adjusted through the reinforcement learning model to nonlinearly map the marginal effect of resource utilization. For example, when αj When it is >1, the impact of resource usage growth on overall utilization is exponentially amplified.

[0106] σ ik The calculation formula for the historical execution time volatility of task k on node i is:

[0107] in, represents the tth execution time, Indicates the average execution time.

[0108] σ avg The average historical execution time volatility of all tasks in the cluster is used for normalization. The calculation formula is:

[0109] Where N represents the total number of cluster nodes, m i Indicates the number of tasks on node i.

[0110] ρ i The network congestion probability of node i predicted by the long short-term memory network (LSTM) is calculated by the following steps:

[0111] Using historical bandwidth data Train the LSTM model;

[0112] Predict bandwidth usage within the future Δt time window

[0113] Calculate the congestion probability:

[0114] Where θ represents the congestion threshold, M represents the length of the prediction time window, χ(·) represents the indicator function, and B i represents the total network bandwidth capacity of node i.

[0115] This embodiment constructs a node resource utilization calculation model from a multi-dimensional perspective, taking into account the three key dimensions of resources, tasks, and networks, and comprehensively and accurately reflecting the node resource usage status. The weight and sensitivity coefficient settings of the resource dimension can adapt to the differences in the importance of different resource types in business scenarios; the task dimension combines priority and execution time fluctuations to achieve a quantitative assessment of the relationship between task execution efficiency and resource occupancy; the network dimension adjusts the bandwidth usage weight based on the predicted congestion probability, so that the resource utilization calculation fits the actual operation status of the network. Calculating resource utilization through this formula provides objective data support for task scheduling and resource allocation strategy formulation, helps to discover resource usage bottlenecks, optimize resource allocation, improve the overall operating efficiency and resource utilization benefits of the computing network, and enhance the adaptability and stability of the system under complex business loads.

[0116] In one embodiment, a long short-term memory network is used to predict the link congestion probability. If the probability exceeds a preset threshold, the path selection is adjusted to generate a flow control strategy table. The following steps may be included:

[0117] Step S501 : calculating an initial link with the shortest physical path or the least number of hops based on a shortest path algorithm for a task source node and a target node in the network link flow data.

[0118] Step S502 : extract the historical traffic sequence of the initial link and input it into the long short-term memory network, and output the link congestion probability in the future time window.

[0119] Step S503: If the link congestion probability exceeds a preset threshold, the link switching mechanism is triggered to select a suboptimal path from the candidate links and generate a flow control policy table of link path priorities and congestion prediction results.

[0120] Furthermore, for the task source node and target node in the network link traffic data, the initial link with the shortest physical path or the fewest hops is calculated based on the shortest path algorithm (such as the Dijkstra algorithm or the Bellman-Ford algorithm); the historical traffic sequence of the initial link (including timing characteristics such as bandwidth utilization, packet transmission rate, and delay jitter) is extracted and input into the long short-term memory network (LSTM) for training, and the predicted value of the link congestion probability in the future time window is output; if the predicted link congestion probability exceeds a preset threshold (such as 80%), the link switching mechanism is triggered, and the suboptimal path (such as the path with the second shortest physical path length and congestion probability below the threshold) is selected from the pre-calculated alternative link set. The path priority sorting and congestion prediction results are combined to generate a flow control policy table containing link ID, physical path information, congestion probability, and priority level.

[0121] This method achieves efficiency optimization and congestion avoidance in link scheduling by combining the shortest path algorithm with a long-short-term memory network: the initial link is determined based on the shortest path algorithm to ensure the physical path efficiency of data transmission; the long-short-term memory network is used to learn historical traffic sequences to predict the link congestion probability and identify potential congestion risks in advance; the link switching mechanism is triggered by a preset threshold and the suboptimal path is selected, generating a flow control policy table that includes path priority and congestion prediction results, thereby improving network resource utilization and data transmission reliability.

[0122] In one embodiment, updating the resource pool network constraints according to the resource utilization table and the flow control policy table, remapping tasks with the effective computing power of the nodes as weights, and generating a scheduling execution plan may include the following steps:

[0123] Step S601: perform weighted fusion on the node computing power parameters in the resource utilization table and the comprehensive evaluation value of the data center to obtain the node effective computing power weight.

[0124] Step S602: construct a consistent hash ring based on the effective computing power weights of the nodes, and allocate virtual nodes according to the weight ratio to quantify the node scheduling priority.

[0125] Step S603 : extracting the link path priority and the link congestion threshold from the flow control policy table, and generating a network bandwidth matrix including inter-node communication delay constraints.

[0126] Step S604 : Dynamically map and match the network bandwidth matrix with the hash ring, and superimpose to generate a task migration path table that carries the load status and link constraints.

[0127] Step S605 : optimizing virtual node distribution based on the task migration path table and taking the link congestion threshold as a boundary condition, and generating a scheduling execution plan including traffic balancing parameters in combination with the data center priority.

[0128] Specifically, the node computing power parameters (such as CPU occupancy and memory occupancy) in the resource utilization table are first weighted and fused with the comprehensive evaluation value of the data center. The node effective computing power weight is calculated by pre-set weight coefficients to quantify the actual capacity of the node in scheduling. Secondly, a consistent hash ring is constructed based on the node effective computing power weight, and the number of virtual nodes is allocated to each node according to the weight ratio. The node scheduling priority is quantified by the proportion of virtual nodes to achieve load balancing of task allocation. Then, the link path priority (such as shortest path, suboptimal path) and link congestion threshold (such as maximum allowed bandwidth utilization) are extracted from the traffic control policy table to generate a network bandwidth matrix containing inter-node communication delay constraints and clarify the network transmission restriction conditions. Subsequently, the network bandwidth matrix is dynamically mapped and matched with the hash ring, and a task migration path table carrying node load status and link constraints is generated by superimposing the priority matching algorithm to associate task routing with resource status. Finally, based on the task migration path table, the distribution of virtual nodes in the hash ring is optimized with the link congestion threshold as the boundary condition. Nodes are screened in combination with the data center priority to generate a scheduling execution plan containing inter-node traffic balancing parameters to achieve coordinated scheduling of computing power resources and network links.

[0129] This embodiment realizes the coordinated optimization scheduling of computing power resources and network links through multi-dimensional resource parameter fusion and dynamic mapping mechanism: quantifies the node scheduling priority through weighted fusion of node computing power parameters and comprehensive evaluation values of data centers; builds a load-balanced task allocation basis based on consistent hash ring and virtual node allocation; generates a network bandwidth matrix by combining link path priority and congestion threshold to clarify transmission constraints; generates a task migration path table with dual constraints through dynamic mapping and matching of the matrix and hash ring; finally, optimizes the virtual node distribution with link congestion threshold as boundary condition, and generates a traffic-balanced scheduling execution plan by combining data center priority, realizing full-process resource coordinated scheduling from node selection to path planning, and improving overall network resource utilization and task execution efficiency.

[0130] In one embodiment, after generating the scheduling execution plan, the following steps may also be included:

[0131] Step S701: Obtain the execution delay rate and resource occupancy rate in the task feedback data after executing the scheduling execution plan.

[0132] Step S702: Calculate a multi-dimensional scheduling feature vector according to the execution delay rate and the resource occupancy rate.

[0133] Step S703: Input the multi-dimensional scheduling feature vector into the weighted scoring model and output the scheduling effect score.

[0134] Step S704: Generate a dynamic optimization parameter set based on the scheduling effect score, and extract the reward function adjustment factor and computing power weight correction coefficient.

[0135] Step S705: Update the weight distribution of the reinforcement learning model according to the reward function adjustment factor to obtain a strategy evaluation matrix.

[0136] Step S706: construct a resource allocation scheduling decision tree based on the computing power weight correction coefficient and the policy evaluation matrix.

[0137] Step S707: Generate an iterated scheduling execution plan based on the scheduling decision tree.

[0138] Specifically, after executing the scheduling execution plan, the execution delay rate and resource occupancy rate in the task feedback data are collected. Then, based on the execution delay rate and resource occupancy rate, a multi-dimensional scheduling feature vector is generated through a specific calculation method to quantitatively reflect the actual effect of the scheduling plan. Next, the multi-dimensional scheduling feature vector is input into the weighted scoring model, and the scheduling effect score is output after the model operation to quantitatively evaluate the scheduling plan. Based on the scheduling effect score, a dynamic optimization parameter set is generated, and the reward function adjustment factor and computing power weight correction coefficient are extracted from it. The reward function adjustment factor is used to update the weight distribution of the reinforcement learning model, and the policy evaluation matrix is obtained to optimize the reinforcement learning strategy. Finally, combined with the computing power weight correction coefficient and the policy evaluation matrix, a scheduling decision tree for resource allocation is constructed, and the iterative scheduling execution plan is generated based on the decision tree to complete the optimization and update of the scheduling plan.

[0139] This iterative optimization mechanism achieves dynamic improvement of computing network scheduling solutions through a closed-loop process of data feedback, evaluation, and optimization. A multidimensional scheduling feature vector is generated based on task execution delay rates and resource utilization, providing a comprehensive and objective data foundation for scheduling solution evaluation. The application of a weighted scoring model quantifies scheduling effectiveness, allowing the pros and cons of solutions to be intuitively reflected. The generation of dynamic optimization parameter sets and the updating of reinforcement learning models directly link scheduling performance with algorithm parameter adjustments, effectively improving the adaptability of reinforcement learning strategies. A scheduling decision tree is constructed and iterative solutions are generated, enabling targeted optimization of resource allocation strategies based on actual execution results, forming a virtuous cycle of continuous improvement. This helps improve the accuracy and efficiency of computing network resource scheduling, enhances the system's responsiveness to dynamic load changes, and ensures the long-term stable and efficient operation of the computing network.

[0140] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0141] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of a computing power network resource scheduling method as described above are implemented.

[0142] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0143] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the components described as separate parts may or may not be physically separated, and the parts displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0144] The above-described embodiments merely represent several implementation methods of the embodiments of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the concept of the embodiments of the present application, and these modifications and improvements fall within the scope of protection of the embodiments of the present application.

Claims

1. A computing power network resource scheduling method, characterized in that: The method comprises: Obtain the floating-point computing performance parameters, operating status, and energy index data of each node processor and the data center in which it is located; calculate the data according to the preset weights and the standardized method to generate a node list; Based on the node list, a global resource pool is constructed, and each node is divided into three computing power areas using an algorithm to generate a resource distribution table containing the total computing power of the area; Obtain the computing requirements and latency constraints of the task queue and combine them with the resource distribution table, extract feature vectors, and use a deep reinforcement learning framework to build a task requirement-region computing power decision to generate a resource utilization table; Obtain network link traffic data and use long short-term memory networks to predict link congestion probability. If it exceeds a preset threshold, adjust the path selection and generate a flow control policy table; The resource pool network constraints are updated according to the resource utilization table and the flow control strategy table, the tasks are remapped with the effective computing power of the nodes as weights, and a scheduling execution plan is generated.

2. The method according to claim 1, characterized in that The step of calculating the data according to the preset weights and the standardization method to generate a node list includes: Normalizing the floating-point operation peak performance parameter data according to a preset performance weight coefficient to obtain a standardized theoretical computing power value; Performing dynamic attenuation calculation on the operating status data according to a preset state weight coefficient to generate a dynamically adjusted actual computing power value; The theoretical computing power value and the actual computing power value are linearly superimposed to generate a node computing power matrix containing a unified computing power index; Obtain the energy indicator data and computing power resource rate of the data center to which each node belongs, and calculate the comprehensive evaluation value of the data center; the energy indicator data includes but is not limited to electricity price data, new energy utilization rate, and green electricity utilization rate; The nodes are screened according to the comprehensive evaluation value of the data center, and the screened nodes are combined with the node computing power matrix to generate a node list including resource load levels and data center labels.

3. The method according to claim 1, characterized in that The global resource pool is constructed based on the node list, and each node is divided into three types of computing power areas using an algorithm to generate a resource distribution table containing the total computing power of the area, including: Obtaining node computing power indicators in the node list; the node computing power indicators include memory capacity and computing throughput; Calculate a quantile threshold based on the node computing power indicator; the quantile threshold includes a memory capacity quantile and a computing throughput quantile; Initializing the cluster center of the K-means algorithm based on the quantile threshold; the cluster center includes initial values of the three computing power dimensions of high, medium, and low; Iteratively clustering the node computing power indicators according to the cluster center to generate a region division result; the region division result includes a region label corresponding to each node; Aggregating the total computing power of each region according to the region label and generating a resource distribution table including the region label and the total computing power; the total computing power includes the total memory and the total computing power; If the difference in total computing power between regions in the resource distribution table exceeds a preset threshold, the quantile threshold is readjusted and secondary clustering is triggered to obtain a secondary clustering result. The region labels and total computing power values of the resource distribution table are updated according to the secondary clustering result to obtain a final resource distribution table.

4. The method according to claim 1, wherein The computing requirements and latency constraints of the task queue are obtained and combined with the resource distribution table, feature vectors are extracted, and a deep reinforcement learning model is used to construct a task requirement-region computing power decision to generate a resource utilization table, including: Obtaining the computing resource type and task priority of the task queue, combining the node computing power status and regional bandwidth capacity in the resource distribution table, extracting feature vectors to generate a matching matrix between tasks and regions; If the variance of the eigenvector in the matching matrix exceeds a preset threshold, a deep network algorithm is used to dynamically adjust the task allocation ratio and generate a node mapping decision table using task requirements and regional computing power as the state space; Calculate the resource utilization of each node according to the node mapping decision table, and when it exceeds the preset range, migrate low-priority tasks to low-load nodes to generate an optimized task distribution; Calculate dynamic weight allocation parameters based on the maximum allowable delay and historical delay fluctuation rate of the task distribution, and input them into the deep reinforcement learning model to generate a regional computing power decision table; Based on the regional computing power decision table, the task requirements and node resources are matched, and the load balancing coefficient is updated to generate a resource utilization table.

5. The method according to claim 4, characterized in that The resource utilization of each node is calculated using the following formula: Among them, U i represents the resource utilization of node i, n represents the total number of resource types, r ij represents the current usage of resource type j on node i, c ij represents the total capacity of resource type j on node i, ω j represents the weight of resource type j, α j represents the sensitivity coefficient of resource type j, m represents the total number of tasks running on node i, and p k represents the priority of task k, t ik represents the current execution time of task k on node i, s ik represents the estimated total execution time of task k on node i, σ ik represents the historical execution time volatility of task k on node i, σ avg represents the cluster average volatility, b i represents the current network bandwidth usage of node i, B i represents the total network bandwidth capacity of node i, ρ i represents the network congestion probability of node i predicted based on the long short-term memory network.

6. The method according to claim 1, characterized in that The method uses a long short-term memory network to predict the link congestion probability, and if it exceeds a preset threshold, adjusts the path selection to generate a flow control strategy table, including: Calculating an initial link with the shortest physical path or the least number of hops based on a shortest path algorithm for a task source node and a target node in the network link flow data; Extracting the historical traffic sequence of the initial link and inputting it into a long short-term memory network, and outputting the link congestion probability in a future time window; If the link congestion probability exceeds a preset threshold, the link switching mechanism is triggered to select a suboptimal path from the alternative links, and a flow control policy table of link path priorities and congestion prediction results is generated.

7. The method according to claim 1, characterized in that The updating of the resource pool network constraints according to the resource utilization table and the flow control strategy table, remapping the tasks with the effective computing power of the nodes as weights, and generating a scheduling execution plan include: Performing weighted fusion on the node computing power parameters in the resource utilization table and the comprehensive evaluation value of the data center to obtain the node effective computing power weight; A consistent hash ring is constructed based on the effective computing power weights of the nodes, and virtual nodes are allocated according to the weight ratio to quantify the node scheduling priority; Extracting link path priorities and link congestion thresholds from the flow control policy table to generate a network bandwidth matrix including inter-node communication delay constraints; Dynamically mapping and matching the network bandwidth matrix with the hash ring, and superimposing them to generate a task migration path table carrying load status and link constraints; The virtual node distribution is optimized according to the task migration path table with the link congestion threshold as a boundary condition, and a scheduling execution plan including traffic balancing parameters is generated in combination with the data center priority.

8. The method according to claim 7, characterized in that After generating the scheduling execution plan, the method further includes: Obtaining the execution delay rate and resource occupancy rate in the task feedback data after executing the scheduling execution plan; Calculating a multidimensional scheduling feature vector according to the execution delay rate and resource occupancy rate; Inputting the multidimensional scheduling feature vector into a weighted scoring model and outputting a scheduling effect score; Generate a dynamic optimization parameter set based on the scheduling effect score, and extract the reward function adjustment factor and computing power weight correction coefficient; Updating the weight distribution of the reinforcement learning model according to the reward function adjustment factor to obtain a strategy evaluation matrix; Constructing a scheduling decision tree for resource allocation based on the computing power weight correction coefficient and the strategy evaluation matrix; An iterative scheduling execution plan is generated according to the scheduling decision tree.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images