Computing power routing method and device and medium

Abstract

The present disclosure provides a computing power routing method, device and medium, relating to the technical field of network, and used for solving the problem of how to cope with the complex computing network resource scheduling of different network topologies. The method comprises: receiving a first computing network resource scheduling request initiated by a first computing power network; obtaining a plurality of first graph features of a first topology structure of the first computing power network; obtaining a first topology classification of the first topology structure according to the plurality of first graph features; obtaining a first computing power routing model according to the first topology classification; and selecting a first computing power node and a first route to the first computing power node in the first topology structure for the first computing network resource scheduling request according to the first computing power routing model. The present disclosure obtains a computing power routing model based on different network topologies. The computing power routing model can adapt to the computing network resource scheduling of complex business scenarios, and improves the ability to cope with complex and variable computing power network business scenarios.

Description

Technical Field

[0001] This disclosure relates at least to the field of network technology, and in particular to a computing power routing method, a computing power routing device, and a computer-readable storage medium. Background Technology

[0002] The development of computing power is inseparable from the support of computing networks. Therefore, the industry has proposed the concept of "computing networks." Routing plays a crucial role in network communication. Routing is the transmission channel through which network infrastructure selects and delivers information generated by a source node to a destination node. It determines the transmission path of information in the network and directly impacts network performance such as throughput, latency, and packet loss.

[0003] However, with the increasing complexity of computing network environments, traditional routing algorithms often fail to consider the impact of differences in network topology on routing, and can no longer meet the business needs of unified scheduling of computing and network resources and sudden changes in traffic. Summary of the Invention

[0004] The technical problem to be solved by this disclosure is to provide a computing power routing method, a computing power routing device, and a computer-readable storage medium to address the above-mentioned shortcomings and solve the problem of complex computing network resource scheduling with network topology differences.

[0005] In a first aspect, this disclosure provides a computing power routing method, the method comprising:

[0006] Receive the first computing network resource scheduling request initiated by the first computing power network;

[0007] Obtain several first graph features of the first topology of the first computing power network;

[0008] The first topological classification of the first topological structure is obtained based on several features of the first graph.

[0009] The first computing power routing model is obtained based on the first topology classification;

[0010] According to the first computing power routing model, for the first computing network resource scheduling request, the first computing power node and the first route to the first computing power node are selected in the first topology.

[0011] Furthermore, receiving the first computing network resource scheduling request initiated by the first computing power network specifically includes:

[0012] The computing power network scheduling center receives the first computing power network resource scheduling request from the user equipment. The first computing power network resource scheduling request is a triple (C, N, R), where: C represents the user's computing power resource request, N represents the user's network resource request, and R includes the source node corresponding to the user equipment and the requested first computing power network.

[0013] Furthermore, several first graph features of the first topology of the first computing power network are obtained, specifically including:

[0014] Obtain the current real-time topology of the first computing power network from the computing power network scheduling center;

[0015] The first average node degree and the median value of the first link in the first topology are extracted using the NetworkX network analysis library in Python as features of the first graph.

[0016] Wherein, the first average node degree is the average number of edges connected to each computing node in the first topology, and the first link median value is the average number of edges traversed between any two computing nodes in the first topology.

[0017] Furthermore, the method also includes:

[0018] Obtain several second graph features for each of the multiple second topologies;

[0019] Topological classification of multiple second topological structures based on several features of the second graph;

[0020] For each topology classification, a corresponding computing power routing model is trained to obtain its own model.

[0021] Furthermore, several second graph features are obtained for each of the multiple second topologies, specifically including:

[0022] Obtain multiple second topologies from TopologyZoo, and extract the second average node degree and the second link median value for each of the multiple second topologies.

[0023] Topological classification of multiple second topological structures is performed based on several features of the second graph, specifically including:

[0024] The second distance between multiple second topologies is calculated based on the second average node degree and the second link median value. Based on the K-means clustering algorithm, the second topologies of multiple computing power networks are divided into K topology categories according to the second distance.

[0025] For each topology classification, a corresponding computational power routing model is trained, specifically including:

[0026] The computational power routing model for each topology category is trained using the central second topology structure of each topology category, and the generalization performance of the computational power routing model in the corresponding topology category is verified using the non-central second topology structure of each topology category.

[0027] Furthermore, a first topological classification of the first topological structure is obtained based on several features of the first graph, specifically including:

[0028] Calculate the first KL divergence between the first and second topologies and the second KL divergence between the second and first topologies according to the following formula (1):

[0029]

[0030] The first distance between the first topology and the second topology is calculated according to the following formula (2):

[0031]

[0032] In the formula, P represents the first topology, Q represents the second topology, P1 represents the first average node degree of P, P2 represents the first link median value of P, Q1 represents the second average node degree of Q, Q2 represents the second link median value of Q, KL(P|Q) represents the first KL divergence, and KL(Q|P) represents the second KL divergence.

[0033] In response to a first distance being less than a preset distance threshold, the topology classification of the pre-obtained second topology structure is acquired and used as the first topology classification of the first topology structure.

[0034] Furthermore, according to the first computing power routing model, for the first computing network resource scheduling request, the first computing power node and the first route to the first computing power node are selected in the first topology, specifically including:

[0035] Based on the first computing network resource scheduling request, obtain the computing power resource request, network resource request, and source node; obtain the first topology of the first computing power network; and obtain the first number of specified candidate service nodes and the second number of candidate routes, as input to the first computing power routing model.

[0036] The first computing power routing model obtains a first number of candidate computing power nodes and a second number of candidate routes based on the input and the node selection algorithm and candidate route selection algorithm obtained through pre-training. It also obtains the reward for executing each candidate route based on the ∈-greedy exploration strategy and action scoring function obtained through pre-training. The candidate route with the highest reward is selected as the first route and the corresponding candidate computing power node is selected as the first computing power node.

[0037] Furthermore, among which:

[0038] The node selection algorithm monitors each service node of the first computing power network and selects a first number of candidate computing power nodes based on the available computing power resources of each service node.

[0039] The candidate routing selection algorithm monitors each link of the first computing power network and selects a second number of candidate routes based on the available bandwidth of each link, the path centrality of reachable candidate computing power nodes, and the number of paths to reachable candidate computing power nodes.

[0040] Secondly, this disclosure provides a computing power routing device, the device comprising:

[0041] The receiving module is used to receive the first computing network resource scheduling request initiated by the first computing power network;

[0042] The graph feature module, connected to the receiving module, is used to acquire several first graph features of the first topology of the first computing power network.

[0043] The topology classification module, connected to the graph feature module, is used to obtain the first topology classification of the first topology structure based on several first graph features.

[0044] The model acquisition module is connected to the topology classification module and is used to acquire the first computing power routing model based on the first topology classification.

[0045] The routing module, connected to the model acquisition module, is used to select the first computing power node and the first route to the first computing power node in the first topology structure for the first computing network resource scheduling request, based on the first computing power routing model.

[0046] Thirdly, this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the computing power routing method described above.

[0047] This disclosure provides a computing power routing method, a computing power routing device, and a computer-readable storage medium. It obtains a computing power routing model based on a differentiated network topology. This computing power routing model can adapt to the scheduling of computing network resources in complex business scenarios and improves the ability to cope with complex and ever-changing computing network business scenarios. Attached Figure Description

[0048] Figure 1 This is a flowchart of a computing power routing method according to an embodiment of this disclosure;

[0049] Figure 2 This is a schematic diagram of a computing power network scheduling center according to an embodiment of this disclosure;

[0050] Figure 3 This is a schematic diagram of a computing power routing model structure according to an embodiment of this disclosure;

[0051] Figure 4 This is a schematic diagram of the result of topological classification according to an embodiment of this disclosure;

[0052] Figure 5 This is a schematic diagram illustrating the effects of several node selection algorithms according to embodiments of this disclosure;

[0053] Figure 6This is a dot plot showing the experimental results of applying several candidate routing algorithms from the embodiments of this disclosure to a central topology;

[0054] Figure 7 This is a CDF diagram showing the experimental results of several candidate routing algorithms from the embodiments of this disclosure applied to a central topology;

[0055] Figure 8 This is a dot plot showing the experimental results of several candidate routing algorithms from the embodiments of this disclosure applied to a non-central topology;

[0056] Figure 9 This is a CDF diagram showing the experimental results of several candidate routing algorithms from the embodiments of this disclosure applied to a non-central topology;

[0057] Figure 10 This is a schematic diagram of the structure of a computing power routing device according to an embodiment of the present disclosure. Detailed Implementation

[0058] To enable those skilled in the art to better understand the technical solutions of this disclosure, the embodiments of this disclosure will be further described in detail below with reference to the accompanying drawings.

[0059] It is understood that the specific embodiments and accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of this disclosure.

[0060] It is understood that, without conflict, the various embodiments and features in the embodiments of this disclosure can be combined with each other.

[0061] It is understood that, for ease of description, only the parts relevant to this disclosure are shown in the accompanying drawings, while parts unrelated to this disclosure are not shown in the drawings.

[0062] It is understood that each unit or module involved in the embodiments of this disclosure may correspond to only one entity structure, or may be composed of multiple entity structures, or multiple units or modules may be integrated into one entity structure.

[0063] It is understood that, without conflict, the functions and steps marked in the flowcharts and block diagrams of this disclosure may occur in a different order than that marked in the accompanying drawings.

[0064] It is understood that the flowcharts and block diagrams of this disclosure illustrate the architecture, functions, and operations of possible implementations of systems, apparatuses, devices, and methods according to various embodiments of this disclosure. Each block in a flowchart or block diagram may represent a unit, module, program segment, or code, containing executable instructions for implementing the specified function. Furthermore, each block or combination of blocks in the block diagrams and flowcharts may be implemented using a hardware-based system to implement the specified function, or using a combination of hardware and computer instructions.

[0065] It is understood that the units and modules involved in the embodiments of this disclosure can be implemented by software or by hardware, for example, the units and modules can be located in a processor.

[0066] Example 1:

[0067] like Figure 1 As shown, this disclosure provides a computing power routing method, the method comprising:

[0068] S1. Receive the first computing network resource scheduling request initiated by the first computing power network;

[0069] S2. Obtain several first graph features of the first topology of the first computing power network;

[0070] S3. Obtain the first topological classification of the first topological structure based on several features of the first graph;

[0071] S4. Obtain the first computing power routing model based on the first topology classification;

[0072] S5. Based on the first computing power routing model, for the first computing network resource scheduling request, select the first computing power node and the first route to the first computing power node in the first topology.

[0073] In this embodiment, a computing power routing model is obtained based on differentiated network topology. This computing power routing model can adapt to the scheduling of computing network resources in complex business scenarios, thereby improving the ability to cope with complex and ever-changing computing network business scenarios.

[0074] Specifically, network topology is classified based on graph features, and different models are selected for different categories of network topology. This allows the model to better select computing power routing strategies based on network features, select the optimal computing power node and route according to user requests, improve the model's scalability and fault tolerance, and enable it to cope with complex and ever-changing computing power network business scenarios.

[0075] For example, this computing power routing model can adapt to computing power routing operations on unfamiliar network topologies, or it can still obtain effective computing power routing results when the network topology changes due to link failures, node disconnections, or other events in the computing power network.

[0076] More specifically, as a platform integrating computing and network resources, a computing power network can provide users with more flexible, efficient, and reliable services compared to traditional cloud service providers. For example... Figure 2 As shown, this embodiment designs a computing power network scheduling center. Based on this, and combining the advantages of the computing power network, a computing power resource overall scheduling model is implemented to achieve computing power routing while selecting nodes. Furthermore, to better learn the characteristics of the computing power network and improve the model's scalability and fault tolerance, graph neural networks are introduced into reinforcement learning, proposing the CPRM (Computingpower Routing Model). The goal of the routing model is to select the optimal node and the path to that node from multiple computing power nodes that meet user needs, thereby maximizing network traffic and improving bandwidth resource utilization. Considering that finding the optimal node and route configuration for users in a computing power network is an NP (Non-deterministic Polynomial) difficult problem, traditional routing algorithms can no longer efficiently solve this type of problem. Therefore, an alternative solution needs to be explored. The emergence of Hierarchical Reinforcement Learning (DRL) provides a new approach to solving this type of NP-hard problem. Based on DRL proxy objects, the optimal node and route configuration can be found according to the current network and node states to provide efficient and reliable computing power services.

[0077] In one embodiment, S1, receiving a first computing network resource scheduling request initiated by the first computing power network specifically includes:

[0078] The computing power network scheduling center receives the first computing power network resource scheduling request from the user equipment. The first computing power network resource scheduling request is a triple (C, N, R), where: C represents the user's computing power resource request, N represents the user's network resource request, and R includes the source node corresponding to the user equipment and the requested first computing power network.

[0079] In this embodiment, the user initiates a user request to the computing power network scheduling center. The user request is a triple Q(C, N, R), where C represents the computing power resources requested by the user, including CPU resources, GPU resources, and storage resources; N represents the network resources requested by the user, specifically bandwidth resources; and R includes the source node S corresponding to the requesting user and the requested computing power network, which includes a set of service nodes D that meet the computing power requirements.

[0080] The dispatch center receives user requests and selects the optimal computing power node and traffic route from multiple computing power nodes that meet the user's needs, maximizing the traffic through the network and satisfying as many business needs as possible.

[0081] like Figure 2 As shown, a computing power network contains a topology that includes network links and computing power nodes. Links are the edges in the topology graph. Network links have specific bandwidth capacities, while computing power nodes have limited computing power resources. The scheduling center contains a global view of the current node and link states of the computing power network, enabling intelligent routing orchestration and comprehensive scheduling of computing network resources. When a user request arrives, the scheduling center first uses a specific node selection algorithm to find a set of computing power service nodes that can meet the business requirements. Then, the computing network resource scheduling model determines the final computing power node providing the service and the route to that node.

[0082] In one embodiment, S2, obtaining several first graph features of the first topology of the first computing power network, specifically includes:

[0083] Obtain the current real-time topology of the first computing power network from the computing power network scheduling center;

[0084] The first average node degree and the median value of the first link in the first topology are extracted using the NetworkX network analysis library in Python as features of the first graph.

[0085] Wherein, the first average node degree is the average number of edges connected to each computing node in the first topology, and the first link median value is the average number of edges traversed between any two computing nodes in the first topology.

[0086] In this embodiment, routing is essentially the process of finding subgraphs within the network topology. Therefore, to differentiate network topologies and describe their differences, two graph features are extracted: average node degree (the number of edges associated with a node) and average link median (distance). A network topology model is maintained by the scheduling center, and then the NetworkX network analysis library in Python is used to extract the average node degree and average link median. Average node degree is often related to network topology density; a higher average node degree indicates a denser network topology. The link median describes network connectivity; a high link median indicates a poor path distribution, with multiple paths crossing the same links.

[0087] In one embodiment, the method further includes:

[0088] Obtain several second graph features for each of the multiple second topologies;

[0089] Topological classification of multiple second topological structures based on several features of the second graph;

[0090] For each topology classification, a corresponding computing power routing model is trained to obtain its own model.

[0091] In this embodiment, as Figure 3 As shown, the computational routing model is pre-trained by the DRL agent. A corresponding computational routing model is trained for each topology classification. The training process and the usage process are relative; both involve first extracting the corresponding graph features, then classifying the topology based on these features, and finally obtaining a computational routing model for each topology classification. However, the use of DRL requires caution. As network scale becomes more complex and computational resources become more diverse, the training difficulty of the model gradually increases. Therefore, it is necessary to rationally plan the model training environment and action space.

[0092] In one embodiment, obtaining several second graph features for each of the multiple second topologies specifically includes:

[0093] Obtain multiple second topologies from TopologyZoo, and extract the second average node degree and the second link median value for each of the multiple second topologies.

[0094] Topological classification of multiple second topological structures is performed based on several features of the second graph, specifically including:

[0095] The second distance between multiple second topologies is calculated based on the second average node degree and the second link median value. Based on the K-means clustering algorithm, the second topologies of multiple computing power networks are divided into K topology categories according to the second distance.

[0096] For each topology classification, a corresponding computational power routing model is trained, specifically including:

[0097] The computational power routing model for each topology category is trained using the central second topology structure of each topology category, and the generalization performance of the computational power routing model in the corresponding topology category is verified using the non-central second topology structure of each topology category.

[0098] In this embodiment, as Figure 3 As shown, model training is performed based on differentiated network topology. In order to better capture the graph features in the network, a graph neural network (GNN) is introduced into the computational network resource scheduling model to pass actions to the DRL agent in the form of a graph.

[0099] It's important to note that introducing GNNs complicates model training, requiring consideration of both the traffic matrix and network topology during model design. However, given that network operators can discretize user traffic demands (currently, operators offer fixed bandwidth packages, such as 1Mbps, 2Mbps, etc., rather than allowing users to input arbitrary bandwidth), this embodiment only needs to focus on the impact of network topology on the model. Currently, routing model training processes often rarely consider the impact of differences in network topologies on routing. To improve model scalability and usability, this embodiment considers how to train the model based on differentiated network topologies.

[0100] For example, the network topologies that can be selected come from TopologyZoo, a dataset containing hundreds of real-world network topologies, such as the Internet, data centers, and communication networks. The CPRM model is implemented using Python TensorFlow, with the DRL module based on DQN (Deep Q-Network) and the agent component based on GNN. The environment is based on the OpenAIGym framework (an open-source reinforcement learning toolkit released by OpenAI). Some graph-related operations are involved, using the NetworkX and matplotlib libraries. Table 1 shows the hyperparameters used during the DRL agent training phase.

[0101] Table 1 Examples of DRL agent training hyperparameters

[0102]

[0103] Experiments were conducted on the same problem domain to evaluate the performance of the CPRM model and the basic routing algorithm. By comparing the experimental results, the advantages and disadvantages of different routing schemes were observed, thus providing some suggestions for model optimization.

[0104] 1. Topology classification preprocessing

[0105] To improve the scalability of the CPRM model across different network topologies, 107 network topologies under TopologyZoo were first classified using a clustering algorithm. Representative topologies from the three categories were then selected for model training to enhance the model's adaptability to different networks. For each network topology, a binary tuple containing the average link median and average node degree was constructed. The traditional K-means algorithm was used for clustering, and the clustering results based on the binary tuples are shown below. Figure 4 As shown, clustering is performed using K=3 to divide the network topology into... Figure 4 Among the three categories of yellow, purple, and green, the network topologies closest to the cluster center are AGIS, Globalcenter, and Amres, respectively (indicated by red × in the figure).

[0106] 2. Selection of computing nodes

[0107] To limit the dimensionality of the action space, the number of selectable computing nodes needs to be limited. For example, a maximum of three candidate computing nodes can be selected. Considering that the AGIS map has the most topological categories compared to Globalcenter and Amres, and is therefore more general, a node selection control experiment was conducted based on the AGIS map to test the experimental effects of applying different node selection algorithms to the CPRM model. Figure 5 The results demonstrate the scores of the CPRM model during training. alg.1 represents the node selection algorithm based on average path, alg.2 represents the node selection algorithm based on available resources, and alg.3 represents the random node selection algorithm. The experimental results show that all three node selection algorithms tend to stabilize after a similar number of training iterations, and the training time costs are relatively similar. However, alg.2 outperforms the other two algorithms in terms of score; therefore, the node selection algorithm based on available resources will be used uniformly in subsequent experiments.

[0108] 3. Routing Model Comparison Experiment

[0109] In this experiment, the CPRM model was trained on the AGIS network and compared with traditional load balancing (LB), shortest path first (SF), and theoretical fluid dynamics (FLUID) routing strategies. The theoretical fluid dynamics model allows traffic to be distributed proportionally to different candidate paths, with the proportion calculated based on the maximum available bandwidth of the current path. This routing mode resulted in the model achieving the highest score on the AGIS network. However, the theoretical fluid dynamics model is only suitable for networks with simple structures and decomposable traffic. In computing power network scenarios, it can be considered an optimal solution, but it cannot be used as the final routing strategy. Figure 6 A dot plot of 1000 evaluated experimental results can be observed in the middle. Figure 7 The cumulative distribution function (CDF) of the scores for different routing strategies is presented. Analysis of the experimental results shows that the CPRM is closer to the routing score under the theoretical fluid model than the traditional routing strategy, and can be used as an alternative to the traditional routing strategy.

[0110] 4. Analysis of the generalization ability of the routing model

[0111] The generalization ability of a routing model can be divided into homogeneous topology generalization and heterogeneous topology generalization. Homogeneous topology generalization refers to the CPRM model's ability to generalize on network topologies of the same type. Heterogeneous topology generalization refers to the CPRM model's ability to generalize on network topologies of different types. For experiments on homogeneous topology generalization ability, the VisionNet network model was used. Figure 8 A dot plot of 1000 evaluated experimental results can be observed in the middle. Figure 9 The cumulative distribution function of scores for different routing strategies is shown. Considering that the VisionNet network topology is more complex than the AGIS network topology, the average score is lower than that of AGIS under the same traffic matrix. However, analysis of the experimental results shows that CPRM still scores higher than traditional routing strategies on new network topologies of the same type, and is close to the score of routing strategies based on theoretical fluid models. This proves that the model has a certain generalization ability.

[0112] In one embodiment, S3, obtaining a first topological classification of the first topological structure based on several first graph features, specifically includes:

[0113] Calculate the first KL divergence between the first and second topologies and the second KL divergence between the second and first topologies according to the following formula (1):

[0114]

[0115] The first distance between the first topology and the second topology is calculated according to the following formula (2):

[0116]

[0117] In the formula, P represents the first topology, Q represents the second topology, P1 represents the first average node degree of P, P2 represents the first link median value of P, Q1 represents the second average node degree of Q, Q2 represents the second link median value of Q, KL(P|Q) represents the first KL divergence, and KL(Q|P) represents the second KL divergence.

[0118] In response to a first distance being less than a preset distance threshold, the topology classification of the pre-obtained second topology structure is acquired and used as the first topology classification of the first topology structure.

[0119] In this embodiment, based on two network features, average node degree and link median, a clustering algorithm can be used to classify the network topology before model training. Then, network topologies close to the cluster center under different categories can be selected for model training, enabling the model to adapt to network topologies that have not been encountered or have experienced link failures, thereby improving its ability to cope with complex and ever-changing computing network service scenarios.

[0120] Classification can be performed by calculating KL divergence based on average node degree and link median, calculating distance in clustering algorithms based on KL divergence, and performing topology classification and topology selection based on distance.

[0121] Alternatively, distance can be used directly for classification. For example, k data points can be randomly selected as initial cluster centers. For each data point, the distance to all cluster centers is calculated, and then it is assigned to the cluster corresponding to the nearest cluster center. The distance metric used is usually Euclidean distance. Assuming there are two different network topologies T1(x1,y1) and T2(x2,y2), where the average node degree is x, the average link median is y, and the formula for calculating the topological distance d in the clustering algorithm is: For each cluster, calculate the mean of all points within the cluster, and use this mean as the new cluster center; specifically, if cluster c contains points {T1, T2, ..., T...} n The formula for calculating the new cluster centers is as follows: Repeat the above operation until the change in cluster centers is less than a certain threshold or the maximum number of iterations is reached.

[0122] In one embodiment, S5, according to the first computing power routing model, for the first computing network resource scheduling request, selecting a first computing power node and a first route to the first computing power node in the first topology, specifically including:

[0123] Based on the first computing network resource scheduling request, obtain the computing power resource request, network resource request, and source node; obtain the first topology of the first computing power network; and obtain the first number of specified candidate service nodes and the second number of candidate routes, as input to the first computing power routing model.

[0124] The first computing power routing model obtains a first number of candidate computing power nodes and a second number of candidate routes based on the input and the node selection algorithm and candidate route selection algorithm obtained through pre-training. It also obtains the reward for executing each candidate route based on the ∈-greedy exploration strategy and action scoring function obtained through pre-training. The candidate route with the highest reward is selected as the first route and the corresponding candidate computing power node is selected as the first computing power node.

[0125] In this embodiment, the optimization scheme for the computing power network resource scheduling model is as follows:

[0126] 1. Q-value function based on graph neural networks

[0127] This paper models the Q-value function in DRL (Deep Reinforcement Learning) based on a graph neural network. The Q-value function is used to score actions in deep reinforcement learning, and a computational power routing (CPRM) model based on a graph neural network is implemented. The model structure is as follows. Figure 3As shown. In each interaction between the agent and the environment, the DRL agent receives a network topology (a graph structure) and a compute power request (a request for power). During the training phase, the goal of the GNN is to learn the network graph features to help the agent calculate the Q-value of the routing action in the current network state. Then, based on the ∈-greedy exploration strategy, the next action is determined and applied to the environment to obtain a new network state. Using the ∈-greedy exploration strategy means that the probability of executing a random action is λ, while the probability of choosing an action with a higher q-value is (1-λ). This strategy is simple and effective, and can balance exploration and exploitation. The agent is implemented based on the Message Passing Neural Network model, which includes three parts: message passing, message aggregation, and message updating.

[0128] 2. Environment and Status

[0129] In a computing power network, the routing model needs to consider both computing power and traffic requirements. The computing power nodes are the destination nodes for the routes. Each time a user request arrives, one or more candidate computing power nodes need to be selected. Candidate routes are selected based on network conditions, which are defined by the characteristics of the topology links. These characteristics mainly include link capacity, link centrality (path centrality to reachable candidate computing power nodes), and the number of reachable paths. Link capacity represents the available bandwidth on the current link; link centrality describes how many paths traverse the current link (a path consists of multiple links from the origin to the destination), and its value equals the number of paths through the link (reachable candidate computing power nodes) divided by the total number of paths (reachable candidate computing power nodes in the entire topology), i.e., the centrality of the current link among the paths to reachable candidate computing power nodes; the number of reachable paths equals the number of candidate paths from the current link that meet the user's requirements.

[0130] 3. Motion space

[0131] In computing power networks, the number of end-to-end paths that meet user needs is often very large, leading to the need to handle a high-dimensional action space during model training, making routing problems based on deep reinforcement learning extremely complex. To address this issue, this embodiment, based on a node selection strategy using available resources, determines up to three candidate computing power nodes as target nodes, and each node retains two shortest paths to the source node as candidate routes (this can be adjusted based on actual conditions; it is recommended that candidate nodes × candidate routes < 6, otherwise very high computing power is required). For each user request, up to six candidate routes can be generated as the action space.

[0132] 4. Agency and Rewards

[0133] Deep reinforcement learning agents primarily train their models by interacting with the network environment. The environment first receives a traffic demand, represented by a triple Q(C, N, R). The agent explores candidate paths to all computing nodes in D, calculates the q-value for each path, and then uses an ∈-greedy exploration strategy to select the next action. The action is then applied to the environment, resulting in a new state and reward. If the current link satisfies the allocation conditions, it immediately receives a reward proportional to the current traffic demand; otherwise, the reward is zero.

[0134] Table 2 illustrates the specific workflow of the deep reinforcement learning agent. Lines 1-5 initialize the environment by providing the scheduling center with computing network topology information and business request information. Lines 7-8 calculate candidate paths corresponding to all candidate node sets. Lines 9-13 calculate the q-value for all routes that satisfy the business request. Lines 14-15 select the action to execute based on the q-value and the ε_greedy decision method, i.e., selecting an end-to-end traffic route. Line 16 allocates resources in the computing network, obtains the latest environment state, and the reward corresponding to this action execution. Lines 17-20 update the state and accumulate experience to support the next stage of model training.

[0135] Table 2 Examples of Computing Power Routing Models

[0136]

[0137] In one embodiment, wherein:

[0138] The node selection algorithm monitors each service node of the first computing power network and selects a first number of candidate computing power nodes based on the available computing power resources of each service node.

[0139] The candidate routing selection algorithm monitors each link of the first computing power network and selects a second number of candidate routes based on the available bandwidth of each link, the path centrality of reachable candidate computing power nodes, and the number of paths to reachable candidate computing power nodes.

[0140] In this embodiment, the computing power routing model mainly includes:

[0141] 1. Node selection strategy in computing power networks

[0142] In a computing power network, the scheduling center needs to select appropriate computing power nodes to execute tasks based on user needs and node resource availability. A well-designed node selection algorithm can improve resource utilization and achieve reasonable traffic allocation. However, as computing power network service scenarios become more complex, the number of nodes meeting user computing power requests increases, and each computing power node contains multiple routes that meet network bandwidth requirements. The interaction between these two factors leads to the model agent needing to handle a high-dimensional action space, making model training difficult to converge. Solving the routing resource scheduling problem based on deep reinforcement learning becomes extremely complex. To address this issue, a carefully designed node selection strategy is needed to reduce the dimensionality of the action space. This embodiment compares the following node selection strategies:

[0143] (1) Node selection algorithm based on average path: Calculate the average path length of the candidate routes corresponding to each computing power node that meets the user's needs, and prioritize the selection of nodes with shorter average path lengths.

[0144] (2) Node selection algorithm based on available resources: This method requires the scheduling center to monitor and evaluate the available computing resources of each node at all times, and allocate the node with the most available resources to perform the task.

[0145] (3) Random node selection algorithm: Randomly select computing nodes to perform tasks. This strategy is fair, simple and easy to implement, but it cannot take into account the load and performance differences of the nodes.

[0146] The selection of computing nodes needs to consider the actual scenario; there is no single optimal selection strategy. This embodiment primarily determines the three candidate routing strategies based on interpretability and feasibility. Since all strategies require data already present in the model, they have virtually no impact on the model's execution efficiency. The three algorithms are at the same level; only one is selected for each topology scenario, and the specific selection is determined by the model's score. Selecting nodes with shorter average paths or more available resources can improve overall resource utilization. Because the data required by these strategies is already present in the model, implementation is relatively simple, requiring no complex calculations or significant model adjustments.

[0147] 2. Routing strategies in computing power networks

[0148] Table 3 provides a formal description of link state propagation in the model, where link features serve as input and the output is the action evaluation value q. Lines 1-2 iterate through and initialize the feature vectors of all links, including available bandwidth x1, link centrality x2, and the number of reachable computing nodes x3. Lines 3-6 perform state propagation. Line 5 describes the message function m(·) constructed by the fully connected network, aggregating the hidden states of the main edge and its neighbors. Line 6 corresponds to a message update function u(·), which learns the dependencies and temporal correlations in the sequence based on the RNN, capturing long-term dependencies in the sequence. Line 7 models the Readout function r(·) based on the fully connected neural network and outputs the model's evaluation value q. In Table 3: l - an edge in the topology, each l has a corresponding state s; L - the set of all edges; i - the i-th neighbor of the l-th edge; N(l) - the set of neighbors of the l-th edge.

[0149] Table 3 Examples of Routing Selection Algorithms

[0150]

[0151] The beneficial effects of this embodiment 1 are as follows: It constructs a computing power network resource scheduling center to achieve unified scheduling of computing power resources and network resources, providing users with more flexible, efficient, and reliable services; it implements a computing power network resource scheduling model (CPRM) that simultaneously selects nodes and routes, efficiently utilizing computing power and bandwidth resources to meet user needs; it determines three node selection strategies from both interpretability and feasibility perspectives, avoiding high-dimensional action spaces and reducing the training difficulty of the model; it introduces graph neural networks into the CPRM, allowing the model to better learn network features and improving its scalability and fault tolerance; it clusters the network topology based on two graph features—average node degree and average link median—and trains the model on different categories of network topologies, enabling it to cope with complex and ever-changing computing power network service scenarios.

[0152] Example 2:

[0153] like Figure 10 As shown, this disclosure provides a computing power routing device, the device comprising:

[0154] Receiving module 1 is used to receive a first computing network resource scheduling request initiated by the first computing power network;

[0155] Graph feature module 2, connected to receiving module 1, is used to acquire several first graph features of the first topology of the first computing power network;

[0156] Topology classification module 3, connected to graph feature module 2, is used to obtain a first topology classification of the first topology structure based on several first graph features;

[0157] Model acquisition module 4 is connected to topology classification module 3 and is used to acquire the first computing power routing model according to the first topology classification.

[0158] The routing module 5, connected to the model acquisition module 4, is used to select the first computing power node and the first route to the first computing power node in the first topology structure for the first computing network resource scheduling request, based on the first computing power routing model.

[0159] In one embodiment, the receiving module 1 is specifically used for:

[0160] The computing power network scheduling center receives the first computing power network resource scheduling request from the user equipment. The first computing power network resource scheduling request is a triple (C, N, R), where: C represents the user's computing power resource request, N represents the user's network resource request, and R includes the source node corresponding to the user equipment and the requested first computing power network.

[0161] In one embodiment, the graph feature module 2 specifically includes:

[0162] The topology acquisition unit is used to obtain the current real-time topology of the first computing power network from the computing power network scheduling center.

[0163] The feature extraction unit, connected to the topology acquisition unit, is used to extract the first average node degree and the first link intermediate value in the first topology using the NetworkX network analysis library in Python, as the first graph feature.

[0164] Wherein, the first average node degree is the average number of edges connected to each computing node in the first topology, and the first link median value is the average number of edges traversed between any two computing nodes in the first topology.

[0165] In one embodiment, the device includes:

[0166] Graph feature module 2 is also used to obtain several second graph features for each of the multiple second topologies;

[0167] Topology classification module 3 is also used to perform topology classification on multiple second topological structures based on several second graph features;

[0168] Model acquisition module 4 is also used to train and acquire the respective computing power routing model for each topology classification.

[0169] In one embodiment, the graph feature module 2 specifically includes:

[0170] The topology acquisition unit is also used to acquire multiple second topologies from TopologyZoo; the feature extraction unit is also used to extract the second average node degree and the second link median value of each of the multiple second topologies.

[0171] Topology classification module 3 also includes:

[0172] The distance calculation unit is used to calculate the second distance between multiple second topologies based on the second average node degree and the second link median value; the clustering calculation unit is connected to the distance calculation unit and is used to divide the second topologies of multiple computing power networks into K topology categories based on the second distance using the K-means clustering algorithm.

[0173] Model acquisition module 4 also includes:

[0174] The model training unit is used to train the computing power routing model for each topology category using the central second topology structure of each topology category; the model validation unit is connected to the model training unit and is used to validate the generalization performance of the computing power routing model in the corresponding topology category using the non-central second topology structure of each topology category.

[0175] In one embodiment, the topology classification module 3 specifically includes:

[0176] The divergence calculation unit is used to calculate the first KL divergence between the first topology and the second topology, and the second KL divergence between the second topology and the first topology, according to the following formula (1):

[0177]

[0178] The distance calculation unit, connected to the divergence calculation unit, is used to calculate the first distance between the first topology and the second topology according to the following formula (2):

[0179]

[0180] In the formula, P represents the first topology, Q represents the second topology, P1 represents the first average node degree of P, P2 represents the first link median value of P, Q1 represents the second average node degree of Q, Q2 represents the second link median value of Q, KL(P|Q) represents the first KL divergence, and KL(Q|P) represents the second KL divergence.

[0181] The classification unit, connected to the distance calculation unit, is used to obtain the topological classification of the pre-obtained second topological structure in response to a first distance being less than a preset distance threshold, and use it as the first topological classification of the first topological structure.

[0182] In one embodiment, the routing module 5 specifically includes:

[0183] The input unit is used to obtain computing power resource requests, network resource requests, and source nodes according to the first computing network resource scheduling request, obtain the first topology of the first computing power network, and obtain the first number of specified candidate service nodes and the second number of candidate routes as input to the first computing power routing model.

[0184] The output unit, connected to the input unit, is used by the first computing power routing model to obtain a first number of candidate computing power nodes and a second number of candidate routes based on the input and the node selection algorithm and candidate route selection algorithm obtained through pre-training. Based on the ∈-greedy exploration strategy and action scoring function obtained through pre-training, the reward for executing each candidate route is obtained, and the candidate route with the highest reward is obtained as the first route and the corresponding candidate computing power node is obtained as the first computing power node.

[0185] In one embodiment, wherein:

[0186] The node selection algorithm monitors each service node of the first computing power network and selects a first number of candidate computing power nodes based on the available computing power resources of each service node.

[0187] The candidate routing selection algorithm monitors each link of the first computing power network and selects a second number of candidate routes based on the available bandwidth of each link, the path centrality of reachable candidate computing power nodes, and the number of paths to reachable candidate computing power nodes.

[0188] Example 3:

[0189] Embodiment 3 of this disclosure provides a computer-readable storage medium storing a computer program. When the computer program is run by a processor, it implements the computing power routing method as described in Embodiment 1, or the computing power routing device as described in Embodiment 2.

[0190] The computer-readable storage medium includes volatile or non-volatile, removable or non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, computer program modules, or other data). Computer-readable storage media include, but are not limited to, RAM (Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory or other memory technologies, CD-ROM (Compact Disc Read-Only Memory), DVD or other optical disc storage, cartridges, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer.

[0191] In addition, this disclosure may also provide a computer device including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the computing power routing method as described in Embodiment 1, and the computer device may be the computing power routing device as described in Embodiment 2.

[0192] The memory is connected to the processor. The memory can be flash memory, read-only memory or other types of memory. The processor can be a central processing unit or a microcontroller.

[0193] Embodiments 1-3 of this disclosure provide a computing power routing method, a computing power routing device, and a computer-readable storage medium. The computing power routing model is obtained based on a differentiated network topology. This computing power routing model can adapt to the scheduling of computing network resources in complex business scenarios, thereby improving the ability to cope with complex and ever-changing computing network business scenarios.

[0194] It is understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of this disclosure, and this disclosure is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of this disclosure, and these modifications and improvements are also considered to be within the scope of protection of this disclosure.

Claims

1. A computing power routing method, characterized in that, The method includes: Receive the first computing network resource scheduling request initiated by the first computing power network; Obtain several first graph features of the first topology of the first computing power network; The first topological classification of the first topological structure is obtained based on several features of the first graph. The first computing power routing model is obtained based on the first topology classification; According to the first computing power routing model, for the first computing network resource scheduling request, the first computing power node and the first route to the first computing power node are selected in the first topology; The acquisition of several first graph features of the first topology of the first computing power network specifically includes: Obtain the current real-time topology of the first computing power network from the computing power network scheduling center; The first average node degree and the median value of the first link in the first topology are extracted using the NetworkX network analysis library in Python as features of the first graph. Wherein, the first average node degree is the average number of edges connected to each computing node in the first topology, and the first link median value is the average number of edges traversed between any two computing nodes in the first topology.

2. The method according to claim 1, characterized in that, Receiving a first computing network resource scheduling request initiated by the first computing power network specifically includes: The computing power network scheduling center receives the first computing power network resource scheduling request from the user equipment. The first computing power network resource scheduling request is a triple (C, N, R), where: C represents the user's computing power resource request, N represents the user's network resource request, and R includes the source node corresponding to the user equipment and the requested first computing power network.

3. The method according to claim 1, characterized in that, Before receiving a first computing network resource scheduling request initiated by the first computing power network, the method further includes: Obtain several second graph features for each of the multiple second topologies; Topological classification of multiple second topological structures based on several features of the second graph; For each topology classification, a corresponding computing power routing model is trained to obtain its own model.

4. The method according to claim 3, characterized in that, Obtain several second graph features for each of the multiple second topologies, specifically including: Obtain multiple second topologies from TopologyZoo, and extract the second average node degree and the second link median value for each of the multiple second topologies. Topological classification of multiple second topological structures is performed based on several features of the second graph, specifically including: The second distance between multiple second topologies is calculated based on the second average node degree and the second link median value. Based on the K-means clustering algorithm, the second topologies of multiple computing power networks are divided into K topology categories according to the second distance. For each topology classification, a corresponding computational power routing model is trained, specifically including: The computational power routing model for each topology category is trained using the central second topology structure of each topology category, and the generalization performance of the computational power routing model in the corresponding topology category is verified using the non-central second topology structure of each topology category.

5. The method according to claim 4, characterized in that, The first topological classification of the first topological structure is obtained based on several features of the first graph, specifically including: Calculate the first KL divergence between the first and second topologies and the second KL divergence between the second and first topologies according to the following formula (1): The first distance between the first topology and the second topology is calculated according to the following formula (2): In the formula, P represents the first topology, Q represents the second topology, P1 represents the first average node degree of P, P2 represents the first link median value of P, Q1 represents the second average node degree of Q, Q2 represents the second link median value of Q, KL(P|Q) represents the first KL divergence, and KL(Q|P) represents the second KL divergence. In response to a first distance being less than a preset distance threshold, the topology classification of the pre-obtained second topology structure is acquired and used as the first topology classification of the first topology structure.

6. The method according to any one of claims 3-5, characterized in that, According to the first computing power routing model, for the first computing network resource scheduling request, the first computing power node and the first route to the first computing power node are selected in the first topology, specifically including: Based on the first computing network resource scheduling request, obtain the computing power resource request, network resource request, and source node; obtain the first topology of the first computing power network; and obtain the first number of specified candidate service nodes and the second number of candidate routes, as input to the first computing power routing model. The first computing power routing model obtains a first number of candidate computing power nodes and a second number of candidate routes based on the input and the node selection algorithm and candidate route selection algorithm obtained through pre-training. It also obtains the reward for executing each candidate route based on the ∈-greedy exploration strategy and action scoring function obtained through pre-training. The candidate route with the highest reward is selected as the first route and the corresponding candidate computing power node is selected as the first computing power node.

7. The method according to claim 6, characterized in that, in: The node selection algorithm monitors each service node of the first computing power network and selects a first number of candidate computing power nodes based on the available computing power resources of each service node. The candidate routing selection algorithm monitors each link of the first computing power network and selects a second number of candidate routes based on the available bandwidth of each link, the path centrality of reachable candidate computing power nodes, and the number of paths to reachable candidate computing power nodes.

8. A computing power routing device, characterized in that, The device includes: The receiving module is used to receive the first computing network resource scheduling request initiated by the first computing power network; The graph feature module, connected to the receiving module, is used to acquire several first graph features of the first topology of the first computing power network. The topology classification module, connected to the graph feature module, is used to obtain the first topology classification of the first topology structure based on several first graph features. The model acquisition module is connected to the topology classification module and is used to acquire the first computing power routing model based on the first topology classification. The routing module, connected to the model acquisition module, is used to select the first computing power node and the first route to the first computing power node in the first topology structure for the first computing network resource scheduling request, based on the first computing power routing model. The graph feature module specifically includes: The topology acquisition unit is used to obtain the current real-time topology of the first computing power network from the computing power network scheduling center. The feature extraction unit, connected to the topology acquisition unit, is used to extract the first average node degree and the first link intermediate value in the first topology using the NetworkX network analysis library in Python, as the first graph feature. Wherein, the first average node degree is the average number of edges connected to each computing node in the first topology, and the first link median value is the average number of edges traversed between any two computing nodes in the first topology.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the computing power routing method as described in any one of claims 1-7.

Images