A resource collaborative scheduling method based on graph neural network in computing power network

Abstract

The present invention discloses a resource collaborative scheduling method based on graph neural network in a computing power network, and belongs to the field of computer science. The present invention enables the computing power network to operate efficiently and improve network performance while comprehensively considering the resource usage status of each computing node in the network; the network environment is modeled into a graph structure, and GNN is used to update and extract status information in the network environment in real time; in order to maximize the performance of the computing power network and meet the diverse needs of users, the present invention combines GNN and DDPG, uses a dual neural network architecture to separate strategy learning and value learning, reduces the risk of overestimation, and introduces the experience replay method to make the algorithm easier to converge, improve the stability and learning efficiency of the algorithm, and enhance the generalization ability and robustness of the model.

Description

Technical Field

[0001] The present invention belongs to the field of computers, and specifically relates to a resource collaborative scheduling method based on graph neural networks in a computing power network. Background Art

[0002] Compute First Networking (CFN) is a new type of network architecture that aims to integrate various computing resources (such as edge computing devices, cloud computing centers, smart terminal devices, etc.) to form a huge computing network, and realize on-demand allocation and flexible scheduling of computing resources and network resources between cloud-edge-end. It makes more efficient use of the massive distributed computing resources at the edge of the network and provides customers with more efficient and flexible computing services.

[0003] However, due to the dynamics and uncertainty of the network environment, when the network topology changes (for example, new computing nodes join or existing nodes leave the network), traditional deep reinforcement learning-based algorithms need to retrain the model to adapt to the new environment, which is time-consuming and inefficient. The present invention discloses a method for processing real-time optimization problems in a dynamic network environment. Currently, most of the research on the collaborative scheduling of computing resources and network resources in computing power networks does not involve the problem of resource balancing when the network topology changes. Therefore, the present invention combines the Graph Neural Network (GNN) with the Deep Deterministic Policy Gradient (DDPG) algorithm to disclose a collaborative scheduling method that maintains resource balance in a dynamic network. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a GNN-based method for collaborative scheduling of computing resources and network resources in response to the shortcomings of the background technology; in order to maximize the performance of the computing network and meet the diverse needs of users, the present invention combines GNN and DDPG to publish an algorithm for efficiently utilizing computing resources in a dynamic network environment.

[0005] The present invention adopts the following technical solutions to solve the above technical problems:

[0006] A resource collaborative scheduling method based on a graph neural network in a computing network, including an application scenario of resource collaborative scheduling based on a graph neural network, wherein the application scenario of resource collaborative scheduling based on a graph neural network includes a cloud-edge-end collaborative computing network environment and an intelligent agent and network node communication and computing environment;

[0007] The cloud-edge-end collaborative computing network environment includes a cloud computing server cluster, an edge device set, and a terminal user set;

[0008] The cloud computing server cluster includes elastic servers and professional servers;

[0009] The edge device set includes an edge server and a base station connecting the network and the terminal device;

[0010] The terminal user set includes terminal devices such as smart phones, mobile tablets, and laptops;

[0011] When a terminal sends a service request to the computing network, the agent will interact with the cloud computing server cluster and edge device collection to allocate a suitable computing node;

[0012] The communication and computing environment between the intelligent agent and the network node includes the intelligent agent evaluating the resource status of the current network node according to the received service request, including available resource capacity, network bandwidth, and network delay; at the same time, the graph neural network GNN is used to combine the real-time status of network resources and the requirements of service requests to calculate the optimal resource allocation plan and send instructions to the computing nodes involved in the plan. The computing nodes that receive the instructions will feedback the actual use of resources, and then the graph neural network GNN will update the current network status information according to the received feedback information. The intelligent agent will adjust the resource allocation plan in time after obtaining the new network status information to ensure the effective use of computing resources and the efficient processing of service requests to adapt to the dynamic changes of network resources and the diversity of service requests;

[0013] The specific steps include:

[0014] Step 1: Use the message passing neural network MPNN framework in the graph neural network GNN to effectively model the network environment; and obtain the state information of the graph through the message passing neural network MPNN;

[0015] Step 2: Obtain a more reliable resource allocation solution through the DDPG algorithm;

[0016] Step 3: At each decision moment, the agent uses the DDPG algorithm to determine the optimal resource allocation strategy based on the current network environment status and service request requirements, and allocates the most appropriate resources to the task based on the determined optimal strategy;

[0017] Step 4: After resource allocation, the agent will continuously monitor the system's operating status and performance, collect feedback information, and adjust the resource allocation strategy based on the feedback information to ensure the efficiency and stability of the system.

[0018] As a further preferred solution of a resource collaborative scheduling method based on graph neural network in a computing power network of the present invention, in step 1, the nodes in the established graph structure framework represent the various entities in the network, including computing nodes, servers, and terminal devices, while the edges represent the connections and interaction relationships between these entities.

[0019] As a further preferred solution of the resource collaborative scheduling method based on graph neural network in a computing network of the present invention, in step 2, the DDPG algorithm consists of two core parts: an Actor network for decision-making and a Critic network for value evaluation; a dual neural network architecture, namely the current network and the target network, is used, that is, DDPG actually introduces four networks for making decisions and value evaluation; the specific division of labor of the four networks, the Actor network and the Critic network, is as follows: The purpose of the Actor current network is to make decisions based on the current network status information. Make a decision to choose the current action ; At the same time, interact with the network environment to update the network status information to obtain the new status and rewards ; In this model, the input of the Actor's current network is the state information of the network environment , choose the time step to be The purpose of the Actor target network is to replay the next state sampled in the pool based on experience , choose the best next action ; Network status information is periodically obtained from the Actor's current network; The input of the Actor's target network includes the experience in the experience replay pool and the network status information copied from the Actor's current network;

[0020] The main responsibility of the Critic current network is to evaluate the value of the current state. According to the action selected by the Actor current network and the reward of environmental feedback, the target Q value is calculated as the basis for action selection. The network updates the value network parameters through iteration. , optimize the accuracy of value assessment; the Critic target network is used to evaluate and calculate the Q value function of the state-action at the next moment, and the network parameters Regularly utilize the parameters of the Critic current network Updated to generate more accurate target value estimates, helping to optimize performance across the network.

[0021] As a further preferred solution of the resource collaborative scheduling method based on graph neural network in a computing power network of the present invention, in step 3, the implementation of the optimal resource allocation strategy includes: obtaining status information through GNN; the intelligent agent obtains the optimal allocation strategy; the intelligent agent allocates resources and continuously monitors the system status information.

[0022] As a further preferred solution of the resource collaborative scheduling method based on graph neural network in a computing network of the present invention, in step 3, the state information is obtained through GNN, as follows: the network environment essentially has the characteristics of a graph structure, in which nodes represent various entities in the network, including computing nodes, servers, and terminal devices, and edges represent the connection and interaction relationship between these entities; that is, the network topology can be defined as ,in represents a collection of network entities, Represents the relationship between entities; uses the message passing neural network framework in GNN to effectively model the network environment; the forward propagation of MPNN is generally divided into two stages: the information transmission stage and the reading stage:

[0023] When the information transmission phase operates for t time steps, each node According to its neighbor nodes and its own hidden state and the edge features between the two Function passing via message generate The message received at time step t+1 ,in For Node exist Hidden state at time step; received message back Will be based on its own hidden state and Message Using the update function Get its own hidden state at time step t+1 ; After the information is transmitted, the node Current hidden state Use the read function R to calculate the eigenvector of the entire graph , Indicates the status of the decision step of the current task to be decided ; Among them, the node hidden state is a vector describing node features in the MPNN framework; computing resources are used arrive Representation, including the computing type, computing energy consumption, available resource capacity and nodes allocated to the service request;

[0024] In the DDPG algorithm, the environment needs to return the reward of the current action after executing the corresponding action. Consider optimizing both computing resources and network resources to achieve resource balance and design the reward function as follows: ;in, Indicates that due to the execution of an action The resulting computing resource utilization and network resource utilization The reward value, Represents actions for low energy consumption and low latency The reward value, To calculate the energy consumption, Represents the total amount of computing resources, Indicates network delay Used to measure execution actions back and Target utilization the degree of difference between is the weight coefficient; is a normalization factor to ensure that these indicators are calculated on the same dimension.

[0025] As a further preferred solution of the resource collaborative scheduling method based on graph neural network in a computing network of the present invention, in step 3, the intelligent agent obtains the optimal allocation strategy, which is as follows: the DDPG algorithm is adopted, and the optimization goal is to maximize the expected cumulative reward, and the merits of the strategy are measured by the reward, and then the optimal resource allocation strategy is obtained by the maximum cumulative reward; in DDPG, a deep neural network is used to approximate the strategy function and value function , where the policy function is used to output the optimal action under a given network state, and the value function is used to calculate the Q value at different times; the DDPG algorithm based on graph neural network is implemented as follows:

[0026] (1) A dual neural network architecture is used for both the policy function and the value function. Therefore, the policy optimization model includes the Actor current network and the Actor target network, the Critic current network and the Critic target network. First, initialize the network parameters of the four networks: Randomly initialize the Actor current network parameters and Critic's current network parameters , and update the corresponding target network parameters and ,initialization , the number of training time steps per round Considering the complexity of network models in practical applications, to ensure that the model has enough opportunities to learn the potential patterns in the data, the maximum number of training rounds is set to is the number of nodes; at the same time, to avoid overfitting of the model, set the verification frequency , tolerance period ,counter , that is, every 35 Perform a validation. If the current performance does not exceed the best performance, increase the counter, otherwise reset the counter. If the counter reaches the tolerance limit, stop training. Avoid overfitting and save computing resources.

[0027] (2) In step 1, the state of the decision step of the task to be decided in the network environment has been obtained through MPNN , the Actor current network receives the status from the network environment , based on the state Select current action , Indicates that at the moment of decision The random noise added during the learning process increases the coverage of learning; then the Actor's current network interacts with the network environment, and the environment responds to the action. After allocating corresponding resources to the subtask, the new status is obtained And rewards , and then the current state at each decision moment ,action ,award and the next state The experience tuple is stored in the experience replay pool; at the same time, the Actor target network is in the experience replay collection Sampling is performed in , choose the best next action , these actions are used as one of the inputs of the Critic target network, which then replays the set of experiences Random sampling Tuples are used as samples for training to evaluate the Q value function of the state-action at the next moment The result is input as a parameter into the current network of Critic to calculate the Q value of the current target. Considering that the influence of future rewards on current decisions will gradually decrease with the increase of time steps, the attenuation factor is introduced. Reduce the influence of future rewards on current decisions; the specific calculation formula can be expressed as: ;

[0028] (3) Since the network topology environment, computing resources and network resources are changing dynamically, the Actor network and the Critic network need to constantly adapt to the changing network environment to provide accurate evaluation or decision-making. The specific implementation method is to continuously update the network parameters; through the mean square error loss function Use the gradient back propagation of the neural network to update all parameters of the Critic current network , combined with the Q value function of the Critic current network, using the Actor strategy gradient function Update the current network parameters of the Actor; in order to enhance the stability and convergence of training, the target network in DDPG uses soft update to update the network parameters. The target network update method of Critic is , the Actor target network update method is ,in is the soft update coefficient;

[0029] (4) Update the state to the next state and repeat the above three steps. If the termination condition is reached, the loop is exited and the specific actions of the optimal resource allocation plan obtained through continuous training, the corresponding Q value estimation, and the network parameters of the Actor network and the Critic network are returned.

[0030] As a further preferred solution of a resource collaborative scheduling method based on graph neural network in a computing power network of the present invention, in step 4, the intelligent agent obtains the specific actions of the optimal resource allocation plan through the DDPG algorithm at each decision-making moment according to the current network environment status and service request requirements, and allocates the most suitable resources to each task; the computing node that receives the task will feedback the actual usage of the resources, and then use MPNN to update the current network status information according to the received feedback information. The intelligent agent will adjust the resource allocation plan in time when obtaining the new network status information to ensure the effective utilization of computing power resources and efficient processing of service requests, so as to adapt to the dynamic changes of network resources and the diversity of service requests.

[0031] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0032] The present invention discloses a GNN-based collaborative scheduling method for computing resources and network resources. On the one hand, the present invention utilizes GNN's powerful information processing capability, topological adaptability, optimized scheduling capability, and collaborative decision-making capability to enable the computing network to operate efficiently and improve network performance while comprehensively considering the resource usage status of each computing node in the network. On the other hand, the network environment is modeled into a graph structure, and GNN is used to update and extract status information in the network environment in real time. In order to maximize the performance of the computing network and meet the diverse needs of users, the present invention combines GNN and DDPG to disclose an algorithm for efficiently utilizing computing resources in a dynamic network environment.

[0033] The present invention uses GNN to process the topological structure and complex relationships between nodes in the computing network, so as to support the present invention to adapt to the changes in the network topological structure and optimize the network performance; according to the state information of the graph provided by MPNN, the DDPG algorithm is used to enable the intelligent agent to continuously coordinate and optimize the computing resources and computing power resources in the dynamic network environment to achieve the purpose of resource balance, so that the application scenario is more in line with the actual situation and can achieve better results;

[0034] The present invention uses a dual neural network architecture to separate strategy learning and value learning, which reduces the risk of over-estimation, and introduces experience replay to make the algorithm easier to converge, improves the stability and learning efficiency of the algorithm, and enhances the generalization ability and robustness of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0036] Figure 1 It is an application scenario of the present invention, including a cloud-edge-end collaborative computing network environment and an agent and network node communication and computing environment;

[0037] Figure 2 It is a framework combining GNN and DDPG algorithms in this invention;

[0038] Figure 3 It is a flow chart of a resource collaborative scheduling method based on graph neural network in a computing power network of the present invention. DETAILED DESCRIPTION

[0039] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings:

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. The present invention is described in detail below based on the drawings and preferred embodiments, and the purpose and effect of the present invention will become clearer. It should be understood that the specific embodiments described here are only used to explain the present invention and are not used to limit the present invention.

[0041] A GNN-based resource collaborative scheduling method in a computing network. The GNN-based resource collaborative scheduling application scenario includes a cloud-edge-end collaborative computing network environment and an intelligent agent and network node communication and computing environment. Figure 1As shown, the cloud-edge-end collaborative computing network environment includes cloud computing server clusters such as elastic servers and professional servers; edge device collections such as edge servers and base stations connecting the network and terminal devices; and terminal user device collections such as smart phones, mobile tablets, and laptops. When the terminal sends a service request to the computing network, the intelligent agent will interact with the cloud computing server cluster and the edge device collection to allocate suitable computing nodes.

[0042] The communication and computing environment between the intelligent agent and the network nodes includes that after receiving a service request, the intelligent agent will evaluate the resource status of the current network node; at the same time, GNN will be used to combine the real-time status of network resources and the requirements of service requests to calculate the optimal resource allocation plan and send instructions to the computing nodes involved in the plan. The computing nodes that receive the instructions will feedback the actual usage of the resources, and then GNN will update the current network status information based on the received feedback information. The intelligent agent will adjust the resource allocation plan in time when it obtains the new network status information to ensure the effective use of computing resources and efficient processing of service requests to adapt to the dynamic changes of network resources and the diversity of service requests.

[0043] like Figure 3 As shown, the resource collaborative scheduling method based on graph neural network specifically includes the following steps: obtaining status information through GNN:

[0044] Specifically, the network environment is essentially a graph structure, where nodes represent entities in the network, including computing nodes, servers, and terminal devices, and edges represent the connections and interactions between these entities; that is, the network topology can be defined as ,in represents a collection of network entities, Represents the relationship between entities; uses the message passing neural network framework in GNN to effectively model the network environment; the forward propagation of MPNN is generally divided into two stages: the information transmission stage and the reading stage:

[0045] When the information transmission phase operates for t time steps, each node According to its neighbor nodes and its own hidden state and the edge features between the two Function passing via message generate The message received at time step t+1 ,in For Node exist Hidden state at time step; received message back Will be based on its own hidden state and Message Using the update function Get its own hidden state at time step t+1 ; After the information is transmitted, the node Current hidden state Use the read function R to calculate the eigenvector of the entire graph , Indicates the status of the decision step of the current task to be decided ; Among them, the node hidden state is a vector describing node features in the MPNN framework.

[0046] These features provide the intelligent agent with key information about the network environment and node status, thereby helping the intelligent agent make more reasonable decisions. In this example, the hidden state is given the meaning shown in Table 1. arrive Indicates the role of network status and bandwidth allocation. Link capacity indicates the threshold of available bandwidth. Link betweenness indicates the number of paths flowing through the link divided by the total number of paths, reflecting the priority of the link. The larger the link betweenness, the greater the possibility that the link will become a critical path in the topology. Network delay refers to the time required for data to be transmitted from one node to another in the network. The larger the network delay, the worse the network performance. arrive It represents the computing type, computing energy consumption, available resource capacity and nodes allocated to the service request; Table 1 shows the meaning of the symbols in the hidden state of the node.

[0047] Table 1

[0048]

[0049] In the DDPG algorithm, the environment needs to return the reward of the current action after executing the corresponding action. Consider optimizing both computing resources and network resources to achieve resource balance and design the reward function as follows: ;in, Indicates that due to the execution of an action The resulting computing resource utilization and network resource utilization The reward value, Represents actions for low energy consumption and low latency The reward value, To calculate the energy consumption, Represents the total amount of computing resources, Indicates network delay Used to measure execution actions back and Target utilization the degree of difference between is the weight coefficient; is a normalization factor to ensure that these indicators are calculated on the same dimension.

[0050] The agent obtains the optimal allocation strategy, as follows:

[0051] This paper adopts the DDPG algorithm, and the optimization goal is to maximize the expected cumulative reward. The reward is used to measure the quality of the strategy, and then the optimal resource allocation strategy is obtained through the maximum cumulative reward. In DDPG, a deep neural network is used to approximate the strategy function. and value function , where the policy function is used to output the optimal action under a given network state, and the value function is used to calculate the Q value at different times; in this implementation scheme, the DDPG algorithm based on graph neural network is implemented as follows:

[0052] (1) A dual neural network architecture is used for both the policy function and the value function. Therefore, the policy optimization model includes the Actor current network and the Actor target network, the Critic current network and the Critic target network. First, initialize the network parameters of the four networks: Randomly initialize the Actor current network parameters and Critic's current network parameters , and update the corresponding target network parameters and ,initialization , the number of training time steps per round Considering the complexity of network models in practical applications, to ensure that the model has enough opportunities to learn the potential patterns in the data, the maximum number of training rounds is set to is the number of nodes; at the same time, to avoid overfitting of the model, set the verification frequency , tolerance period ,counter , that is, every 35 Perform a validation. If the current performance does not exceed the best performance, increase the counter, otherwise reset the counter. If the counter reaches the tolerance limit, stop training. Avoid overfitting and save computing resources.

[0053] (2) In step 1, the state of the decision step of the task to be decided in the network environment has been obtained through MPNN , the Actor current network receives the status from the network environment , based on the state Select current action , Indicates that at the moment of decision The random noise added during the learning process increases the coverage of learning; then the Actor's current network interacts with the network environment, and the environment responds to the action. After allocating corresponding resources to the subtask, the new status is obtained And rewards , and then the current state at each decision moment ,action ,award and the next state The experience tuple is stored in the experience replay pool; at the same time, the Actor target network is in the experience replay collection Sampling is performed in , choose the best next action , these actions are used as one of the inputs of the Critic target network, which then replays the set of experiences Random sampling Tuples are used as samples for training to evaluate the Q value function of the state-action at the next moment The result is input as a parameter into the current network of Critic to calculate the Q value of the current target. Considering that the influence of future rewards on current decisions will gradually decrease with the increase of time steps, the attenuation factor is introduced. Reduce the influence of future rewards on current decisions; the specific calculation formula can be expressed as: ;

[0054] (3) Since the network topology environment, computing resources and network resources are changing dynamically, the Actor network and the Critic network need to constantly adapt to the changing network environment to provide accurate evaluation or decision-making. The specific implementation method is to continuously update the network parameters; through the mean square error loss function Use the gradient back propagation of the neural network to update all parameters of the Critic current network , combined with the Q value function of the Critic current network, using the Actor strategy gradient function Update the current network parameters of the Actor; in order to enhance the stability and convergence of training, the target network in DDPG uses soft update to update the network parameters. The target network update method of Critic is , the Actor target network update method is ,in is the soft update coefficient;

[0055] (4) The algorithm proposed in this invention is suitable for dynamically changing network environments and therefore requires continuous iteration: update the state to the next state and repeat the above three steps. If the termination condition is reached, the loop is exited and the specific actions of the optimal resource allocation solution obtained through continuous training, the corresponding Q value estimation, and the network parameters of the Actor network and the Critic network are returned. These outputs help the intelligent agent to achieve optimized resource scheduling and allocation and improve the utilization efficiency of computing resources.

[0056] The agent allocates resources and continuously monitors system status information: the agent According to the current network environment status and service request requirements, the DDPG algorithm is used to obtain the specific actions of the optimal resource allocation plan, and the most appropriate resources are allocated to each task. The computing node that receives the task will feedback the actual use of resources, and then use MPNN to update the current network status information according to the received feedback information. When the intelligent agent obtains the new network status information, it will adjust the resource allocation plan in time to ensure the effective use of computing resources and efficient processing of service requests, so as to adapt to the dynamic changes of network resources and the diversity of service requests.

[0057] Complete the computing task and return the computing result: According to the strategy determined by the above algorithm, the intelligent agent allocates corresponding resources to the task. After receiving the assigned task and data, the computing node starts to execute the computing task. After completing the computing task, the computing result is returned to the user terminal through the communication link. In the whole process, the dynamic nature of the network environment is taken into account, which is more in line with the actual application scenario, realizes the efficient use of computing resources, meets the diverse needs of users, and improves the performance of the computing network.

[0058] Those skilled in the art can understand that the above are only preferred examples of the invention and are not intended to limit the invention. Although the invention is described in detail with reference to the above examples, those skilled in the art can still modify the technical solutions recorded in the above examples or replace some of the technical features with equivalents. Any modification, equivalent replacement, etc. made within the spirit and principle of the invention should be included in the protection scope of the invention. All technical features in this embodiment can be freely combined according to actual needs.

[0059] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A resource collaborative scheduling method based on graph neural network in a computing network, characterized by: It includes a resource collaborative scheduling application scenario based on graph neural network, and the resource collaborative scheduling application scenario based on graph neural network includes a cloud-edge-end collaborative computing network environment and an intelligent agent and network node communication and computing environment; When a terminal sends a service request to the computing network, the agent will interact with the cloud computing server cluster and edge device collection to allocate a suitable computing node; The communication and computing environment between the intelligent agent and the network node includes the intelligent agent evaluating the resource status of the current network node according to the received service request, including the available resource capacity, network bandwidth, and network delay; using the graph neural network GNN in combination with the real-time status of the network resources and the requirements of the service request, calculating the optimal resource allocation plan and sending instructions to the computing nodes involved in the plan. The computing nodes that receive the instructions will feedback the actual usage of the resources, and the graph neural network GNN will update the current network status information according to the received feedback information. The intelligent agent obtains the new network status information and adjusts the resource allocation plan; include: Step 1: Use the message passing neural network MPNN framework in the graph neural network GNN to model the network environment; and obtain the state information of the graph through the message passing neural network MPNN; Step 2: Obtain resource allocation plan through DDPG algorithm; Step 3: At each decision moment, the agent uses the DDPG algorithm to determine the optimal resource allocation strategy based on the current network environment status and service request requirements, and allocates resources to the task based on the determined optimal strategy; Step 4: After resource allocation, the agent will continuously monitor the system's operating status and performance, collect feedback information, and adjust the resource allocation strategy based on the feedback information.

2. According to claim 1, a resource collaborative scheduling method based on graph neural network in a computing power network is characterized in that: In step 1, the nodes in the established graph structure framework represent the various entities in the network, including computing nodes, servers, and terminal devices, while the edges represent the connections and interactions between these entities.

3. The resource collaborative scheduling method based on graph neural network in a computing network according to claim 1 is characterized in that: In step 2, the DDPG algorithm consists of two core parts: the Actor network for decision-making and the Critic network for value evaluation. Using a dual neural network architecture, namely the current network and the target network, DDPG actually introduces four networks for decision-making and value evaluation. The specific division of labor of the four networks, the Actor network and the Critic network, is as follows: 1) The purpose of the Actor current network is to use the current network status information s t Make a decision to choose the current action a t ; Interact with the network environment to update network status information to obtain a new status s t+1 and reward R t ; The input of the Actor's current network is the state information s of the network environment t , select the action when the time step is t; 2) The purpose of the Actor target network is to replay the next state s sampled in the pool based on experience t+1 , choose the best next action a t+1 ; Network status information is periodically obtained from the Actor's current network; The input of the Actor's target network includes the experience in the experience replay pool and the network status information copied from the Actor's current network; 3) The main responsibility of the Critic network is to evaluate the value of the current state. According to the action selected by the Actor network and the reward of the environment feedback, the target Q value is calculated as the basis for action selection. The network updates the value network parameter θ through iteration. Q , optimize the accuracy of value assessment; 4) The Critic target network is used to evaluate the Q value function of the state-action at the next moment, and the network parameter θ Q ′Regularly use the parameters θ of the current network of Critic Q Updated to generate more accurate target value estimates, helping to optimize performance across the network.

4. The resource collaborative scheduling method based on graph neural network in a computing network according to claim 1, characterized in that: In step 3, the optimal resource allocation strategy is implemented by: obtaining status information through GNN; the agent obtains the optimal allocation strategy; the agent allocates resources and continuously monitors the system status information.

5. The resource collaborative scheduling method based on graph neural network in a computing network according to claim 1, characterized in that: In step 3, the state information is obtained through GNN, as follows: the network environment essentially has the characteristics of a graph structure, where nodes represent various entities in the network, including computing nodes, servers, and terminal devices, and edges represent the connections and interactions between these entities; that is, the network topology can be defined as G = (V, E), where V = {v i |i=1,2,...,n} represents the set of network entities, Represents the relationship between entities; uses the message passing neural network framework in GNN to effectively model the network environment; the forward propagation of MPNN is divided into two stages: information transmission stage and reading stage: 1) When the information transmission phase operates for t time steps, each node v i According to its neighbor node v j and its own hidden state and the edge features between the two By message passing function φ t Generate v i The message received at time step t+1 in For node v i Hidden state at time step t; received message After v i Will be based on its own hidden state and Message Using the update function U t Get its own hidden state at time step t+1 2) After the information is transmitted, in the reading phase, according to the node v i Current hidden state Use the read function R to calculate the eigenvector of the entire graph Represents the state s of the decision step of the current decision task t ; Among them, the node hidden state is a vector describing node characteristics in the MPNN framework; computing resources are represented by b5 to b8, including the computing type, computing energy consumption, available resource capacity and nodes allocated for the service request; In the DDPG algorithm, the environment needs to return the reward of the current action after executing the corresponding action. Considering the optimization of computing resources and network resources, the reward function is designed to achieve resource balance: R t (s t ,a t )=ι·Utilization(s t , at )+α·Efficiency(s t ,a t )+β·Balance(s t ,a t ); in, Indicates that due to the execution of action a t The resulting computing resource utilization U C，t and network resource utilization U N，t The reward value, Represents actions for low energy consumption and low latency a t The reward value, E is the computing energy consumption, N total represents the total amount of computing resources, and D represents network latency Used to measure the execution of action a t After U C，t with U N，t and target utilization U target the degree of difference between ξ, is the weight coefficient; a, β, ι, ω are standardization factors to ensure that these indicators are calculated on the same dimension.

6. The resource collaborative scheduling method based on graph neural network in a computing network according to claim 1, characterized in that: In step 3, the agent obtains the optimal allocation strategy as follows: The DDPG algorithm is used, and the optimization goal is to maximize the expected cumulative reward. The reward is used to measure the quality of the strategy, and then the optimal resource allocation strategy is obtained through the maximum cumulative reward. In DDPG, a deep neural network is used to approximate the policy function λ and the value function Q, where the policy function is used to output the optimal action under a given network state, and the value function is used to calculate the Q value at different times. The DDPG algorithm based on graph neural network is implemented as follows: (1) A dual neural network architecture is used for both the policy function and the value function. Therefore, the policy optimization model includes the Actor current network and the Actor target network, the Critic current network and the Critic target network. First, the network parameters of the four networks are initialized: randomly initialize the Actor current network parameter η μ and the current network parameter θ of Critic Q , update the corresponding target network parameter η μ ′ and θ Q ', initialize episode = 1, the number of time steps per training round is t = 1, and set the maximum number of training rounds to episode max =n-1, n is the number of nodes; to avoid overfitting of the model, set the verification frequency f=35, tolerance period patience=20, counter flag=0, that is, verification is performed every 35 episodes. If the current performance does not exceed the best performance, the counter is increased, otherwise the counter is reset; if the counter reaches the tolerance period setting value, the training is stopped to avoid overfitting and save computing resources; (2) In step 1, the state s of the decision step of the task to be decided in the network environment has been obtained through MPNN t , the Actor current network receives the state s from the network environment t , based on the state s t Select current action ε t represents the random noise added at the decision time τ to increase the coverage of learning; then the Actor current network interacts with the network environment, and the environment responds to the action a t After allocating corresponding resources to the subtasks, the new state s is obtained t+1 And the reward R t , and then the current state s at each decision moment t 、Action a t , Reward R t and the next state s t+1 Stored as an experience tuple in the experience replay pool; sampled with this Actor target network in the experience replay set W to obtain s t+1 , choose the best next action a t+1 , these actions are used as one of the inputs of the Critic target network, which then randomly samples x tuples from the experience replay set W as samples for training to evaluate the Q-value function of the state-action at the next moment The result is input as a parameter into the current network of Critic to calculate the Q value of the current target. Considering that the influence of future rewards on the current decision will gradually decrease with the increase of time steps, the attenuation factor v is introduced to reduce the influence of future rewards on the current decision. The specific calculation formula can be expressed as: (3) Since the network topology environment, computing resources and network resources are changing dynamically, the Actor network and the Critic network need to constantly adapt to the changing network environment to provide accurate evaluation or decision-making. The specific implementation method is to continuously update the network parameters; through the mean square error loss function Use the gradient back propagation of the neural network to update all parameters θ of the Critic current network Q , combined with the Q value function of the Critic current network, using the Actor strategy gradient function Update the current network parameters of the Actor; in order to enhance the stability and convergence of training, the target network in DDPG uses soft update to update the network parameters. The target network update method of Critic is The Actor target network update method is in is the soft update coefficient; (4) Update the state to the next state and repeat the above three steps. If the termination condition is met, the loop is exited and the specific actions of the optimal resource allocation solution obtained through continuous training, the corresponding Q value estimation, and the network parameters of the Actor network and the Critic network are returned.

7. The resource collaborative scheduling method based on graph neural network in a computing network according to claim 1 is characterized in that: In step 4, at each decision moment, the agent uses the DDPG algorithm to obtain the specific actions of the optimal resource allocation plan according to the current network environment status and service request requirements, and allocates resources to each task; the computing node that receives the task will feedback the actual usage of the resources, and then use MPNN to update the current network status information according to the received feedback information. The agent will adjust the resource allocation plan in time after obtaining the new network status information to ensure the effective use of computing resources and efficient processing of service requests, so as to adapt to the dynamic changes of network resources and the diversity of service requests.

Images