Distributed computing power network task scheduling method, device and equipment

Abstract

The present disclosure provides a distributed computing power network task scheduling method, device and equipment, wherein the distributed computing power network includes a user-side edge router and a computing power-side edge router, and the method comprises: the user-side edge router acquires a target computing power demand and determines a current computing power state of the distributed computing power network according to a multi-protocol border gateway protocol; the user-side edge router inputs the target computing power demand and the current computing power state into a first intelligent agent reinforcement learning model corresponding to the user-side edge router, and outputs an initial node address corresponding to the target computing power demand via the first intelligent agent reinforcement learning model; the user-side edge router sends the initial node address to the computing power-side edge router, the computing power-side edge router judges whether the initial node address needs to be updated, and determines a target node address according to the judgment result; and the computing power-side edge router offloads the target computing power demand to a target node corresponding to the target node address.

Description

Technical Field

[0001] This disclosure relates to the field of data processing, and in particular to a method, apparatus and device for scheduling tasks in a distributed computing network. Background Technology

[0002] With the increasing demand for distributed computing, especially in complex computing environments with multiple users and multiple terminals, traditional task scheduling methods have shown obvious limitations when facing problems such as dynamic changes in resources, uneven node load, and competition for computing resources.

[0003] Existing scheduling schemes are difficult to effectively adapt to the high requirements of computing resource allocation and task execution efficiency in different scenarios. In particular, when the coordination between computing nodes is poor and the scheduling latency is high, problems such as resource waste and task execution failure are common. Summary of the Invention

[0004] In view of this, the purpose of this disclosure is to propose a distributed computing power network task scheduling method, apparatus and equipment to solve or partially solve the above problems.

[0005] To achieve the above objectives, the first aspect of this disclosure provides a task scheduling method for a distributed computing power network, wherein the distributed computing power network includes user-side edge routers and computing power-side edge routers, and the method includes:

[0006] The user-side edge router obtains the target computing power requirement and determines the current computing power status of the distributed computing power network according to the multi-protocol border gateway protocol. The current computing power status represents the task allocation status of each computing power node in the current distributed computing power network.

[0007] The user-side edge router inputs the target computing power requirement and the current computing power status into the first agent reinforcement learning model corresponding to the user-side edge router, and outputs the initial node address corresponding to the target computing power requirement through the first agent reinforcement learning model.

[0008] The user-side edge router sends the initial node address to the computing power-side edge router. The computing power-side edge router determines whether the initial node address needs to be updated and determines the target node address based on the determination result.

[0009] The computing power edge router offloads the target computing power requirement to the target node corresponding to the target node address.

[0010] Based on the same inventive concept, a second aspect of this disclosure proposes a distributed computing power network task scheduling device, comprising:

[0011] The data acquisition module is configured to acquire the target computing power requirement from the user-side edge router and determine the current computing power status of the distributed computing power network according to the multi-protocol border gateway protocol, wherein the current computing power status represents the task allocation status of each computing power node in the current distributed computing power network.

[0012] The initial node address determination module is configured such that the user-side edge router inputs the target computing power requirement and the current computing power status to the first agent reinforcement learning model corresponding to the user-side edge router, and outputs the initial node address corresponding to the target computing power requirement through the first agent reinforcement learning model.

[0013] The target node address determination module is configured such that the user-side edge router sends the initial node address to the computing power-side edge router, the computing power-side edge router determines whether the initial node address needs to be updated, and determines the target node address based on the determination result;

[0014] The offloading module is configured to allow the computing power side edge router to offload the target computing power requirement to the target node corresponding to the target node address.

[0015] Based on the same inventive concept, a third aspect of this disclosure proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor, when executing the computer program, implements the distributed computing network task scheduling method as described above.

[0016] Based on the same inventive concept, a fourth aspect of this disclosure proposes a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the distributed computing network task scheduling method described above.

[0017] As can be seen from the above, this disclosure proposes a method, apparatus, and device for task scheduling in a distributed computing power network. The user-side edge router obtains the target computing power requirement and determines the current computing power state of the distributed computing power network according to a multi-protocol border gateway protocol. The current computing power state represents the task allocation state of each computing power node in the current distributed computing power network, which is then used by the agent corresponding to the user-side edge router to make decisions and determine suitable computing power resource nodes. The user-side edge router inputs the target computing power requirement and the current computing power state into a first agent reinforcement learning model corresponding to the user-side edge router. The first agent reinforcement learning model outputs an initial node address corresponding to the target computing power requirement. The node corresponding to the initial node address is the node that matches the current computing power state and the user's target computing power requirement. The user-side edge router sends the initial node address to the computing power-side edge router. The computing power-side edge router determines whether the initial node address needs to be updated and determines the target node address based on the determination result. That is, because the computing power offloading decision given by the user-side model may no longer be applicable after several hops, the model deployed on the computing power-side edge router can redirect the demand offloading before the final offloading of the computing power requirement. The computing power-side edge router offloads the target computing power demand to the target node corresponding to the target node address, making the final determined target node address more accurate. Through the intelligent agents of the user-side edge router and the computing power-side edge router, efficient task scheduling in a distributed environment is achieved, meeting the efficiency and reliability requirements of task execution in different scenarios. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this disclosure or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a distributed computing network task scheduling method according to an embodiment of the present disclosure;

[0020] Figure 2 This is a structural block diagram of a distributed computing power network task scheduling device according to an embodiment of the present disclosure;

[0021] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0023] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this disclosure should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in the embodiments of this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0024] The following are definitions of terms used in this disclosure:

[0025] MP-BGP: Multiprotocol Border Gateway Protocol (MP-BGP) is an extension of traditional BGP, supporting the exchange of routing information for multiple protocol families (such as IPv4 and IPv6). By introducing Address Family Identifier (AFI) and Subsequent Address Family Identifier (SAFI), it can simultaneously transmit routing information for multiple protocols within the same BGP session, such as unicast, multicast, and MPLS VPN. MP-BGP provides flexible routing control and management in multiprotocol environments and is widely used in IPv6 migration, MPLS VPNs, and complex multiprotocol networks, achieving efficient and flexible routing information distribution and path selection.

[0026] MARL (Multi-Agent Reinforcement Learning) is a technique that applies reinforcement learning to multi-agent systems, aiming to enable multiple agents to learn and make decisions collaboratively in the same environment to solve cooperative or competitive tasks. Each agent can acquire information by interacting with the environment and other agents, and update its policy based on the feedback received. MARL is often used in complex distributed task scheduling, autonomous driving, and robot collaboration scenarios involving multiple agents working together or competing with each other. MADDPG, as a type of multi-agent reinforcement learning, has each agent with its own policy network (Actor) for making decisions based on current observations. However, the agent's value function (Critic) is centralized, allowing access to the observations and actions of other agents. This architecture allows agents to utilize global information to evaluate the value of their actions when making local decisions, thereby improving learning efficiency.

[0027] MADDPG (Multi-Agent Deep Deterministic Policy Gradient) is a reinforcement learning algorithm used in multi-agent environments. It is an extension of the Deep Deterministic Policy Gradient (DDPG) algorithm. MADDPG is primarily used to solve cooperation and competition problems in multi-agent environments, especially when the interactions between agents can be very complex.

[0028] OSPF: Open Shortest Path First. OSPF is a link-state routing protocol used in Internet Protocol (IP) networks.

[0029] With the increasing demand for distributed computing, especially in complex computing environments with multiple users and multiple terminals, traditional task scheduling methods have shown obvious limitations when facing problems such as dynamic changes in resources, uneven node load, and competition for computing resources.

[0030] As computing networks evolve from centralized to distributed systems, they still face challenges such as uneven resource distribution, low computing power utilization efficiency, and a lack of effective computing power scheduling. Particularly in the training of large AI models, decentralized distributed computing networks have a greater chance of being implemented in model inference, and the potential for future growth is also substantial. However, because distributed computing technology is still in the research stage, it faces a series of challenges, including communication latency, data privacy, and resource scheduling.

[0031] Existing scheduling schemes are difficult to effectively adapt to the high requirements of computing resource allocation and task execution efficiency in different scenarios. In particular, when the coordination between computing nodes is poor and the scheduling latency is high, problems such as resource waste and task execution failure are common.

[0032] Based on the above description, this embodiment proposes a distributed computing power network task scheduling method, such as... Figure 1 As shown, the distributed computing power network includes user-side edge routers and computing power-side edge routers, and the method includes:

[0033] Step 101: The user-side edge router obtains the target computing power requirement and determines the current computing power status of the distributed computing power network according to the multi-protocol border gateway protocol, wherein the current computing power status represents the task allocation status of each computing power node in the current distributed computing power network.

[0034] In practical implementation, distributed computing networks represent a novel computing model that transforms traditional centralized computing architectures into distributed computing. By connecting various computing nodes, a vast computing network is formed. Its key features include resource distribution and sharing, dynamic and efficient task scheduling, and parallel and distributed processing capabilities, improving resource utilization and task execution efficiency. Furthermore, it possesses strong fault tolerance and high availability, avoiding single points of failure through task migration and redundancy design, while relying on low-latency, high-bandwidth network connections to ensure efficient communication between nodes.

[0035] The user-side edge router obtains the user's target computing power requirements and determines the current computing power status of the distributed computing power network according to the multi-protocol border gateway protocol. The current computing power status represents the task allocation status of each computing power node in the current distributed computing power network.

[0036] The following explains the meaning of the fields in the MP-BGP protocol:

[0037] The MP-BGP extended community attribute is an 8-byte path attribute used to append additional routing information to BGP Update messages. This attribute consists of four parts: a 1-byte type field (Type) identifying the attribute category; a 1-byte sub-type field further refining the type's purpose; and a 6-byte value attribute carrying specific numerical or policy information. This structure allows BGP to flexibly extend its functionality and carry specific service requirements.

[0038] The Type field typically consists of two parts: the high 4 bits represent the type, and the low 4 bits represent other characteristics of the subtype or attribute. For information on user computing power requirements and resources, the Type field is set to 0x03 (Opaque ExtendedCommunity) for custom purposes.

[0039] For computing power demand, the Sub-Type field is set to 0x01. For computing power resource information, the Sub-Type field is set to 0x02 (Route Target).

[0040] For a 6-byte data field, when carrying computing power requirements, the highest three bits represent the user's CPU requirements, and the third bit represents the user's storage requirements. When carrying computing power resource information, since the computing power resource information is a 7-tuple, the unit billing standard for CPU and storage is combined into one value. The 6 bytes represent the six-tuple of computing power resources, with each value occupying one byte.

[0041] Step 1022: The user-side edge router inputs the target computing power requirement and the current computing power status into the first agent reinforcement learning model corresponding to the user-side edge router, and outputs the initial node address corresponding to the target computing power requirement through the first agent reinforcement learning model.

[0042] In practice, a first agent reinforcement learning model is deployed in the user-side edge router. The user-side edge router inputs the target computing power requirement and the current computing power status into the first agent reinforcement learning model deployed in the user-side edge router. The first agent reinforcement learning model is used to make decisions and output the initial node address corresponding to the target computing power requirement.

[0043] Step 103: The user-side edge router sends the initial node address to the computing power-side edge router. The computing power-side edge router determines whether the initial node address needs to be updated and determines the target node address based on the determination result.

[0044] In practice, the user-side edge router sends the initial node address obtained from the decision to the computing power-side edge router, which also deploys an agent reinforcement learning model. Because the computing power state may change during the process of data packets being sent from the user-side edge router to the computing power-side edge router accessed by the computing power resource node, the computing power-side edge router determines whether the initial node address needs to be updated and determines the target node address based on the determination result.

[0045] Step 104: The computing power edge router offloads the target computing power requirement to the target node corresponding to the target node address.

[0046] In practice, once the edge router on the computing power side determines the target node address, it can offload the user's target computing power demand to the target node corresponding to the target node address, so as to realize intelligent collaborative scheduling between computing power nodes.

[0047] Through the above scheme, the user-side edge router obtains the target computing power requirement and determines the current computing power status of the distributed computing power network according to the multi-protocol border gateway protocol. The current computing power status represents the task allocation status of each computing power node in the current distributed computing power network, which is then used by the agent corresponding to the user-side edge router to make decisions and determine suitable computing power resource nodes. The user-side edge router inputs the target computing power requirement and the current computing power status into the first agent reinforcement learning model corresponding to the user-side edge router. The first agent reinforcement learning model outputs the initial node address corresponding to the target computing power requirement. The node corresponding to the initial node address is the node that matches the current computing power status and the user's target computing power requirement. The user-side edge router sends the initial node address to the computing power side edge router. The computing power side edge router determines whether the initial node address needs to be updated and determines the target node address based on the determination result. That is, because the computing power offloading decision given by the user-side model may no longer be applicable after several hops, the model deployed on the computing power side edge router can redirect the demand offloading before the final unloading of the computing power requirement. The computing power-side edge router offloads the target computing power demand to the target node corresponding to the target node address, making the final determined target node address more accurate. Through the intelligent agents of the user-side edge router and the computing power-side edge router, efficient task scheduling in a distributed environment is achieved, meeting the efficiency and reliability requirements of task execution in different scenarios.

[0048] In some embodiments, resource conflicts may occur in multi-user, multi-terminal scenarios, where different users' computing power needs are scheduled to the same computing power resource node, and the computing power needs arrive at the computing power resource node at different times, causing the scheduling decisions given by the user-side router agent to become inapplicable.

[0049] In some embodiments, the distributed computing network further includes a backbone router, and step 103 specifically includes:

[0050] Step 1031: Determine the target message path using the Open Shortest Path First protocol and the initial node address.

[0051] Step 1032: Determine whether there is a backbone router among at least one intermediate node contained in the target message path. The intermediate node is any node other than the initial target node corresponding to the initial node address and the current node where the user-side edge router is located.

[0052] Step 1033: In response to the existence of a backbone router, the backbone router determines a new initial node address based on the target computing power requirement, and the backbone router sends the new initial node address to the computing power side edge router.

[0053] or,

[0054] Step 1034: In response to the absence of a backbone router, the user-side edge router sends the initial node address to the computing power-side edge router.

[0055] In practice, the user-side edge router uses the OSPF protocol and the initial node address to determine the target packet path based on its own routing table. The target packet path is the shortest path from the current node to the initial node address.

[0056] Identify all intermediate nodes along the target packet path. These intermediate nodes are all nodes except the initial target node corresponding to the initial node address and the current node where the user-side edge router is located. Determine if any of the intermediate nodes have a backbone router deployed to determine whether the initial node address needs to be updated.

[0057] If a backbone router exists, it will be used to make decisions based on the target computing power requirements and determine a new initial node address. The specific method for determining the new initial node address is as follows:

[0058] The current computing power status of the distributed computing network is obtained based on a message queue and used as the first computing power status. The target computing power requirement and the first computing power status are input into the second agent reinforcement learning model corresponding to the backbone router. The second agent reinforcement learning model makes a decision and outputs a new initial node address corresponding to the target computing power requirement.

[0059] In this embodiment, the distributed computing power network synchronizes computing power information through the Redis-Streams message queue. Computing power resource nodes act as message producers, monitoring the status of their own computing power resources and periodically writing computing power information to the streams. Routers deploying intelligent agents act as message consumers, subscribing to message topics to periodically obtain the latest computing power resource information and transmit it to the intelligent agents.

[0060] If there is no backbone router, the initial node address is sent directly to the edge router on the computing power side.

[0061] The above scheme addresses the issue that the computing power status may change as data packets travel from the user-side edge router through the backbone router and finally reach the computing power resource node's access edge router. Therefore, when passing through the backbone router, a second-agent reinforcement learning model deployed on the backbone router is used to re-determine a new initial node address. This new address better matches the computing power status of each node in the current distributed computing power network, resulting in a more accurate new initial node address.

[0062] In some embodiments, a backbone router is defined as a router with a degree greater than 2. A router with a degree of 2 will only receive packets from one port and send them out from another port, and does not have the conditions for intelligent scheduling.

[0063] In some embodiments, step 103 specifically includes:

[0064] Step 1031: The edge router on the computing power side determines the computing power resources corresponding to the initial node address.

[0065] Step 1032: Determine whether the computing power resources meet the target computing power requirements.

[0066] Step 1033: In response to the computing power resources meeting the target computing power requirement, it is determined that there is no need to update the initial node address, and the initial node address is used as the target node address.

[0067] or,

[0068] Step 1034: In response to the fact that the computing power resources do not meet the target computing power requirements, it is determined that the initial node address needs to be updated. The computing power side edge router updates the initial node address until the computing power resources meet the target computing power requirements, and obtains the updated node address. The updated node address is then used as the target node address.

[0069] In practice, the edge router on the computing power side determines the computing power resources of the edge node corresponding to the initial node address and judges whether the computing power resources can meet the user's target computing power requirements.

[0070] If the computing power resources are sufficient to meet the target computing power requirements, it means that the edge node can meet the user's needs. In this case, there is no need to update the initial node address, and the initial node address is used as the target node address.

[0071] If the computing power resources do not meet the target computing power requirements, it means that the edge node cannot meet the user's needs, and it is determined that the initial node address needs to be updated. The initial node address is updated using a computing power-side edge router. The specific update process includes:

[0072] The edge router on the computing power side obtains the computing power status of the distributed computing power network at the current moment based on the message queue, and uses it as the second computing power status.

[0073] The edge router on the computing power side is deployed with a third agent reinforcement learning model. The target computing power requirement and the second computing power status are input into the third agent reinforcement learning model, and the new node address is output through the third agent reinforcement learning model.

[0074] The edge router corresponding to the new node address reassesses whether its computing resources are sufficient to meet the target computing power requirement. If it is determined again that it cannot meet the target computing power requirement, a new node address is determined based on the edge router corresponding to the new node address, until the computing resources of the node corresponding to the new node address meet the target computing power requirement, and the new node address is used as the updated node address.

[0075] The above approach addresses the issue that the computing power offloading decision provided by the user-side model may become inapplicable after several hops. Therefore, the model deployed on the computing power edge router can redirect the demand offloading before the final offloading. By using a third-agent reinforcement learning model deployed on the computing power edge router to redetermine the new initial node address, the new address is more accurately matched to the computing power status of each node in the current distributed computing power network.

[0076] In some embodiments, after step 1033, the method further includes:

[0077] Step A: Count the number of times the initial node address is updated;

[0078] Step B: In response to the fact that the number of updates is greater than a preset update number threshold and the computing power resources of the node corresponding to the new node address do not meet the target computing power requirement, the node address obtained after the last update is used as the updated node address.

[0079] In practice, when a data packet arrives at the edge router on the computing power side, the intelligent agent deployed on the edge router may encounter a scenario where the data packet continues to propagate in the network because the resources of each computing power resource node do not meet the computing power requirements.

[0080] Therefore, when the number of updates to the initial node address is counted, if the number of updates exceeds the preset update threshold and the computing power resources of the node corresponding to the new node address do not meet the target computing power requirement, then the new node address will not be re-determined, and the node address obtained after the last update will be directly used as the updated node address.

[0081] The above scheme stops updating the initial node address when the number of updates reaches a certain limit, thus preventing data packets from continuously propagating in the network and causing network congestion and other problems.

[0082] This embodiment provides a distributed computing power network task scheduling method. This scheme utilizes a multi-agent reinforcement learning model deployed on routers for decision-making, allocating appropriate computing power resource nodes to offload the computing power demands of accessing users. Before the task scheduling process, the network distributes computing power information via the MP-BGP protocol, ensuring that each router possesses global computing power information. During computing power task scheduling, routing in the network is implemented using the default OSPF protocol, with reinforcement learning models deployed only on user-side edge routers, network backbone routers, and computing power-side edge routers. Considering the constantly changing computing power resource status in multi-user access scenarios, the computing power offloading decision given by the user-side model may become inapplicable after several hops. Therefore, the models deployed on the network backbone routers and computing power-side edge routers can redirect the computing power demand before final offloading. When the number of computing power offloading redirections on the computing power-side edge routers reaches a certain threshold, the edge routers no longer reschedule the computing power tasks but instead add them to the task queue of the last allocated computing power node, thus avoiding the situation where computing power demands are frequently scheduled due to insufficient resources.

[0083] In some embodiments, step 102 specifically includes:

[0084] Step 1021: The user-side edge router obtains the resource node information of the distributed computing network, and determines the available computing resources and available storage resources corresponding to each resource node based on the resource node information.

[0085] Step 1022: Input the target computing power requirement and the current computing power status into the first agent reinforcement learning model corresponding to the user-side edge router. The first agent reinforcement learning model includes constraints and optimization objectives. The constraints are the target computing volume and target storage task corresponding to the target computing power requirement. The optimization objective is to maximize the target reward function of the first agent reinforcement learning model.

[0086] Step 1023: Calculate the first agent reinforcement learning model to obtain the initial node address corresponding to the target computing power requirement.

[0087] In practice, the user-side edge router obtains the resource node information of the distributed computing network and determines the available computing resources and available storage resources corresponding to each resource node based on the resource node information.

[0088] For example, a distributed computing power network contains M users, each user u∈{1,2,3,...,M} has several computing power tasks. The user's computing power task model can be described as a set. For each computing task t u,kBoth are represented as a binary tuple {CPU} u,k Storage u,k} represent the FLOPs (number of floating-point operations) and storage tasks, respectively, in GB.

[0089] In a distributed computing network, there are N computing resource nodes simultaneously. Each computing resource node j∈{1,2,3,...,N} consists of one or two types of servers. Resource nodes composed of two types of servers have both computing and storage capabilities, while resource nodes composed of one type of server have only one capability.

[0090] For a computing server, its triple is {CPU} j Cost cpu,j Load cpu,j}, where CPU j Total computing power of the server, measured in FLOPS, Cost cpu,j For usage costs, charges are based on the unit of computation. Load cpu,j This refers to server load. For storage servers, the triplet is {Storage}. j Cost storage,j Load storage,j Band j}, where Storage j This refers to the server's storage capacity, measured in GB (GB). Cost storage,j For usage costs, billing is based on unit usage. Load storage,j For server load conditions, Band j This represents the server's storage bandwidth, measured in GB / s.

[0091] Therefore, each computing resource node is represented as:

[0092] R j ={CPU j Cost cpu,j Load cpu,j Storage j Cost storage,j Load storage,j Band j}

[0093] Based on the resource node information, the available computing resources and available storage resources corresponding to each resource node are determined, wherein the available computing resources and available storage resources are expressed by the following formula:

[0094]

[0095] in, Available computing resources Available storage resources.

[0096] The target computing power requirement and the current computing power status are input into the first agent reinforcement learning model corresponding to the user-side edge router. The first agent reinforcement learning model includes constraints and optimization objectives. The constraints are the target computing volume and target storage task corresponding to the target computing power requirement. The optimization objective is to maximize the target reward function of the first agent reinforcement learning model. The first agent reinforcement learning model is calculated to obtain the initial node address corresponding to the target computing power requirement.

[0097] Based on the above example, the constraints of the first agent reinforcement learning model specifically include:

[0098] Determine the target computing power requirement and the target storage task corresponding to the target computing power requirement. The constraints are expressed by the following formula:

[0099]

[0100] Among them, each computing task t in the target computing power requirement u,k Both are represented as a binary tuple {CPU} u,k Storage u,k CPU u,k Storage u,k Store the target task. Available computing resources Available storage resources.

[0101] The optimization objective of the first agent reinforcement learning model specifically includes:

[0102] In the cooperative multi-agent model (MADDPG), each agent's state contains both local information and partial global information, enabling efficient scheduling of multi-user tasks. The local state s of agent i... i,local It contains the computing power tasks t that currently need to be scheduled. u,k The agent's global state s, along with its own routing table, contains a list of next-hop nodes and reachability information. i,global This includes all information about global computing power resource nodes.

[0103] The load of each resource node in the distributed computing network is determined, and the global load imbalance of the agent is determined based on the load of all resource nodes. The global load imbalance is expressed by the formula:

[0104]

[0105] Where B represents the global load imbalance, N represents the total number of resource nodes in the distributed computing network, j represents each resource node in the distributed computing network, and Load cpu,j Lood is responsible for the server load of computing servers in a distributed computing network. sttorage,j This refers to the server load of storage servers in a distributed computing network. This is the average load of all resource nodes, determined based on the load of all resource nodes.

[0106] Therefore, the state space S of agent i is i =s i,local ∪s i,global Each agent's action is to select and schedule computing tasks to a specific computing resource node. The action space A∈{a1,a2,a3,...,a...} i}, where a i =j, indicating that agent i will allocate computing power to task t. u,k Scheduled for execution on computing resource node j. Cooperative reward function R. cooperative The cooperative reward function of the first agent's reinforcement learning model is determined based on the global load imbalance, which measures whether the agent's allocation decisions optimize global resource utilization, task cost, latency, and load balancing. The cooperative reward function is expressed by the following formula:

[0107]

[0108] Among them, R cooperative Let M be the collaborative reward function, and M be the total number of users in the distributed computing network. For computing power task t u,k Resource utilization rate on computing resource node j For computing power task t u,k The cost of completing a task on computing resource node j The time delay of the task is represented by α, β, and γ, which are preset coefficients. δ is the penalty weight for imbalance and is used to balance local rewards and global equilibrium.

[0109] The and The specific determination process can be expressed using a formula as follows:

[0110]

[0111]

[0112] The The time delay of the task is represented by the network propagation delay from the agent to the computing resource node (measured based on hop count), the queuing delay of the task on the computing resource node, and the processing delay on the computing resource node. The specific determination process is expressed by the formula:

[0113]

[0114] in, Let be the number of hops from agent i to target node j. The queuing delay of computing power requirement uk in the queue of agent j.

[0115] The collaborative MADDPG includes a centralized Critic network Q. cooperative (S,a) uses the global state set S and the action set a of all agents to evaluate the joint Q-value of all agent actions, which is used to guide policy optimization.

[0116] The target reward function of the first agent reinforcement learning model is determined based on the collaborative reward function, and the target reward function is expressed by the formula:

[0117]

[0118] in, For the target reward function, π i (a i |s i ) is a distributed Actor network, π i (a i |s i The policy is updated based on the local state and some global information, where S is the set of global states, a is the set of actions of the agent, and Q is the set of actions of the agent. cooperative (S,a) is a centralized Critic network, γQ cooperative (S′,a′|θ Q θ represents the reward of the action-value function in the next state S′ and the action a′ taken by the agent in that state. Q For the parameters of the Q-network, it represents the parameters of the neural network model of the action-value function Q(s,a). Here, S′ represents the parameters of the policy network (i.e., the action selection mechanism) of the agent, a′ represents the state, a′ represents the action chosen by the agent in state S′, and E represents the mathematical expectation.

[0119] Through the above scheme, cooperation is mainly achieved through the global reward function R. cooperativeThis is achieved through a centralized Critic network and global information sharing. The local reward function includes global objectives such as resource utilization, task cost, latency, and load imbalance, guiding agents to contribute to the overall system optimization in their behavioral choices. Within the CTDE (Centralized Training and Distributed Execution) framework, the centralized Critic can access the states and behaviors of all agents during training, evaluating the contribution of joint actions to the global reward and encouraging agents to collaboratively optimize during policy updates. Simultaneously, agents can share global information such as load status during execution, thus achieving collaborative effects in local decision-making.

[0120] In this embodiment, multi-agent reinforcement learning is employed, deploying multiple agents on different nodes in the network. Each agent can learn and make decisions independently. Compared to centralized scheduling methods, distributed agents can make rapid scheduling decisions based on local information, significantly improving the system's response speed and task scheduling efficiency. Through reinforcement learning, agents can optimize their decision-making strategies through continuous interaction and feedback, gradually learning how to better allocate tasks and avoid resource conflicts. This learning mechanism allows agents to be adaptive in the face of dynamically changing network environments, enabling them to quickly respond to fluctuations in network traffic and changes in computing resources.

[0121] Meanwhile, in distributed computing networks, computing resources are scattered across different physical locations. Traditional centralized scheduling methods easily become bottlenecks when task requests increase, leading to single points of failure and scheduling delays. Multi-agent reinforcement learning, by distributing scheduling tasks across various agent nodes, not only avoids overloading the central node but also improves the system's scalability and fault tolerance. Even if some nodes fail, other agents can continue task scheduling based on the current network state, ensuring continuous system operation. This invention uses a message queue mechanism to ensure the synchronization of computing information in distributed computing network scenarios. Furthermore, the reinforcement learning-based scheduling method can dynamically adjust based on real-time feedback in the network. Each agent adjusts its scheduling strategy based on its own observations and the global reward function, ensuring that tasks are rationally allocated across different nodes, avoiding resource overload or idleness. Over time, the agents' decision-making capabilities gradually improve, ultimately achieving global optimization. This adaptive scheduling not only improves network resource utilization but also significantly enhances task execution efficiency and quality.

[0122] The distributed computing network task scheduling method proposed in this embodiment is suitable for complex environments with multiple users and multiple computing nodes. For example, in large-scale cloud computing or edge computing networks, the diversity and complexity of tasks place higher demands on scheduling. Through MARL, customized scheduling strategies can be created for different types of tasks and resources, ensuring that user needs for task latency, bandwidth, and computing power are met. This method features high adaptability, distributed architecture advantages, efficient resource utilization, and strong scalability, providing a promising solution for task scheduling in large-scale distributed computing environments.

[0123] It should be noted that the method of this disclosure embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this disclosure embodiment, and the multiple devices will interact with each other to complete the method described.

[0124] It should be noted that the above description describes some embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0125] Based on the same inventive concept, corresponding to any of the above embodiments, this disclosure also provides a distributed computing power network task scheduling device.

[0126] refer to Figure 2 , Figure 2 The distributed computing network task scheduling device for this embodiment includes:

[0127] The data acquisition module 201 is configured to acquire the target computing power requirement from the user-side edge router and determine the current computing power status of the distributed computing power network according to the multi-protocol border gateway protocol, wherein the current computing power status represents the task allocation status of each computing power node in the current distributed computing power network.

[0128] The initial node address determination module 202 is configured to input the target computing power requirement and the current computing power status into the first agent reinforcement learning model corresponding to the user-side edge router, and output the initial node address corresponding to the target computing power requirement through the first agent reinforcement learning model.

[0129] The target node address determination module 203 is configured such that the user-side edge router sends the initial node address to the computing power-side edge router, the computing power-side edge router determines whether the initial node address needs to be updated, and determines the target node address based on the determination result;

[0130] The offloading module 204 is configured to allow the computing power side edge router to offload the target computing power requirement to the target node corresponding to the target node address.

[0131] In some embodiments, the distributed computing network further includes a backbone router, and the target node address determination module 203 specifically includes:

[0132] The message path determination unit is configured to determine the target message path using the Open Shortest Path First protocol and the initial node address;

[0133] The judgment unit is configured to determine whether at least one intermediate node contained in the target packet path is a backbone router, wherein the intermediate node is any node other than the initial target node corresponding to the initial node address and the current node where the user-side edge router is located.

[0134] The update unit is configured to, in response to the presence of a backbone router, use the backbone router to determine a new initial node address based on the target computing power requirement, and send the new initial node address to the computing power-side edge router; or...

[0135] The sending unit is configured to send the initial node address to the computing power edge router in response to the absence of a backbone router.

[0136] In some embodiments, the update unit is specifically configured as follows:

[0137] The current computing power status of the distributed computing power network is obtained based on the message queue and used as the first computing power status.

[0138] The target computing power requirement and the first computing power status are input into the second agent reinforcement learning model corresponding to the backbone router, and the second agent reinforcement learning model outputs the new initial node address corresponding to the target computing power requirement.

[0139] In some embodiments, the target node address determination module 203 specifically includes:

[0140] The resource determination unit is configured to determine the computing resources corresponding to the initial node address by the computing power side edge router.

[0141] The judgment unit is configured to determine whether the computing power resources meet the target computing power requirement;

[0142] The first target node address determination unit is configured to, in response to the computing power resources meeting the target computing power requirement, determine that it is not necessary to update the initial node address and use the initial node address as the target node address; or...

[0143] The second target node address determination unit is configured to, in response to the computing power resources not meeting the target computing power requirement, determine that the initial node address needs to be updated, and the computing power side edge router updates the initial node address until the computing power resources meet the target computing power requirement, thereby obtaining the updated node address, and using the updated node address as the target node address.

[0144] In some embodiments, the second target node address determination unit is specifically configured as follows:

[0145] The edge router on the computing power side obtains the computing power status of the distributed computing power network at the current moment based on the message queue, and uses it as the second computing power status.

[0146] The target computing power requirement and the second computing power status are input into the third agent reinforcement learning model corresponding to the computing power side edge router, and the new node address is output through the third agent reinforcement learning model.

[0147] The new node address will be used as the updated node address until the computing power resources of the node corresponding to the new node address meet the target computing power requirement.

[0148] In some embodiments, the initial node address determination module 202 is specifically configured as follows:

[0149] The user-side edge router obtains the resource node information of the distributed computing network and determines the available computing resources and available storage resources corresponding to each resource node based on the resource node information.

[0150] The target computing power requirement and the current computing power status are input into the first agent reinforcement learning model corresponding to the user-side edge router. The first agent reinforcement learning model includes constraints and optimization objectives. The constraints are the target computing volume and target storage task corresponding to the target computing power requirement. The optimization objective is to maximize the target reward function of the first agent reinforcement learning model.

[0151] The reinforcement learning model of the first agent is calculated to obtain the initial node address corresponding to the target computing power requirement.

[0152] In some embodiments, the constraints of the first agent reinforcement learning model specifically include:

[0153] Determine the target computing power requirement and the target storage task corresponding to the target computing power requirement. The constraints are expressed by the following formula:

[0154]

[0155] Among them, each computing task t in the target computing power requirement u,k Both are represented as a binary tuple {CPU} u,k Storage u,k CPU u,k Storage u,k Store the target task. Available computing resources Available storage resources;

[0156] The optimization objective of the first agent reinforcement learning model specifically includes:

[0157] The load of each resource node in the distributed computing network is determined, and the global load imbalance of the agent is determined based on the load of all resource nodes. The global load imbalance is expressed by the formula:

[0158]

[0159] Where B represents the global load imbalance, N represents the total number of resource nodes in the distributed computing network, j represents each resource node in the distributed computing network, and Load cpu,j Load is the server load of computing servers in a distributed computing network. storage,j This refers to the server load of storage servers in a distributed computing network. This refers to the average load of all resource nodes, determined based on the load of all resource nodes.

[0160] The cooperative reward function of the first agent reinforcement learning model is determined based on the global load imbalance, and the cooperative reward function is expressed by the formula:

[0161]

[0162] Among them, R cooperative Let M be the collaborative reward function, and M be the total number of users in the distributed computing network. For computing power task t u,k Resource utilization rate on computing resource node j For computing power task t u,k The cost of completing a task on computing resource node j The time delay of the task is represented by α, β, and γ, which are preset coefficients, and δ is the penalty weight for the imbalance.

[0163] The target reward function of the first agent reinforcement learning model is determined based on the collaborative reward function, and the target reward function is expressed by the formula:

[0164]

[0165] in, For the target reward function, π i (a i |s i ) is a distributed Actor network, where S is the global state set, a is the action set of the agent, and Q is the action set of the agent. cooperative (S,a) is a centralized Critic network, γQ cooperative (S′,a′|θ Q θ represents the reward of the action-value function in the next state S′ and the action a′ taken by the agent in that state. Q For the parameters of the Q-network, it represents the parameters of the neural network model of the action-value function Q(s,a). Here, S′ represents the parameters of the policy network (i.e., the action selection mechanism) of the agent, a′ represents the state, a′ represents the action chosen by the agent in state S′, and E represents the mathematical expectation.

[0166] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, in implementing this disclosure, the functions of each module can be implemented in one or more software and / or hardware.

[0167] The apparatus described above is used to implement the corresponding distributed computing network task scheduling method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0168] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the distributed computing network task scheduling method described in any of the above embodiments.

[0169] Figure 3 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0170] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0171] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0172] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0173] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0174] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0175] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0176] The electronic devices described above are used to implement the corresponding distributed computing network task scheduling methods in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0177] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this disclosure also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to execute the distributed computing network task scheduling method as described in any of the above embodiments.

[0178] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0179] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the distributed computing network task scheduling method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0180] It is understood that before using the technical solutions of the various embodiments in this disclosure, users will be informed of the type, scope of use, and usage scenarios of the personal information involved in an appropriate manner, and user authorization will be obtained.

[0181] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media performing the operations of this disclosed technical solution.

[0182] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0183] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0184] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this disclosure (including the claims) is limited to these examples; within the framework of this disclosure, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this disclosure as described above, which are not provided in detail for the sake of brevity.

[0185] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this disclosure, the provided drawings may or may not show well-known power / ground connections to integrated circuit (IC) chips and other components. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this disclosure, and this also takes into account the fact that the details of implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this disclosure will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuitry) have been set forth to describe exemplary embodiments of this disclosure, it will be apparent to those skilled in the art that the embodiments of this disclosure may be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0186] Although this disclosure has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0187] This disclosure is intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A distributed computing power network task scheduling method, characterized in that, The distributed computing power network includes user-side edge routers and computing power-side edge routers, including: The user-side edge router obtains the target computing power requirement and determines the current computing power status of the distributed computing power network according to the multi-protocol border gateway protocol. The current computing power status represents the task allocation status of each computing power node in the current distributed computing power network. The user-side edge router inputs the target computing power requirement and the current computing power status into the first agent reinforcement learning model corresponding to the user-side edge router, and outputs the initial node address corresponding to the target computing power requirement through the first agent reinforcement learning model. The user-side edge router sends the initial node address to the computing power-side edge router. The computing power-side edge router determines whether the initial node address needs to be updated and determines the target node address based on the determination result. The computing power edge router offloads the target computing power requirement to the target node corresponding to the target node address; The computing power-side edge router determines whether the initial node address needs to be updated, and determines the target node address based on the determination result, including: The edge router on the computing power side determines the computing power resources corresponding to the initial node address; Determine whether the computing resources meet the target computing power requirement; In response to the computing power resources meeting the target computing power requirement, it is determined that there is no need to update the initial node address, and the initial node address is used as the target node address; or... In response to the fact that the computing power resources do not meet the target computing power requirements, it is determined that the initial node address needs to be updated. The computing power side edge router updates the initial node address until the computing power resources meet the target computing power requirements, and obtains the updated node address. The updated node address is then used as the target node address. The computing power-side edge router updates the initial node address until the computing power resources meet the target computing power requirement, obtaining the updated node address, including: The edge router on the computing power side obtains the computing power status of the distributed computing power network at the current moment based on the message queue, and uses it as the second computing power status. The target computing power requirement and the second computing power status are input into the third agent reinforcement learning model corresponding to the computing power side edge router, and the new node address is output through the third agent reinforcement learning model. The new node address will be used as the updated node address until the computing power resources of the node corresponding to the new node address meet the target computing power requirement.

2. The method according to claim 1, characterized in that, The distributed computing network also includes backbone routers. The user-side edge router sends the initial node address to the computing power-side edge router, including: The target message path is determined using the Open Shortest Path First (OSF) protocol and the initial node address. Determine whether there is a backbone router among at least one intermediate node contained in the target message path. The intermediate node is any node other than the initial target node corresponding to the initial node address and the current node where the user-side edge router is located. In response to the existence of a backbone router, the backbone router determines a new initial node address based on the target computing power requirement, and sends the new initial node address to the computing power-side edge router; or... In response to the absence of a backbone router, the user-side edge router sends the initial node address to the computing power-side edge router.

3. The method according to claim 2, characterized in that, The step of determining a new initial node address using a backbone router based on the target computing power requirement includes: The current computing power status of the distributed computing power network is obtained based on the message queue and used as the first computing power status. The target computing power requirement and the first computing power status are input into the second agent reinforcement learning model corresponding to the backbone router, and the second agent reinforcement learning model outputs the new initial node address corresponding to the target computing power requirement.

4. The method according to claim 1, characterized in that, After the new node address is output by the reinforcement learning model of the third agent, the following is also included: Count the number of times the initial node address is updated; In response to the fact that the number of updates exceeds a preset update number threshold and the computing power resources of the node corresponding to the new node address do not meet the target computing power requirement, the node address obtained after the last update is used as the updated node address.

5. The method according to claim 1, characterized in that, The user-side edge router inputs the target computing power requirement and the current computing power status into the first agent reinforcement learning model corresponding to the user-side edge router, and outputs the initial node address corresponding to the target computing power requirement through the first agent reinforcement learning model, including: The user-side edge router obtains the resource node information of the distributed computing network and determines the available computing resources and available storage resources corresponding to each resource node based on the resource node information. The target computing power requirement and the current computing power status are input into the first agent reinforcement learning model corresponding to the user-side edge router. The first agent reinforcement learning model includes constraints and optimization objectives. The constraints are the target computing volume and target storage task corresponding to the target computing power requirement. The optimization objective is to maximize the target reward function of the first agent reinforcement learning model. The reinforcement learning model of the first agent is calculated to obtain the initial node address corresponding to the target computing power requirement.

6. The method according to claim 5, characterized in that, The constraints of the first agent reinforcement learning model specifically include: Determine the target computing power requirement and the target storage task corresponding to the target computing power requirement. The constraints are expressed by the following formula: Each computing task in the target computing power requirement Both are represented as pairs. , For the target computational load, Store the target task. Available computing resources Available storage resources; The optimization objective of the first agent reinforcement learning model specifically includes: The load of each resource node in the distributed computing network is determined, and the global load imbalance of the agent is determined based on the load of all resource nodes. The global load imbalance is expressed by the formula: in, For global load imbalance, This represents the total number of resource nodes in the distributed computing network. For each resource node in a distributed computing network, This refers to the server load of computing servers in a distributed computing network. This refers to the server load of storage servers in a distributed computing network. This refers to the average load of all resource nodes, determined based on the load of all resource nodes. The cooperative reward function of the first agent reinforcement learning model is determined based on the global load imbalance, and the cooperative reward function is expressed by the formula: in, Let M be the collaborative reward function, and M be the total number of users in the distributed computing network. For computing power tasks Resource utilization rate on computing resource node j For computing power tasks The cost of completing a task on computing resource node j Indicates the time delay of the task. , , These are preset coefficients, It is the penalty weight for imbalance; The target reward function of the first agent reinforcement learning model is determined based on the collaborative reward function, and the target reward function is expressed by the formula: in, For the target reward function, This is a distributed Actor network, where S is the global state set and a is the set of actions of the agent. It is a centralized Critic network. The action value function in the next state And the actions taken by the agent in this state. The return For the parameters of the Q-network, it represents the parameters of the neural network model of the action-value function Q(s,a). These are the parameters of the policy network, representing the parameters of the agent's policy network (i.e., the action selection mechanism). For state, In the state The action chosen by the agent is E, where E is the mathematical expectation.

7. A distributed computing power network task scheduling device, characterized in that, include: The data acquisition module is configured to acquire the target computing power requirement from the user-side edge router and determine the current computing power status of the distributed computing power network according to the multi-protocol border gateway protocol, wherein the current computing power status represents the task allocation status of each computing power node in the current distributed computing power network. The initial node address determination module is configured such that the user-side edge router inputs the target computing power requirement and the current computing power status to the first agent reinforcement learning model corresponding to the user-side edge router, and outputs the initial node address corresponding to the target computing power requirement through the first agent reinforcement learning model. The target node address determination module is configured such that the user-side edge router sends the initial node address to the computing power-side edge router, the computing power-side edge router determines whether the initial node address needs to be updated, and determines the target node address based on the determination result; The offloading module is configured to allow the computing power side edge router to offload the target computing power requirement to the target node corresponding to the target node address; The computing power-side edge router determines whether the initial node address needs to be updated, and determines the target node address based on the determination result, including: The edge router on the computing power side determines the computing power resources corresponding to the initial node address; Determine whether the computing resources meet the target computing power requirement; In response to the computing power resources meeting the target computing power requirement, it is determined that there is no need to update the initial node address, and the initial node address is used as the target node address; or... In response to the fact that the computing power resources do not meet the target computing power requirements, it is determined that the initial node address needs to be updated. The computing power side edge router updates the initial node address until the computing power resources meet the target computing power requirements, and obtains the updated node address. The updated node address is then used as the target node address. The computing power-side edge router updates the initial node address until the computing power resources meet the target computing power requirement, obtaining the updated node address, including: The edge router on the computing power side obtains the computing power status of the distributed computing power network at the current moment based on the message queue, and uses it as the second computing power status. The target computing power requirement and the second computing power status are input into the third agent reinforcement learning model corresponding to the computing power side edge router, and the new node address is output through the third agent reinforcement learning model. The new node address will be used as the updated node address until the computing power resources of the node corresponding to the new node address meet the target computing power requirement.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as claimed in any one of claims 1 to 6.

Images