Distributed computing power network task scheduling method, device and equipment

Abstract

The invention provides a distributed computing power network task scheduling method, device and equipment, a distributed computing power network comprises a user side edge router and a computing power side edge router, and the method comprises the steps that the user side edge router obtains a target computing power demand, determining a current computing power state corresponding to the distributed computing power network according to the multi-protocol border gateway protocol; the user side edge router inputs the target computing power demand and the current computing power state to a first agent reinforcement learning model corresponding to the user side edge router, and an initial node address corresponding to the target computing power demand is output through the first agent reinforcement learning model; the user side edge router sends the initial node address to the computing power side edge router, and the computing power side edge router judges whether the initial node address needs to be updated or not and determines a target node address according to a judgment result; and the computing power side edge router unloads the target computing power demand to a target node corresponding to the target node address.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and in particular to a distributed computing network task scheduling method, device and equipment. Background Art

[0002] With the increasing demand for distributed computing, especially in complex computing environments with multiple users and terminals, traditional task scheduling methods have shown obvious limitations when faced with problems such as dynamic resource changes, uneven node loads, and competition for computing resources.

[0003] Existing scheduling schemes are difficult to effectively adapt to the high requirements of computing resource allocation and task execution efficiency in different scenarios, especially when the coordination between computing nodes is poor and the scheduling delay is high, which can easily lead to frequent problems such as resource waste and task execution failure. Summary of the invention

[0004] In view of this, the purpose of the present disclosure is to propose a distributed computing network task scheduling method, device and equipment to solve or partially solve the above-mentioned problems.

[0005] Based on the above purpose, the first aspect of the present disclosure provides a distributed computing network task scheduling method, wherein the distributed computing network includes a user-side edge router and a computing-side edge router, and the method includes:

[0006] The user-side edge router obtains the target computing power demand and determines the current computing power state corresponding to the distributed computing power network according to the multi-protocol border gateway protocol, wherein the current computing power state represents the task allocation state of each computing power node in the current distributed computing power network;

[0007] The user-side edge router inputs the target computing power requirement and the current computing power state into a first agent reinforcement learning model corresponding to the user-side edge router, and outputs an initial node address corresponding to the target computing power requirement via the first agent reinforcement learning model;

[0008] The user-side edge router sends the initial node address to the computing power-side edge router, and the computing power-side edge router determines whether the initial node address needs to be updated, and determines the target node address according to the determination result;

[0009] The computing power side edge router offloads the target computing power demand to the target node corresponding to the target node address.

[0010] Based on the same inventive concept, the second aspect of the present disclosure proposes a distributed computing power network task scheduling device, including:

[0011] The data acquisition module is configured to obtain the target computing power demand for the user-side edge router, and determine the current computing power state corresponding to the distributed computing power network according to the multi-protocol border gateway protocol, wherein the current computing power state represents the task allocation state of each computing power node in the current distributed computing power network;

[0012] An initial node address determination module is configured so that the user-side edge router inputs the target computing power requirement and the current computing power state into a first agent reinforcement learning model corresponding to the user-side edge router, and outputs an initial node address corresponding to the target computing power requirement via the first agent reinforcement learning model;

[0013] The target node address determination module is configured so that the user-side edge router sends the initial node address to the computing power-side edge router, and the computing power-side edge router determines whether the initial node address needs to be updated, and determines the target node address according to the determination result;

[0014] The unloading module is configured as a computing power side edge router to unload the target computing power demand to the target node corresponding to the target node address.

[0015] Based on the same inventive concept, the third aspect of the present disclosure proposes an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the distributed computing power network task scheduling method as described above when executing the computer program.

[0016] Based on the same inventive concept, the fourth aspect of the present disclosure proposes a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the distributed computing power network task scheduling method as described above.

[0017] As can be seen from the above, the present disclosure proposes a distributed computing network task scheduling method, device and equipment, the user side edge router obtains the target computing power demand, and determines the current computing power state corresponding to the distributed computing power network according to the multi-protocol border gateway protocol, wherein the current computing power state represents the task allocation state of each computing power node in the current distributed computing power network, so as to provide the intelligent agent corresponding to the subsequent user side edge router for decision-making and determine the appropriate computing power resource node. The user side edge router inputs the target computing power demand and the current computing power state into the first intelligent agent reinforcement learning model corresponding to the user side edge router, and outputs the initial node address corresponding to the target computing power demand through the first intelligent agent reinforcement learning model, and the node corresponding to the initial node address is the node that matches the current computing power state and the user's target computing power demand. The user side edge router sends the initial node address to the computing power side edge router, and the computing power side edge router determines whether the initial node address needs to be updated, and determines the target node address according to the judgment result, that is, because the computing power unloading decision given by the user side model may no longer be applicable after several jumps, the model deployed on the computing power side edge router can redirect the demand unloading before the computing power demand is finally unloaded. The edge router on the computing power side unloads the target computing power demand to the target node corresponding to the target node address, so that the final determined target node address is more accurate. Through the intelligent agents of the user-side edge router and the computing power side edge router, efficient task scheduling is achieved in a distributed environment, meeting the efficiency and reliability requirements of task execution in different scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A flowchart of a distributed computing network task scheduling method according to an embodiment of the present disclosure;

[0020] Figure 2 It is a structural block diagram of a distributed computing power network task scheduling device according to an embodiment of the present disclosure;

[0021] Figure 3 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0023] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0024] The terms used in this disclosure are explained as follows:

[0025] MP-BGP: Multi-protocol Border Gateway Protocol (MP-BGP) is an extension of traditional BGP, supporting the exchange of routing information of multiple protocol families (such as IPv4, IPv6). By introducing the Address Family Identifier (AFI) and the Subsequent Address Family Identifier (SAFI), it can simultaneously transmit routing information of multiple protocols in the same BGP session, such as unicast, multicast, MPLS VPN, etc. MP-BGP provides flexible routing control and management in a multi-protocol environment, and is widely used in IPv6 migration, MPLS virtual private networks and complex multi-protocol networks, realizing efficient and flexible routing information distribution and path selection.

[0026] MARL: Multi-Agent Reinforcement Learning (MARL) is a technique for applying reinforcement learning in multi-agent systems, aiming to allow multiple agents to learn and make decisions together in the same environment to solve cooperative or competitive tasks. Each agent can obtain information by interacting with the environment and other agents, and update its strategy based on the feedback obtained. MARL is often used in complex scenarios such as distributed task scheduling, autonomous driving, and robot collaboration, involving scenarios where multiple agents work together or compete with each other. As a type of multi-agent reinforcement learning, MADDPG has each agent with its own policy network (Actor) to make decisions based on current observations. However, the value function (Critic) of the agent is centralized, and it can access the observation and action information of other agents. This architecture allows agents to use global information to evaluate the value of their actions when making local decisions, thereby improving learning efficiency.

[0027] MADDPG: Multi-Agent Deep Deterministic Policy Gradient (MADDPG) is a reinforcement learning algorithm used in multi-agent environments. This algorithm is an extension of the Deep Deterministic Policy Gradient (DDPG) algorithm. MADDPG is mainly used to solve cooperation and competition problems in multi-agent environments, especially when the interactions between agents may be very complex.

[0028] OSPF: Open Shortest Path First (OSPF), OSPF routing protocol is a link-state routing protocol used in Internet Protocol (IP) networks.

[0029] With the increasing demand for distributed computing, especially in complex computing environments with multiple users and terminals, traditional task scheduling methods have shown obvious limitations when faced with problems such as dynamic resource changes, uneven node loads, and competition for computing resources.

[0030] In the process of evolving from centralized to distributed computing networks, computing networks still face challenges such as uneven resource distribution, low computing efficiency, and lack of effective scheduling. Especially in the training of large AI models, decentralized distributed computing networks have a better chance of landing in model reasoning, and it can be predicted that the incremental space in the future is also large enough. Since the development of distributed computing technology is still in the research stage, it faces a series of problems such as communication delay, data privacy, and resource scheduling.

[0031] Existing scheduling schemes are difficult to effectively adapt to the high requirements of computing resource allocation and task execution efficiency in different scenarios, especially when the coordination between computing nodes is poor and the scheduling delay is high, which can easily lead to frequent problems such as resource waste and task execution failure.

[0032] Based on the above description, this embodiment proposes a distributed computing network task scheduling method, such as Figure 1 As shown, the distributed computing power network includes a user-side edge router and a computing power-side edge router, and the method includes:

[0033] Step 101: The user-side edge router obtains the target computing power requirement and determines the current computing power state corresponding to the distributed computing power network according to the multi-protocol border gateway protocol, wherein the current computing power state represents the task allocation state of each computing power node in the current distributed computing power network.

[0034] In specific implementation, the distributed computing network is a new computing model that transforms the traditional centralized computing architecture into distributed computing, and forms a huge computing network by connecting various computing nodes. Its main features include resource distribution and sharing, dynamic and efficient task scheduling, parallel and distributed processing capabilities, which improve resource utilization and task execution efficiency. In addition, it has strong fault tolerance and high availability, avoiding single point failures through task migration and redundant design, and relying on low-latency, high-bandwidth network connections to ensure efficient communication between nodes.

[0035] The user-side edge router obtains the user's target computing power requirement and determines the current computing power state corresponding to the distributed computing power network according to the multi-protocol border gateway protocol, wherein the current computing power state represents the task allocation state of each computing power node in the current distributed computing power network.

[0036] The following is an explanation of the fields in the MP-BGP protocol:

[0037] The extended community attribute of MP-BGP is an 8-byte path attribute used to attach additional routing information to the BGP Update message. The attribute consists of four parts: a 1-byte type field (Type) used to identify the attribute category; a 1-byte subtype field (Sub-Type) to further refine the type usage; and a 6-byte value attribute used to carry specific numerical values ​​or policy information. This structure allows BGP to flexibly expand its functions and carry specific business demand information.

[0038] The Type field usually consists of two parts: the upper 4 bits represent the type, and the lower 4 bits represent the subtype or other characteristics of the attribute. For carrying user computing power requirements and computing power resource information, the Type field is set to 0x03 (Opaque Extended Community) for custom purposes.

[0039] For computing power demand bearer, the Sub-Type field is set to 0x01. For computing power resource information, the Sub-Type field is set to 0x02 (Route Target).

[0040] For the 6-byte data field, when carrying computing power requirements, the upper three bits represent the user's CPU requirements, and the third bit represents the user's storage requirements; when carrying computing power resource information, since the computing power resource information is a 7-tuple, the unit billing standards of CPU and storage are combined into one value. The 6 bytes represent the six-tuple of computing power resources, and each value occupies one byte.

[0041] In step 1022, the user-side edge router inputs the target computing power requirement and the current computing power state into the first agent reinforcement learning model corresponding to the user-side edge router, and outputs the initial node address corresponding to the target computing power requirement via the first agent reinforcement learning model.

[0042] During specific implementation, a first intelligent agent reinforcement learning model is deployed in the user-side edge router. The user-side edge router inputs the target computing power demand and the current computing power status into the first intelligent agent reinforcement learning model corresponding to the user-side edge router, uses the first intelligent agent reinforcement learning model to make decisions, and outputs the initial node address corresponding to the target computing power demand.

[0043] Step 103: The user-side edge router sends the initial node address to the computing power-side edge router. The computing power-side edge router determines whether the initial node address needs to be updated and determines the target node address based on the determination result.

[0044] In specific implementation, the user-side edge router sends the initial node address obtained by the decision to the computing power side edge router, which is also deployed with an intelligent agent reinforcement learning model. Because the computing power state may change during the process of the data packet being sent from the user-side edge router to the computing power side edge router connected to the computing power resource node. Therefore, the computing power side edge router determines whether the initial node address needs to be updated and determines the target node address based on the judgment result.

[0045] Step 104: The computing power side edge router unloads the target computing power demand to the target node corresponding to the target node address.

[0046] In specific implementation, after the edge router on the computing power side determines the target node address, it can offload the user's target computing power demand to the target node corresponding to the target node address to achieve intelligent collaborative scheduling between computing power nodes.

[0047] Through the above scheme, the user-side edge router obtains the target computing power demand, and determines the current computing power state corresponding to the distributed computing power network according to the multi-protocol border gateway protocol, wherein the current computing power state represents the task allocation state of each computing power node in the current distributed computing power network, so as to provide the intelligent agent corresponding to the subsequent user-side edge router for decision-making and determine the appropriate computing power resource node. The user-side edge router inputs the target computing power demand and the current computing power state into the first intelligent agent reinforcement learning model corresponding to the user-side edge router, and outputs the initial node address corresponding to the target computing power demand through the first intelligent agent reinforcement learning model. The node corresponding to the initial node address is the node that matches the current computing power state and the user's target computing power demand. The user-side edge router sends the initial node address to the computing power side edge router, and the computing power side edge router determines whether the initial node address needs to be updated, and determines the target node address according to the judgment result, that is, because the computing power unloading decision given by the user-side model may no longer be applicable after several jumps, the model deployed on the computing power side edge router can redirect the demand unloading before the computing power demand is finally unloaded. The edge router on the computing power side unloads the target computing power demand to the target node corresponding to the target node address, so that the final determined target node address is more accurate. Through the intelligent agents of the user-side edge router and the computing power side edge router, efficient task scheduling is achieved in a distributed environment, meeting the efficiency and reliability requirements of task execution in different scenarios.

[0048] In some embodiments, in a multi-user-multi-terminal scenario, resource conflicts may occur, that is, different user computing power demands are scheduled to the same computing power resource node, and the computing power demands arrive at the computing power resource node at different times, resulting in the scheduling decision made by the user-side router intelligent agent no longer being applicable.

[0049] In some embodiments, the distributed computing network further includes a backbone router, and step 103 specifically includes:

[0050] Step 1031, using the open shortest path first protocol and the initial node address, determine the target message path.

[0051] Step 1032, determining whether there is a backbone router in at least one intermediate node included in the target message path, wherein the intermediate node is a node other than the initial target node corresponding to the initial node address and the current node where the user-side edge router is located.

[0052] Step 1033, in response to the existence of a backbone router, the backbone router is used to determine a new initial node address according to the target computing power requirement, and the backbone router sends the new initial node address to the edge router on the computing power side.

[0053] or,

[0054] Step 1034, in response to the absence of a backbone router, the user-side edge router sends the initial node address to the computing power-side edge router.

[0055] In specific implementation, the user-side edge router uses the OSPF protocol and the initial node address to determine the target message path according to its own routing table, wherein the target message path is the shortest path from the current node to the initial node address.

[0056] Determine all intermediate nodes on the target message path, where the intermediate nodes are nodes other than the initial target node corresponding to the initial node address and the current node where the user-side edge router is located. Determine whether there is a backbone router deployed in the intermediate nodes to determine whether the initial node address needs to be updated.

[0057] If there is a backbone router, the backbone router is used to make a decision based on the target computing power requirements to determine the new initial node address. The specific method for determining the new initial node address is:

[0058] Based on the message queue, the computing power state corresponding to the distributed computing power network at the current moment is obtained as the first computing power state. The target computing power requirement and the first computing power state are input into the second agent reinforcement learning model corresponding to the backbone router, and the second agent reinforcement learning model makes a decision and outputs a new initial node address corresponding to the target computing power requirement.

[0059] In this embodiment, the computing power information of the distributed computing power network is synchronized through the redis-streams message queue. The computing power resource node acts as a message producer, monitors the status of its own computing power resources, and regularly writes computing power information to streams. The router deployed with the intelligent agent acts as a message consumer, and by subscribing to the message topic, it regularly obtains the latest computing power resource information and passes it to the intelligent agent.

[0060] If there is no backbone router, the initial node address is sent directly to the edge router on the computing power side.

[0061] Through the above scheme, the computing power status may change in the process of the data packet being sent from the user-side edge router, passing through the backbone router in the network, and finally reaching the computing power side edge router accessed by the computing power resource node. Therefore, when passing through the backbone router, the second agent reinforcement learning model deployed on the backbone router is used to re-determine the new initial node address, which is more consistent with the computing power status of each node in the current distributed computing power network, and the new initial node address obtained is more accurate.

[0062] In some embodiments, a backbone router is defined as a router with a degree greater than 2. For a router with a degree equal to 2, data packets are only transmitted from one port and sent from another port, and the conditions for intelligent scheduling are not met.

[0063] In some embodiments, step 103 specifically includes:

[0064] Step 1031: The edge router on the computing power side determines the computing power resources corresponding to the initial node address.

[0065] Step 1032, determine whether the computing power resources meet the target computing power requirements.

[0066] Step 1033: In response to the computing power resources satisfying the target computing power requirement, it is determined that there is no need to update the initial node address, and the initial node address is used as the target node address.

[0067] or,

[0068] Step 1034, in response to the computing power resources not meeting the target computing power requirements, it is determined that the initial node address needs to be updated, and the computing power side edge router updates the initial node address until the computing power resources meet the target computing power requirements, obtains an updated node address, and uses the updated node address as the target node address.

[0069] During specific implementation, the edge router on the computing power side determines the computing power resources of the edge node corresponding to the initial node address, and determines whether the computing power resources can meet the user's target computing power requirements.

[0070] If the computing power resources are sufficient to meet the target computing power requirements, it means that the edge node can meet the user's needs. At this time, there is no need to update the initial node address, and the initial node address is used as the target node address.

[0071] If the computing power resources do not meet the target computing power requirements, it means that the edge node cannot meet the user's needs, and it is determined that the initial node address needs to be updated. The initial node address is updated using the computing power side edge router. The specific update process includes:

[0072] The edge router on the computing power side obtains the computing power status corresponding to the distributed computing power network at the current moment based on the message queue as the second computing power status.

[0073] The edge router on the computing power side is deployed with a third-agent reinforcement learning model, the target computing power requirement and the second computing power state are input into the third-agent reinforcement learning model, and a new node address is output through the third-agent reinforcement learning model.

[0074] The computing power side edge router corresponding to the new node address re-judges whether its computing power resources are sufficient to meet the target computing power requirements. If it is determined again that the target computing power requirements cannot be met, a new node address is determined again based on the computing power side edge router corresponding to the new node address until the computing power resources of the node corresponding to the new node address meet the target computing power requirements, and the new node address is used as the updated node address.

[0075] Through the above scheme, since the computing power offloading decision given by the user-side model may no longer be applicable after several hops, the model deployed on the edge router on the computing power side can redirect the demand offloading before the computing power demand is finally offloaded. The new initial node address is re-determined using the third-agent reinforcement learning model deployed on the edge router on the computing power side, which better matches the computing power status of each node in the current distributed computing power network, and the new initial node address is more accurate.

[0076] In some embodiments, after step 1033, the method further includes:

[0077] Step A, counting the number of updates of the initial node address;

[0078] Step B, in response to the number of updates being greater than a preset update number threshold, and the computing power resources of the node corresponding to the new node address do not meet the target computing power requirements, the node address obtained after the last update is used as the updated node address.

[0079] In specific implementation, when a data packet reaches the edge router on the computing power side, the intelligent agent deployed on the edge router on the computing power side will reschedule resources. This may cause the data packet to continue to propagate in the network because the resources of each computing power resource node do not meet the computing power requirements.

[0080] Therefore, the number of updates of the initial node address is counted. When the number of updates is greater than the preset update number threshold, and the computing power resources of the node corresponding to the new node address do not meet the target computing power requirements, the new node address will no longer be re-determined, and the node address obtained after the last update will be directly used as the updated node address.

[0081] Through the above scheme, when the number of initial node address updates reaches a certain limit, further updates are stopped to avoid data packets from continuing to propagate in the network, causing network congestion and other problems.

[0082] This embodiment provides a distributed computing network task scheduling method. The scheme deploys a multi-agent reinforcement learning model on a router to make decisions, and allocates appropriate computing resource nodes to the computing power requirements of the access users for demand unloading. Before the task scheduling process, the network distributes computing power information through the MP-BGP protocol to ensure that each router has global computing power information. In the computing power task scheduling process, the routing in the network is implemented through the default OSPF protocol, and the reinforcement learning model is deployed only on the edge router on the user side, the backbone router of the network, and the edge router on the computing power side. Considering that the computing power resource status is constantly changing in the scenario of multi-user access, the computing power unloading decision given by the user-side model may no longer be applicable after several hops. Therefore, the model deployed on the network backbone router and the computing power side edge router can redirect the demand unloading before the computing power demand is finally unloaded. When the number of computing power unloading redirections on the computing power side edge router reaches a certain threshold, the edge router no longer reschedules the computing power task, but chooses to add it to the last allocated computing power node task queue to wait, so as to avoid the situation where the computing power demand has been frequently scheduled due to insufficient resources.

[0083] In some embodiments, step 102 specifically includes:

[0084] Step 1021: The user-side edge router obtains resource node information of the distributed computing network, and determines available computing resources and available storage resources corresponding to each resource node according to the resource node information;

[0085] Step 1022: input the target computing power requirement and the current computing power state into a first agent reinforcement learning model corresponding to the user-side edge router, wherein the first agent reinforcement learning model includes constraints and optimization objectives, the constraints are the target computing amount and the target storage task corresponding to the target computing power requirement, and the optimization objective is to maximize the target reward function of the first agent reinforcement learning model;

[0086] Step 1023, calculate the first agent reinforcement learning model to obtain the initial node address corresponding to the target computing power requirement.

[0087] During specific implementation, the user-side edge router obtains the resource node information of the distributed computing network, and determines the available computing resources and available storage resources corresponding to each resource node based on the resource node information.

[0088] For example, the distributed computing network contains M users, and each user u∈{1,2,3,...,M} has several computing tasks. The computing task model of the user can be described as a set For each computing task t u,k, are represented as a tuple {CPU u,k ,Storage u,k}, representing the FLOP (floating point operations) and storage tasks of the task, in GB.

[0089] There are N computing resource nodes in the distributed computing network at the same time, and each computing resource node j∈{1,2,3,...,N} is composed of one or two servers. The resource nodes composed of two types have both computing and storage capabilities, and the resource nodes composed of one type of server only have one capability.

[0090] For computing servers, the triple is {CPU j ,Cost cpu,j ,Load cpu,j}, where CPU j is the total computing power of the server in FLOPS, Cost cpu,j The usage cost is charged by unit computing volume. cpu,j is the server load. For storage servers, the triple is {Storage j ,Cost storage,j ,Load storage,j ,Band j}, where Storage j is the storage capacity of the server, in GB, Cost storage,j The usage cost is charged by unit usage. storage,j Band is the server load condition. j The storage bandwidth of the server, in GB / s.

[0091] Therefore, each computing resource node is represented as:

[0092] R j ={CPU j ,Cost cpu,j ,Load cpu,j ,Storage j ,Cost storage,j ,Load storage,j ,Band j}.

[0093] Determine the available computing resources and available storage resources corresponding to each resource node according to the resource node information, wherein the available computing resources and available storage resources are expressed as follows:

[0094]

[0095] in, is the available computing resources, Available storage resources.

[0096] The target computing power requirement and the current computing power status are input into the first agent reinforcement learning model corresponding to the user-side edge router, wherein the first agent reinforcement learning model includes constraints and optimization objectives, the constraints are the target computing amount and target storage task corresponding to the target computing power requirement, the optimization objective is to maximize the target reward function of the first agent reinforcement learning model, and the first agent reinforcement learning model is calculated to obtain the initial node address corresponding to the target computing power requirement.

[0097] Based on the above example, the constraints of the first agent reinforcement learning model specifically include:

[0098] Determine the target computing amount and target storage task corresponding to the target computing power requirement. The constraint condition is expressed by the formula:

[0099]

[0100] Among them, each computing task t in the target computing power requirement u,k , are represented as a tuple {CPU u,k ,Storage u,k}, CPU u,k The target computing amount, Storage u,k Store tasks for the target, is the available computing resources, Available storage resources.

[0101] The optimization objectives of the first agent reinforcement learning model specifically include:

[0102] In collaborative multi-agent MADDPG, the state of each agent contains both local information and partial global information for efficient scheduling of multi-user tasks. The local state s of agent i is i,local Contains the computing tasks that need to be scheduled currently u,k , and its own routing table, which contains the next-hop node list and reachability information. The global state of the agent s i,global Includes all information about global computing resource nodes.

[0103] Determine the load of each resource node in the distributed computing network, and determine the global load imbalance of the agent according to the load of all resource nodes, wherein the global load imbalance is expressed by the formula:

[0104]

[0105] Among them, B is the global load imbalance, N is the total number of resource nodes in the distributed computing network, j is each resource node in the distributed computing network, and Load cpu,j Lood is the server load of the computing server in the distributed computing network. sttorage,j The server load of the storage server in the distributed computing network, The average load of all resource nodes is determined based on the load of all resource nodes.

[0106] Therefore, the state space S of agent i i =s i,local ∪s i,global The action of each agent is to choose to dispatch the computing task to a computing resource node. The action space A∈{a1,a2,a3,...,a i}, where a i =j, indicating that agent i will calculate task t u,k Schedule to the computing resource node j for execution. Collaboration reward function R cooperative It is used to measure whether the allocation decision of the agent optimizes the global resource utilization, task cost, delay and load balance. The collaborative reward function of the first agent reinforcement learning model is determined according to the global load imbalance. The collaborative reward function is expressed by the formula:

[0107]

[0108] Among them, R cooperative is the collaborative reward function, M is the total number of users in the distributed computing network, is the computing task t u,k Resource utilization on computing resource node j, is the computing task t u,k The usage cost of completing the task on computing resource node j, represents the time delay of the task, α, β, γ are preset coefficients respectively, δ is the penalty weight of imbalance, and δ is used to balance local rewards and global equilibrium.

[0109] Said and The specific determination process is expressed by the formula:

[0110]

[0111]

[0112] Said It represents the time delay of the task, including the network propagation delay from the agent to the computing resource node (measured based on the number of hops), the queuing delay of the task at the computing resource node, and the processing delay of the computing resource node. The specific determination process is expressed by the formula:

[0113]

[0114] in, is the number of hops from agent i to target node j, is the queuing delay of computing power demand uk in the queue of agent j,

[0115] Collaborative MADDPG includes a centralized critic network Q cooperative (S,a) uses the global state set S and the action set a of all agents to evaluate the joint Q value of all agent actions to guide policy optimization.

[0116] The target reward function of the first agent reinforcement learning model is determined according to the collaborative reward function. The target reward function is expressed by the formula:

[0117]

[0118] in, is the target reward function, π i (a i |s i ) is a distributed Actor network, π i (a i |s i ) Update the strategy based on the local state and some global information, S is the global state set, a is the action set of the agent, Q cooperative (S,a) is a centralized critic network, γQ cooperative (S′,a′|θ Q ) is the action value function in the next state S′ and the reward of the action a′ taken by the agent in this state, θ Q is the parameter of the Q network, which represents the parameter of the neural network model of the action value function Q(s,a), is the parameter of the policy network, which represents the parameter of the agent's policy network (i.e., action selection mechanism), S′ is the state, a′ is the action selected by the agent in state S′, and E is the mathematical expectation.

[0119] Through the above scheme, cooperation is mainly achieved through the global reward function R cooperative, centralized critic network and global information sharing. The local reward function contains global target items such as resource utilization, task cost, latency and load imbalance, guiding the agent to make an optimized contribution to the overall system in terms of behavior selection. Under the CTDE (centralized training and distributed execution) framework, the centralized critic can access the status and behavior of all agents during training, evaluate the contribution of joint behavior to the global reward, and encourage the agents to collaboratively optimize when updating strategies. At the same time, agents can share global information such as load status during execution, thereby forming a collaborative effect in local decision-making.

[0120] In this embodiment, by adopting multi-agent reinforcement learning, multiple agents are deployed on different nodes in the network, and each agent can learn and make decisions independently. Compared with centralized scheduling methods, distributed agents can make fast scheduling decisions based on local information, greatly improving the response speed of the system and the efficiency of task scheduling. Through reinforcement learning, the agent can optimize its decision-making strategy in continuous interaction and feedback, and gradually learn how to better allocate tasks and avoid resource conflicts. This learning mechanism allows the agent to be adaptive when facing a dynamically changing network environment, and can quickly respond to fluctuations in network traffic and changes in computing resources.

[0121] At the same time, in a distributed computing power network, computing power resources are scattered in different physical locations. The traditional centralized scheduling method is easy to become a bottleneck of the system when the number of task requests increases, resulting in single point failures and scheduling delays. Multi-agent reinforcement learning not only avoids overload of central nodes by distributing scheduling tasks to each agent node, but also improves the scalability and fault tolerance of the system. Even if some nodes fail, other agents can continue to schedule tasks according to the current network status to ensure the continuous operation of the system. The invention ensures the synchronization of computing power information in the distributed computing power network scenario through a message queue mechanism. On the other hand, the scheduling method based on reinforcement learning can also be dynamically adjusted according to real-time feedback in the network. Each agent will adjust the scheduling strategy according to its own observations and global reward functions to ensure that tasks are reasonably allocated among different nodes and avoid resource overload or idleness. Over time, the decision-making ability of the agent gradually improves, and ultimately achieves global optimization. This adaptive scheduling can not only improve the utilization of network resources, but also significantly improve the efficiency and quality of task execution.

[0122] The distributed computing network task scheduling method proposed in this embodiment is suitable for complex environments with multiple users and multiple computing nodes. For example, in large-scale cloud computing or edge computing networks, the diversity and complexity of tasks place higher demands on scheduling. Through MARL, customized scheduling strategies can be created for different types of tasks and resources to ensure that user requirements for task latency, bandwidth, computing power, etc. are met. This method has high adaptability, distributed architecture advantages, efficient resource utilization, and strong scalability, and provides a promising solution for task scheduling in large-scale distributed computing environments.

[0123] It should be noted that the method of the embodiment of the present disclosure can be performed by a single device, such as a computer or a server. The method of the present embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present disclosure, and the multiple devices will interact with each other to complete the described method.

[0124] It should be noted that the above describes some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0125] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a distributed computing power network task scheduling device.

[0126] refer to Figure 2 , Figure 2 The distributed computing power network task scheduling device of the embodiment includes:

[0127] The data acquisition module 201 is configured to obtain the target computing power demand for the user-side edge router, and determine the current computing power state corresponding to the distributed computing power network according to the multi-protocol border gateway protocol, wherein the current computing power state represents the task allocation state of each computing power node in the current distributed computing power network;

[0128] The initial node address determination module 202 is configured so that the user-side edge router inputs the target computing power requirement and the current computing power state into the first agent reinforcement learning model corresponding to the user-side edge router, and outputs the initial node address corresponding to the target computing power requirement via the first agent reinforcement learning model;

[0129] The target node address determination module 203 is configured such that the user-side edge router sends the initial node address to the computing power-side edge router, and the computing power-side edge router determines whether the initial node address needs to be updated, and determines the target node address according to the determination result;

[0130] The unloading module 204 is configured as a computing power side edge router to unload the target computing power demand to the target node corresponding to the target node address.

[0131] In some embodiments, the distributed computing network further includes a backbone router, and the target node address determination module 203 specifically includes:

[0132] A message path determination unit, configured to determine a target message path using an open shortest path first protocol and the initial node address;

[0133] A judging unit configured to judge whether there is a backbone router at least one intermediate node included in the target message path, wherein the intermediate node is a node other than the initial target node corresponding to the initial node address and the current node where the user-side edge router is located;

[0134] An updating unit is configured to, in response to the existence of a backbone router, determine a new initial node address according to the target computing power requirement using the backbone router, and the backbone router sends the new initial node address to the computing power side edge router; or,

[0135] The sending unit is configured to send the initial node address from the user-side edge router to the computing power-side edge router in response to the absence of the backbone router.

[0136] In some embodiments, the updating unit is specifically configured to:

[0137] Based on the message queue, the computing power state corresponding to the distributed computing power network at the current time is obtained as the first computing power state;

[0138] The target computing power requirement and the first computing power state are input into a second agent reinforcement learning model corresponding to the backbone router, and a new initial node address corresponding to the target computing power requirement is output via the second agent reinforcement learning model.

[0139] In some embodiments, the target node address determination module 203 specifically includes:

[0140] A resource determination unit, configured to determine the computing power resources corresponding to the initial node address by the computing power side edge router;

[0141] A determination unit, configured to determine whether the computing power resources meet the target computing power requirement;

[0142] A first target node address determining unit is configured to, in response to the computing power resource satisfying the target computing power requirement, determine that the initial node address does not need to be updated, and use the initial node address as the target node address; or

[0143] The second target node address determination unit is configured to determine that the initial node address needs to be updated in response to the computing power resources not meeting the target computing power requirement, and the computing power side edge router updates the initial node address until the computing power resources meet the target computing power requirement, obtains an updated node address, and uses the updated node address as the target node address.

[0144] In some embodiments, the second target node address determination unit is specifically configured to:

[0145] The edge router on the computing power side obtains the computing power state corresponding to the distributed computing power network at the current moment based on the message queue as the second computing power state;

[0146] Input the target computing power requirement and the second computing power state into a third agent reinforcement learning model corresponding to the computing power side edge router, and output a new node address through the third agent reinforcement learning model;

[0147] Until the computing power resources of the node corresponding to the new node address meet the target computing power requirement, the new node address is used as the updated node address.

[0148] In some embodiments, the initial node address determination module 202 is specifically configured to:

[0149] The user-side edge router obtains resource node information of the distributed computing power network, and determines the available computing resources and available storage resources corresponding to each resource node according to the resource node information;

[0150] Input the target computing power requirement and the current computing power state into a first agent reinforcement learning model corresponding to the user-side edge router, wherein the first agent reinforcement learning model includes constraints and optimization objectives, the constraints are the target computing amount and the target storage task corresponding to the target computing power requirement, and the optimization objective is to maximize the target reward function of the first agent reinforcement learning model;

[0151] The first agent reinforcement learning model is calculated to obtain an initial node address corresponding to the target computing power requirement.

[0152] In some embodiments, the constraints of the first agent reinforcement learning model specifically include:

[0153] Determine the target computing amount and target storage task corresponding to the target computing power requirement. The constraint condition is expressed by the formula:

[0154]

[0155] Among them, each computing task t in the target computing power requirement u,k , are represented as a tuple {CPU u,k ,Storage u,k}, CPU u,k The target computing amount, Storage u,k Store tasks for the target, is the available computing resources, is the available storage resources;

[0156] The optimization objectives of the first agent reinforcement learning model specifically include:

[0157] Determine the load of each resource node in the distributed computing network, and determine the global load imbalance of the agent according to the load of all resource nodes, wherein the global load imbalance is expressed by the formula:

[0158]

[0159] Among them, B is the global load imbalance, N is the total number of resource nodes in the distributed computing network, j is each resource node in the distributed computing network, and Load cpu,j is the server load of the computing server in the distributed computing network, Load storage,j The server load of the storage server in the distributed computing network, is the average load of all resource nodes determined according to the load of all resource nodes;

[0160] The collaborative reward function of the first agent reinforcement learning model is determined according to the global load imbalance, and the collaborative reward function is expressed by the formula:

[0161]

[0162] Among them, R cooperative is the collaborative reward function, M is the total number of users in the distributed computing network, is the computing task t u,k Resource utilization on computing resource node j, is the computing task t u,k The usage cost of completing the task on computing resource node j, represents the time delay of the task, α, β, γ are preset coefficients, and δ is the penalty weight of imbalance;

[0163] The target reward function of the first agent reinforcement learning model is determined according to the collaborative reward function. The target reward function is expressed by the formula:

[0164]

[0165] in, is the target reward function, π i (a i |s i ) is a distributed Actor network, S is a global state set, a is an action set of the agent, Q cooperative (S,a) is a centralized critic network, γQ cooperative (S′,a′|θ Q ) is the action value function in the next state S′ and the reward of the action a′ taken by the agent in this state, θ Q is the parameter of the Q network, which represents the parameter of the neural network model of the action value function Q(s,a), is the parameter of the policy network, which represents the parameter of the agent's policy network (i.e., action selection mechanism), S′ is the state, a′ is the action selected by the agent in state S′, and E is the mathematical expectation.

[0166] For the convenience of description, the above device is described by dividing it into various modules according to its functions. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0167] The device of the above-mentioned embodiment is used to implement the corresponding distributed computing power network task scheduling method in any of the above-mentioned embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0168] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the distributed computing power network task scheduling method described in any of the above embodiments is implemented.

[0169] Figure 3 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0170] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0171] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0172] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0173] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0174] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0175] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0176] The electronic device of the above-mentioned embodiment is used to implement the corresponding distributed computing power network task scheduling method in any of the above-mentioned embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0177] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the distributed computing power network task scheduling method described in any of the above embodiments.

[0178] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0179] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the distributed computing network task scheduling method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0180] It is understandable that before using the technical solutions of each embodiment of the present disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0181] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0182] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0183] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0184] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of simplicity.

[0185] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present disclosure difficult to understand, the known power / ground connections to the integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present disclosure difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present disclosure will be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it is apparent to those skilled in the art that the embodiments of the present disclosure can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0186] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0187] The embodiments of the present disclosure are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A distributed computing network task scheduling method, characterized in that: The distributed computing power network includes a user-side edge router and a computing power-side edge router, including: The user-side edge router obtains the target computing power demand and determines the current computing power state corresponding to the distributed computing power network according to the multi-protocol border gateway protocol, wherein the current computing power state represents the task allocation state of each computing power node in the current distributed computing power network; The user-side edge router inputs the target computing power requirement and the current computing power state into a first agent reinforcement learning model corresponding to the user-side edge router, and outputs an initial node address corresponding to the target computing power requirement via the first agent reinforcement learning model; The user-side edge router sends the initial node address to the computing power-side edge router, and the computing power-side edge router determines whether the initial node address needs to be updated, and determines the target node address according to the determination result; The computing power side edge router offloads the target computing power demand to the target node corresponding to the target node address.

2. The method according to claim 1, characterized in that The distributed computing network also includes backbone routers. The user-side edge router sends the initial node address to the computing power-side edge router, including: Determine the target message path by using the open shortest path first protocol and the initial node address; Determine whether there is a backbone router at least one intermediate node included in the target message path, wherein the intermediate node is a node other than the initial target node corresponding to the initial node address and the current node where the user-side edge router is located; In response to the existence of a backbone router, a new initial node address is determined according to the target computing power requirement by using the backbone router, and the backbone router sends the new initial node address to the computing power side edge router; or, In response to the absence of a backbone router, the user-side edge router sends the initial node address to the computing power-side edge router.

3. The method according to claim 2, characterized in that The determining of a new initial node address according to the target computing power requirement by using a backbone router includes: Based on the message queue, the computing power state corresponding to the distributed computing power network at the current time is obtained as the first computing power state; The target computing power requirement and the first computing power state are input into a second agent reinforcement learning model corresponding to the backbone router, and a new initial node address corresponding to the target computing power requirement is output via the second agent reinforcement learning model.

4. The method according to claim 1, characterized in that: The computing power side edge router determines whether the initial node address needs to be updated, and determines the target node address according to the determination result, including: The edge router on the computing power side determines the computing power resources corresponding to the initial node address; Determine whether the computing power resources meet the target computing power requirements; In response to the computing power resource satisfying the target computing power requirement, determining that the initial node address does not need to be updated, and using the initial node address as the target node address; or, In response to the computing power resources not meeting the target computing power requirements, it is determined that the initial node address needs to be updated, and the computing power side edge router updates the initial node address until the computing power resources meet the target computing power requirements, obtains an updated node address, and uses the updated node address as the target node address.

5. The method according to claim 4, characterized in that The computing power side edge router updates the initial node address until the computing power resource meets the target computing power requirement, and obtains an updated node address, including: The edge router on the computing power side obtains the computing power state corresponding to the distributed computing power network at the current moment based on the message queue as the second computing power state; Input the target computing power requirement and the second computing power state into a third agent reinforcement learning model corresponding to the computing power side edge router, and output a new node address through the third agent reinforcement learning model; Until the computing power resources of the node corresponding to the new node address meet the target computing power requirement, the new node address is used as the updated node address.

6. The method according to claim 5, characterized in that After outputting a new node address via the third agent reinforcement learning model, the method further includes: Counting the number of updates of the initial node address; In response to the number of updates being greater than a preset update number threshold, and the computing power resources of the node corresponding to the new node address not meeting the target computing power requirement, the node address obtained after the last update is used as the updated node address.

7. The method according to claim 1, characterized in that The user-side edge router inputs the target computing power requirement and the current computing power state into a first agent reinforcement learning model corresponding to the user-side edge router, and outputs an initial node address corresponding to the target computing power requirement via the first agent reinforcement learning model, including: The user-side edge router obtains resource node information of the distributed computing power network, and determines the available computing resources and available storage resources corresponding to each resource node according to the resource node information; Input the target computing power requirement and the current computing power state into a first agent reinforcement learning model corresponding to the user-side edge router, wherein the first agent reinforcement learning model includes constraints and optimization objectives, the constraints are the target computing amount and the target storage task corresponding to the target computing power requirement, and the optimization objective is to maximize the target reward function of the first agent reinforcement learning model; The first agent reinforcement learning model is calculated to obtain an initial node address corresponding to the target computing power requirement.

8. The method according to claim 7, characterized in that The constraints of the first agent reinforcement learning model specifically include: Determine the target computing amount and target storage task corresponding to the target computing power requirement. The constraint condition is expressed by the formula: Among them, each computing task t in the target computing power requirement u,k , are represented as a tuple {CPU u,k ,Storage u,k }, CPU u,k The target computing amount, Storage u,k Store tasks for the target, is the available computing resources, For available storage resources; The optimization objectives of the first agent reinforcement learning model specifically include: Determine the load of each resource node in the distributed computing network, and determine the global load imbalance of the agent according to the load of all resource nodes, wherein the global load imbalance is expressed by the formula: Among them, B is the global load imbalance, N is the total number of resource nodes in the distributed computing network, j is each resource node in the distributed computing network, and Load cpu,j is the server load of the computing server in the distributed computing network, Load storage,j The server load of the storage server in the distributed computing network, is the average load of all resource nodes determined according to the load of all resource nodes; The collaborative reward function of the first agent reinforcement learning model is determined according to the global load imbalance, and the collaborative reward function is expressed by the formula: Among them, R cooperative is the collaborative reward function, M is the total number of users in the distributed computing network, is the computing task t u,k Resource utilization on computing resource node j, is the computing task t u,k The usage cost of completing the task on computing resource node j, represents the time delay of the task, α, β, γ are preset coefficients, and δ is the penalty weight of imbalance; The target reward function of the first agent reinforcement learning model is determined according to the collaborative reward function. The target reward function is expressed by the formula: Q cooperative (S,a|θ Q )=E[R cooperative +γQ cooperative (S′,a′|θ Q )] in, is the target reward function, π i (a i |s i ) is a distributed Actor network, S is a global state set, a is an action set of the agent, Q cooperative (S,a) is a centralized critic network, γQ cooperative (S′,a′|θ Q ) is the action value function in the next state S′ and the reward of the action a′ taken by the agent in this state, θ Q is the parameter of the Q network, which represents the parameter of the neural network model of the action value function Q(s,a), is the parameter of the policy network, which represents the parameter of the agent's policy network (i.e., action selection mechanism), S′ is the state, a′ is the action selected by the agent in state S′, and E is the mathematical expectation.

9. A distributed computing network task scheduling device, characterized in that: include: The data acquisition module is configured to obtain the target computing power demand for the user-side edge router, and determine the current computing power state corresponding to the distributed computing power network according to the multi-protocol border gateway protocol, wherein the current computing power state represents the task allocation state of each computing power node in the current distributed computing power network; An initial node address determination module is configured so that the user-side edge router inputs the target computing power requirement and the current computing power state into a first agent reinforcement learning model corresponding to the user-side edge router, and outputs an initial node address corresponding to the target computing power requirement via the first agent reinforcement learning model; The target node address determination module is configured so that the user-side edge router sends the initial node address to the computing power-side edge router, and the computing power-side edge router determines whether the initial node address needs to be updated, and determines the target node address according to the determination result; The unloading module is configured as a computing power side edge router to unload the target computing power demand to the target node corresponding to the target node address.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 8 when executing the program.

Images