Method, system and device for scheduling computing power network resources and storage medium

Abstract

The invention relates to the technical field of computing power networks, and provides a computing power network resource scheduling method, system and device and a storage medium, and the method comprises the steps: determining computing power resource demand information according to a first computing power service request; and according to the computing power resource demand information and the network state information, determining a target computing node, a first link path and a first resource allocation strategy based on the minimum service time delay and the link load balance degree. The first computing power service request is allocated to the target computing power service domain after being subjected to computing power resource type matching, the solution space is effectively reduced, the complexity of obtaining an optimal or approximately optimal solution in the solution space is reduced, the resource scheduling efficiency is improved, each computing node serving as an intelligent agent can independently make a resource scheduling decision, the computing power service request is responded in time, and the resource scheduling efficiency is improved. The resource scheduling processing duration is shortened, the real-time processing capability is enhanced, the resource scheduling decision is determined based on the principle of minimizing the service and the link load balance degree, and the resource utilization rate is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computing power networks, and in particular, to a method, system, device, and storage medium for scheduling computing power network resources. Background Art

[0002] A computing power network is a network architecture with the core goal of integrating distributed computing power resources and achieving efficient resource scheduling. Its main task is to uniformly manage the computing power of each heterogeneous computing node, such as cloud servers, edge devices, and terminal devices, and through resource scheduling, allocate the computing tasks of computing power services to the most suitable computing nodes, thereby improving the computing efficiency and resource utilization rate of the entire system.

[0003] The resource scheduling algorithm is the core algorithm of the computing power network and affects the performance of the computing power network. However, the existing resource scheduling algorithms have the following problems: Large solution space: Due to the numerous node states, task characteristics, and environmental change factors in the network, the resource algorithm needs to search for the optimal or near-optimal solution in a huge solution space. Moreover, with the expansion of the scale of the computing power network, the increase in the number of computing nodes will lead to an increase in the computational complexity of the algorithm, thereby reducing the resource scheduling efficiency; Low resource utilization rate: During the resource scheduling process, attention is paid to the computing power resources provided by computing nodes, but other resources such as bandwidth in the computing power network are ignored, resulting in the underutilization of the resources of the computing power network; Long algorithm processing time: Facing a large number of computing power service requests and complex scheduling algorithms, the computing power service control node requires a long processing time for the overall resource scheduling process, thereby affecting the real-time processing ability of the entire computing power network.

[0004] Therefore, in the process of resource scheduling in the computing power network, how to improve the scheduling efficiency, increase the resource utilization rate, and shorten the processing duration is an urgent problem to be solved. Summary of the Invention

[0005] The present invention provides a method, system, device, and storage medium for scheduling computing power network resources to solve the problems in the prior art that in the process of resource scheduling in the computing power network, there are problems such as a large solution space resulting in low resource scheduling efficiency, low resource utilization rate, and long algorithm processing time.

[0006] The present invention provides a method for scheduling computing power network resources, which is applied to a computing node acting as an agent, and includes: Responding to a first computing power service request, and determining computing power resource demand information according to the first computing power service request; Based on the computing power resource demand information and network state information, and based on minimizing service latency and link load balancing degree, determining a target computing node, a first link path, and a first resource allocation strategy; Determine that the target computing node is the node itself: Determine the target client according to the first computing power service request, interact with the target client according to the first link path, and configure the computing power resources corresponding to the target client according to the first resource allocation policy; Determine that the target computing node is another node: Send the first computing power service request, the first link path, and the first resource allocation policy to the target computing node, so that the target computing node interacts with the client based on the first link path and the target computing node configures the computing power resources corresponding to the client based on the first resource allocation policy; Wherein, the first computing power service request is sent by the access node after determining the target computing power service domain according to the type of computing power resource requirement of the first computing power service request; the target computing power service domain includes multiple computing nodes.

[0007] According to a computing power network resource scheduling method provided by the present invention for a computing node serving as an agent, the method for determining a target computing node, a first link path, and a first resource allocation policy based on the computing power resource requirement information and network status information, including: According to the computing power resource requirement information and the network status information, determine the transmission delay and bandwidth utilization rate of the links between each network node, and determine the computing delay, queuing delay, and resource status of each computing node within the same computing power service domain; Use the transmission delay, the computing delay, the queuing delay, the bandwidth utilization rate, and the resource status as the input of a neural network model to obtain a target computing node, a first link path, and a first resource allocation policy; Wherein, the neural network model is obtained by multi-agent deep reinforcement learning, and the neural network model is used to minimize the service delay and link load balancing degree; the service delay is determined by the transmission delay, the computing delay, and the queuing delay; the link load balancing degree is determined by the bandwidth utilization rate of each link.

[0008] According to a computing power network resource scheduling method provided by the present invention for a computing node serving as an agent, the neural network model includes a first neural network and a second neural network; after sending the first computing power service request, the first link path, and the first resource allocation policy to the target computing node, it further includes: Generate experience data according to the network status information and store the experience data in an experience replay buffer, where the experience data represents the influence of resource scheduling decision actions; Randomly extract a preset number of the experience data from the experience replay buffer as training data; According to the training data, update the first weight parameter of the first neural network, and according to the first weight parameter, update the second weight parameter of the second neural network based on a preset transfer coefficient until a preset training termination condition is satisfied; Wherein, the first neural network is used to update the weight parameter; the second neural network is used to output the target computing node, the first link path, and the first resource allocation strategy according to the computing power resource demand information and the network state information.

[0009] According to a computing power network resource scheduling method provided by the present invention and applied to a computing node as an agent, it further includes: In response to a second computing power service request, obtain a second link path and a second resource allocation strategy; Determine a target client according to the second computing power service request; Establish a session with the target client according to the second link path, construct a computing power service instance corresponding to the target client, and configure the computing power resources of the computing power service instance according to the second resource allocation strategy; Announce the status information of the computing power service instance to the computing nodes in the same computing power service domain; Determine that the computing power service instance has finished execution, release the computing power resources of the computing power service instance, and announce the status of its own node to the computing nodes in the same computing power service domain; Wherein, the second computing power service request, the second link path, and the second resource allocation strategy are generated by the computing nodes in the same computing power service domain.

[0010] The present invention also provides a computing power network resource scheduling method applied to an access node, including: In response to a computing power service request from a client, determine the type of computing power resource demand of the computing power service request; According to the type of computing power resource demand, determine a target computing power service domain, and send the computing power service request to the target computing power service domain so that the agent in the target computing power service domain executes a computing power network resource scheduling method applied to a computing node as an agent.

[0011] According to a computing power network resource scheduling method provided by the present invention and applied to an access node, the determining a target computing power service domain according to the type of computing power resource demand and sending the computing power service request to the target computing power service domain includes: According to the type of computing power resource demand, retrieve a computing power service domain that provides corresponding computing power resources; Determine that the computing power service domain can provide corresponding various types of computing power resources simultaneously, and execute the steps: Take the computing power service domain as the target computing power service domain; Send the computing power service request to the target computing power service domain; Determine that there is no computing power service domain that can provide corresponding various types of computing power resources simultaneously, and execute the steps: Split the computing power service request based on the required type of computing power resources to form computing power service sub-requests; For each of the computing power service sub-requests, take the computing power service domain that provides the corresponding type of computing power resources as the target computing power service domain; Send each of the computing power service sub-requests to the corresponding target computing power service domain.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements a computing power network resource scheduling method as described in any one of the above.

[0013] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements a computing power network resource scheduling method as described in any one of the above.

[0014] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements a computing power network resource scheduling method as described in any one of the above.

[0015] A computing power network resource scheduling method, system, device, and storage medium provided by the present invention have at least the following beneficial effects: After the first computing power service request passes through the access node for computing power resource type matching, it is allocated to the target computing power service domain, reducing the solution space to the target service domain. In the target service domain, the computing nodes acting as agents respond to the first computing power service request to determine the computing power resource demand information. Combining the network status information, based on the principle of minimizing service delay and link load balancing degree, the target computing node that specifically provides the computing power service is determined, and the first link path and the first resource allocation strategy for computing power service interaction are determined. When the target computing node is the node itself, it interacts with the target client according to the first link path and the first resource allocation strategy to provide the computing power service; when the target computing node is other nodes, the first computing power service request, the first link path, and the first resource allocation strategy are sent to the target computing node, so that the target computing node interacts with the target client according to the first link path and the first resource allocation strategy to provide the computing power service. In this way, after the first computing power service request is allocated to the target computing power service domain through computing power resource type matching, the solution space can be effectively reduced, which is beneficial to reducing the complexity of obtaining the optimal or approximate optimal solution in the solution space and improving the resource scheduling efficiency. At the same time, in the target computing power service domain, each computing node acting as an agent can independently respond to the first computing power service request and make resource scheduling decisions, that is, each agent can perform the function of computing power service management and control, can respond to the computing power service request in a timely manner, which is beneficial to shortening the resource scheduling processing time and enhancing the real-time processing ability. In addition, the resource scheduling decision is determined based on the principle of minimizing service and link load balancing degree, comprehensively considering computing power resources and link resources, which is beneficial to making full use of the resources of the computing power network and improving the resource utilization rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0017] Figure 1 It is a flowchart showing a computing power network resource scheduling method applied to a computing node acting as an agent provided by the present invention.

[0018] Figure 2 It is a flowchart showing a computing power network resource scheduling method applied to an access node provided by the present invention.

[0019] Figure 3 It is a schematic structural diagram of a computing power network resource scheduling system provided by the present invention.

[0020] Figure 4 It is a schematic structural diagram of the electronic device provided by the present invention. Specific Embodiments

[0021] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0022] To facilitate the understanding of the technical solution of the present invention, the basic architecture of the computing power network will be described first, as Figure 3 shown. The computing power network includes a client, a routing and forwarding device, and a computing node. Among them, the client is the initiator of the computing power service request; the routing and forwarding device is used for routing and forwarding data information to realize communication between the computing service node and the client, providing a basis for the computing power service; the computing node has computing power resources to provide computing power services to the client, and the computing node includes a cloud computing node, an edge computing node, and an end node.

[0023] Next, with reference to Figure 1 and Figure 3 a computing power network resource scheduling method of the present invention will be described. Applied to a computing node as an agent, it includes: S100: In response to a first computing power service request, determine computing power resource demand information according to the first computing power service request; S200: According to the computing power resource demand information and network status information, based on minimizing service delay and link load balancing degree, determine a target computing node, a first link path, and a first resource allocation strategy; S310: Determine that the target computing node is the node itself: Determine a target client according to the first computing power service request, interact with the target client according to the first link path, and configure the computing power resources corresponding to the target client according to the first resource allocation strategy; S320: Determine that the target computing node is another node: Send the first computing power service request, the first link path, and the first resource allocation strategy to the target computing node, so that the target computing node interacts with the client based on the first link path and the target computing node configures the computing power resources corresponding to the client based on the first resource allocation strategy; Among them, the first computing power service request is sent by the access node after determining the target computing power service domain according to the computing power resource demand type of the first computing power service request; the target computing power service domain includes multiple computing nodes.

[0024] After the first computing power service request is matched with the computing power resource type by the access node, it is allocated to the target computing power service domain, so that the solution space is reduced to the target service domain. In the target service domain, the computing nodes acting as agents respond to the first computing power service request to determine the computing power resource demand information, and based on the principle of minimizing service delay and link load balancing degree in combination with the network state information, determine the target computing nodes that specifically provide computing power services, and determine the first link path and the first resource allocation strategy for the computing power service interaction. When the target computing node is the node itself, it interacts with the target client according to the first link path and the first resource allocation strategy to provide computing power services; when the target computing node is other nodes, it sends the first computing power service request, the first link path and the first resource allocation strategy to the target computing node, so that the target computing node interacts with the target client according to the first link path and the first resource allocation strategy to provide computing power services.

[0025] In this way, after the first computing power service request is matched with the computing power resource type and allocated to the target computing power service domain, it can effectively reduce the solution space, which is beneficial to reducing the complexity of obtaining the optimal or approximate optimal solution in the solution space and improving the resource scheduling efficiency. At the same time, in the target computing power service domain, each computing node acting as an agent can independently respond to the first computing power service request and make resource scheduling decisions, that is, each agent can play the function of computing power service management and control, can respond to the computing power service request in time, which is beneficial to shortening the resource scheduling processing time and enhancing the real-time processing ability. In addition, the resource scheduling decision is determined based on the principle of minimizing service and link load balancing degree, comprehensively considering computing power resources and link resources, which is beneficial to making full use of the resources of the computing power network and improving the resource utilization rate.

[0026] The computing nodes include cloud computing nodes, edge computing nodes and end nodes. In some embodiments of the present invention, the cloud computing nodes and edge computing nodes can be used as agents, so that each cloud computing node and edge computing node can independently make resource scheduling decisions. Even with the expansion of the computing power network and the increase in the number of computing nodes, since the number of computing nodes acting as agents will also increase accordingly, the efficiency of resource scheduling can be maintained, and the response speed and real-time performance of resource scheduling can be guaranteed. In some embodiments of the present invention, each agent can be based on the method of multi-agent deep reinforcement learning (MADRL), cooperate with each other and achieve the effect of independently making resource scheduling decisions.

[0027] It can be understood that the target computing node is selected from the computing power service domain to which it belongs as the computing node of the intelligent agent. The target computing node can be the computing node itself serving as the intelligent agent, or other computing nodes in the same computing power service domain.

[0028] The computing power resource demand types of the first computing power service request can include various required computing power resource types, and the computing power service domain can provide various computing power resource types. Refer to Figure 3 , the computing nodes in the computing power service domain provide some or all of the computing power resource types of the computing power service domain. However, computing nodes with the same computing power resource type are not necessarily in the same computing power service domain. In some embodiments of the present invention, the computing power service domain can be divided based on the position of the computing node in the computing power network and the computing power resource type provided by the computing node. When there are two or more computing power service domains providing the same computing power resource type, the access node can, based on the computing power resource demand type, combine with the transmission delay to the computing power service domain, and use the computing power service domain that meets the computing power resource demand type and has the lowest transmission delay as the target computing power service domain.

[0029] The network state information includes the link state information between nodes and the node state information of each node. In some embodiments of the present invention, the network state information can be in the form of an undirected graph. In the undirected graph , the graph node V in the figure corresponds to the network node in the computing power network, the graph edge L in the undirected graph corresponds to the link between network nodes in the computing power network, the node set M represents the physical computing nodes, M ∈ V, and the set M includes mutually exclusive end node subsets Md, edge computing node subsets Me, and cloud computing node subsets Mc. The physical computing nodes and The link between them is represented by . , and The bandwidth between them is represented by .

[0030] It should be noted that a physical computing node can be virtualized into multiple computing nodes. The physical computing node m can correspond to multiple virtual computing nodes n, and the corresponding relationship can be represented by the matrix . The element in the matrix means that the virtual computing node n corresponds to the physical computing node m. In some embodiments of the present invention, the state of the computing node n can be represented by , where represents the CPU capability of the computing node n, with the unit of OPS (operations / s); represents the GPU capability of the computing node n; represents the bandwidth resource of the computing node n, It represents the waiting operation time of computing node n, which is the sum of the operation times of computing tasks in the waiting queue. It can be understood that the state of computing node n may also include other performance information such as memory and storage space.

[0031] During the operation of the computing power network, as the computing power network expands, new computing nodes will gradually be connected to the computing power network. When a computing node connects to the computing power network, it undergoes a computing power registration process: The computing node sends the computing power resources it possesses to the nearest routing node. The routing node announces the computing power status, enabling each routing node to establish and maintain a computing power status table after receiving the computing power resource information of the computing node. Specifically, the routing node obtains the CPU computing power, computer hardware parameters, etc. of the end node, and simultaneously obtains the channel parameters between each end node and its corresponding edge computing node, such as channel gain, transmit power, signal-to-interference-plus-noise ratio, etc.; obtains the bandwidth resources, bandwidth resource allocation status, computing power resources, computing power resource allocation status, etc. of the edge computing node; obtains information such as the computing power resources of the cloud computing center; Based on the computing power resources provided by the computing node, such as CPU service resources, GPU service resources, etc., the computing node is divided into different computing power service domains based on SID (Service Identifiers), forming a mapping relationship table between virtual computing nodes and actual physical computing nodes; The access node, which is the first node accessed by the client, calculates the transmission delay of each network node within the computing power service domain based on the computing power resource situation, network status information, etc. of the computing node, so as to subsequently determine the target computing power service domain corresponding to the client's computing power service request.

[0032] The first computing power service request and the second computing power service request described later are essentially both computing power service requests. "First" and "second" are only for conveniently distinguishing the sources of the computing power service requests. The first computing power service request comes from the access node, which obtains the computing power service request from the client, and the second computing power service request comes from other computing nodes within the same computing power service domain.

[0033] The computing power resource demand information of the computing power service request can be represented by where, represents the data transmission volume (bit) of computing task k; represents the number of CPU cycles required to complete computing task k; represents the number of GPU cycles required to complete computing task k; represents the delay constraint of computing task k, that is, the service delay requirement for completing the client's computing task k. When the computing power service request has no requirement for a certain computing power resource, the corresponding computing power resource demand value is 0.

[0034] The allocation of computing tasks can be represented using a matrix The elements in the matrix indicate that computing task k is allocated to computing node n, indicating that computing task k is not allocated to computing node n.

[0035] In some embodiments of a computing power network resource scheduling method where the present invention is applied to a computing node serving as an agent, S200 includes: According to the computing power resource demand information and the network status information, determine the transmission delay and bandwidth utilization rate of the links between each network node, and determine the computing delay, queuing delay, and resource status of each computing node within the same computing power service domain; Use the transmission delay, the computing delay, the queuing delay, the bandwidth utilization rate, and the resource status as the input of a neural network model to obtain a target computing node, a first link path, and a first resource allocation strategy; Among them, the neural network model is obtained by multi-agent deep reinforcement learning, and the neural network model is used to minimize the service delay and the link load balancing degree; the service delay is determined by the transmission delay, the computing delay, and the queuing delay; the link load balancing degree is determined by the bandwidth utilization rate of each link.

[0036] According to the network status information, determine the link status information of each link. According to the computing power resource demand information and the network status information, determine the computing node status information of each computing node. The link status information includes the transmission delay and the bandwidth utilization rate, and the computing node status information includes the computing delay, the queuing delay, and the resource status. Input the determined information into the neural network model for processing, and obtain the output target computing node, the first link path, and the first resource allocation strategy as the resource scheduling decision. The neural network model is obtained by multi-agent deep reinforcement learning, which can enable the agents to cooperate with each other and achieve the effect of independently making resource scheduling decisions. The target computing node represents the specific node that provides computing power services to the client. The first link path represents the routing path for the target computing node to communicate with the client for data information transmission, and the first resource allocation strategy represents the computing power resource allocation when providing computing power services to the client.

[0037] The first link path and the second link path described later are essentially link paths. "First" and "Second" are only for convenience in distinguishing the sources. The first link path is generated by the computing node itself serving as an agent, and the second link path comes from other computing nodes within the same computing power service domain. Similarly, the first resource allocation strategy and the second resource allocation strategy described later are essentially resource allocation strategies. The former is generated by the computing node itself serving as an agent, and the latter comes from other computing nodes within the same computing power service domain.

[0038] In some embodiments of the present invention, based on the network state information being an undirected graph in the implementation manner, the state of computing node n is represented by and the computing power resource demand information is represented by . The computing delay can be calculated by the following formula: where is the computing delay for computing node n to process computing task k; is the proportion of CPU resources allocated by computing node n for computing task k; is the proportion of GPU resources allocated by computing node n for computing task k; and are mainly determined according to the ratio between the computing power resource demand of computing task k and the total resource amount of computing node n. Additionally, the above formula is only used when the remaining computing power resources of the computing node can meet the requirements of the computing task.

[0039] The queuing delay can be calculated by the following formula: where is the queuing delay for computing node n to process computing task k, that is, the duration when computing task k arrives at computing node n and is in an unprocessed state; is the set of computing tasks waiting to be processed in computing node n at time slot t. The formula indicates that the queuing delay of computing task k is equal to the sum of the computing delays of all computing tasks in front of the waiting queue.

[0040] The transmission delay can be calculated by the following formula: where is the transmission delay for computing node n to process computing task k; is the set of links between source node m and computing node n; is the link between node i and node j; is the data transmission volume of computing task k; is the data transmission rate for transmitting computing task k between node i and node j.

[0041] Since the link can be divided into a wired link and a wireless link, the data transmission rate can be calculated by the following formula: where is the bandwidth ratio allocated to computing task k, ; is the bandwidth between node i and node j; is the signal-to-interference ratio between node i and node j. Since the returned calculation result is much smaller than the data transmission amount, the return delay can be ignored.

[0042] Service delay It can be calculated by the following formula: in, The service latency for computing node n to process computing task k; The computational latency of computing node n for processing computing task k; The queuing delay of computing node n for processing computing task k; The transmission delay of computing task k processed by computing node n.

[0043] Total service delay of computing tasks in the computing service domain in time slot t for: in, is the total service delay of each computing node in the computing service domain to process computing tasks; N is the total number of computing nodes; K is the total number of computing tasks; Assigning elements of the matrix to computational tasks, Indicates that computing task k is assigned to computing node n. If Indicates that computing task k is not assigned to computing node n; The service latency for computing node n to process computing task k.

[0044] Link load balancing It can be calculated by the following formula: in, is the link load balance, which indicates the balance of link utilization in the computing network. The smaller the value, the higher the balance degree and the more fully the link resources are utilized; is the set of links, is the total number of links; is the bandwidth utilization of the mth link; is the average bandwidth utilization of each link; is the bandwidth utilization of the link between node i and node j, and It is a one-to-one correspondence, which can be understood as is numbered m; is the bandwidth utilization rate allocated to computing task k for the link between node i and node j; K is the total amount of computing tasks carried by the link.

[0045] Define the utility function as: Among them, is the total service delay weight; is the link load balancing degree weight. Let the utility function be minimized to achieve the effect of minimizing both the service delay and the link load balancing. By adjusting and the proportion of the total service delay and the link load balancing degree can be adjusted.

[0046] At the same time, when solving the optimal solution that minimizes the utility function , the following constraint conditions need to be satisfied: Among them, is the computing delay for computing node n to process computing task k; is the queuing delay for computing node n to process computing task k; is the transmission delay for computing node n to process computing task k; is the delay constraint for computing task k; is the bandwidth utilization rate of the link between node i and node j; is the bandwidth utilization rate allocated to computing task k for the link between node i and node j; is the CPU usage rate of computing node n; is the CPU usage rate allocated to computing task k by computing node n; is the GPU usage rate of computing node n; is the GPU usage rate allocated to computing task k by computing node n; K is the total amount of computing tasks in the computing node.

[0047] In some embodiments of the present invention, the optimization of the neural network model requires maximizing the reward function during the training process of multi-agent deep reinforcement learning, while the goal of resource scheduling is to minimize the above-mentioned utility function , therefore, the reward function of the neural network model Set to , that is . In this way, the neural network model processes based on the input information to minimize the service delay and link load balancing degree, so as to determine the target computing node, the first link path, and the first resource allocation strategy.

[0048] In some embodiments of the present invention, as an agent computing node, it can represent some network state information as follows: Among them, is the node state information of computing node n at time slot t; is the CPU usage rate of computing node n at time slot t; is the GPU usage rate of computing node n at time slot t; is the bandwidth utilization rate of computing task k at time slot t for computing node n.

[0049] Part of the state space formed by the agent computing node's network observation can be represented as: Among them, is the node state space at time slot t; is the node state information of computing node n at time slot t; N is the total number of computing nodes in the service domain with the same computing power.

[0050] It can be understood that the network state information may also include link state information, etc., and the state space may include link state space in addition to the node state space.

[0051] In some embodiments of the present invention, some of the resource scheduling decisions made by the agent computing node can be represented as: Among them, is part of the resource scheduling decision made at time slot t; is the task allocation matrix determined at time slot t, and the element in the matrix represents that computing task k is allocated to computing node n, and the computing node n to which computing task k is allocated is determined by the matrix, that is, the target computing node is determined; is the CPU usage rate determined at time slot t for computing node n to be allocated to computing task k; is the GPU usage rate determined at time slot t for computing node n to be allocated to computing task k; is the bandwidth utilization rate determined at time slot t for computing node n to be allocated to computing task k.

[0052] It can be understood that in addition to the above In addition to characterizing the target computing node and the resource allocation policy, the resource scheduling decision further includes a link path, and a link set can be used. It represents the link path adopted by computing task k.

[0053] It should be noted that in some embodiments of the present invention, the network state information of the computing node may not include the complete state information of the computing power network, but include the node state information of each computing node in the computing power service domain to which it belongs, and the link state information of each link in the network area where it is located.

[0054] In some embodiments of a computing power network resource scheduling method in which the present invention is applied to a computing node acting as an agent, the neural network model includes a first neural network and a second neural network; after sending the first computing power service request, the first link path, and the first resource allocation policy to the target computing node, it further includes: Generating experience data according to the network state information and storing the experience data in an experience replay buffer, where the experience data characterizes the influence of the resource scheduling decision action; Randomly extracting a preset number of the experience data from the experience replay buffer as training data; Updating the first weight parameter of the first neural network according to the training data, and updating the second weight parameter of the second neural network based on a preset transfer coefficient according to the first weight parameter until a preset training termination condition is met; Among them, the first neural network is used to update the weight parameter; the second neural network is used to output the target computing node, the first link path, and the first resource allocation policy according to the computing power resource demand information and the network state information.

[0055] After the computing node acting as an agent determines the target computing node, the first link path, and the first resource allocation policy, that is, after making a resource scheduling decision, it can obtain the effect of the resource scheduling decision based on the network state information after the resource scheduling decision action, and generate experience data to record the current resource scheduling decision and the corresponding effect. When meeting the preset update conditions, such as reaching the update period and other conditions, randomly extract the recorded experience data from the experience replay buffer as training data to train the first neural network, and then update the first weight parameter of the first neural network. Update the second weight parameter of the second neural network corresponding to the first weight parameter of the first neural network according to a preset transfer coefficient, such as 0.01, etc., until the preset training termination condition is met, and complete the update of the neural network model. In this way, through the neural network model including the first neural network and the second neural network, by updating the first weight parameter of the first neural network and correspondingly updating the second weight parameter of the second neural network based on the preset transfer coefficient, while optimizing the second neural network, the effect of maintaining the performance stability of the second neural network is achieved, avoiding too large an update adjustment amplitude, making the second neural network more stable and reliable in optimization. The second neural network is used to make resource scheduling decisions, which is conducive to more stable and reliable optimization of the resource scheduling effect.

[0056] In some embodiments of the present invention, the first neural network may include a first policy network and a first Q network, and the second neural network includes a second policy network and a second Q network. Train the first policy network and the first Q network according to the training data to update the first policy network and the first Q network. For the first Q network, calculate the mean square error between the predicted Q value and the target Q value. For the first policy network, use gradient ascent to try to maximize the expected Q value. Use the backpropagation algorithm to update the weights of the first policy network and the first Q network, and update the weights of the second policy network and the second Q network correspondingly based on the preset transfer coefficient.

[0057] In some embodiments of a computing power network resource scheduling method in which the present invention is applied to a computing node acting as an agent, it further includes: In response to a second computing power service request, obtain a second link path and a second resource allocation policy; Determine a target client according to the second computing power service request; Establish a session with the target client according to the second link path, construct a computing power service instance corresponding to the target client, and configure the computing power resources of the computing power service instance according to the second resource allocation policy; Announce the status information of the computing power service instance to the computing nodes in the same computing power service domain; Determine that the computing power service instance has finished execution, release the computing power resources of the computing power service instance, and notify other computing nodes in the same computing power service domain of its own node status; Among them, the second computing power service request, the second link path, and the second resource allocation policy are generated by computing nodes in the same computing power service domain.

[0058] Upon obtaining the second computing power service request, the second link path, and the second resource allocation policy sent by other computing nodes acting as agents, after determining the target client, establish a session through communication and interaction with the client according to the second link path and construct a computing power service instance to provide computing power services to the client. Configure the computing power resources of the computing power service instance according to the second resource allocation policy, such as the CPU usage rate and GPU usage rate occupied. Notify other computing nodes of the status information of the computing power service instance, so that other computing nodes can update the network status information, and facilitate computing nodes making resource scheduling decisions to generate empirical data. After the computing power service instance has finished execution, that is, after providing computing power services to the target client, notify other computing nodes of its own node status, so as to facilitate other computing nodes to update the network status information.

[0059] In this way, computing nodes in the same computing power service domain can cooperate with each other, providing a basis for computing nodes acting as agents to independently make resource scheduling decisions, and achieving the purpose of reliably and efficiently scheduling resources in the computing power network.

[0060] Next, a computing power network resource scheduling method applied to an access node provided by the present invention will be described. The following description of a computing power network resource scheduling method applied to an access node can be correspondingly referred to in relation to the above-described computing power network resource scheduling method applied to a computing node acting as an agent.

[0061] Refer to Figure 2 and Figure 3 The present invention also provides a computing power network resource scheduling method applied to an access node, including: S010: In response to a computing power service request from a client, determine the type of computing power resource requirements of the computing power service request; S020: According to the type of computing power resource requirements, determine the target computing power service domain, and send the computing power service request to the target computing power service domain, so that the agent in the target computing power service domain executes the above-described computing power network resource scheduling method applied to a computing node acting as an agent.

[0062] As the first node for a client to access the computing power network, the access node responds to the computing power service request of the client. Based on the type of computing power resource requirements needed for the computing power service request, it determines the target computing power service domain that can meet the computing power service request, and then sends the computing power service request to the target computing power service domain. In this way, the access node initially allocates the computing power service request, which can effectively reduce the solution space for subsequent resource scheduling, facilitate reducing the complexity of obtaining the optimal or approximate optimal solution in the solution space, and improve the resource scheduling efficiency.

[0063] After the computing node acting as an agent in the target computing power service domain obtains the computing power service request sent by the access node, that is, the first computing power service request, it determines the computing power resource requirement information. Combining with the network status information, based on the principle of minimizing service delay and link load balancing degree, it determines the target computing node that specifically provides the computing power service, and determines the first link path and the first resource allocation strategy for the computing power service interaction, and then realizes resource scheduling. The resource scheduling decision is determined based on the principle of minimizing service and link load balancing degree, comprehensively considering computing power resources and link resources, which is conducive to making full use of the resources of the computing power network and improving resource utilization rate.

[0064] It should be noted that for the same computing power service domain, the first computing power service requests from different access nodes may be transmitted to different computing nodes acting as agents, that is, they will not be fixedly transmitted to one node. Each computing node acting as an agent makes independent resource scheduling decisions to realize resource scheduling and provide computing power services for the client. Even with the expansion of the computing power network and the increase in the number of computing nodes, since the number of computing nodes acting as agents will also increase accordingly, each computing node acting as an agent makes independent resource scheduling decisions, which can maintain the efficiency of resource scheduling and ensure the response speed and real-time performance of resource scheduling.

[0065] In some embodiments of a method for scheduling computing power network resources applied to an access node in the present invention, the S020 includes: According to the type of computing power resource requirements, retrieve the computing power service domain that provides the corresponding computing power resources; Determine that there is a computing power service domain that can simultaneously provide the corresponding various types of computing power resources, and execute the steps: Take the computing power service domain as the target computing power service domain; Send the computing power service request to the target computing power service domain; Determine that there is no computing power service domain that can simultaneously provide the corresponding various types of computing power resources, and execute the steps: Split the computing power service request based on the required type of computing power resources to form computing power service sub-requests; For each of the computing power service sub-requests, use the computing power service domain that provides the corresponding computing power resource type as the target computing power service domain; Send each of the computing power service sub-requests to the corresponding target computing power service domain.

[0066] The access node retrieves the computing power service domains with the corresponding computing power resource types according to the computing power resource demand types. When there is a computing power service domain that can meet all the computing power resource demand types of the computing power service request, use this computing power service domain as the target computing power service domain; when there is no computing power service domain that meets all the computing power resource demand types, split the computing power service request based on the computing power resource types to form at least two computing power service sub-requests, so that there is a computing power service domain that meets all its computing power resource demands for the computing power service sub-requests. Determine the corresponding target computing power service domain for each computing power service sub-request, and then send the computing power service sub-requests to the corresponding target computing power service domains. In this way, the computing power service request is initially allocated to achieve the purpose of reducing the solution space, and at the same time, it can ensure that the target computing power service domain allocated for the computing power service request can meet the resource requirements of the computing task, which is beneficial to improving the reliability of the computing power service.

[0067] It can be understood that the computing power service sub-request is essentially equivalent to an independent computing power service request. It is only for the convenience of explanation that it is obtained by splitting. When the computing power service sub-request is transmitted to the computing node acting as an intelligent agent, it will also be used as the first computing power service request.

[0068] It should be noted that when there are multiple computing power service domains that meet all the computing power resource demand types of the computing power service request at the same time, the computing power service domain that meets the computing power resource demand type and has the lowest transmission delay can be further used as the target computing power service domain.

[0069] For the convenience of understanding, an illustrative example is given: The computing power resource types in the computing power network can be divided into three categories: A, B, and C. Among them, the computing power service domain 1 has computing power resources A and B, the computing power service domain 2 has computing power resources B and C, the computing power service request m requires computing power resource A, and the computing power service request n requires computing power resources A, B, and C. Then the computing power service request m is preferentially allocated to the computing power service domain 1 for processing. Since there is no computing power service domain that simultaneously meets the computing power resource demand types of the computing power service request n, the computing power service request n needs to be split into computing power service sub-requests n1 and n2 according to the computing power resource types, and then scheduled to the computing power service domain 1 and the computing power service domain 2 for processing respectively.

[0070] Next, a computing power network resource scheduling system provided by the present invention is described. The computing power network resource scheduling system described below can be mutually corresponding and referred to with the computing power network resource scheduling method described above.

[0071] Reference Figure 3 , the present invention further provides a computing power network resource scheduling system, including: An access node for obtaining a computing power service request from a client, and the access node is communicatively connected to a routing node; A computing node for providing computing power services, the computing node is communicatively connected to the routing node, and the computing node includes a cloud computing node, an edge computing node, and an end node; Wherein, the access node executes a computing power network resource scheduling method applied to the access node; the cloud computing node and the edge computing node act as agents to execute a computing power network resource scheduling method applied to the computing node acting as an agent.

[0072] The computing nodes providing the same type of computing power resources form a computing power service domain. The access node, as the first node for the client to access the computing power network, responds to the computing power service request of the client, determines the target computing power service domain that can meet the computing power service request based on the type of computing power resource requirements required by the computing power service request, and then sends the computing power service request to the target computing power service domain. In this way, the access node initially allocates the computing power service request, which can effectively reduce the solution space for subsequent resource scheduling, is beneficial to reducing the complexity of obtaining the optimal or approximate optimal solution in the solution space, and improves the resource scheduling efficiency.

[0073] After the computing node acting as an agent in the target computing power service domain obtains the computing power service request sent by the access node, that is, the first computing power service request, it determines the computing power resource requirement information, combines the network status information, and based on the principle of minimizing service delay and link load balancing degree, determines the target computing node that specifically provides computing power services, and determines the first link path and the first resource allocation strategy for computing power service interaction, thereby realizing resource scheduling. The resource scheduling decision is determined based on the principle of minimizing service and link load balancing degree, comprehensively considering computing power resources and link resources, which is beneficial to making full use of the resources of the computing power network and improving resource utilization rate.

[0074] For the same computing power service domain, the first computing power service requests from different access nodes may be transmitted to different computing nodes acting as agents, that is, they will not be fixedly transmitted to one node. Each computing node acting as an agent independently makes a resource scheduling decision to realize resource scheduling and provide computing power services for the client. Even with the expansion of the computing power network and the increase in the number of computing nodes, since the number of computing nodes acting as agents will also increase accordingly, each computing node acting as an agent independently makes a resource scheduling decision, which can maintain the efficiency of resource scheduling and ensure the response speed and real-time performance of resource scheduling.

[0075] Figure 4 Illustrates a schematic diagram of the physical structure of an electronic device, such asFigure 4 As shown in Figure 4 , the electronic device may include: a processor 810, a communications interface 820, a memory 830, and a communication bus 840. Among them, the processor 810, the communications interface 820, and the memory 830 complete communication with each other through the communication bus 840. The processor 810 may call the logical instructions in the memory 830 to execute the above-mentioned method for scheduling computing power network resources.

[0076] In some embodiments, the electronic device provided by the present invention may act as an access node and execute the above-mentioned method for scheduling computing power network resources applied to an access node; in some embodiments, it may act as a computing node and execute the above-mentioned method for scheduling computing power network resources applied to a computing node acting as an agent.

[0077] In addition, when the logical instructions in the above-mentioned memory 830 are implemented in the form of a software functional unit and sold or used as an independent product, they may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, may be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0078] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute a method for scheduling computing power network resources provided by the above-mentioned various methods.

[0079] On yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute a method for scheduling computing power network resources provided by the above-mentioned various methods.

[0080] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative effort.

[0081] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0082] In the description of the present invention, it should be understood that the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In the description of the present invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0083] In the description of this specification, the descriptions referring to terms such as "an embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0084] All actions of obtaining signals, information, or data in this application are carried out on the premise of complying with the corresponding data protection regulations and policies of the location and obtaining the authorization given by the owner of the corresponding device.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A computing power network resource scheduling method, characterized in that, Applied to a computing node acting as an agent, including: In response to a first computing power service request, determine computing power resource requirement information according to the first computing power service request; According to the computing power resource requirement information and network status information, based on minimizing service latency and link load balancing degree, determine a target computing node, a first link path, and a first resource allocation strategy; Determine that the target computing node is the node itself: Determine a target client according to the first computing power service request, interact with the target client according to the first link path, and configure the computing power resources corresponding to the target client according to the first resource allocation strategy; Determine that the target computing node is another node: Send the first computing power service request, the first link path, and the first resource allocation strategy to the target computing node, so that the target computing node interacts with the client based on the first link path and the target computing node configures the computing power resources corresponding to the client based on the first resource allocation strategy; Wherein, the first computing power service request is obtained by the access node after determining a target computing power service domain according to the type of computing power resource requirement of the first computing power service request; the target computing power service domain includes multiple computing nodes.

2. The method for scheduling computing power network resources according to claim 1, wherein, The determining the target computing node, the first link path, and the first resource allocation strategy based on minimizing service latency and link load balancing degree according to the computing power resource requirement information and network status information includes: According to the computing power resource requirement information and the network status information, determine the transmission latency and bandwidth utilization rate of the links between each network node, and determine the computing latency, queuing latency, and resource status of each computing node within the same computing power service domain; Use the transmission latency, the computing latency, the queuing latency, the bandwidth utilization rate, and the resource status as the input of a neural network model, and obtain a target computing node, a first link path, and a first resource allocation strategy; Wherein, the neural network model is obtained by multi-agent deep reinforcement learning, and the neural network model is used to minimize service latency and link load balancing degree; the service latency is determined by the transmission latency, the computing latency, and the queuing latency; the link load balancing degree is determined by the bandwidth utilization rate of each link.

3. A computing power network resource scheduling method according to claim 2, characterized in that, The neural network model includes a first neural network and a second neural network; After sending the first computing power service request, the first link path, and the first resource allocation strategy to the target computing node, further include: Generate experience data according to the network status information and store the experience data in an experience replay buffer, where the experience data represents the influence of resource scheduling decision actions; Randomly extract a preset number of the experience data from the experience replay buffer as training data; According to the training data, update the first weight parameter of the first neural network, and according to the first weight parameter, update the second weight parameter of the second neural network based on a preset transfer coefficient until a preset training termination condition is satisfied; Among them, the first neural network is used to update weight parameters; the second neural network is used to output the target computing node, the first link path, and the first resource allocation policy according to the computing power resource demand information and network status information.

4. A computing power network resource scheduling method according to claim 1, characterized in that, It further includes: In response to a second computing power service request, obtain a second link path and a second resource allocation policy; Determine a target client according to the second computing power service request; Establish a session with the target client according to the second link path, construct a computing power service instance corresponding to the target client, and configure the computing power resources of the computing power service instance according to the second resource allocation policy; Announce the status information of the computing power service instance to the computing nodes in the same computing power service domain; Determine that the computing power service instance has been executed, release the computing power resources of the computing power service instance, and announce the node status of itself to the computing nodes in the same computing power service domain; Among them, the second computing power service request, the second link path, and the second resource allocation policy are generated by the computing nodes in the same computing power service domain.

5. A computing power network resource scheduling method, characterized in that, Applied to an access node, it includes: In response to a computing power service request from a client, determine the type of computing power resource demand of the computing power service request; According to the type of computing power resource demand, determine a target computing power service domain, and send the computing power service request to the target computing power service domain, so that the agent in the target computing power service domain executes a computing power network resource scheduling method according to any one of claims 1 to 4.

6. A computing power network resource scheduling method according to claim 5, characterized in that, The determining the target computing power service domain according to the type of computing power resource demand and sending the computing power service request to the target computing power service domain includes: According to the type of computing power resource demand, retrieve the computing power service domain that provides the corresponding computing power resources; Determine that there is a computing power service domain that can provide various corresponding computing power resource types at the same time, and execute the steps: Take the computing power service domain as the target computing power service domain; Send the computing power service request to the target computing power service domain; Determine that there is no computing power service domain that can provide various corresponding computing power resource types at the same time, and execute the steps: Split the computing power service request based on the required computing power resource types to form computing power service sub-requests; For each computing power service sub-request, take the computing power service domain that provides the corresponding computing power resource type as the target computing power service domain; Send each computing power service sub-request to the corresponding target computing power service domain.

7. A computing power network resource scheduling system, characterized in that It includes: An access node, used to obtain a computing power service request from a client, and the access node is communicatively connected to a routing node; A computing node, used to provide computing power services, the computing node is communicatively connected to the routing node, and the computing node includes a cloud computing node, an edge computing node, and an end node; Among them, the access node executes a computing power network resource scheduling method according to claim 5 or 6; the cloud computing node and the edge computing node act as agents and execute a computing power network resource scheduling method according to any one of claims 1 to 4.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements a computing power network resource scheduling method according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements a computing power network resource scheduling method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements a computing power network resource scheduling method according to any one of claims 1 to 6.

Images