Distributed resource collaborative scheduling method, device, equipment and medium for integrated computing

Abstract

The present invention provides a distributed resource collaborative scheduling method, device, equipment and medium for general computing fusion, including: obtaining a distributed computing offloading communication model and a distributed computing offloading calculation model corresponding to a network scenario of general computing fusion; determining a distributed computing power load balancing model based on the distributed computing offloading communication model and the distributed computing offloading calculation model; modeling the computing power collaborative scheduling process as a Markov decision process according to the distributed computing power load balancing model; solving the Markov decision process using a preset distributed computing power scheduling algorithm to obtain an offloading service strategy; wherein the preset distributed computing power scheduling algorithm is a distributed computing power scheduling algorithm based on MADDPG. The present invention can maintain the node load balance in the network while ensuring the user experience, and has certain effectiveness and scalability.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things, and in particular to a distributed resource collaborative scheduling method, device, equipment and medium with integrated computing. Background Art

[0002] At present, in order to cope with the explosive traffic demand generated by emerging business scenarios such as the Internet of Things and mobile Internet, it has become an important trend to transfer various communication, computing, and storage resources to the cloud. However, the rapid increase in the types of computing tasks generated by mobile smart devices poses a challenge to the efficient offloading of computing requests and the rational allocation of network resources. A single network architecture can no longer meet the diverse business needs. Considering the iteration cost and service coverage of network facilities, a single network structure also faces a series of problems such as limited scope of application and low utilization of network resources, making it difficult to support the computing needs generated by the continuous updating of application scenarios.

[0003] With the introduction of high-density heterogeneous networks, the way mobile smart devices process computing tasks is no longer limited to nearby computing nodes. Instead, they comprehensively consider the load status of each node in the system, the task transmission cost and other factors, and select the best node to process the computing requests generated by the user, so as to better balance the load of each node in the network. Therefore, there is an urgent need for an efficient service offloading strategy so that the collaborative work between computing nodes can not only meet the needs of the user, but also maintain network load balance and fully utilize network resources. Summary of the invention

[0004] Embodiments of the present invention provide a distributed resource collaborative scheduling method, apparatus, device and medium for general computing fusion to overcome the defects of the prior art.

[0005] In order to achieve the above object, the present invention adopts the following technical scheme.

[0006] In a first aspect, the present invention provides a distributed resource collaborative scheduling method for general computing fusion, comprising: obtaining a distributed computing offloading communication model and a distributed computing offloading calculation model corresponding to a general computing fusion network scenario;

[0007] Determine a distributed computing power load balancing model based on the distributed computing offloading communication model and the distributed computing offloading calculation model;

[0008] Modeling the computing power collaborative scheduling process as a Markov decision process according to the distributed computing power load balancing model;

[0009] The Markov decision process is solved using a preset distributed computing power scheduling algorithm to obtain an unloading service strategy; wherein the preset distributed computing power scheduling algorithm is a distributed computing power scheduling algorithm based on MADDPG.

[0010] Optionally, the obtaining of a distributed computing offloading communication model and a distributed computing offloading calculation model corresponding to a network scenario integrated with general computing includes:

[0011] According to the load corresponding to the current computing power node where the user end is located in the network scenario of general computing fusion, the data transmission rate corresponding to each computing task in the user end is determined, and the determined data transmission rate is used as the distributed computing offloading communication model;

[0012] Determine whether the computing task is processed locally based on the computing power of the user end;

[0013] When it is determined that the computing task is processed locally, a first cost is calculated according to the latency and energy consumption of the computing task;

[0014] If it is determined that the computing task is not processed locally, determine whether the computing task is processed on the current computing node based on the computing capacity corresponding to the current computing node where the user terminal is located;

[0015] When it is determined that the computing task is processed in the current computing power node, the second cost is calculated according to the latency and energy consumption of the computing task and the latency and energy consumption corresponding to the data transmission to the current computing power node;

[0016] When it is determined that the computing task is not processed by the current computing power node, the third cost is calculated according to the latency and energy consumption of the computing task, the latency and energy consumption corresponding to the data transmission to the current computing power node, and the latency and energy consumption corresponding to the data transmission to the remote computing power node;

[0017] A distributed computing offloading computing model is determined according to the first cost, the second cost, and the third cost.

[0018] Optionally, determining a distributed computing power load balancing model based on the distributed computing offloading communication model and the distributed computing offloading calculation model includes:

[0019] The load coefficient and load balancing coefficient of the current computing node are calculated based on the computing power of the current computing node where the user terminal is located and the computing power corresponding to all computing nodes in the integrated network scenario;

[0020] The distributed computing power load balancing model is determined according to the distributed computing offloading communication model, the distributed computing offloading calculation model, the load coefficient of the current computing power node and the load balancing coefficient.

[0021] Optionally, modeling the computing power collaborative scheduling process as a Markov decision process according to the distributed computing power load balancing model includes:

[0022] Set the system state space for M node computing controllers;

[0023] Set a behavior space for offloading decisions corresponding to N client computing tasks;

[0024] A reward function is set according to the system state space and the behavior space, and the reward function is used to represent the minimization of the comprehensive overhead of delay and energy consumption of each computing task under the premise of system load balancing.

[0025] Optionally, solving the Markov decision process using a preset distributed computing power scheduling algorithm to obtain an offloading service strategy includes:

[0026] S1, initialize the parameters involved in the preset distributed computing power scheduling algorithm to obtain the initialization parameters, which include initializing the parameters of the M node computing power controllers, initializing the actor network, initializing the critic network, and initializing the random variable N for action exploration. t And receive the state information of each agent as the initial state s=(s1,s2...s M );

[0027] S2, based on the initialization parameters, the agent conducts action exploration to obtain real-time rewards and new environment states;

[0028] S3, using the experience replay mechanism to store the old environment state, multi-agent joint actions, real-time rewards, and new environment state into the replay memory bank, and extract samples from the replay memory bank;

[0029] S4, update the parameters in the initialized actor network and the initialized critic network according to the extracted samples;

[0030] S2-S4 are repeatedly executed until the preset conditions are met, thereby obtaining the trained actor network and critic network, and obtaining the offloading service strategy based on the trained actor network and critic network.

[0031] Optionally, the network scenario of the general computing fusion includes a user terminal, a base station, a node computing power controller and a computing power sensor;

[0032] The node computing power controller is respectively communicatively connected with the base station, the user terminal and the computing power sensor; each node computing power controller corresponds to multiple user terminals, and the node computing power controller and the multiple user terminals corresponding to the node computing power controller constitute a computing power node.

[0033] In a second aspect, the present invention further provides a distributed resource collaborative scheduling device with general computing integration, comprising:

[0034] A communication model and computing model acquisition module, used to acquire a distributed computing offloading communication model and a distributed computing offloading computing model corresponding to a network scenario integrated with general computing;

[0035] A load balancing model determination module, used to determine a distributed computing power load balancing model based on the distributed computing offloading communication model and the distributed computing offloading calculation model;

[0036] A process conversion module, used for modeling the computing power collaborative scheduling process as a Markov decision process according to the distributed computing power load balancing model;

[0037] A solution module is used to solve the Markov decision process using a preset distributed computing power scheduling algorithm to obtain an unloading service strategy; wherein the preset distributed computing power scheduling algorithm is a distributed computing power scheduling algorithm based on MADDPG.

[0038] In a third aspect, the present invention also provides an electronic device, including a memory and a processor, the processor and the memory communicate with each other, the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the above-mentioned distributed resource collaborative scheduling method with general computing integration.

[0039] In a fourth aspect, the present invention further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned distributed resource collaborative scheduling method with general computing fusion.

[0040] Beneficial effects of the present invention: The distributed resource collaborative scheduling method, device, equipment and medium for general computing integration provided by the present invention comprehensively considers factors such as the load status of each node in the system and the task transmission cost, and selects the best node to process the computing request generated by the user end, thereby better balancing the load of each node in the network. The collaborative work between computing nodes can not only meet the needs of the user end, but also maintain network load balance and realize the full utilization of network resources. In addition, a reasonable unloading service strategy is formulated for the computing service requests generated by multiple user ends through a distributed computing power scheduling algorithm based on MADDPG, which maintains the node load balance in the network while ensuring the user experience, and has certain effectiveness and scalability.

[0041] Additional aspects and advantages of the present invention will be given in part in the following description, which will become obvious from the following description, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0043] Figure 1 A schematic diagram of a flow chart of a distributed resource collaborative scheduling method with general computing integration provided by an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of a network scenario of general computing fusion provided by an embodiment of the present invention;

[0045] Figure 3 A schematic diagram of a flow chart of a preset distributed computing power scheduling algorithm provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0047] Those skilled in the art will appreciate that, unless expressly stated otherwise, the singular forms "a", "an", "" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other elements, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or couplings. The term "and / or" as used herein includes any unit and all combinations of one or more of the associated listed items.

[0048] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.

[0049] Example 1

[0050] Figure 1 A flow chart of a distributed resource collaborative scheduling method with general computing integration provided by an embodiment of the present invention; Figure 1 As shown, a distributed resource collaborative scheduling method with general computing integration includes the following steps:

[0051] S101, obtaining a distributed computing offloading communication model and a distributed computing offloading calculation model corresponding to a network scenario integrated with general computing.

[0052] S102, determining a distributed computing power load balancing model based on the distributed computing offloading communication model and the distributed computing offloading calculation model.

[0053] S103: Modeling the computing power collaborative scheduling process as a Markov decision process according to the distributed computing power load balancing model.

[0054] S104, using a preset distributed computing power scheduling algorithm to solve the Markov decision process to obtain an offloading service strategy.

[0055] Among them, the preset distributed computing power scheduling algorithm is a distributed computing power scheduling algorithm based on MADDPG.

[0056] Specifically, DDPG is an algorithm designed for continuous action space problems. MADDPG, as an algorithm designed for multi-agent tasks and continuous action space problems, can be seen as an extension of DDPG in the multi-agent case and can be applied to collaborative learning among multiple agents.

[0057] According to the distributed resource collaborative scheduling method with general computing integration provided by the embodiment of the present invention, factors such as the load status of each node in the system and the task transmission cost are comprehensively considered, and the best node is selected to process the computing request generated by the user terminal, so as to better balance the load of each node in the network. The collaborative work between computing nodes can not only meet the needs of the user terminal, but also maintain the balance of network load and realize the full utilization of network resources. In addition, a reasonable unloading service strategy is formulated for the computing service requests generated by multiple user terminals through the distributed computing power scheduling algorithm based on MADDPG, which maintains the node load balance in the network while ensuring the user experience, and has certain effectiveness and scalability.

[0058] Furthermore, if Figure 2 As shown, the network scenario of the comprehensive computing fusion includes a user terminal, a base station, a node computing power controller and a computing power sensor.

[0059] The node computing power controller is respectively communicatively connected with the base station, the user terminal and the computing power sensor; each node computing power controller corresponds to multiple user terminals, and the node computing power controller and the multiple user terminals corresponding to the node computing power controller constitute a computing power node.

[0060] Specifically, the network scenario of integrated computing has M computing nodes, and each computing node has N mobile smart devices (i.e., user terminals).

[0061] The user end can be a device with certain computing capabilities, such as a smart phone, a computer, or a wearable device. The mobile smart devices within a certain geographical range are regarded as a user end set User = {u1,u2...u n ...u N Multiple user terminals communicate wirelessly through a macro base station, and the base station can be divided into multiple sub-channels for use by the user terminals.

[0062] The node computing power controller is connected to the base station through optical fiber, and at the same time, it is also connected to several hardware infrastructures with computing and storage capabilities through optical fiber, and can process the computing tasks sent by the user end. In addition to receiving the computing task information to be executed sent by the terminal device, including the amount of data uploaded by the task, the computing resources required to execute the task, the amount of data returned, and the terminal device's own status, the node computing power controller also receives the load status information of each computing power node sent by the computing power sensor to make corresponding decisions, and decide where to execute the computing power service request generated by the user end, whether the user end directly processes it or unloads it to the node for execution, whether it is executed directly at the current node or needs to be transmitted to other computing power nodes for execution. The controller returns the decision result to the terminal device, and the terminal device transmits the computing task to the corresponding location according to the unloading decision.

[0063] The computing power sensor is the core module of the entire network. Although it does not directly participate in the processing of tasks, it monitors the load of each computing power node in the network and sends it to the node computing power controller that needs to make an unloading decision, thereby achieving load balancing of the entire network and improving the resource utilization of the entire network.

[0064] In the network scenario of general computing fusion, each node controller corresponds to N user terminals and can make reasonable offloading decisions for the computing tasks sent by these user terminals. There are M node computing controllers in the system, denoted as Control = {p1, p2...p m ...p M The node computing power controller and several computing power infrastructures together constitute the computing power node, denoted as V = {V1, V2...V m ...V M}, the computing power of the computing power node can be expressed as C = {C1, C2...C M Typically, computing tasks generated by mobile smart devices (i.e., user terminals) can be divided according to different execution modes, i.e., local processing or offloaded computing. All computing tasks cannot be split and only one execution mode can be selected.

[0065] Furthermore, the acquisition of the distributed computing offloading communication model and the distributed computing offloading calculation model corresponding to the network scenario integrated with the general computing includes:

[0066] According to the load situation corresponding to the current computing power node where the user end is located in the network scenario of general computing fusion, the data transmission rate corresponding to each computing task in the user end is determined, and the determined data transmission rate is used as the distributed computing offloading communication model.

[0067] Determine whether the computing task is processed locally based on the computing power of the user end.

[0068] When it is determined that the computing task is processed locally, the first cost is calculated based on the latency and energy consumption of the computing task.

[0069] When it is determined that the computing task is not processed locally, it is determined whether the computing task is processed in the current computing node based on the computing power corresponding to the current computing node where the user terminal is located.

[0070] When it is determined that the computing task is processed in the current computing node, the second cost is calculated based on the latency and energy consumption of the computing task, and the latency and energy consumption corresponding to the transmission of data to the current computing node.

[0071] When it is determined that the computing task is not processed on the current computing node, the third cost is calculated based on the latency and energy consumption of the computing task, the latency and energy consumption corresponding to the data transmission to the current computing node, and the latency and energy consumption corresponding to the data transmission to the remote computing node.

[0072] A distributed computing offloading computing model is determined according to the first cost, the second cost, and the third cost.

[0073] Specifically, the computationally intensive task is regarded as a set consisting of several computational subtasks, denoted as A = {A1, A2, A3...A n}. Use a tuple to compute task A i Describe, that is:

[0074] A i ={d i ,c i}

[0075] Among them, d iIndicates the length of input data, including relevant information of computing tasks, system parameters, etc. When the request is offloaded to the computing power node for calculation, d i It needs to be uploaded to the corresponding computing device through a communication link for calculation. i Indicates execution of subtask A i The required computing resources, that is, the number of CPU clock cycles. At the same time, we introduce τ Max Describe the computing task A, τ Max Indicates the maximum allowed delay.

[0076] When there is a computing task to be processed, it is necessary to consider not only the status of the local device, including network fluctuations, the remaining power percentage of the device, the CPU load rate, and the computing service A i Information, etc., must also be considered, in order to make a reasonable computing offloading decision. A computing task can only choose one processing method. Therefore, a vector set is used to represent all the offloading decisions that the nth user end may take:

[0077] ζ n (t) = {ζ n,0 (t),ζ n,1 (t),ζ n,2 (t)...ζ n,M (t)}

[0078] Among them, n,m (t)∈{0,1},ζ n,m (t) = 0 means u n The service request generated is not executed on the mth computing node, n,m (t) = 1 means u n The service request generated is offloaded to the mth computing node for execution. In particular, when ζ n,0 (t) = 1 means that the service request is executed on the user side without uninstallation. n For example, the constraints of the offloading vector can be expressed as:

[0079]

[0080] User side n Connected to the node computing power controller via a wireless communication channel, there are two unloading situations for computing power service requests that need to be migrated to the node for execution. The first situation is that the computing power infrastructure directly connected to the controller has sufficient computing resources, and it can be executed directly at the current computing power node. The other situation is that the current computing power node is overloaded and the computing task needs to be transferred to other nodes for execution.

[0081] First, let’s discuss the first case. In this case, multiple local terminal devices share one base station, so the impact of interval interference is not considered. Then, at this time, the user end u n The computing task A is sent via uplink i The input data is transmitted to the current computing power node V m The rate R n It can be expressed as:

[0082]

[0083] Among them, B nm Indicates the channel transmission bandwidth allocated to the nth user end uploading data to the mth computing power node. nm Represents the channel coefficient of the communication link allocated to the nth user terminal. Indicates the transmit power of the local terminal device. N0 indicates the noise power in the uplink communication link.

[0084] For the second case, when the service request needs to be transmitted to other computing nodes for processing, it is necessary to consider the data transmission between the two computing nodes. i By computing power node V m Transmit to computing power node V k When the data transmission rate is r mk It can be expressed as:

[0085]

[0086] Among them, B mk Represents the computing power node V m To computing power node V k The transmission channel bandwidth. mk Represents the channel coefficient between two computing nodes. Represents the computing power node V m N0 represents the channel noise power.

[0087] After obtaining the transmission rates in the above two cases (i.e., the distributed computing offloading communication model), the computing model is determined. There are two cases of distributed computing offloading computing models, which are divided into local processing models and offloading computing models. The offloading computing model is further divided into the current node computing model and the remote node computing model. Finally, the computing optimization problem is modeled.

[0088] Among them, the local processing model is for the user end u n Generated service request A i , when represents the unloading decision variable ζ n,0When (t) = 1, the computing task will be executed at the local device without computing offload, and the delay and energy consumption will be generated locally. Assuming that the computing power of the local device is sufficient, the total delay and total energy consumption of the computing task executed locally only considers the time and energy consumed by processing the task, without considering the queuing of the task. i,local To process computing task A i The total delay is:

[0089]

[0090] Among them, C n,local Indicates the user end u n The local computing power is the number of CPU clock cycles.

[0091] Remember E i,local For the user end n Processing computing task A i The total energy consumption is:

[0092]

[0093] Among them, α and β are coefficients related to the specific CPU model.

[0094] Computational Task A i The total cost of the combined latency and energy required for local execution i,local :

[0095] Cost i,local =λ T T i,local +λ E E i,local

[0096] Where 0≤λ T ,λ E ≤1 respectively represent the weight coefficients of latency and energy consumption for executing computing tasks, which the user end can adjust according to its own needs.

[0097] Unloading calculation model, for user side u n The generated computing task A i , assuming that the user terminal communicates with the computing power node V through a wireless channel m There are two cases of unloading computing tasks: n,m When (t) = 1, it means that the user end u n The generated computing task A i Unload to computing node V m In this case, in addition to considering the delay and energy consumption of task processing, it is also necessary to consider the delay and energy consumption caused by data transmission to the node through the wireless channel. n,m(t) = 0, ζ n,k When (t) = 1, it means that the user end u n The generated computing task A i Not in the current computing node V m is executed at the node, and is transmitted to other nodes with better load status V k In addition to considering the latency and energy consumption caused by task processing and data transmission from the user end to the computing node, it is also necessary to consider the latency and energy consumption caused by data transmission between different nodes. Next, the above two offloading situations are discussed separately.

[0098] Current node calculation model: When uninstalling variable ζ n,m When (t) = 1, the user end u n Send service request A i Transmit to computing power node V m Ignoring the return delay of the calculation results, the calculation at the node adopts the first-come-first-served principle. At this time, the total delay includes the calculation task transmission delay, waiting delay and processing delay, which is recorded as

[0099]

[0100] in, Indicates the transmission delay when the user uploads data to the node computing power controller through the wireless channel allocated by the base station. Indicates a calculation request A i Waiting delay at the computing power node. Indicates the latency of a computing node in processing computing tasks. It can be expressed as:

[0101]

[0102] Assume that the computing task A i There are k subtasks waiting to be executed before, so the user-side computing task A i In the computing power node V m The waiting and calculation delays are expressed as:

[0103]

[0104]

[0105] The computational energy consumption in this case can be obtained:

[0106]

[0107] in, It indicates the energy consumption of uploading data from the user end to the node computing power controller through the wireless channel allocated by the base station. Indicates the waiting energy consumption of the user-side computing task at the computing power node. Indicates the energy consumption of computing nodes for processing computing tasks, where:

[0108]

[0109]

[0110] The total cost of this method is:

[0111]

[0112] Remote node computing model: When the variable ζ is offloaded n,m (t) = 0 and ζ n,k When (t) = 1, it means that the user end u n The generated computing task A i In the computing power node V k Execute at this time, calculate the data d i Need to get from the computing power node V m Transmitted to node V via link k Similarly, node V k The calculation at the location is based on the first-come-first-served principle, without considering the return transmission consumption of the calculation results. Then the total delay in this case includes transmission delay (transmission delay from the user end to the computing power node and transmission delay between nodes), queuing delay and processing delay, recorded as

[0113]

[0114] in, Indicates that the data is in the computing power node V m Transmit to node V k delay.

[0115] Computational Task A i By computing power node V m Transmit to node V k Delay:

[0116]

[0117] Similarly, the energy consumption under the calculation offloading condition can be obtained:

[0118]

[0119] Transmission energy consumption It can be expressed as:

[0120]

[0121] The calculation method of other delays and energy consumption is the same as that of the current node offloading model. The total cost under this method is:

[0122]

[0123] In summary, when the computing task needs to be executed at the node, the total cost of computing offloading can be shown as:

[0124]

[0125] stC1:1≤m,k≤K and m≠k

[0126] C2:ζ n,m (t),ζ n,k (t)∈{0,1}

[0127] Among them, constraint C1 is a constraint on the computing power node that executes the computing task. This constraint indicates two processing methods after the computing request is offloaded, either to choose to execute it on the current node or to choose to transfer it to other computing power nodes for execution. Constraint C2 is a constraint on the value of the offloading decision variable.

[0128] Computational optimization problem modeling: As can be seen from the above, the user end u n The computing power service request A generated i There are only two processing methods: directly executing on the user side or offloading to the node side for processing. The node computing power controller makes scientific and reasonable offloading decisions for computing tasks by analyzing the cost of different computing methods. n Complete calculation task A i The total cost can be expressed as:

[0129] Cost n,i =ζ n,0 (t)Cost i,local +(1-ζ n,0 (t))Cost i,off

[0130] Among them, when n,0 When (t) = 0, it means that the computing service request generated by the user end is directly executed at the local device. n,0 When (t) = 1, it means that the computing service request generated by the user end is offloaded to the computing node for execution. The user end set in the system is User = {u1,u2...u n ...u N}, the calculation task is A={A1,A2,A3...A n}. Then in the current system, the total cost consumed by all users to complete all computing tasks can be expressed as:

[0131]

[0132] During the whole process of computing offloading, the user side hopes to minimize the combined overhead of latency and energy consumption in processing computing tasks. For the entire system, it hopes to reasonably distribute the computing tasks generated by the user side to various computing nodes for processing, so that the entire system can achieve load balancing and minimize the total cost of the system. Therefore, the optimization problem can be modeled as:

[0133] minCost n,i

[0134] stC1:ζ n,0 (t)∈ζ n (t),ζ n (t)∈{0,1}

[0135]

[0136]

[0137]

[0138] The optimization goal is to minimize the weighted total cost of latency and energy consumption of computing tasks processed by the user. Among them, C1 is a constraint on the offloading decision. C2 is a constraint on the load of all computing nodes in the system. C3 defines the collection range of the user, computing nodes, and computing subtasks. C4 constrains the total latency of the system.

[0139] Furthermore, the distributed computing power load balancing model is determined based on the distributed computing offloading communication model and the distributed computing offloading calculation model, including:

[0140] The load coefficient and load balancing coefficient of the current computing power node are calculated based on the computing power of the current computing power node where the user end is located and the computing power corresponding to all computing power nodes in the integrated network scenario.

[0141] The distributed computing power load balancing model is determined according to the distributed computing offloading communication model, the distributed computing offloading calculation model, the load coefficient of the current computing power node and the load balancing coefficient.

[0142] Specifically, a distributed computing load balancing model is constructed to transform the computational offloading problem of multiple users under computing resource constraints into a constraint problem for processing, including:

[0143] 1. Design node load factor:

[0144] In a system, the computing infrastructure in each computing node has a certain storage and computing capacity, and can provide services to multiple clients at the same time. When computing tasks of multiple clients need to be processed on the same node, these tasks will be queued for computing on a first-come, first-served basis. Assume that at this time, node V m There are k user-side tasks waiting in queue for computation, and the current computing power node V m The computing power is denoted as C m . Set the current computing power node V m The ratio of the computational cycles required for all computational tasks at a node to the computational capacity of the node is defined as the node load factor, denoted as Load m ,Right now:

[0145]

[0146] In order to facilitate the evaluation of the overall load of the system, the average value of the load factor of all nodes in the system is expressed as:

[0147]

[0148] Among them, M is the number of computing power nodes in the entire system.

[0149] 2. Design the load balancing factor of the node:

[0150] In order to maximize the utilization of resources at each computing node, measures need to be taken to balance the load at each computing node in the system, so as to avoid all computing tasks being concentrated on a few nodes. In order to evaluate the degree of load balancing, the variance of the load coefficients of each computing node in the system is defined as the load balancing coefficient LB, that is:

[0151]

[0152] Furthermore, the computing power collaborative scheduling process is modeled as a Markov decision process according to the distributed computing power load balancing model, including:

[0153] Set the system state space for M node computing controllers;

[0154] Set a behavior space for offloading decisions corresponding to N client computing tasks;

[0155] A reward function is set according to the system state space and the behavior space, and the reward function is used to represent the minimization of the comprehensive overhead of delay and energy consumption of each computing task under the premise of system load balancing.

[0156] Specifically, 1. Set the system state space for the M node computing power controllers:

[0157] Assume that in the current state, there are n∈{1,2...N} clients generating computing tasks. n The generated computing task A i , the node computing power controller needs to comprehensively consider the local device status, computing service related information and the load of all computing power nodes in the system to make reasonable unloading decisions. Therefore, the system status corresponding to the current time t includes the status of the smart mobile device And the load of computing nodes , which can be expressed as:

[0158]

[0159] The state of smart mobile devices Including the current network fluctuation status of the local device, the remaining battery percentage of the device, the current CPU load rate of the device and related information of the computing task, namely:

[0160]

[0161] Among them, n t Indicates the network fluctuation of device t at the current time, q t Indicates the remaining power percentage of the device at the current time t, δ t Indicates the CPU load rate of the device at the current time t, q t , δ t Satisfy 0≤q t ,δ t ≤1, A i Indicates information related to computing tasks generated by the user.

[0162] The computing power nodes in the current system can be expressed as V = {V1, V2...V M}, that is, the number of available computing nodes in the system is M, and the number of computing nodes is V m The load factor is expressed as Load m , then when the user u n Generate computing task A i The load of all computing nodes in the system It can be expressed as:

[0163]

[0164] Since the number of node computing controllers in the entire system is M, the state space S of the entire system at the current time t is t It can be expressed as:

[0165] S t ={s 1,t ,s 2,t ...s M,t}

[0166] 2. Set the behavior space for offloading decisions of N client computing tasks:

[0167] At the current time t, the set of offloading decisions of the nth user-side computing task is defined as the behavior space A n . It can be seen that:

[0168] A n (t) = {A n,0 (t),A n,1 (t)...A n,M (t)}

[0169]

[0170] Since a single node controller is connected to N clients, at the current time t, the node computing power controller p m The behavior space can be expressed as:

[0171] A m,t ={A1(t),A2(t)...A N (t)}

[0172] The behavior space of the entire system can be expressed as:

[0173] A t ={A 1,t ,A 2,t ...A M,t}

[0174] 3. Set the reward function to minimize the latency and energy consumption of each user-side computing task under the premise of system load balancing:

[0175] At the current time t, the environment state s t The following actions are taken t Recorded as a set of state-action pairs (s t ,a t ), when the system takes an action in this state, it will get a real-time reward value for the action, denoted as R(s t ,a t). The pros and cons of the behavior taken by the system in the current state can be intuitively reflected by the size of the reward value. Considering that reinforcement learning takes the maximization of long-term cumulative reward value as the primary goal of solving problems, it is necessary to establish a relationship between the objective function of the problem to be solved and the reward function. The comprehensive overhead after the introduction of the weight value and the inverse of the load balancing coefficient are used as the reward function of the decision, and the load balancing coefficient is used as a constraint to achieve better decision-making results. If the behavior strategy selected by the agent in the current state causes the load of the entire system to be unbalanced, the penalty will increase. The reward function can be expressed as:

[0176] R(s t ,a t ) n =-ρ Cost Cost n,i -ρ LB LB

[0177] This formula comprehensively considers the total cost of the delay and energy consumption of the user-side computing offloading process and the load condition of the entire system. The specific value of the reward function depends on the current state of the system on the one hand, and on the strategy executed by the agent in this state on the other hand. Cost , LB They represent the comprehensive cost of completing the computing request and the weight value of the load balancing coefficient of the entire system. Since the goal is to minimize the comprehensive cost of latency and energy consumption under the premise of load balancing of each node in the system, the entire reward function is a negative number, and the minimum value of the objective function is converted into the maximum value of the entire system reward.

[0178] Furthermore, if Figure 3 As shown, the Markov decision process is solved by using a preset distributed computing power scheduling algorithm to obtain an unloading service strategy, including:

[0179] S1, initialize the parameters involved in the preset distributed computing power scheduling algorithm to obtain the initialization parameters, which include initializing the parameters of the M node computing power controllers, initializing the actor network, initializing the critic network, and initializing the random variable N for action exploration. t And receive the state information of each agent as the initial state s=(s1,s2...s M );

[0180] S2, based on the initialization parameters, the agent conducts action exploration to obtain real-time rewards and new environment states;

[0181] S3, using the experience replay mechanism to store the old environment state, multi-agent joint actions, real-time rewards, and new environment state into the replay memory bank, and extract samples from the replay memory bank;

[0182] S4, update the parameters in the initialized actor network and the initialized critic network according to the extracted samples;

[0183] S2-S4 are repeatedly executed until the preset conditions are met, thereby obtaining the trained actor network and critic network, and obtaining the offloading service strategy based on the trained actor network and critic network.

[0184] Specifically, S1, initialize the DCPAM algorithm parameters: initialize the parameters of M controllers, including the playback memory library M m Initialize the actor network, including the online policy network and target strategy network , initialize the critic network, including the online Q network and target Q network , initialize the random variable N used for action exploration t , receiving the state information of each agent as the initial state s=(s1,s2...s M ).

[0185] S2, the agent conducts action exploration: selects actions based on the current strategy and exploration mechanism , execute multi-agent joint action a=(a1,a2...a M ) to obtain real-time reward r = (r1, r2...r M ) and the new environment state s′=(s′1,s′2...s′ M ).

[0186] S3, experience replay mechanism: store (s, a, r, s′) in the replay memory bank and update the system state: s←s′, randomly extract a set of samples (s j ,a j ,r j ,s′ j ),set up

[0187] S4, neural network parameter update: gradient descent of the Loss function to update the critic network:

[0188]

[0189] Update the actor network using the policy gradient method:

[0190]

[0191] Update the target-strategy network structure parameters of the agent:

[0192]

[0193] Update the structural parameters of the agent's target-Q network:

[0194]

[0195] Finally, the above S2-S4 are repeatedly executed until the iteration upper limit reaches the set value or reaches the threshold, and finally the optimal resource allocation deployment can be obtained.

[0196] Example 2

[0197] On the basis of Embodiment 1, this Embodiment 2 provides a distributed resource collaborative scheduling device with general computing integration, which corresponds to the distributed resource collaborative scheduling with general computing integration described above, and includes:

[0198] A communication model and computing model acquisition module, used to acquire a distributed computing offloading communication model and a distributed computing offloading computing model corresponding to a network scenario integrated with general computing;

[0199] A load balancing model determination module, used to determine a distributed computing power load balancing model based on the distributed computing offloading communication model and the distributed computing offloading calculation model;

[0200] A process conversion module, used for modeling the computing power collaborative scheduling process as a Markov decision process according to the distributed computing power load balancing model;

[0201] A solution module is used to solve the Markov decision process using a preset distributed computing power scheduling algorithm to obtain an unloading service strategy; wherein the preset distributed computing power scheduling algorithm is a distributed computing power scheduling algorithm based on MADDPG.

[0202] For specific details, please refer to the description of the distributed resource collaborative scheduling method for general computing fusion, which will not be repeated here.

[0203] Example 3

[0204] Embodiment 3 of the present invention provides an electronic device, including a memory and a processor, the processor and the memory communicate with each other, the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute a distributed resource collaborative scheduling method for general computing fusion, the method including the following process steps:

[0205] Obtain a distributed computing offloading communication model and a distributed computing offloading calculation model corresponding to a network scenario integrated with general computing;

[0206] Determine a distributed computing power load balancing model based on the distributed computing offloading communication model and the distributed computing offloading calculation model;

[0207] Modeling the computing power collaborative scheduling process as a Markov decision process according to the distributed computing power load balancing model;

[0208] The Markov decision process is solved using a preset distributed computing power scheduling algorithm to obtain an unloading service strategy; wherein the preset distributed computing power scheduling algorithm is a distributed computing power scheduling algorithm based on MADDPG.

[0209] Example 4

[0210] Embodiment 4 of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, a distributed resource collaborative scheduling method for general computing fusion is implemented. The method includes the following process steps:

[0211] Obtain a distributed computing offloading communication model and a distributed computing offloading calculation model corresponding to a network scenario integrated with general computing;

[0212] Determine a distributed computing power load balancing model based on the distributed computing offloading communication model and the distributed computing offloading calculation model;

[0213] Modeling the computing power collaborative scheduling process as a Markov decision process according to the distributed computing power load balancing model;

[0214] The Markov decision process is solved using a preset distributed computing power scheduling algorithm to obtain an unloading service strategy; wherein the preset distributed computing power scheduling algorithm is a distributed computing power scheduling algorithm based on MADDPG.

[0215] In summary, the distributed resource collaborative scheduling method with integrated computing provided by the embodiment of the present invention comprehensively considers factors such as the load status of each node in the system and the task transmission cost, and selects the best node to process the computing request generated by the user terminal, so as to better balance the load of each node in the network. The collaborative work between computing nodes can not only meet the needs of the user terminal, but also maintain the balance of network load and realize the full utilization of network resources. In addition, a reasonable unloading service strategy is formulated for the computing service requests generated by multiple user terminals through the distributed computing power scheduling algorithm based on MADDPG, which maintains the node load balance in the network while ensuring the user experience, and has certain effectiveness and scalability.

[0216] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0217] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the method or device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The method and device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0218] The above are only preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A distributed resource collaborative scheduling method with general computing integration, characterized in that: include: Obtain a distributed computing offloading communication model and a distributed computing offloading calculation model corresponding to a network scenario integrated with general computing; Determine a distributed computing power load balancing model based on the distributed computing offloading communication model and the distributed computing offloading calculation model; Modeling the computing power collaborative scheduling process as a Markov decision process according to the distributed computing power load balancing model; The Markov decision process is solved by using a preset distributed computing power scheduling algorithm to obtain an unloading service strategy; wherein the preset distributed computing power scheduling algorithm is a distributed computing power scheduling algorithm based on MADDPG; The acquisition of the distributed computing offloading communication model and the distributed computing offloading calculation model corresponding to the network scenario integrated with the general computing includes: According to the load corresponding to the current computing power node where the user end is located in the network scenario of general computing fusion, the data transmission rate corresponding to each computing task in the user end is determined, and the determined data transmission rate is used as the distributed computing offloading communication model; Determine whether the computing task is processed locally based on the computing power of the user end; When it is determined that the computing task is processed locally, a first cost is calculated according to the latency and energy consumption of the computing task; When it is determined that the computing task is not processed locally, determine whether the computing task is processed in the current computing node according to the computing capacity corresponding to the current computing node where the user terminal is located; When it is determined that the computing task is processed in the current computing power node, the second cost is calculated according to the latency and energy consumption of the computing task and the latency and energy consumption corresponding to the data transmission to the current computing power node; When it is determined that the computing task is not processed by the current computing power node, the third cost is calculated according to the latency and energy consumption of the computing task, the latency and energy consumption corresponding to the data transmission to the current computing power node, and the latency and energy consumption corresponding to the data transmission to the remote computing power node; Determine a distributed computing offloading computing model according to the first cost, the second cost, and the third cost; The determining of the distributed computing power load balancing model based on the distributed computing offloading communication model and the distributed computing offloading calculation model includes: The load coefficient and load balancing coefficient of the current computing node are calculated based on the computing power of the current computing node where the user terminal is located and the computing power corresponding to all computing nodes in the integrated network scenario; Determine a distributed computing power load balancing model according to the distributed computing offloading communication model, the distributed computing offloading calculation model, the load coefficient of the current computing power node and the load balancing coefficient; The method of solving the Markov decision process by using a preset distributed computing power scheduling algorithm to obtain an unloading service strategy includes: S1, initialize the parameters involved in the preset distributed computing power scheduling algorithm to obtain the initialization parameters, which include initializing the parameters of the M node computing power controllers, initializing the actor network, initializing the critic network, and initializing the random variable N for action exploration. t And receive the state information of each agent as the initial state s=(s1,s2...s M ); S2, based on the initialization parameters, the agent conducts action exploration to obtain real-time rewards and new environment states; S3, using the experience replay mechanism to store the old environment state, multi-agent joint actions, real-time rewards, and new environment state into the replay memory bank, and extract samples from the replay memory bank; S4, update the parameters in the initialized actor network and the initialized critic network according to the extracted samples; S2-S4 are repeatedly executed until the preset conditions are met, thereby obtaining the trained actor network and critic network, and obtaining the offloading service strategy based on the trained actor network and critic network.

2. The distributed resource collaborative scheduling method with integrated computing according to claim 1 is characterized in that: The method of modeling the computing power collaborative scheduling process as a Markov decision process according to the distributed computing power load balancing model includes: Set the system state space for M node computing controllers; Set a behavior space for offloading decisions corresponding to N client computing tasks; A reward function is set according to the system state space and the behavior space, and the reward function is used to represent the minimization of the comprehensive overhead of delay and energy consumption of each computing task under the premise of system load balancing.

3. The distributed resource collaborative scheduling method with integrated computing according to claim 1 or 2, characterized in that: The network scenario of the general computing fusion includes a user terminal, a base station, a node computing power controller and a computing power sensor; The node computing power controller is respectively communicatively connected with the base station, the user terminal and the computing power sensor; each node computing power controller corresponds to multiple user terminals, and the node computing power controller and the multiple user terminals corresponding to the node computing power controller constitute a computing power node.

4. A distributed resource collaborative scheduling device with general computing integration, characterized in that: include: A communication model and computing model acquisition module, used to acquire a distributed computing offloading communication model and a distributed computing offloading computing model corresponding to a network scenario integrated with general computing; A load balancing model determination module, used to determine a distributed computing power load balancing model based on the distributed computing offloading communication model and the distributed computing offloading calculation model; A process conversion module, used for modeling the computing power collaborative scheduling process as a Markov decision process according to the distributed computing power load balancing model; A solution module, used to solve the Markov decision process using a preset distributed computing power scheduling algorithm to obtain an unloading service strategy; wherein the preset distributed computing power scheduling algorithm is a distributed computing power scheduling algorithm based on MADDPG; The communication model and calculation model acquisition module is specifically used for: According to the load corresponding to the current computing power node where the user end is located in the network scenario of general computing fusion, the data transmission rate corresponding to each computing task in the user end is determined, and the determined data transmission rate is used as the distributed computing offloading communication model; Determine whether the computing task is processed locally based on the computing power of the user end; When it is determined that the computing task is processed locally, a first cost is calculated according to the latency and energy consumption of the computing task; If it is determined that the computing task is not processed locally, determine whether the computing task is processed on the current computing node based on the computing capacity corresponding to the current computing node where the user terminal is located; When it is determined that the computing task is processed in the current computing power node, the second cost is calculated according to the latency and energy consumption of the computing task and the latency and energy consumption corresponding to the data transmission to the current computing power node; When it is determined that the computing task is not processed by the current computing power node, the third cost is calculated according to the latency and energy consumption of the computing task, the latency and energy consumption corresponding to the data transmission to the current computing power node, and the latency and energy consumption corresponding to the data transmission to the remote computing power node; Determine a distributed computing offloading computing model according to the first cost, the second cost, and the third cost; The load balancing model determination module is specifically used for: The load coefficient and load balancing coefficient of the current computing node are calculated based on the computing power of the current computing node where the user terminal is located and the computing power corresponding to all computing nodes in the integrated network scenario; Determine a distributed computing power load balancing model according to the distributed computing offloading communication model, the distributed computing offloading calculation model, the load coefficient of the current computing power node and the load balancing coefficient; The solution module is specifically used for: S1, initialize the parameters involved in the preset distributed computing power scheduling algorithm to obtain the initialization parameters, which include initializing the parameters of the M node computing power controllers, initializing the actor network, initializing the critic network, and initializing the random variable N for action exploration. t And receive the state information of each agent as the initial state s=(s1,s2...s M ); S2, based on the initialization parameters, the agent conducts action exploration to obtain real-time rewards and new environment states; S3, using the experience replay mechanism to store the old environment state, multi-agent joint actions, real-time rewards, and new environment state into the replay memory bank, and extract samples from the replay memory bank; S4, update the parameters in the initialized actor network and the initialized critic network according to the extracted samples; S2-S4 are repeatedly executed until the preset conditions are met, thereby obtaining the trained actor network and critic network, and obtaining the offloading service strategy based on the trained actor network and critic network.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it implements the distributed resource collaborative scheduling method with comprehensive computing integration as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: It stores a computer program, which, when executed by a processor, implements the distributed resource collaborative scheduling method with comprehensive computing integration as described in any one of claims 1 to 3.

Images