Edge collaborative power distribution network task unloading game optimization method and device

Abstract

The invention relates to an edge collaborative power distribution network task unloading game optimization method and device, and the method comprises the steps: determining a remote edge server utility function and a local edge server utility function on the basis of considering the task priority; constructing a Stackelberg game model by taking maximization of a remote edge server utility function and a local edge server utility function as a target; an MADDPG algorithm is adopted to optimize the Stackelberg game model, and an MADDPG model of each agent is obtained; after the local edge server generates a calculation task, system state information of the local edge server and the remote edge server is obtained, calculation is carried out based on the MADDPG model of each agent, an optimal task unloading game strategy is obtained, and task unloading is carried out based on the optimal task unloading game strategy. According to the method, edge-to-edge cooperative task unloading of the power distribution network calculation task can be realized, and the task calculation efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of distribution network edge computing, and particularly to an edge collaborative distribution network task offloading game optimization method and device. Background Art

[0002] In recent years, limited by factors such as the current computing, communication capabilities, and data processing efficiency of the distribution network, the distribution network still faces severe challenges in aspects such as task real-time response and processing efficiency. Edge collaborative computing can significantly reduce task computing latency and enhance the real-time response ability of the distribution network by optimizing the computing resource allocation of the distribution network. In addition, data-driven strategies do not rely on precise mathematical models, can flexibly cope with the complex changes of the distribution network, and dynamically adjust the allocation of computing resources to optimize the overall operation of the system. Therefore, the data-driven distribution network edge collaborative computing strategy is one of the effective technical means to solve the problem of insufficient computing power of the distribution network and provide low-latency and high-efficiency computing services.

[0003] Currently, in the methods of edge collaborative computing in the power Internet of Things, if the edge-cloud collaborative task computing method is adopted, although the cloud server has powerful computing capabilities and edge-cloud collaboration can give play to the advantages of the respective system resources, there are also disadvantages such as long latency, high requirements for communication bandwidth and data security, and high construction and usage costs. If the edge-edge collaborative task computing method is adopted, there are still few current research methods, especially in aspects such as its collaborative mechanism, which need to be further explored. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide an edge collaborative distribution network task offloading game optimization method and device, which can realize edge-edge collaborative task offloading of distribution network computing tasks and improve task computing efficiency.

[0005] The technical solution adopted by the present invention to solve its technical problems is: to provide an edge collaborative distribution network task offloading game optimization method, including the following steps:

[0006] On the basis of considering task priorities, determine the utility function of the remote edge server based on the revenue obtained from providing computing services by the remote edge server and the data storage cost, and determine the utility function of the local edge server based on the revenue obtained from task computing by the local edge server, the cost of paying for edge-edge collaborative computing services, and the data storage cost;

[0007] Construct a Stackelberg game model with the maximization of the utility function of the remote edge server and the utility function of the local edge server as the goal;

[0008] Optimize the Stackelberg game model using the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm to obtain the multi-agent deep deterministic policy models of each agent;

[0009] After the local edge server generates a computing task, obtain the system state information of the local edge server and the remote edge server, and perform calculations based on the multi-agent deep deterministic policy models of each agent to obtain the optimal task offloading game strategy.

[0010] The utility function of the remote edge server is expressed as: Where, is the utility function of the remote edge server, p is the unit price of the computing service provided by the remote edge server, is the priority of the task, expressed as: D i is the data volume of the task, G i is the number of CPU cycles required to complete the task, T i max is the maximum latency allowed for the task, Q n is the task volume of the edge server after considering the task priority, ω1, ω2, ω3, and ω4 are all weight coefficients, and λ is the offloading ratio of the local edge server, is the cache cost occupied during the queuing of the task in the remote edge server.

[0011] The cache cost occupied during the queuing of the task in the remote edge server is expressed as: Where, is the unit price of the edge server providing cache for the task data, is the task volume of the remote edge server after considering the task priority, is the computing power of the remote edge server.

[0012] The utility function of the local edge server is expressed as: Where, is the utility function of the local edge server, γ is the conversion coefficient for converting the task computing latency into economic benefits, is the priority of the task, expressed as: D i is the data volume of the task, G i is the number of CPU cycles required to complete the task, T i max is the maximum latency allowed for the task, Q n is the task volume of the edge server after considering the task priority, ω1, ω2, ω3, and ω4 are all weight coefficients, t total,i is the execution time of the local edge server without edge collaboration, is the execution time of the task on the local edge server, is the execution time of the task on the remote edge server, is the cache cost occupied during the queuing of the task on the local edge server, p is the unit price of the computing service provided by the remote edge server, and λ is the offloading ratio of the local edge server.

[0013] The cache cost occupied during the queuing of the task on the local edge server is expressed as: where, is the unit price of the edge server providing cache for the task data, is the task volume of the local edge server after considering the task priority, is the computing power of the local edge server.

[0014] The execution time of the local edge server without edge-edge collaboration is expressed as: The execution time of the task on the local edge server is expressed as: The execution time of the task on the remote edge server is expressed as: where, is the queuing delay of the task on the local edge server, expressed as: is the queuing task volume of the local edge server after considering the task priority, is the computing power of the local edge server; is the computing delay of the task on the local edge server, expressed as: is the computing delay of the task on the remote edge server, expressed as: where, is the computing power of the remote edge server; is the queuing delay of the task on the remote edge server, expressed as: is the queuing task volume of the remote edge server after considering the task priority; is the transmission time for the task to be offloaded to the remote edge server, expressed as: R n is the transmission rate offloaded to the remote edge server, expressed as: R n =w·log2(1 + P n ·h n / (N0·w)), w is the signal transmission bandwidth, P n is the edge-edge signal transmission power, h n is the channel gain, and N0 is the Gaussian white noise spectral density.

[0015] The multi-agent deep deterministic policy gradient algorithm is used to optimize the Stackelberg game model, and the multi-agent deep deterministic policy models of each agent are obtained, specifically including:

[0016] In the distribution network, the edge server that generates computing tasks is used as the main agent. Each main agent is set with an actor network and a first critic network. Among them, the actor network is used to make task offloading decisions based on maximizing its own reward, and the first critic network is used to evaluate the Q value of the action selected by the actor network; each subordinate agent is set with a second critic network, and the second critic network is used to evaluate the reward of task collaborative computing.

[0017] Each agent calculates its own current state, action, reward, and next state respectively, and forms a sequence as a sample to be stored in the experience replay buffer.

[0018] In each round of training, using the samples in the experience replay buffer, the main agent continuously updates its own policy, and evaluates the value of its task offloading decision through the first critic network. The subordinate agent evaluates the benefit of task offloading through the second critic network. During the training process, the actor network, the first critic network, and the second critic network perform parameter updates. After the training is completed, the multi-agent deep deterministic policy models of each agent are obtained.

[0019] The technical solution adopted by the present invention to solve its technical problems is: to provide an edge collaborative distribution network task offloading game optimization device, including:

[0020] A determination module, which is used to determine the utility function of the remote edge server based on the revenue obtained from providing computing services by the remote edge server and the data storage cost on the basis of considering task priorities, and determine the utility function of the local edge server based on the revenue obtained from task computing by the local edge server, the cost of paying for edge-edge collaborative computing services, and the data storage cost.

[0021] A construction module, which is used to construct a Stackelberg game model with the maximization of the utility function of the remote edge server and the utility function of the local edge server as the goal.

[0022] An optimization training module, which is used to optimize the Stackelberg game model by using the multi-agent deep deterministic policy gradient algorithm to obtain the multi-agent deep deterministic policy models of each agent.

[0023] A task offloading module, which is used to obtain the system state information of the local edge server and the remote edge server after a computing task is generated by the local edge server, and perform calculations based on the multi-agent deep deterministic policy models of each agent to obtain the optimal task offloading game strategy.

[0024] The utility function of the remote edge server determined by the determination module is as follows: where is the utility function of the remote edge server, p is the unit price of the computing service provided by the remote edge server, is the priority of the task, expressed as: D i is the data volume of the task, G i is the number of CPU cycles required to complete the task, T i max is the maximum latency allowed for the task, Q n is the task volume of the edge server after considering the task priority, ω1, ω2, ω3, and ω4 are all weight coefficients, λ is the offloading ratio of the local edge server, is the cache cost occupied during the queuing of the task in the remote edge server.

[0025] The cache cost occupied during the queuing of the task in the remote edge server is expressed as: where is the unit price of the edge server providing cache for the task data, is the task volume of the remote edge server after considering the task priority, is the computing power of the remote edge server.

[0026] The utility function of the local edge server determined by the determination module is as follows: where is the utility function of the local edge server, γ is the conversion coefficient for converting the task computing latency into economic benefits, is the priority of the task, expressed as: D i is the data volume of the task, G i is the number of CPU cycles required to complete the task, T i max is the maximum latency allowed for the task, Q n is the task volume of the edge server after considering the task priority, ω1, ω2, ω3, and ω4 are all weight coefficients, t total,i is the execution time of the local edge server without edge collaboration, is the execution time of the task on the local edge server, is the execution time of the task on the remote edge server, is the cache cost occupied during the queuing of the task in the local edge server, p is the unit price of the computing service provided by the remote edge server, and λ is the offloading ratio of the local edge server.

[0027] The cache cost occupied during the queuing of the task in the local edge server is expressed as: where is the unit price for the edge server to provide caching for task data, is the task volume of the local edge server after considering task priority, is the computing power of the local edge server.

[0028] The execution time of the local edge server without edge-edge collaboration is expressed as: The execution time of the task on the local edge server is expressed as: The execution time of the task on the remote edge server is expressed as: Among them, is the queuing delay of the task on the local edge server, expressed as: is the queuing task volume of the local edge server after considering task priority, is the computing power of the local edge server; is the computing delay of the task on the local edge server, expressed as: is the computing delay of the task on the remote edge server, expressed as: Among them, is the computing power of the remote edge server; is the queuing delay of the task on the remote edge server, expressed as: is the queuing task volume of the remote edge server after considering task priority; is the transmission time for the task to be offloaded to the remote edge server, expressed as: R n is the transmission rate for offloading to the remote edge server, expressed as: R n = w·log2(1 + P n ·h n / (N0·w)), where w is the signal transmission bandwidth, P n is the edge-edge signal transmission power, h n is the channel gain, and N0 is the Gaussian white noise spectral density.

[0029] The optimization training module includes:

[0030] An initialization unit, which is used to use the edge server that generates the computing task as the main agent, and set an actor network and a first critic network for each main agent, and set a second critic network for each slave agent. Among them, the actor network is used to make task offloading decisions according to maximizing its own reward, the first critic network is used to evaluate the Q value of the action selected by the actor network, and the second critic network is used to evaluate the reward of task collaborative computing;

[0031] The acquisition unit is used for data acquisition. During acquisition, it controls each agent to calculate its respective current state, action, reward, and next state, and forms a sequence as a sample to be stored in the experience replay buffer.

[0032] The training unit is used for training each agent. In each round of training, using the samples in the experience replay buffer, the main agent continuously updates its policy, and evaluates the value of its task offloading decision through the first critic network. The slave agents evaluate the benefits of task offloading through the second critic network. During the training process, the actor network, the first critic network, and the second critic network perform parameter updates. After the training is completed, a multi-agent deep deterministic policy model of each agent is obtained.

[0033] The technical solution adopted by the present invention to solve its technical problems is: to provide an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned edge collaborative distribution network task offloading game optimization method are implemented.

[0034] The technical solution adopted by the present invention to solve its technical problems is: to provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned edge collaborative distribution network task offloading game optimization method are implemented.

[0035] Beneficial effects

[0036] Due to the adoption of the above technical solution, compared with the prior art, the present invention has the following advantages and positive effects: The present invention models the edge-edge collaborative task offloading of the distribution network as a Stackelberg game model considering task priorities, and proposes a master-slave game model optimization method based on the multi-agent deep deterministic policy gradient algorithm. This method enables each edge server to share its computing resources to a greater extent through the establishment of an appropriate incentive mechanism, improves the computing efficiency of local services while increasing the benefits of other edge servers, realizes win-win for multiple parties, solves the problem of distribution network computing task offloading, and improves the processing efficiency of distribution network tasks. Brief description of the drawings

[0037] Figure 1 is a schematic diagram of the edge-cloud collaborative architecture of the distribution network;

[0038] Figure 2 is a flowchart of the edge collaborative distribution network task offloading game optimization method in the first embodiment of the present invention;

[0039] Figure 3 is a schematic diagram of task offloading in the first embodiment of the present invention;

[0040] Figure 4 It is a schematic diagram of the main agent reward in the MADDPG training process in an embodiment of the present invention;

[0041] Figure 5 It is a schematic diagram of the optimal target server and its offloading ratio in an embodiment of the present invention;

[0042] Figure 6 It is a comparison chart of task calculation results under different strategies in an embodiment of the present invention;

[0043] Figure 7 It is the edge-edge collaboration result under different local edge servers in an embodiment of the present invention. Specific Embodiments

[0044] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms also fall within the scope defined by the appended claims of this application.

[0045] The edge-cloud collaboration architecture of the distribution network is as Figure 1 shown. Taking the improved IEEE 33-node as an example, multiple edge servers are configured in the distribution network to serve the task calculations within their respective regions. The edge servers (ESs) are interconnected through a communication network.

[0046] In the distribution network, multiple edge servers (ESs) are generally deployed. When a computing task is generated, the local ES can either perform local computing or offload the task to other ESs for computing. Therefore, it is crucial to reasonably utilize the computing resources in the network to reduce its own computing pressure and improve the task execution quality. The Stackelberg game model formulates the optimal task offloading strategy based on the information and interests of both sides of the game, thereby realizing the distributed decision-making of the system. This enables the strategy to adapt to the characteristics and changes of different nodes, improving the flexibility of task offloading. Secondly, its Nash equilibrium solution is the balance point that maximizes the overall interests of both sides of the game, ensuring the balance of the interests of both sides of the game.

[0047] The key issues in the task offloading of the distribution network are to select a suitable target server and determine the offloading ratio of tasks to achieve efficient edge-edge collaborative computing. To this end, the first embodiment of the present invention relates to an edge collaborative distribution network task offloading game optimization method. This method first establishes an edge server utility function considering task priorities. On this basis, the edge-edge collaborative task offloading is modeled as a Stackelberg game model. Among them, the local server where the task is generated acts as the leader, aiming to optimize the comprehensive performance of task execution delay and computing cost; while the offloading target server acts as the follower, aiming to maximize its own revenue by providing computing services. Finally, a Stackelberg game model optimization method based on MADDPG is proposed, and edge-edge collaborative task offloading is carried out based on the MADDPG model.

[0048] As Figure 2 shown, the edge collaborative distribution network task offloading game optimization method of this embodiment includes the following steps:

[0049] Step 1, on the basis of considering task priorities, determine the remote edge server utility function based on the revenue obtained from the computing services provided by the remote edge server and the data storage cost, and determine the local edge server utility function based on the revenue obtained from task computing by the local edge server, the cost of paying for edge-edge collaborative computing services, and the data storage cost.

[0050] Let the set of edge servers in the distribution network system be S = {S1, S2, S3, ··· S i , ···}. Each edge server in S is represented as where p n is the unit price (￥ / (cycle·s)) of the edge server S n providing task offloading services, f n is the computing power (cycle / s) of the edge server S n , Q n is the queued task volume (cycle) of the edge server S n . is the unit price (￥ / (bit·s)) of the edge server S n providing cache for task data. Let the set of tasks generated in the distribution transformer area be J = {J1, J2, J3, ··· J i , ···}. Each task in J is represented as J i = {D i , G i , T i max}, where D i is the data volume size (bit) of the task J i , G iThe number of CPU cycles (cycles) required to complete the task, T i max The maximum latency (s) allowed for the task.

[0051] Considering the priorities of tasks helps optimize the allocation of computing resources during task offloading, improve task processing efficiency, and enhance the overall response ability of the system. By synthesizing the characteristics of different tasks J i such as the data volume size D i 、computing volume G i 、maximum allowed latency T i max , and the current edge server status, establish its priority as follows:

[0052]

[0053] where ω1, ω2, ω3, and ω4 are all weight coefficients.

[0054] Equation (1) enables the task priority model to reflect task differences while having certain adjustability and stability by introducing the smoothing term of the exponential function.

[0055] In the edge-edge collaborative computing mode, for task J i the computing latencies at the local and remote ESs are respectively:

[0056]

[0057] where λ is the offloading ratio of task J i to the remote ES. and are the computing capabilities of the local and remote ESs respectively.

[0058] For task J i the queuing latencies waiting for computing at the local ES and the remote ES are and respectively:

[0059]

[0060] where and are the queuing task volumes of the local and remote ESs considering task priorities respectively.

[0061] According to Shannon's theorem, the transmission rate (bit / s) of task J i offloaded to another edge server S n is:

[0062] R n =w·log2(1 + P n·h n / (N0·w)) (6)

[0063] where w is the signal transmission bandwidth, P n is the edge-to-edge signal transmission power, h n is the channel gain, and N0 is the Gaussian white noise spectral density.

[0064] Therefore, the transmission time for task J i offloaded to other ESs is:

[0065]

[0066] Since the amount of result data returned after task execution is very small, its transmission time is generally ignored.

[0067] In summary, the execution times of task J i on the local and remote ESs and are respectively:

[0068]

[0069] Under edge-to-edge collaboration, the final execution time of task J i is:

[0070]

[0071] Since the task data to be processed needs to occupy precious server cache resources, therefore, the cost of data occupying the cache needs to be considered under edge-to-edge collaborative computing. The cache costs i occupied by task J during queuing on the local and remote ESs and are respectively:

[0072]

[0073] Under edge-to-edge collaborative computing, the goal of the local ES to execute task offloading is to partially offload task J i to other edge servers to reduce the task execution time. At the same time, this ES needs to pay the corresponding edge-to-edge collaborative computing service cost. The revenue of the local ES is the difference between the revenue obtained from the task calculation delay and the payment for the edge-to-edge collaborative computing service cost and data storage cost, that is:

[0074]

[0075] where γ is the conversion coefficient for converting the task calculation delay into economic revenue. p is the unit price of the computing service provided by the remote ES (￥ / cycle). t total,i is the execution time of the local ES without edge-to-edge collaboration, that is:

[0076]

[0077] Under edge-edge collaborative computing, the goal of the remote ES to execute task offloading is to provide computing services and obtain benefits through reasonable pricing. The benefits of the remote ES include the difference between the benefits obtained from providing computing services and the data storage cost, that is:

[0078]

[0079] In this embodiment, the benefits of the local ES are used as the utility function of the local edge server, and the benefits of the remote ES are used as the utility function of the remote edge server

[0080] Step 2: Construct a Stackelberg game model with the maximization of the remote edge server utility function and the local edge server utility function as the goal.

[0081] In the task offloading of the distribution network, both the local edge server and the remote edge server hope to maximize their own interests according to equations (13) and (15). There is a cooperation and competition relationship between the two, resulting in a situation of mutual game.

[0082] Therefore, the task offloading of the edge-edge collaborative computing in the distribution network system is modeled as a Stackelberg game model Among them, M is the leader, that is, the local ES side; N is the follower, that is, the remote ES side; (λ, x) is the strategy set of the local edge server, that is, the offloading ratio and the target server of the local ES; are the utility functions of the leader and the follower respectively.

[0083] In the Stackelberg game model, the goal of the leader (local ES) is to reduce the task execution time by offloading tasks to the remote ES to obtain the optimal benefit, that is:

[0084]

[0085] s.t. 0 ≤ λ ≤ 1 (16)

[0086] 0 ≤ x ≤ X max (17)

[0087]

[0088] Among them, D l is the current cache of the local edge server, is the maximum cache of the local edge server.

[0089] The follower (remote ES) aims to maximize its own revenue by providing computing services, that is:

[0090]

[0091] s.t. 0 ≤ λ ≤ 1 (20)

[0092]

[0093]

[0094] where D r,x is the current cache of the remote edge server x, and is the maximum cache of the remote server.

[0095] Equation (16) is the constraint condition for the offloading ratio; Equation (17) indicates that the offloading target server does not exceed the maximum number of remote servers; Equations (18) and (21) indicate that the local and remote execution times do not exceed the maximum allowed time of the task; Equations (19) and (22) indicate that the sum of the task sizes and the current caches of the local and remote servers does not exceed their cache capacity limits.

[0096] When the two players in the game reach Nash equilibrium, each player reaches its maximum value, and unilaterally changing the strategy of a player will not increase its revenue, that is:

[0097] There is at least one Nash equilibrium in the non - cooperative game, that is:

[0098] (1) The strategy space X(λ) is a non - empty, bounded, closed, and convex set;

[0099] (2) The utility function is continuous and quasi - concave in X.

[0100] Obviously, for all remote edge servers, the strategy space X of task offloading is a non - empty, bounded, closed, and convex set in Euclidean space. It is known that a bounded closed set in Euclidean space is compact, and the utility function is continuous in its strategy space. Next, it is proved that the utility function satisfies the characteristics of a concave function.

[0101] For the utility function the first - order and second - order partial derivatives with respect to λ are obtained as follows:

[0102]

[0103] From Equation (24), it can be seen that Therefore, the utility function is strictly concave, and the established game model has a Nash equilibrium solution.

[0104] Step 3: Optimize the Stackelberg game model using the MADDPG algorithm to obtain the multi-agent deep deterministic policy model for each agent.

[0105] The MADDPG algorithm is a multi-agent deep reinforcement learning algorithm, especially suitable for dealing with the cooperative control problems among multiple agents. Therefore, the established Stackelberg game model can be optimized through the MADDPG framework. Among them, the master agent makes decisions by maximizing its own reward. The slave agents indirectly affect the policy of the master agent by evaluating the task execution effect, thus promoting the game to reach equilibrium.

[0106] In a multi-agent environment, the state, action, and reward directly affect the training and control performance of MADDPG. Due to the differences between the master and slave agents in aspects such as task offloading objectives, it is necessary to design their state spaces, action spaces, and reward functions respectively to ensure reasonable cooperation and optimization during the task offloading process.

[0107] (1) State spaces of the master and slave agents. The state space of the master agent includes the current task state and the states of the master and slave agents; the state space of the slave agent includes the state of the slave agent. That is, the state spaces of the master and slave agents are designed as follows:

[0108]

[0109] Among them, is the state space of the master agent, is the state space of the slave agent, A ma is the current task state, and A sl,n is the state of each slave agent.

[0110] (2) Action space. The action space of the master agent includes its decision variables: the target offloading server and the offloading ratio. The slave agents only execute the tasks of the master agent without making decisions, that is:

[0111]

[0112] Among them, is the action space of the master agent. The target offloading server x is an integer variable, and λ is a continuous variable with a value range of [0, 1].

[0113] (3) Reward. The reward functions of the master and slave agents respectively correspond to their respective objective functions (as shown in Equations (13) and (15)). Since the goals of the master and slave agents are both to maximize their own benefits, the reward functions are designed as follows:

[0114]

[0115] Among them, r ma is the reward function of the main agent, and r sl is the reward function of the subordinate agent.

[0116] The MADDPG algorithm realizes multi-agent collaborative control by introducing a centralized training framework and a distributed execution strategy. The edge-edge collaborative task offloading method based on MADDPG is as follows:

[0117] (1) Initialization phase. In the distribution network, the edge servers that generate computing tasks are used as the main agents. Each main agent sets an actor network and a critic network. Among them, the actor network is responsible for making task offloading decisions according to maximizing its own reward, while the critic network is used to evaluate the Q value of the action selected by the actor network, so as to guide the actor network to optimize the decision. Since in the game structure designed in this embodiment, the subordinate agents do not directly participate in the decision-making process, the subordinate agents only need to set a critic network to evaluate the reward of task collaborative computing, guide them to accept the computing tasks assigned by the main agent, and update the corresponding rewards according to the evaluation results.

[0118] Since the decision variables of the main agent are mixed discrete and continuous actions, in this embodiment, a softmax layer is introduced into the actor network to output the probability distribution of discrete actions to handle the mixed action space, thus avoiding adding an additional complex network architecture. Further, in order to achieve edge collaborative computing of distribution network computing tasks, each task is regarded as an episode in this embodiment. Whenever a new task is generated, the relevant system states need to be initialized.

[0119] (2) Sampling phase. Each agent calculates its current state s, action a, reward r, and the next state s' respectively, and forms a sequence (s, a, r, s') as a sample and stores it in the experience replay buffer. When the number of samples in the experience replay buffer is greater than the threshold, the network starts to learn.

[0120] In order to improve the adaptability of the training model to random task changes, noise is superimposed on the output of the actor network to increase randomness, prompting the model to learn a more robust strategy:

[0121]

[0122] Among them, λ (n) , Δλ (n) are the loads and their noises of each agent at time t, and ε and σ are the expectation and standard deviation respectively.

[0123] (3) Training phase. In each round of training, the main agent continuously updates its own policy and evaluates the value of its task offloading decision through the critic network. The slave agents evaluate the benefits of task offloading through the critic network. The update of the critic network of each agent is as follows:

[0124]

[0125] The update of the actor network of the main agent is as follows:

[0126]

[0127] Among them, a′ represents the target action at the next moment, μ′() represents the target policy network, s′ represents the state at the next moment, θ μ′ represents the parameters of the target actor network, y represents the target state-action value function value (Q value), γ represents the discount factor, done represents whether the current state is a terminal state, Q′() represents the target Q value network, μ() represents the policy network, θ μ represents the actor network parameters, L c represents the loss function of the critic network, q represents the Q value output by the current critic network, q new represents the target Q value used to update the critic network, Q() represents the current critic network, a new represents the newly calculated action, θ μ represents the parameters of the critic network, θ Q′ represents the parameters of the target critic network.

[0128] Furthermore, the target networks of each agent are slowly adjusted using a soft update method, and the soft update strategy is as follows:

[0129]

[0130] In the formula, τ represents the soft update coefficient of the target network. represents the parameters of the target critic network of agent i, represents the parameters of the critic network of agent i. represents the parameters of the target actor network of agent i, represents the parameters of the actor network of agent i.

[0131] Since the reward functions of the master and slave agents are both aimed at maximizing their own interests, and in the process of network update, each agent is committed to maximizing its own reward, thus forming a master-slave game relationship. After several training rounds, the strategies of the master agent and the slave agent interact with each other and gradually tend to the cooperation equilibrium point, and finally the optimal task offloading scheme can be achieved.

[0132] Step 4: After the local edge server generates a computing task, obtain the system state information of the local edge server and the remote edge server, and perform calculations based on the MADDPG models of each agent to obtain the optimal task offloading game strategy. Finally, perform task offloading based on this optimal task offloading game strategy.

[0133] Based on the established edge collaborative task offloading game optimization strategy based on deep reinforcement learning, the edge-edge collaborative task offloading process of the distribution network is as Figure 3 shown. In the model training stage, based on the master-slave game model of task offloading, use the MADDPG algorithm to train the network models of different edge servers as the master and slave agents respectively to obtain the MADDPG models of each agent. In the task offloading execution stage, when the master agent (local edge server) generates a computing task, first obtain the system state information of the local and remote servers, and then calculate based on the MADDPG models of each agent to obtain the optimal task offloading game strategy, complete the task offloading and obtain the corresponding benefits of the master and slave edge servers.

[0134] To verify the effectiveness of the method in this embodiment, an improved IEEE 33-node distribution network system is used as the test system for verification. It is assumed that the computing tasks generated in the distribution network system are completed by 3 edge servers ES0 to ES2, as Figure 1 shown. The simulation is implemented through Pycharm / Matlab, and the parameters of the edge servers are shown in Table 1.

[0135] Table 1 Edge server parameters

[0136]

[0137]

[0138] The computing tasks of the distribution network can generally be divided into three categories according to the real-time requirements. One category is tasks with high real-time requirements, such as distribution network topology analysis, power grid dynamic stability calculation, fault monitoring, etc. The data and computing volume of this type of task are relatively small. The second category is conventional state monitoring tasks, such as electrical data collection of the substation area, data processing, state monitoring, etc. Although the computing volume of a single point of this type of task is small, the scale is large. The data and computing volume of this type of task are medium, and the real-time requirements are also average. The third category of tasks is some dispatching services, such as intraday dispatching, load forecasting, new energy grid connection dispatching, etc. This type of task has the lowest real-time requirements, but its computing load is the largest. The types and parameter settings of the distribution network computing tasks are shown in Table 2.

[0139] Table 2 Computing Task Parameters

[0140]

[0141] First, task offloading is performed with a single edge server (ES0), that is, it is assumed that all computing tasks are generated by it. 100 distribution network computing tasks are randomly generated according to Table 2, and the task types include 40 high-real-time tasks, 40 medium-real-time tasks, and 20 low-real-time tasks. The main agent reward in the MADDPG training process is as Figure 4 shown, and the task calculation results are as Figure 5 shown.

[0142] Furthermore, in order to compare the performance of task offloading under different methods, two other common task offloading strategies are used for comparison respectively:

[0143] (1) Random offloading strategy (random task offloading, RTO): Randomly select the target ES and randomly set the offloading ratio to perform task offloading;

[0144] (2) Greedy offloading strategy (greedy task offloading, GTO): Set the offloading ratio through a decreasing sequence, and select the target ES with the lowest task execution time as the goal.

[0145] (3) Do not perform task offloading, and rely entirely on local execution of tasks (without task offloading, WTO).

[0146] (4) The method proposed in this embodiment (game-based DRL, G-DRL).

[0147] The optimal task offloading strategies under different strategies are as Figure 6 and Table 3 shows.

[0148] Table 3 Comparison of offloading results under different strategies

[0149]

[0150]

[0151] From Figures 4 to 6 and Table 3, it can be seen that for more choices of task execution, server ES1 is selected because the computing power of the remote server ES1 is higher, which can better improve the task execution performance. The value of the main agent utility function under the proposed method (G-DRL) is the highest, indicating that its decision-making maximally improves the comprehensive performance of the local server. Compared with the commonly used random offloading algorithm (RTO), the utility value of the local ES under the proposed method has increased by 66.7%, and the average execution time of tasks has decreased by 14.4%. Compared with the method of only local computing (WTO), the performance improvement is more significant. The greedy algorithm (GTO) aims to minimize the task execution time, but the economic cost it pays is the highest, resulting in a decrease in the overall utility function value. In this embodiment, the computing tasks of the distribution network are modeled as a Stackelberg game model, and the optimal results are obtained by taking the optimal comprehensive performance of task execution as the goals of the master and slave servers, thus taking into account both the computing time and cost. In the optimization goal solving stage, this embodiment models the master and slave game parties as the master and slave agents respectively, and realizes the master and slave game through the MADDPG framework. The master and slave agents cooperate through two parameters: the target server and the task offloading ratio, so as to optimize the task offloading.

[0152] In the foregoing assumptions, all the computing tasks of the distribution network come from the task offloading requests of the same edge server ES0. Based on the above 100 tasks, the task generation server is randomly selected, and the edge-edge collaborative task calculation is realized through the proposed method. The calculation results are as Figure 7 and Table 4 show.

[0153] Table 4 Comparison of offloading results under different strategies

[0154] Server G-DRL RTO Local execution Average task fluctuation 0.0623 0.0698 0.0751 Execution time (s) 0.102 0.126 0.147

[0155] From Figure 7 and Table 4, it can be seen that with the cooperation of multiple edge servers, the computing tasks generated by the entire distribution network system can be executed more effectively. In terms of task computing time, compared with the commonly used random offloading algorithm, the average delay of the method in this embodiment has decreased by 19.1%, and at the same time, the average fluctuation of the computing load has also decreased by 10.7%. Since each edge server collaboratively executes computing tasks in a master-slave game manner, the average payment cost of task offloading has been greatly reduced. At the same time, due to the stronger computing power of edge server 2, the cumulative income of the tasks it executes is also the largest.

[0156] The second embodiment of the present invention relates to an edge collaborative distribution network task offloading game optimization device, including:

[0157] A determination module, configured to determine a remote edge server utility function based on the revenue obtained from providing computing services by the remote edge server and the data storage cost, considering the task priority, and determine a local edge server utility function based on the revenue obtained from task computing by the local edge server, the cost of paying for edge-to-edge collaborative computing services, and the data storage cost;

[0158] A construction module, configured to construct a Stackelberg game model with the maximization of the remote edge server utility function and the local edge server utility function as the objective;

[0159] An optimization training module, configured to optimize the Stackelberg game model using the multi-agent deep deterministic policy gradient algorithm to obtain a multi-agent deep deterministic policy model for each agent;

[0160] A task offloading module, configured to, after a computing task is generated by the local edge server, obtain the system status information of the local edge server and the remote edge server, and perform calculations based on the multi-agent deep deterministic policy model of each agent to obtain an optimal task offloading game strategy.

[0161] The remote edge server utility function determined by the determination module is: Where is the remote edge server utility function, p is the unit price of providing computing services by the remote edge server, is the task priority, expressed as: D i is the data volume of the task, G i is the number of CPU cycles required to complete the task, T i max is the maximum latency allowed for the task, Q n is the task volume of the edge server after considering the task priority, ω1, ω2, ω3, and ω4 are all weight coefficients, λ is the offloading ratio of the local edge server, is the cache cost occupied during the queuing of the task on the remote edge server.

[0162] The cache cost occupied during the queuing of the task on the remote edge server is expressed as: Where is the unit price of the edge server providing cache for task data, is the task volume of the remote edge server after considering the task priority, is the computing power of the remote edge server.

[0163] The local edge server utility function determined by the determination module is: Where is the utility function of the local edge server, and γ is the conversion coefficient that converts the task computing delay into economic benefits. is the priority of the task, expressed as: D i is the data volume of the task, G i is the number of CPU cycles required to complete the task, T i max is the maximum latency allowed for the task, Q n is the task volume of the edge server after considering the task priority. ω1, ω2, ω3, and ω4 are all weight coefficients, t total,i is the execution time of the local edge server without edge collaboration. is the execution time of the task on the local edge server. is the execution time of the task on the remote edge server. is the cache cost occupied during the queuing of the task on the local edge server. p is the unit price of the computing service provided by the remote edge server, and λ is the offloading ratio of the local edge server.

[0164] The cache cost occupied during the queuing of the task on the local edge server is expressed as: Among them, is the unit price of the edge server providing cache for the task data. is the task volume of the local edge server after considering the task priority. is the computing power of the local edge server.

[0165] The execution time of the local edge server without edge collaboration is expressed as: The execution time of the task on the local edge server is expressed as: The execution time of the task on the remote edge server is expressed as: Among them, is the queuing delay of the task on the local edge server, expressed as: is the queuing task volume of the local edge server after considering the task priority. is the computing power of the local edge server; is the computing delay of the task on the local edge server, expressed as: is the computing delay of the task on the remote edge server, expressed as: Among them, is the computing power of the remote edge server; is the queuing delay of the task on the remote edge server, expressed as: is the queuing task volume of the remote edge server after considering the task priority; The transmission time for task offloading to a remote edge server, denoted as: R n The transmission rate for offloading to a remote edge server, denoted as R n = w·log2(1 + P n ·h n / (N0·w)), where w is the signal transmission bandwidth, P n is the edge-to-edge signal transmission power, h n is the channel gain, and N0 is the Gaussian white noise spectral density.

[0166] The optimization training module includes:

[0167] An initialization unit, which uses the edge server that generates computing tasks as the main agent, sets an actor network and a first critic network for each main agent, and sets a second critic network for each slave agent. Among them, the actor network is used to make task offloading decisions based on maximizing its own reward, the first critic network is used to evaluate the Q value of the action selected by the actor network, and the second critic network is used to evaluate the reward of task collaborative computing;

[0168] An acquisition unit, which is used for data acquisition. When acquiring data, it controls each agent to calculate its own current state, action, reward, and next state respectively, and forms a sequence as a sample to be stored in the experience replay buffer;

[0169] A training unit, which is used to train each agent. In each round of training, using the samples in the experience replay buffer, the main agent continuously updates its own policy, and evaluates the value of its task offloading decision through the first critic network. The slave agent evaluates the benefit of task offloading through the second critic network. During the training process, the actor network, the first critic network, and the second critic network perform parameter updates. After the training is completed, a multi-agent deep deterministic policy model of each agent is obtained.

[0170] It is not difficult to find that the present invention models the edge-to-edge collaborative task offloading of the distribution network as a Stackelberg game model on the basis of considering task priorities, and proposes an optimization method for the master-slave game model based on the multi-agent deep deterministic policy gradient algorithm. This method enables each edge server to share its computing resources to a greater extent by establishing an appropriate incentive mechanism, improves the computing efficiency of local services while increasing the benefits of other edge servers, achieves win-win results for multiple parties, solves the problem of distribution network computing task offloading, and improves the processing efficiency of distribution network tasks.

[0171] The third embodiment of the present invention relates to an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the edge collaborative distribution network task offloading game optimization method of the first embodiment are implemented.

[0172] The fourth embodiment of the present invention relates to a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the edge collaborative distribution network task offloading game optimization method of the first embodiment are implemented.

[0173] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories and optical memories, etc.) containing computer-usable program codes.

[0174] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0175] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured product including an instruction method, and the instruction method implements the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0176] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide for implementing the functions in Figure 1 one flow or multiple flows and / or blocks Figure 1Steps of functions specified in one or more boxes.

[0177] As described above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An edge collaborative distribution network task offloading game optimization method, characterized in that Including: Based on considering task priorities, determining the utility function of the remote edge server based on the revenue obtained from providing computing services by the remote edge server and the data storage cost, and determining the utility function of the local edge server based on the revenue obtained from task computing by the local edge server, the cost of paying for edge-edge collaborative computing services, and the data storage cost; Constructing a Stackelberg game model with the maximization of the utility function of the remote edge server and the utility function of the local edge server as the goal; Using the multi-agent deep deterministic policy gradient algorithm to optimize the Stackelberg game model to obtain the multi-agent deep deterministic policy model of each agent; After the local edge server generates a computing task, obtaining the system state information of the local edge server and the remote edge server, and performing calculations based on the multi-agent deep deterministic policy model of each agent to obtain the optimal task offloading game strategy.

2. The edge collaborative distribution network task offloading game optimization method according to claim 1, wherein, The utility function of the remote edge server is expressed as: where is the utility function of the remote edge server, p is the unit price of the computing service provided by the remote edge server, is the priority of the task, expressed as: D i is the data volume of the task, G i is the number of CPU cycles required to complete the task, T i max is the maximum latency allowed for the task, Q n is the edge server task volume after considering task priorities. ω1, ω2, ω3, and ω4 are all weight coefficients, and λ is the offloading ratio of the local edge server is the cache cost occupied during the queuing of the task in the remote edge server 3. The edge collaborative distribution network task offloading game optimization method according to claim 2, wherein The cache cost occupied during the queuing of the task at the remote edge server is expressed as: Where is the unit price for the edge server to provide cache for task data, is the task volume of the remote edge server after considering the task priority, is the computing power of the remote edge server.

4. The edge collaborative distribution network task offloading game optimization method according to claim 1, characterized in that The utility function of the local edge server is expressed as: Among them, is the utility function of the local edge server, γ is the conversion coefficient for converting the task computing delay into economic benefits, is the priority of the task, expressed as: D i is the data volume of the task, G i is the number of CPU cycles required to complete the task, T i max is the maximum allowable delay of the task, Q n is the edge server task volume after considering the task priority, ω1, ω2, ω3, and ω4 are all weight coefficients, t total,i is the execution time of the local edge server without edge collaboration, is the execution time of the task on the local edge server, is the execution time of the task on the remote edge server, is the cache cost occupied during the queuing period of the task on the local edge server, p is the unit price of the computing service provided by the remote edge server, and λ is the offloading ratio of the local edge server.

5. The edge collaborative distribution network task offloading game optimization method according to claim 4, wherein The cache cost occupied during the queuing of the task at the local edge server is expressed as: Wherein, is the unit price for the edge server to provide cache for task data, is the task volume of the local edge server after considering task priorities, is the computing power of the local edge server.

6. The edge collaborative distribution network task offloading game optimization method according to claim 4, wherein The execution time of the local edge server without edge-edge collaboration is expressed as: The execution time of the task on the local edge server is expressed as: The execution time of the task on the remote edge server is expressed as: Where, is the queuing delay of the task on the local edge server, expressed as: is the queuing task volume of the local edge server after considering the task priority, is the computing power of the local edge server; is the computing delay of the task on the local edge server, expressed as: is the computing delay of the task on the remote edge server, expressed as: Among them, is the computing power of the remote edge server; is the queuing delay of the task on the remote edge server, expressed as: is the queuing task volume of the remote edge server after considering the task priority; is the transmission time for the task to be offloaded to the remote edge server, expressed as: R n The transmission rate unloaded to the remote edge server, expressed as: R n = w · log2(1 + P n · h n / (N0 · w)), where w is the signal transmission bandwidth, P n is the edge-to-edge signal transmission power, h n is the channel gain, and N0 is the Gaussian white noise spectral density.

7. The edge collaborative distribution network task offloading game optimization method according to claim 1, wherein The step of using the multi-agent deep deterministic policy gradient algorithm to optimize the Stackelberg game model to obtain the multi-agent deep deterministic policy model of each agent specifically includes: In the distribution network, the edge server that generates the computing task is used as the main agent. Each main agent is provided with an actor network and a first critic network. Among them, the actor network is used to make task offloading decisions based on maximizing its own reward, and the first critic network is used to evaluate the Q value of the action selected by the actor network; each subordinate agent is provided with a second critic network, and the second critic network is used to evaluate the reward of task collaborative computing; Each agent calculates its own current state, action, reward, and next state respectively, and forms a sequence as a sample to be stored in the experience replay buffer; In each round of training, using the samples in the experience replay buffer, the main agent continuously updates its own strategy, and evaluates the value of its task offloading decision through the first critic network. The subordinate agent evaluates the revenue of task offloading through the second critic network. During the training process, the actor network, the first critic network, and the second critic network perform parameter updates. After the training is completed, the multi-agent deep deterministic policy model of each agent is obtained.

8. An edge collaborative distribution network task offloading game optimization device, characterized in that Including: A determination module, configured to determine the utility function of the remote edge server based on the revenue obtained from providing computing services by the remote edge server and the data storage cost on the basis of considering task priorities, and determine the utility function of the local edge server based on the revenue obtained from task computing by the local edge server, the cost of paying for edge-edge collaborative computing services, and the data storage cost; A construction module, configured to construct a Stackelberg game model with the maximization of the utility function of the remote edge server and the utility function of the local edge server as the goal; An optimization training module, configured to use the multi-agent deep deterministic policy gradient algorithm to optimize the Stackelberg game model to obtain the multi-agent deep deterministic policy model of each agent; A task offloading module, which is used to obtain the system status information of the local edge server and the remote edge server after generating a computing task on the local edge server, and perform calculations based on the multi-agent deep deterministic policy model of each agent to obtain the optimal task offloading game strategy.

9. The edge collaborative distribution network task offloading game optimization device according to claim 8, characterized in that The utility function of the remote edge server determined by the determination module is as follows: where is the utility function of the remote edge server, p is the unit price of the computing service provided by the remote edge server, is the priority of the task, expressed as: D i is the data volume of the task, G i is the number of CPU cycles required to complete the task, T i max is the maximum latency allowed for the task, Q n is the task volume of the edge server after considering the task priority. ω1, ω2, ω3, and ω4 are all weight coefficients, and λ is the offloading ratio of the local edge server, is the cache cost occupied during the queuing of the task in the remote edge server.

10. The edge collaborative distribution network task offloading game optimization device according to claim 9, characterized in that, The cache cost occupied during the queuing of the task in the remote edge server is expressed as: Where, is the unit price for the edge server to provide cache for task data, is the task volume of the remote edge server after considering the task priority, is the computing power of the remote edge server.

11. The edge collaborative distribution network task offloading game optimization device according to claim 8, wherein The utility function of the local edge server determined by the determining module is: Among them, is the utility function of the local edge server, γ is the conversion coefficient for converting the task computing delay into economic benefits, is the priority of the task, expressed as: D i is the data volume of the task, G i is the number of CPU cycles required to complete the task, T i max is the maximum delay allowed for the task, Q n is the edge server task volume after considering the task priority, ω1, ω2, ω3, and ω4 are all weight coefficients, t total,i is the execution time of the local edge server without edge collaboration, is the execution time of the task on the local edge server, is the execution time of the task on the remote edge server, is the cache cost occupied during the queuing of the task on the local edge server, p is the unit price of the computing service provided by the remote edge server, and λ is the offloading ratio of the local edge server.

12. The edge collaborative distribution network task offloading game optimization device according to claim 11, wherein The cache cost occupied during the queuing of the task at the local edge server is expressed as: Where is the unit price for the edge server to provide cache for task data, is the task volume of the local edge server after considering the task priority, is the computing power of the local edge server.

13. The edge collaborative distribution network task offloading game optimization device according to claim 11, characterized in that, The execution time of the local edge server under edge-edge collaboration is expressed as: The execution time of the task on the local edge server is expressed as: The execution time of the task on the remote edge server is expressed as: Among them, is the queuing delay of the task on the local edge server, expressed as: is the queuing task volume of the local edge server after considering the task priority, is the computing power of the local edge server; is the computing delay of the task on the local edge server, expressed as: is the computing delay of the task on the remote edge server, expressed as: Among them, is the computing power of the remote edge server; is the queuing delay of the task on the remote edge server, expressed as: is the queued task volume of the remote edge server after considering the task priority; is the transmission time for the task to be offloaded to the remote edge server, expressed as: R n The transmission rate unloaded to the remote edge server, expressed as: R n = w·log2(1 + P n ·h n / (N0·w)), where w is the signal transmission bandwidth, P n is the edge-to-edge signal transmission power, h n is the channel gain, and N0 is the Gaussian white noise spectral density.

14. The edge collaborative distribution network task offloading game optimization device according to claim 8, wherein The optimization training module includes: An initialization unit, which is used to use the edge server that generates the computing task as the main agent, and set an actor network and a first critic network for each main agent, and set a second critic network for each slave agent. Among them, the actor network is used to make task offloading decisions according to maximizing its own reward, the first critic network is used to evaluate the Q value of the action selected by the actor network, and the second critic network is used to evaluate the reward of task collaborative computing; A collection unit, which is used to perform data collection. When collecting data, it controls each agent to calculate its own current state, action, reward and next state respectively, and forms a sequence as a sample and stores it in the experience replay buffer; A training unit, which is used to train each agent. In each round of training, using the samples in the experience replay buffer, the main agent continuously updates its own strategy, and evaluates the value of its task offloading decision through the first critic network. The slave agent evaluates the benefits of task offloading through the second critic network. During the training process, the actor network, the first critic network and the second critic network perform parameter updates. After the training is completed, the multi-agent deep deterministic policy model of each agent is obtained.

15. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the edge collaborative distribution network task offloading game optimization method described in any one of claims 1-7.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the edge collaborative distribution network task offloading game optimization method described in any one of claims 1-7.

Images