A blockchain-based mobile edge computing task offloading method

Abstract

This invention belongs to the field of mobile communication technology, specifically relating to a blockchain-based mobile edge computing task offloading method. The method includes establishing a blockchain-based mobile edge computing task offloading model for multi-server dynamic MEC scenarios; aiming to minimize the cost for users to complete computing tasks and maximize the utility gained from user participation in mining, establishing a joint optimization model for task offloading and resource allocation under multi-dimensional resource constraints; considering the stochastic time-varying network environment and the partial observability of the environment state, abstracting the task offloading cost problem and the blockchain mining utility problem into a partially observable Markov decision process (MDP); obtaining the state space and action space of the MDP problem based on the established system model, and constructing a reward function; and employing a multi-agent reinforcement learning algorithm to make optimal offloading and resource allocation decisions. This invention achieves lower task offloading costs and higher blockchain mining utility.

Description

Technical Field

[0001] This invention belongs to the field of mobile communication technology, specifically relating to a method for offloading mobile edge computing tasks based on blockchain. Background Technology

[0002] With the development of the internet, the number of mobile devices has increased dramatically, giving rise to many latency-sensitive and computationally intensive applications, such as virtual reality, interactive online games, facial recognition, and ultra-high-definition video. Due to the limited computing and storage resources of mobile devices, they cannot efficiently complete these latency-sensitive and computationally intensive applications, resulting in a poor user experience. To address these issues, mobile cloud computing was proposed, where users send computing tasks to the cloud, utilize the cloud's abundant computing resources to complete the tasks, and then transmit the results back to the user. However, because users are usually far from the cloud, latency is often significant, making mobile cloud computing unsuitable for latency-sensitive applications.

[0003] To address the high latency issue in mobile cloud computing, Mobile Edge Computing (MEC) emerged. First proposed by the European Telecommunications Standards Institute (ETS) in 2014, MEC offloads computing tasks to edge devices equipped with edge servers. This brings computing and storage resources closer to mobile devices, reducing latency and user energy consumption, improving task offloading efficiency, and enhancing user experience. Consequently, Mobile Edge Computing has garnered widespread attention from academia and industry.

[0004] With the surge in the number of IoT sensors and edge devices, the number of tasks that need to be processed within the coverage area of ​​a single MEC server is increasing. A single MEC server can hardly meet all the computing demands simultaneously, thus necessitating scenarios involving multiple MEC servers working together. In MEC systems, MEC servers typically come from different service providers, leading to potential conflicts of interest and making it difficult to establish trust among them. Furthermore, the heterogeneous nature of edge devices raises security and privacy concerns regarding interactions between heterogeneous edge nodes and service migrations across nodes.

[0005] With its advantages of decentralization, tamper-proofing, transparency, immutability, traceability, and anonymity, blockchain technology can build a secure and trustworthy transaction environment in distributed systems, solving the aforementioned security and privacy issues. In blockchain-assisted MEC systems, asymmetric encryption algorithms and hash algorithms are used in the interaction between users and edge servers to protect the privacy and security of the interaction process. Furthermore, blockchain can use a consensus mechanism to confirm the consistency of transaction records, ensuring their integrity and reliability. In MEC systems, a central controller needs to make offloading decisions; if this controller is attacked, the entire MEC system will be paralyzed, a phenomenon known as a single point of failure. Blockchain, with its distributed nature and consensus mechanism, can maintain normal system operation even if a few nodes are attacked. Therefore, combining blockchain with MEC can improve the security of MEC systems. Summary of the Invention

[0006] To achieve lower task offloading costs and higher blockchain mining utility, this invention proposes a blockchain-based mobile edge computing task offloading method, which specifically includes the following steps:

[0007] For dynamic MEC scenarios with multiple servers, considering that the computing, communication resources and channel states of MEC servers are time-varying, and that the interaction between heterogeneous edge nodes may cause security and privacy issues, a blockchain-based mobile edge computing task offloading model was established.

[0008] With the goal of minimizing the cost for users to complete computing tasks and maximizing the utility gained by users from participating in mining, a joint optimization model for task unloading and resource allocation is established under multi-dimensional resource constraints.

[0009] Considering the random and time-varying network environment and the partial observability of the environment state, the task unloading cost problem and the blockchain mining utility problem are abstracted into a partially observable Markov decision process;

[0010] Based on the established system model, the state space and action space of the MDP problem are obtained, and the reward function is constructed.

[0011] A multi-agent reinforcement learning algorithm is used to make optimal unloading and resource allocation decisions.

[0012] Furthermore, blockchain-based mobile edge computing task offloading models include:

[0013] When an edge device has a computing task offloading requirement, it sends an offloading request to the MEC server layer. After receiving multiple response messages, it queries the reliability table of MEC servers stored in the blockchain to find the reliability of the candidates. Based on the reliability, channel conditions, and available computing resources of the server, it selects an appropriate server for computing offloading. The reliability table is updated periodically based on the verified transactions of computing task offloading.

[0014] The reliability of each MEC server is stored in a blockchain ledger, and the reliability is retrieved from the blockchain when the edge device makes an offloading decision.

[0015] During the consensus process, the master node is responsible for generating blocks, the replica nodes are responsible for verifying blocks, and the edge devices that are not selected are ordinary nodes, which are only responsible for adding the verified blocks to the maintained blockchain ledger.

[0016] Furthermore, the blockchain consensus process includes:

[0017] Signing a block or transaction requires x CPU cycles, verifying a signature requires y CPU cycles, generating a MAC requires z CPU cycles, and verifying a MAC requires z CPU cycles.

[0018] The master node collects unverified transactions from all edge devices and then sorts them by timestamp, assuming a block size of S. b Given that the average size of a transaction is χ, the number of transactions in a block is: During this phase, the master node needs to verify the signatures and MACs of L transactions, so the computational cost of the master node is L(y+z).

[0019] The master node generates a signature and MAC for the block, which is N. s -1 replica nodes generate MACs for pre-prepare messages. Each replica node verifies the MAC of the block, as well as the signatures and MACs of L transactions in the block. The computational cost of the primary node is x+N. s Given that the computational cost of each replica node is z + L(y + z), and assuming that the block transmission time is proportional to the block size, the message transmission time is τ. b S b ;

[0020] After a replica node verifies the pre-prepare message, it sends prepare messages to other consensus nodes. After each consensus node receives 2f prepare messages, it enters the next phase. In this phase, the primary node needs to verify 2f MACs, thus the computational cost is 2fz; each replica node needs to generate N... s-1 MAC and verify 2f MACs, the computational cost of each replica node is (N s -1)z+2fz, message transmission time is τ b S b ;

[0021] After receiving 2f prepare messages, each consensus node sends a commit message to other nodes, including the master node. During this phase, the master and replica nodes need to verify 2f MACs and generate N. s -1 MAC, therefore the computational cost of the primary node and replica node is (N s -1)z+2fz, message transmission time is τ b S b ;

[0022] After receiving 2f commit messages, the master node and replica nodes consider the block valid, add it to the blockchain ledger, and send a reply message containing the verified block to other edge devices. After receiving f+1 reply messages, they update the global view. During this phase, the master node and replica nodes need to generate a MAC for the reply message; therefore, the computational cost for the master node and replica nodes is z, and the message transmission time is τ. b S b .

[0023] Furthermore, establishing a joint optimization model for task unloading and resource allocation includes:

[0024]

[0025] Constraints:

[0026]

[0027]

[0028]

[0029]

[0030]

[0031]

[0032] Where, x n (t) represents the local observation of edge device n in time slot t; p n (t) represents the transmission power of edge device n offloading tasks to MEC server m in time slot t; The computing resources allocated by MEC server m for executing the offloading task of edge device n under time slot t; The computing resources allocated to edge device n for executing tasks under time slot t; The computing resources allocated to edge devices for blockchain mining in time slot t; x represents the consensus utility of edge device n in time slot t; Cost(t) represents the cost of the edge device in time slot t; n (t) represents the offloading decision of edge device n in time slot t. If the computation task is offloaded to MEC server i for execution in time slot t, then x n (t) = i, then x is the task executed locally. n (t) = 0; Let P be the set of edge devices, where N is the number of edge devices; n This represents the maximum transmission power. The system operating time is discretized into a set of T time slots; F n F represents the maximum computing resources available for edge device n. m This represents the maximum computing resources of server m. A collection of MEC servers; φ n τ represents the maximum computing resources available for edge device n to use for blockchain mining. n T represents the maximum tolerable latency for the task. n (t) represents the processing delay of edge device n in time slot t.

[0033] Furthermore, the task offloading cost problem and the blockchain mining utility problem are abstracted into a partially observable Markov decision process, which includes: edge devices acting as agents, and defining a tuple {S,O,A,R} to describe the above Markov game process, where S represents the global state space, the environment of time slot t is the global state s(t)∈S, and O={O1,O2,...,O N} represents the set of observation spaces of the intelligent agent, O n Let A be the value space corresponding to the observation space of edge device n; A = {A1, A2, ..., A...} N Let A be the set of action spaces for the intelligent agent. n Let R be the value space corresponding to the action space of the intelligent agent of edge device n; R = {R1, R2, ..., R...} N Let} be the reward set, R n This represents the value space corresponding to the reward for edge device n; in time slot t, agent N determines the reward based on local observation o. n (t)∈O n , adopt strategy π n :O n →A n Select the corresponding action a n(t)∈A n In order to obtain the corresponding reward r n (t)∈R n .

[0034] Furthermore, the state space of the MDP problem includes the observation information of a single agent in time slot t, which includes the size of the computational task, tolerable latency, channel state, resource state of edge devices, and state of the MEC server. The state space of time slot t is represented as:

[0035] o n (t)={o task (t),o channel (t),o resource (t),o server (t)}

[0036] Among them, o task (t)={C n (t),D n (t),τ n (t)} represents the observation information of the task at time slot t, C n (t) represents the computing resources required for the computation task in time slot t, D n (t) represents the size of the computation task in time slot t, τ n (t) represents the maximum tolerable latency of the computation task in time slot t; channel (t) represents the channel observation information at time slot t. The resource status of edge device n under time slot t; server (t)={μ m (t),F m (t)} represents the state information of the MEC server observed by edge device n in time slot t, μ m (t) represents the reliability of MEC server m in time slot t, F m (t) represents the available computing resources of MEC server m in time slot t.

[0037] Furthermore, the action space of the MDP problem includes the actions of a single agent in time slot t, such as offloading decisions, transmission power selection, computational resource allocation, and consensus resource allocation. The set of actions in time slot t is represented as:

[0038]

[0039] Furthermore, the reward function is expressed as:

[0040]

[0041] Where r(t) represents the reward function value at time slot t; r n(t) represents the reward function value of edge device n in time slot t; N is the number of edge devices.

[0042] Furthermore, a multi-agent reinforcement learning algorithm is employed to make optimal offloading and resource allocation decisions. This algorithm consists of an environment and N agents, each with a centralized training phase and a distributed execution phase. During the training phase, centralized learning is used to train the critic network and the actor network. The critic network requires the state information of the other agents during training. In the distributed execution phase, the actor only needs to know local information and adjusts its local policy based on the estimated policies of the other agents to achieve global optimum. Specifically, this includes the following steps:

[0043] Let π = {π1, π2, ..., π} N Let θ be the set of policies for all agents, where θ = {θ1, θ2, ..., θ} N} represents the parameter set for the corresponding policy, and each agent updates the parameter θ. n To obtain the optimal strategy;

[0044] During the distributed execution phase, at each time slot t, each agent's actor network is based on its local observation state o. n (t) and its own policy selection action are represented as:

[0045] a n (t)=π n o n (t)

[0046] During the intensive training phase, each critic network can obtain observations from other agents. n (t) and action a n (t), then the Q-function of agent n can be expressed as:

[0047]

[0048] The Q function evaluates the actions of the actor network from a global perspective and guides the actor network to select better actions;

[0049] During training, the critic network updates its parameters by minimizing a loss function, which is expressed as:

[0050]

[0051] The actor network updates its parameters and outputs actions based on the loss function calculated by the critic network and its own observation information; the actor updates the network by calculating the gradient of the objective function.

[0052]

[0053] The parameters of the target network are updated using a soft update method, that is:

[0054]

[0055]

[0056] in, γ is the discount factor; E o,a,r,o' [·] indicates the expectation of the expression, where o is the observation set, a is the action set, r is the reward set, and o' is the observation set for the next time slot; r n The reward for edge device n in time slot t; o' n (t) represents the observation of agent n in the next time slot after time slot t; a' n =π n (o n ) is in the observation set of o n Time according to strategy π n The chosen action, π n (·) represents the policy of agent n; E o,a～D [·] indicates that the expression is expected. This represents the parameter θ of the expression with respect to the actor's current network. n Find the gradient, π n (a n |o n ) for the intelligent agent in observing o n Make the following action a n Strategies; θ is the soft update coefficient of the actor network. n θ represents the parameters of the current network for the actor. n 'These are parameters for the target network of the actor. ω is the soft update coefficient of the critic network. n ω' is the parameter of the current network for the critic. n These are the parameters of the target network for the critic.

[0057] This invention studies task offloading and resource allocation methods in multi-server MEC scenarios. Considering the time-varying nature of MEC server computing, communication resources, and channel states, as well as the security and privacy issues arising from interactions between heterogeneous edge nodes, a blockchain-based mobile edge computing task offloading model is established. Then, with the goal of minimizing the cost for users to complete computing tasks and maximizing the utility gained by users participating in mining, a joint optimization model for task offloading and resource allocation is established under multi-dimensional resource constraints. Simultaneously, considering the stochastic and time-varying network environment and the partial observability of environmental states, the task offloading cost problem and the blockchain mining utility problem are abstracted into a partially observable Markov decision process. A deep reinforcement learning algorithm is used to solve the problem. The agent learns historical information about task offloading and resource allocation to make better decisions. This invention achieves lower task offloading costs and higher blockchain mining utility. Attached Figure Description

[0058] Figure 1 A typical MEC system model diagram for consideration in this invention;

[0059] Figure 2 This is a flowchart of the method for unloading and allocating resources for mobile edge computing tasks based on blockchain in this invention;

[0060] Figure 3 This is a diagram of the MADDPG algorithm framework used in this invention;

[0061] Figure 4 This refers to the chain-like data structure of the blockchain used in this invention;

[0062] Figure 5 This describes the consensus process of the PBFT-based consensus algorithm used in this invention. Detailed Implementation

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] This invention proposes a blockchain-based method for offloading mobile edge computing tasks, such as... Figure 2 Specifically, it includes the following steps:

[0065] For dynamic MEC scenarios with multiple servers, considering that the computing, communication resources and channel status of MEC servers are time-varying, and that the interaction between heterogeneous edge nodes may cause security and privacy issues, a blockchain-based mobile edge computing task offloading model is established.

[0066] With the goal of minimizing the cost for users to complete computing tasks and maximizing the utility gained by users from participating in mining, a joint optimization model for task unloading and resource allocation is established under multi-dimensional resource constraints.

[0067] Considering the random and time-varying network environment and the partial observability of the environment state, the task unloading cost problem and the blockchain mining utility problem are abstracted into a partially observable Markov decision process;

[0068] Based on the established system model, the state space and action space of the MDP problem are obtained, and the reward function is constructed.

[0069] A multi-agent reinforcement learning algorithm is used to make optimal unloading and resource allocation decisions.

[0070] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0071] I. System Model

[0072] like Figure 1 As shown, this invention considers a typical MEC system, which consists of four layers: an IoT sensor layer, an edge device layer, a MEC server layer, and a cloud server layer. The IoT sensor layer comprises cameras, smart meters, wearable devices, health monitoring devices, etc., sensing information from the physical environment and generating data that needs to be computed. The IoT sensor acts as a lightweight blockchain node, electing representative edge devices to perform mining and generate blocks. Let the set of edge devices be denoted as . The system comprises N edge devices. Each edge device is responsible for collecting data from a set of IoT sensors it manages, analyzing the data, and determining whether to execute the code on the edge device or offload it to a MEC server. Let the set of MEC servers be . The system comprises M MEC servers, and the system operation time is discretized into T time slots, represented as follows: For simplicity, we assume that each edge device has only one computation task in the current time slot, and we define the task as S. n ={C n D n ,τ n}, C n Indicates completion of task S nRequired computing resources (CPU cycles), D n τ represents the size of the task. n This indicates the maximum tolerable latency for the task.

[0073] Edge devices collect computational data from a set of IoT sensors. By analyzing data size, tolerable latency, channel conditions, and the computing resources of the MEC server, they make offloading decisions. After task processing is complete, if the task is processed on the MEC server, the result is returned to the edge device. The edge device verifies the result and evaluates the performance of the task executor. If the result is verified as valid, the edge device pays the service fee to the MEC and updates the reliability of the MEC server stored on the blockchain.

[0074] 1. Communication model

[0075] definition For the offloading decision of edge device n in time slot t, if the computation task is offloaded to MEC server i for execution in time slot t, then x n (t) = i, then x is the task executed locally. n (t) = 0. The uplink channel gain between the edge device and the MEC server is defined as h. mn (t), where the transmission power of the edge device n offloading the task to the MEC server m in time slot t is p. n (t), the transmission power strategy is in To offload tasks to a collection of edge devices on the MEC server, p n P is the transmission power for edge device n to offload tasks to MEC server m. n Let W be the maximum transmission power for edge device n to offload tasks to MEC server m, and let W be the bandwidth between edge device n and MEC server m. Then, the transmission rate between edge device n and MEC server m in time slot t is:

[0076]

[0077] Where, σ 2 (t) represents the background noise power of the transmission channel between edge device n and MEC server m in time slot t. Within the same time slot, the channel parameters remain unchanged; however, they differ across different time slots. mn Let n be the distance between edge device n and MEC server m, and α be the path loss exponent.

[0078] The transmission latency between edge device n and MEC server m is:

[0079]

[0080] 2. Local Computation Model

[0081] Assumption For time slot t, edge device n is the computing resource allocated to perform the task. The computing resource allocation strategy for edge devices is as follows: Where F n Given the maximum computing resources of edge device n, the latency for edge device n to execute computing tasks locally is:

[0082]

[0083] The energy consumption for performing computing tasks locally is:

[0084]

[0085] Among them, κ n It is the chip-structure-specific energy coefficient in edge device n.

[0086] 3. MEC Server Computing Model

[0087] Assumption Let MEC server m allocate computing resources for executing the offloading task of edge device n at time slot t. Then, the latency of the task execution on MEC server m is:

[0088]

[0089] The total uninstallation latency is:

[0090]

[0091] When unloading a task, the energy consumption cost of edge device n is only related to data transmission, and is given by the following formula:

[0092]

[0093] In an MEC system, service quality mainly includes two aspects: task completion time T. n Energy consumption E n In time slot t, the total delay T of edge device n n and energy consumption E n Represented as:

[0094]

[0095]

[0096] Cost(t) = λ T T n (t)+λ E E n (t) (10)

[0097] Where Cost(t) represents the cost of the edge device in time slot t, and λ T λ is the weighting factor for time delay. E This is the weighting factor for energy consumption.

[0098] 4. Edge Device Model

[0099] When an edge device offloads a computing task to the MEC server, the edge device needs to pay the MEC server a service fee. Assume the service fee for the MEC server to execute a computing task is proportional to the computational load of the task, and assume the unit price of the computing service is q. n The unit price of this service is determined by the MEC server, and its size is C. n The computational cost of the computational task is: q n C n The effectiveness of an unloading task is related to its completion time, as shown in the following formula:

[0100]

[0101] Where, τ n This is the maximum tolerable latency for the task; therefore, the utility of offloading the task from the edge device is:

[0102] U n =u n -q n C n (12)

[0103] 5. MEC Server Model

[0104] During the task unloading process, once the MEC server receives an unloading request, it allocates corresponding resources to process the task. The energy consumption of the MEC server in executing the task is:

[0105]

[0106] The utility of MEC server m executing task n is:

[0107]

[0108] Where ω is the unit price of energy consumption.

[0109] As can be seen from the above formula, the utility of an MEC server depends on the unit price of computing services and the computing resources allocated to tasks. Selfish MEC servers may increase their utility by allocating fewer resources to tasks, thus reducing energy consumption for task execution. However, this reduces the utility of the tasks themselves, preventing them from completing within the maximum tolerable latency and consequently decreasing the utility of the edge devices. To prevent selfish MEC servers from allocating insufficient resources to tasks, a reliability model is established to evaluate the efficiency of MEC servers in executing offloading tasks. The reliability of each MEC server is updated every time slot t. Taking MEC server m as an example, assume that N tasks are offloaded to MEC server m for execution within time slot t. m .

[0110] Define a normalized utility for an unloading task that completes within the maximum tolerable latency:

[0111]

[0112] Wherein, log(1+τ) n () indicates the maximum utility of the task.

[0113] The computational efficiency update expression for MEC server m is:

[0114]

[0115] Where, ρ m (t-1) represents the computational efficiency of the MEC server up to m.

[0116] The task completion rate of MEC server m is:

[0117]

[0118] Where, ρ m (t-1) represents the task completion rate of MEC server m before time slot t, and L m This represents the number of tasks required to be completed within the maximum tolerable delay in time slot t.

[0119] Therefore, the reliability of MEC server m under time slot t is defined as follows:

[0120] μ m (t=ηρ m (t)+(1-η)δ m (t),η∈(0,1) (18)

[0121] Where η is the weighting factor.

[0122] 6. Blockchain Model

[0123] In blockchain-based MEC systems, the consensus process plays a crucial role, and a significant factor affecting blockchain system performance is consensus latency. Therefore, optimizing consensus latency is essential in blockchain consensus. This embodiment employs a chain-like blockchain data structure as follows: Figure 4 As shown in the attached diagram. Existing consensus algorithms (PoW, PoS, etc.) suffer from long latency during the consensus process, which reduces the performance of the blockchain system. Therefore, an enhanced consensus algorithm based on PBFT is proposed to optimize consensus latency. The consensus process is shown in the attached diagram. Figure 5 As shown, consensus nodes are dynamically selected based on the available computing resources and consensus utility of the edge devices, with the number of consensus nodes being N. s Edge devices unload computing tasks, group transaction records, verify them through PBFT consensus, and store them on an immutable and tamper-proof blockchain ledger, thus completing the on-chain block process.

[0124] When an edge device needs to offload a computing task, it first sends an offload request to the MEC server layer. After receiving multiple response messages, it queries the reliability table of MEC servers stored in the blockchain to find the reliability of candidates. Based on the reliability, channel conditions, and available computing resources of the servers, it selects an appropriate server for computation offloading. The reliability table is updated periodically based on verified transactions related to the offloading of computing tasks. By storing the reliability of each MEC server in the blockchain ledger, the edge device can quickly query the reliability from the blockchain when making offloading decisions, thus enabling rapid offloading decisions and improving offloading efficiency.

[0125] In the consensus process, the master node is responsible for generating blocks, the replica nodes are responsible for verifying blocks, and the unselected edge devices act as ordinary nodes, only responsible for adding verified blocks to the maintained blockchain ledger. In blockchain consensus, signatures and Message Authentication Codes (MACs) are used to ensure data integrity and transaction authentication. Signing a block or transaction, verifying a signature, generating a MAC, and verifying a MAC require x, y, z, and z CPU cycles respectively. The main steps of consensus are shown below.

[0126] a)Collect

[0127] The master node collects unverified transactions from all edge devices and then sorts them by timestamp, assuming a block size of S. b Given that the average size of a transaction is χ, the number of transactions in a block is: During this phase, the master node needs to verify the signatures and MACs of L transactions, so the computational cost of the master node is L(y+z).

[0128] b) Pre-prepare

[0129] At this stage, the master node generates a signature and MAC for the block, which is N. s -1 replica nodes generate MACs for pre-prepare messages. Each replica node verifies the MAC of the block, as well as the signatures and MACs of L transactions in the block. The computational cost of the primary node is x+N. s Given that the computational cost of each replica node is z + L(y + z), and assuming that the block transmission time is proportional to the block size, the message transmission time is τ. b S b .

[0130] c) Prepare

[0131] After a replica node verifies the pre-prepare message, it sends prepare messages to other consensus nodes. After each consensus node receives 2f prepare messages, it enters the next phase. In this phase, the primary node needs to verify 2f MACs, thus the computational cost is 2fz; each replica node needs to generate N... s -1 MAC and verify 2f MACs, the computational cost of each replica node is (N s -1)z+2fz, message transmission time is τ b S b f represents the number of problematic nodes (malicious nodes) in the network. The PBFT protocol stipulates that the number of malicious nodes in the network satisfies N≥3f+1, where N is the total number of nodes in the network.

[0132] d)Commit

[0133] After a replica node verifies the pre-prepare message, it sends prepare messages to other consensus nodes. After each consensus node receives 2f prepare messages, it enters the next phase. In this phase, the primary node needs to verify 2f MACs, thus the computational cost is 2fz; each replica node needs to generate N... s -1 MAC and verify 2f MACs, the computational cost of each replica node is (N s -1)z+2fz, message transmission time is τ b S b .

[0134] e)Reply

[0135] After receiving 2f commit messages, the master node and replica nodes consider the block valid, add it to the blockchain ledger, and send a reply message containing the verified block to other edge devices. After receiving f+1 reply messages, they update the global view. During this phase, the master node and replica nodes need to generate a MAC for the reply message; therefore, the computational cost for the master node and replica nodes is z, and the message transmission time is τ. b S b .

[0136] Ultimately, the computation time for the master node is:

[0137]

[0138] in, The computing resources used for consensus in the master node.

[0139] The computation time for a replica node is:

[0140]

[0141] in, These are the computing resources used for consensus within the replica nodes.

[0142] Use TTF to represent the delay in the consensus process:

[0143] T F =T I +T D +T V (twenty one)

[0144] Among them, T I For block spacing, T D =4τ b S b For block transfer time, T V =max{T pri ,T rep} represents the block verification time.

[0145] In a blockchain-assisted MEC system, edge devices act as edge miners. In each consensus round, each IoT sensor user votes on edge miners based on their consensus utility and the resources available for consensus within the edge device. The performance of blockchain consensus primarily depends on the block size and the computational resources available for consensus within the edge device. The consensus utility of edge device n is defined as:

[0146]

[0147] Where, τ n This represents the maximum tolerable latency for the task. Let n be the consensus time for edge device n. The IoT sensor elects edge miners for mining based on the computing resources available for consensus in the edge device. The more computing resources available for consensus, the lower the consensus latency, the higher the consensus utility, and the better the performance of the blockchain system.

[0148] II. Problem Modeling

[0149] In a blockchain-based MEC system, each edge device needs to perform both task offloading and blockchain mining simultaneously. Therefore, system performance evaluation needs to consider both mining performance and task offloading performance. For task offloading, edge devices need to minimize offloading costs to maintain the performance of the task offloading service. For blockchain mining, edge devices need to minimize consensus latency to maintain block mining performance. Therefore, our optimization objective is to maximize the consensus utility of the blockchain across all edge devices and minimize the offloading costs of all edge devices. The optimization problem is as follows:

[0150]

[0151] Constraints:

[0152]

[0153]

[0154]

[0155]

[0156]

[0157]

[0158] Among them, F m φ represents the maximum computing resources of server m. n The maximum computing resources for edge device n used for blockchain mining; The computing resources allocated by edge devices for blockchain mining in time slot t; T n(t) represents the processing latency of edge device n in time slot t; constraint (23a) is a constraint on the offloading decision, constraint (23b) is a constraint on the transmission power, constraint (23c) is a constraint on the computing resources of the edge device, indicating that the edge device should be allocated a positive computing resource to execute the computing task, but cannot exceed the resource budget; constraint (23d) is a constraint on the computing resources allocated by the MEC server for the task; constraint (23e) indicates that the MEC server should allocate a positive computing resource for the task, but cannot exceed the maximum value; constraint (23f) is a constraint on the computing resources used by the edge device for consensus; constraint (23g) indicates that the processing time of the task cannot exceed the maximum value.

[0159] Since the above optimization problem is mixed-integer nonconvex, it is difficult to solve. In dynamic computation offloading scenarios, channel conditions and the available computing resources of edge devices and MEC servers are time-varying. Furthermore, as the number of edge devices gradually increases, the dimension of the system state space becomes very large. Using traditional optimization methods leads to high computational complexity, making it difficult to obtain the optimal offloading strategy and resource allocation strategy. Therefore, this invention uses deep reinforcement learning to solve this problem.

[0160] III. Problem Solving Based on Multi-Agent Deep Reinforcement Learning

[0161] This invention abstracts the aforementioned optimization problem into a partially observable Markov decision process, with edge devices acting as agents. A tuple {S, O, A, R} is defined to describe the Markov game process, where S represents the global state space, the environment at time slot t is the global state s(t) ∈ S, and O = {O1, O2, ..., O...}. N Let A = {A1, A2, ..., A} be the set of observation spaces of the agent. N Let R = {R1, R2, ..., R} be the action space set of the intelligent agent. N} represents the reward set. In time slot t, the agent, based on local observations o... n (t)∈O n , adopt strategy π n :O n →A n Select the corresponding action a n (t)∈A n In order to obtain the corresponding reward r n (t)∈R n .

[0162] 1. State Space

[0163] In time slot t, the observation information of a single agent includes the size of the computational task, tolerable latency, channel state, resource status of edge devices, and the status of the MEC server. Therefore, the observation set in time slot t can be represented as:

[0164] o n (t)={o task (t),o channel (t),o resource (t),o server (t)}

[0165] Among them, o task (t)={C n (t),D n (t),τ n (t)} represents the observation information of the task at time slot t, C n (t) represents the computing resources required for the computational task of edge device n under time slot t, D n (t) represents the size of the computation task of edge device n in time slot t, τ n (t) represents the maximum tolerable latency of the computation task on edge device n under time slot t; channel (t) represents the channel observation information at time slot t. The resource status of edge device n under time slot t; server (t)={μ m (t),F m (t)} represents the state information of MEC server m observed by edge device n in time slot t, μ m (t) represents the reliability of MEC server m in time slot t, F m (t) represents the available computing resources of MEC server m in time slot t.

[0166] O(t) = {o1(t), o2(t), ... o N (t)} is the set of observations at time slot t, consisting of the states of all agents.

[0167] 2. Motion space

[0168] In time slot t, the actions of a single agent include offloading decisions, transmission power selection, computational resource allocation, and consensus resource allocation. Therefore, the set of actions can be represented as:

[0169]

[0170] Where, x n (t) represents the offloading decision of edge device n in time slot t, p n(t) represents the transmission power selected by edge device n to offload tasks to the server in time slot t. A(t) = {a1(t), a2(t), ..., a...} N (t)} is the set of actions in time slot t, which consists of the actions of all agents.

[0171] 3. Reward Function

[0172] Based on the optimization objective, the system reward is defined as the sum of the rewards of all agents in time slot t, and the reward of agent n is defined as r. n (t), then the system reward is:

[0173]

[0174] IV. MADDPG Algorithm

[0175] This section will demonstrate how to use multi-agent deep reinforcement learning methods within a framework of centralized learning and distributed execution to address the aforementioned problems.

[0176] like Figure 3 The MADDPG framework in this invention consists of an environment and N agents, each with a centralized training phase and a decentralized execution phase. During the training phase, centralized learning is used to train the critic network and the actor network; the critic network requires state information from the other agents during training. During the execution phase, the actors only need to know local information and adjust their local policies based on the estimated policies of the other agents to achieve global optimum.

[0177] Let π = {π1, π2, ..., π} N Let θ be the set of policies for all agents, where θ = {θ1, θ2, ..., θ} N} represents the parameter set for the corresponding policy, and each agent updates the parameter θ. n To obtain the optimal strategy.

[0178] During the distributed execution phase, at each time slot t, each agent's actor network is based on its local observation state o. n (t) and its own strategy π n :O n →A n Select action:

[0179] a n (t)=π n o n (t)

[0180] During the intensive training phase, each critic network can obtain observations from other agents. n (t) and action an (t), then the Q-function of agent n can be expressed as:

[0181]

[0182] The Q-function evaluates the actions of the actor network from a global perspective and guides the actor network to select better actions. During training, the critic network updates its parameters by minimizing a loss function, defined as follows:

[0183]

[0184] in, γ is the discount factor; E o,a,r,o' [·] represents the expectation of the expression given that the current agent's observation space is o, the next agent's observation space is o', the agent's action space is a, and the agent's reward space is r.

[0185] Simultaneously, the actor network updates its parameters based on the loss function calculated by the critic network and its own observation information, and outputs action 'a'. The actor updates the network by calculating the gradient of the objective function:

[0186]

[0187] Where D is the set used for experience replay; E o,a-D [·] indicates the expectation of the expression given that the current agent's observation space is o and the distribution of the agent's action space a follows the experience replay set D.

[0188] The parameters of the target network are updated using a soft update method, that is:

[0189]

[0190]

[0191] Where, θ n '、ω' n It is the set of parameters for the updated policy and the set of policies for the agent.

[0192] The pseudocode for task offloading and resource allocation strategies in a MADDPG-based MEC system is as follows:

[0193]

[0194]

[0195] in, This indicates that the expression is related to the policy parameter θ.n Differentiate; A←B means assigning the value of B to A.

[0196] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for offloading mobile edge computing tasks based on blockchain, characterized in that, Specifically, the following steps are included: For dynamic MEC scenarios with multiple servers, considering the time-varying computing and communication resources and channel states of MEC servers, and the security and privacy issues arising from interactions between heterogeneous edge nodes, a blockchain-based mobile edge computing task offloading model is established, including: When an edge device has a computing task offloading requirement, it sends an offloading request to the MEC server layer. After receiving multiple response messages, it queries the reliability table of MEC servers stored in the blockchain to find the reliability of the candidates. Based on the reliability, channel conditions, and available computing resources of the server, it selects an appropriate server for computing offloading. The reliability table is updated periodically based on the verified transactions of computing task offloading. The reliability of each MEC server is stored in a blockchain ledger, and the reliability is retrieved from the blockchain when the edge device makes an offloading decision. During the consensus process, the master node is responsible for generating blocks, the replica nodes are responsible for verifying blocks, and the edge devices that are not selected are ordinary nodes, which are only responsible for adding the verified blocks to the maintained blockchain ledger. The blockchain consensus process includes: For a block, signing a transaction requires x CPU cycles, verifying a signature requires y CPU cycles, generating a MAC requires z CPU cycles, and verifying a MAC requires z CPU cycles. The master node collects unverified transactions from all edge devices and then sorts them by timestamp, assuming a block size of [value missing]. The average size of the transaction is Then the number of transactions in a block is: During this stage, the master node needs to verify the signatures and MAC addresses of L transactions; therefore, the computational cost of the master node is... ; The master node generates a signature and MAC for the block. Each replica node generates a MAC for pre-prepare messages. Each replica node verifies the MAC of the block, as well as the signatures and MACs of L transactions in the block. The computational cost of the primary node is... The computational cost of each replica node is Assuming that the block transmission time is proportional to the block size, then the message transmission time is... , The number of consensus nodes; After the replica node verifies the pre-prepare message, it sends the prepare message to other consensus nodes. Each consensus node receives the prepared message. After receiving a prepare message, the process moves to the next stage, in which the master node needs to verify... There are MACs, therefore the computation cost is Each replica node needs to generate MAC address and verification Each MAC has a computational cost of 1 / 2 * ... The message transmission time is ; Received After each prepare message, each consensus node sends a commit message to other nodes, including the master node. During this phase, the master node and replica nodes need to verify... MAC, generating Therefore, the computational cost for the primary and replica nodes is MAC addresses. The message transmission time is ; Received After a commit message, the master node and replica nodes consider the block valid, add it to the blockchain ledger, and send a reply message containing the verified block to other edge devices. After each reply message, the global view is updated. During this phase, the primary and replica nodes need to generate a MAC for the reply message. Therefore, the computational cost for the primary and replica nodes is z, and the message transmission time is... ; With the goal of minimizing the cost for users to complete computing tasks and maximizing the utility gained by users from participating in mining, a joint optimization model for task unloading and resource allocation is established under multi-dimensional resource constraints. Considering the random and time-varying network environment and the partial observability of the environment state, the task unloading cost problem and the blockchain mining utility problem are abstracted into a partially observable Markov decision process; Based on the established system model, the state space and action space of the MDP problem are obtained, and the reward function is constructed. A multi-agent reinforcement learning algorithm is used to make optimal unloading and resource allocation decisions.

2. The method for offloading mobile edge computing tasks based on blockchain according to claim 1, characterized in that, The joint optimization model for task unloading and resource allocation includes: Constraints: in, This represents the local observation of edge device n in time slot t. This represents the transmission power at which edge device n offloads tasks to MEC server m in time slot t. The computing resources allocated by MEC server m for executing the offloading task of edge device n under time slot t; The computing resources allocated to edge device n for executing tasks under time slot t; The computing resources allocated to edge devices for blockchain mining in time slot t; The consensus utility of edge device n under time slot t; Cost of edge devices under time slot t; This represents the offloading decision for edge device n in time slot t. If the computation task is offloaded to MEC server i for execution in time slot t, then... If the task is executed locally, then ; Let N be the set of edge devices, and N be the number of edge devices. This represents the maximum transmission power. The system operating time is discretized into a set of T time slots; For edge devices Maximum computing resources; For server Maximum computing resources; A collection of MEC servers; For edge devices The largest computing resource used for blockchain mining; Indicates the maximum tolerable latency for the task; For edge devices In the time slot The processing delay.

3. The method for offloading mobile edge computing tasks based on blockchain according to claim 1, characterized in that, Abstracting the task unloading cost problem and the blockchain mining utility problem into a partially observable Markov decision process includes: Edge devices act as intelligent agents and define tuples. Describe the above Markov game process, where S represents the global state space, and the environment in time slot t is the global state. , For the set of observation spaces of intelligent agents, Let n be the value space corresponding to the observation space of the edge device n; For the action space set of the intelligent agent, Let n be the value space corresponding to the action space of the intelligent agent of edge device n; For the reward collection, This represents the value space corresponding to the reward for edge device n; in time slot t, agent N determines the reward based on local observations. , adopt strategies Select the corresponding action In order to obtain the corresponding rewards .

4. The method for offloading mobile edge computing tasks based on blockchain according to claim 1, characterized in that, The state space of the MDP problem includes time slots. The observation information of a single agent includes the size of the computational task, tolerable latency, channel state, resource status of edge devices, and state of the MEC server. The state space of time slot t is represented as: in, For the observation information of the mission in time slot t, The computational resources required for the computation task in time slot t. Let t be the size of the computation task. This represents the maximum tolerable latency of the computation task under time slot t; This refers to the observation information of the channel under time slot t. The resource status of edge device n in time slot t. Let n be the computing resources allocated to edge device n for executing tasks under time slot t. The computing resources allocated to edge device n for blockchain mining under time slot t; This refers to the status information of the MEC server observed by edge device n under time slot t. The reliability of MEC server m under time slot t. The available computing resources for MEC server m under time slot t.

5. A method for offloading mobile edge computing tasks based on blockchain according to claim 1, characterized in that, The action space of the MDP problem includes time slots. The actions of a single agent include offloading decisions, transmission power selection, computational resource allocation, consensus resource allocation, and time slots. The set of actions is represented as: in, For the offloading decision of edge device n in time slot t, The transmission power selected for offloading tasks to the server in time slot t; Let n be the computing resources allocated to edge device n for executing tasks under time slot t. The computing resources allocated to edge device n for blockchain mining under time slot t.

6. The method for offloading mobile edge computing tasks based on blockchain according to claim 1, characterized in that, The reward function is expressed as: in, This represents the reward function value at time slot t; This represents the reward function value of edge device n in time slot t; N is the number of edge devices. The consensus utility of edge device n under time slot t; The cost of the edge device under time slot t.

7. A method for offloading mobile edge computing tasks based on blockchain according to claim 1, characterized in that, A multi-agent reinforcement learning algorithm is employed to make optimal offloading and resource allocation decisions. This algorithm consists of an environment and N agents, each with a centralized training phase and a distributed execution phase. During the training phase, centralized learning is used to train both the critic and actor networks. The critic network requires state information from the other agents during training. In the distributed execution phase, each actor only needs local information and adjusts its local policy based on the estimated policies of the other agents to achieve global optimum. The algorithm includes the following steps: make The set of policies for all agents. Given the set of parameters for the corresponding policy, each agent updates the parameters... To obtain the optimal strategy; During the distributed execution phase, at each time slot t, each agent's actor network is based on its local observation state. And its own strategy selection action, represented as: During the intensive training phase, each critic network can obtain observations from other agents. and actions Then the Q-function of agent n can be expressed as: The Q function evaluates the actions of the actor network from a global perspective and guides the actor network to select better actions; During training, the critic network updates its parameters by minimizing a loss function, which is expressed as: The actor network updates its parameters and outputs actions based on the loss function calculated by the critic network and its own observation information; the actor updates the network by calculating the gradient of the objective function. The parameters of the target network are updated using a soft update method, that is: in, , Discount factor; This represents the expectation of the expression, where For the observation set, For a set of actions, For the reward collection, This is the set of observations for the next time slot; The reward for edge device n in time slot t; For the observations of agent n in the next time slot of time slot t; For the observation set is According to the strategy The chosen action Represents the policy of agent n; This indicates that the expectation of the expression is being calculated. This represents the parameters of the expression with respect to the actor's current network. Find the gradient. For intelligent agents in observation Make an action Strategies; For actor network soft update coefficient, These are the parameters for the current network of the actor. These are the parameters for the target network of the actor. This represents the critic network soft update coefficient. These are the parameters of the current network for the critic. These are the parameters of the target network for the critic.

Images