A collaborative task offloading and block mining method for mobile edge computing

Abstract

The present invention discloses a collaborative task unloading and block mining method for mobile edge computing, including: step 1, constructing a mobile edge computing B-MEC system, and based on the constructed B-MEC system, establishing a data transmission network based on blockchain operation and task unloading; step 2, intelligently allocating transmission power based on a deep reinforcement learning algorithm, and unloading tasks to edge servers according to different proportions when performing task unloading; step 3, establishing an RLR-RAFT consensus protocol based on deep learning reward value election, constructing a multi-objective optimization function, and completing collaborative optimization of task unloading and block mining based on an intelligent agent deep deterministic policy gradient algorithm. The problem of collaborative task unloading and block mining of mobile edge computing enabled by blockchain is solved by deep reinforcement learning technology, and the total delay and total energy consumption of the system are minimized.

Description

Technical Field

[0001] The present invention relates to a collaborative task unloading and block mining method for mobile edge computing, and belongs to the technical field of blockchain consensus mechanism. Background Art

[0002] With the explosive growth of IoT devices and the advent of the big data era, traditional cloud computing models face severe challenges in processing massive data, achieving low-latency responses, and ensuring data security. In order to meet the growing computing needs, mobile edge computing has emerged as a new type of network architecture and computing paradigm. In mobile edge computing, mobile devices offload tasks to edge servers for execution, which reduces the energy consumption and computing latency of mobile devices and improves user experience and service quality. However, the high dynamics and link heterogeneity of the mobile edge environment bring complexity and uncertainty to data security. In addition, the frequent dynamic communication between mobile devices and edge servers exacerbates potential security risks. Blockchain technology has the characteristics of decentralization, immutability, transparency and traceability. It can provide a high degree of security and trust guarantee for mobile edge computing through consensus mechanisms, etc., and has become a solution to improve the security level of mobile edge computing, and has received more and more attention.

[0003] In the above context, the integration of blockchain technology into the mobile edge computing architecture has become a key strategy to improve the security of mobile computing. On the one hand, blockchain technology enhances the security of edge nodes in edge computing by storing key data in blocks. On the other hand, edge computing provides sufficient computing resources for block mining tasks and supports the construction of blockchain. In addition, blockchain performs public verification across edge nodes, further enhancing the fairness, credibility and transparency of data processing. In the existing blockchain-based mobile edge computing optimization scheme, task offloading and block mining optimization are performed separately, resulting in the system performance not being able to achieve optimality. In order to improve the overall performance, it is necessary to combine task offloading and blockchain mining to achieve an efficient blockchain-based mobile edge computing system. Summary of the invention

[0004] The present invention designs and develops a collaborative task offloading and block mining method for mobile edge computing, which solves the collaborative task offloading and block mining problems of mobile edge computing enabled by blockchain through deep reinforcement learning technology, thereby minimizing the total delay and total energy consumption of the system, ensuring the integrity and credibility of traffic data, and improving the overall performance of the intelligent transportation system and road traffic safety.

[0005] The technical solution provided by the present invention is:

[0006] A mobile edge computing collaborative task offloading and block mining method based on blockchain, comprising:

[0007] Step 1: Build a mobile edge computing B-MEC system, and based on the built B-MEC system, establish a data transmission network based on blockchain operations and task offloading;

[0008] Step 2: Intelligently allocate transmission power based on deep reinforcement learning algorithm, and unload tasks to edge servers according to different proportions when offloading tasks;

[0009] Step 3: Establish the RLR-RAFT consensus protocol based on deep learning reward value election, construct a multi-objective optimization function, and complete the collaborative optimization of task offloading and block mining based on the multi-agent deep deterministic policy gradient algorithm.

[0010] Preferably,

[0011] The B-MEC system consists of multiple mobile users and multiple edge servers, and forms three types of data transmission paths, including: between users, between edge servers, and between users and edge servers;

[0012] The set of edge servers is represented as: ;

[0013] The user collection is represented as:

[0014] Preferably, the mobile device is set To the edge server The wireless channel gain is ,

[0015] ;

[0016] In the formula, Distance to edge server Base channel gain at one meter, is the distance between the user and the edge server; mobile device To the edge server The uplink transmission rate is expressed as:

[0017] ;

[0018] In the formula, For mobile devices To the edge server The transmission power, For server bandwidth, is the background noise variance.

[0019] Preferably, it also includes:

[0020] Setting up your mobile device In time period Offload to edge servers The task is , then the mobile device The tasks processed locally are represented as:

[0021] ;

[0022] In the formula, For time period Tasks generated by mobile devices, For mobile devices In time period Offload to edge servers , task.

[0023] Setting up your mobile device The probability of becoming the leader in an election is:

[0024] ;

[0025] In the formula, , , The reward value for each user;

[0026] Setting up your mobile device The transmission power between and and other mobile devices is , is the interference power of other mobile devices, and the wireless channel gain between users is ,

[0027] ;

[0028] In the formula, For mobile devices and The distance between

[0029] mobile device and The communication rate between is expressed as:

[0030] ;

[0031] In the formula, Bandwidth for mobile devices;

[0032] Edge Server and The communication rate between is expressed as:

[0033] ;

[0034] In the formula, For server bandwidth, For edge servers The transmission power between other edge servers, is the interference power between edge servers.

[0035] Preferably, the step 2 comprises:

[0036] For mobile devices , in each time slot Local processing of computing tasks Processing delay It is expressed as:

[0037] ;

[0038] In the formula, for Calculate the number of cycles of one bit of data;

[0039] Based on the time it takes to offload tasks from mobile devices to each edge server and power Calculate the total energy consumption of the mobile device transmission task, the total energy consumption is ,

[0040] ;

[0041] Based on the computing power of the edge server , Edge Server Complete the offload task processing delay for all mobile devices for:

[0042] ;

[0043] In the formula, For mobile devices , In the time period Offload to edge servers Mobile devices for tasks Unloading its tasks The total delay consumed is:

[0044]

[0045] Setting local computation and task offloading in parallel, mobile devices The total computational delay of for:

[0046] ;

[0047] In the formula, For mobile devices Offloading process and offloading to edge servers The sum of execution time;

[0048] mobile device The total energy consumption is the energy consumption of local computing and the energy consumption of offloading tasks, expressed as:

[0049] ;

[0050] In the formula, For mobile devices The total energy consumption, is the energy consumption of local computing, is the energy consumption of the offloading task;

[0051] During the offloading process, based on task loss, set at any time slot The tasks generated in need to be completed in the current time slot, and the unfinished ones will be discarded. These tasks are recorded as , expressed as:

[0052] ;

[0053] In the formula, For mobile devices In time slot Processable cycle.

[0054] Preferably,

[0055] In step 3, the consensus process can be carried out synchronously with the tasks generated by the mobile device. By modeling the election phase and block consensus phase of RLR-RAFT, it is concluded that during the election phase of the mobile device, the leader node in the mobile device needs to execute Voting requests, Election confirmation and The secondary heartbeat signal requires generating blocks and offloading them to the edge server for block verification;

[0056] The total delay for each mobile device in the consensus process is:

[0057] ;

[0058] In the formula, The delay of the leader's transmission status confirmation signal between mobile devices, The delay in producing blocks for the leader, Offloading blocks to edge servers for leaders Delay in uninstalling For mobile devices Probability of being a leader and a follower;

[0059] The total energy consumption of each mobile device during the consensus process is:

[0060] ;

[0061] In the formula, The energy consumption of the leader's transmission status confirmation signal between mobile devices, The energy consumption of generating blocks for the leader, Offloading blocks to edge servers Unloading energy consumption;

[0062] The total delay during the consensus process is:

[0063] ;

[0064] In the formula, The delay in sending the block to other edge servers to complete the block verification process, Block confirmation signal transmission delay for each follower in the edge server;

[0065] The total energy consumption during the consensus process is:

[0066] ;

[0067] In the formula, is the energy consumption for block data transmission in the edge server, Energy consumption of the block confirmation reply process;

[0068] Optimize total delay, total energy consumption and task loss rate.

[0069] Set the objective function as: , transforming the objective function into a maximization problem,

[0070] Set each agent in a time period The reward function in is:

[0071] ;

[0072] In the formula, is the total delay, is the total energy consumption; , , They are the non-negative weighted factors of the three indicators: total delay, total energy consumption, and task loss rate.

[0073] Preferably, based on a multi-agent deep deterministic policy gradient algorithm, collaborative optimization of task offloading and block mining is achieved, including:

[0074] set up As the set of all proxy policies, As the parameters of the corresponding strategy set, each agent updates its parameters To obtain the optimal strategy ;

[0075] based on Framework, each agent corresponds to 4 networks, including: policy network , value network , two target networks and Used for stable training, running in a continuous action space in a multi-agent environment,

[0076] in, Through the behavioral network in each time slot Generate deterministic actions in ,and Value Network Assessment The behavior of the agent during training In By calculating the objective function The gradient of is used to update the behavior network;

[0077] After a certain number of updates, The parameters of the policy network are sent to the user to perform action decisions.

[0078] Preferably, the update process includes:

[0079] Initialize the network parameters of all agents at each time step in each training phase , each agent chooses an action based on the current state , execute the agent’s actions , the observed reward value and the next state , store its experience samples in the experience replay buffer, the trusted third party obtains the experience samples from each agent and combines them into global experience samples, and starts centralized training on the trusted third party;

[0080] Randomly sample a size of Batch samples of , updated by minimizing the loss and policy gradient Network parameters and Network parameters,For the objective function of each agent, the target network parameters are updated by soft update,,controlling the synchronization speed between the target network and the main,network.

[0081] Beneficial effects of the present invention: The present invention proposes a B-MEC system that jointly optimizes task offloading and blockchain operations. Under this framework, each edge server not only processes computing tasks offloaded from mobile devices, but also participates in the block consensus process in the blockchain network. At the same time, based on this system scenario, an RLR-RAFT consensus mechanism in which mobile devices and edge servers participate is proposed, and the reward value election in deep reinforcement learning is used to reach a consensus. In order to maximize system performance, a multi-objective optimization function is constructed by optimizing task offloading decisions, wireless transmission power allocation, and local computing resource allocation, aiming to simultaneously optimize the energy consumption, delay, and task loss rate of task offloading and block mining. In order to solve this multi-dimensional optimization problem, a multi-agent deep deterministic policy gradient algorithm is introduced, and a collaborative task offloading and block mining method in blockchain-based mobile edge computing is proposed.

[0082] The B-MEC system proposed in this invention integrates task offloading and blockchain operations, and achieves a significant improvement in system performance through the RLR-RAFT consensus mechanism and deep reinforcement learning optimization. In practical applications, it can significantly reduce energy consumption, delay and task loss rate, while improving task processing speed and blockchain consensus efficiency, providing users with more efficient, secure and reliable mobile edge computing and blockchain services.

[0083] In the intelligent transportation system, vehicles are mobile devices, and their computing tasks (such as real-time road condition analysis, traffic control algorithms, etc.) are often complex and computationally intensive. The present invention proposes a collaborative task unloading and block mining method for mobile edge computing, which allows vehicles to unload heavy computing tasks to edge servers through wireless networks. The edge servers can quickly process these tasks with their powerful computing power and immediately feed back the processing results to the vehicles. In this way, the computing load of the vehicle is significantly reduced, thereby reducing energy consumption and improving the real-time performance of task processing. In addition, in order to further enhance the security and credibility of the intelligent transportation system, the present method introduces blockchain technology. The edge server can not only perform computing tasks, but also participate in the block mining process, and use its computing power to verify and record traffic data. The present method combines blockchain technology with edge computing, and through the collaborative optimization of the two, the maximum reduction of system energy consumption and delay is achieved. This scheme not only improves the overall performance of the intelligent transportation system, but also ensures the integrity and credibility of traffic data, providing strong technical support for future urban traffic management. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 This is a scenario diagram of mobile edge computing that enables blockchain as described in the present invention.

[0085] Figure 2This is a model framework diagram of the B-MEC system described in the present invention.

[0086] Figure 3 It is a reward function trend diagram of different algorithms described in the present invention.

[0087] Figure 4 This is a comparison chart of average delays of user tasks under different numbers of users described in the present invention.

[0088] Figure 5 This is a comparison chart of average energy consumption of user tasks under different numbers of users described in the present invention.

[0089] Figure 6 This is a comparison chart of the average delay of the consensus process under different numbers of users described in the present invention.

[0090] Figure 7 This is a comparison chart of the average energy consumption of the consensus process under different numbers of users described in the present invention.

[0091] Figure 8 This is a comparison chart of task loss rates of user tasks under different numbers of users described in the present invention.

[0092] Fig. 9 This is a comparison chart of the average delay of the consensus algorithm under different numbers of users described in the present invention. DETAILED DESCRIPTION

[0093] The present invention is further described in detail below in conjunction with the accompanying drawings so that those skilled in the art can implement the invention with reference to the description.

[0094] like Figure 1-9 As shown, the present invention provides a collaborative task offloading and block mining method for mobile edge computing, which optimizes task offloading decisions, wireless transmission power allocation and local computing resource allocation, constructs a multi-objective optimization function, comprehensively considers the energy consumption, delay and task loss rate of task offloading and block mining, so as to achieve the goal of minimizing system energy consumption and delay, and then makes task computing offloading decisions while ensuring low latency and low energy consumption, thereby improving the operating efficiency and resource utilization of the entire system, including:

[0095] like Figure 1 As shown in Figure 1, a blockchain-enabled B-MEC system with multiple edge servers and multiple users is adopted, where blockchain nodes are deployed in both users and servers, and users can offload computing tasks to multiple edge servers. There are three types of wireless links in B-MEC, including: wireless data transmission links between users and edge servers and wireless backhaul links for interactions between users and edge servers; the edge server set is represented as , the user set is represented as ; Each mobile device participates in block mining through a consensus mechanism, and the edge server is responsible for block verification.

[0096] In the edge computing network environment, the backhaul channel between edge servers and the access channel between edge servers and users use different spectrums. The uplink can avoid potential interference from other edge nodes. When multiple devices unload data to the same edge node at the same time, uplink interference will occur, reducing link quality and network performance. Therefore, different mobile devices Need to compete for edge nodes Supported channel resources. Set up mobile devices To the edge server The wireless channel gain is ,in, , Indicates the distance from the edge server Base channel gain at 1 meter, Indicates the distance between the user and the edge server. Mobile device To the edge server The transmission power is defined as , represents the channel noise, represents the server bandwidth, then the mobile device To the edge server The uplink transmission rate can be expressed as:

[0097] ;

[0098] Set in each time period Each mobile device generates a task, and the task is divisible, denoted as ,Mobile devices can keep part of the tasks locally for calculation or choose to offload part of the tasks to the edge server through the wireless channel. The edge server can provide computing services for multiple mobile devices at the same time. Compared with local devices, the edge server has more powerful computing power and more stable power supply. Therefore, two task computing modes are considered: local execution and edge offloading.

[0099] Move your mobile device In time period Offload to edge servers The task is Therefore, mobile devices The tasks processed locally can be described as .

[0100] In the blockchain-based B-MEC system, the blockchain consensus mechanism is a key component, responsible for mining transaction blocks and integrating them into the blockchain to maintain the consistency and security of the distributed ledger. Given that mobile devices are limited by limited computing power and storage resources, users are set as lightweight nodes, responsible for initiating and requesting transactions, and participating in the election phase of the RLR-RAFT consensus mechanism. The edge server has powerful computing processing capabilities and massive storage space. As a complete node, it is responsible for storing blocks and reaching the block verification phase in the RLR-RAFT consensus. After the resource transaction is completed, the blockchain system will record the transaction information (including: timestamp, user ID, edge server ID, computing task information, etc.).

[0101] In the RAFT consensus mechanism, the leader node is responsible for the generation and replication of blocks, which consumes more resources, so the election process needs to be carefully considered. Since there is randomness in the election timeout during the election phase of the original RAFT consensus process, and it is easy for multiple candidates to have the same number of votes, resulting in frequent elections or election failures, all nodes need to wait for a while before restarting the election process, which aggravates consensus delays and energy consumption. Therefore, the RLR-RAFT consensus algorithm based on reinforcement learning reward value is proposed to optimize the election strategy and process to improve the performance of the original RAFT algorithm.

[0102] During the consensus process, leaders and followers need to send status confirmation signals (such as heartbeat packets, voting requests, election requests, etc.) through wireless channels. In order to simplify the communication model of RLR-RAFT, the amount of data for status confirmation is set to , the block size is , the block generation interval is Since the leader consumes more resources, the delay and energy consumption are also higher, so the reinforcement learning reward value is introduced to determine the leader selection. This is because in the set reinforcement learning reward function, the higher the reward value, the lower the delay and energy consumption of the mobile device. At the same time, the reward value can also represent the representative of honest users in this environment.

[0103] In order to optimize the uncertainty of the leader election process in the traditional RAFT algorithm, this paper also proposes a timeout initialization strategy based on reward value. The higher the reward value of the mobile device, the shorter the timeout initialization time, so that it can become a candidate or even a leader faster. Construct a probability-based model to describe the mobile device in the RLR-RAFT consensus mechanism Behavior, set the mobile device The probability of becoming the leader in an election is , .

[0104] ;

[0105] in, The reward value for each user.

[0106] Since all mobile devices and edge servers will become leaders or followers during the election phase and block verification phase, taking a mobile device as an example, the probability of it being a follower is , modeling the relationship between the leader and the follower. Assuming that the backhaul channel between mobile devices and the intervention channel between mobile devices and the server use different spectrums, the mobile devices only need to consider the interference between users. The transmission power between the mobile devices is , is the interference power of other mobile devices, and the wireless channel gain between users is ,

[0107] in, , , Represents a mobile device and The distance between represents the bandwidth of the mobile device. and The communication rate between can be expressed as:

[0108] ;

[0109] Likewise, edge servers and The communication rate between can be expressed as:

[0110] ;

[0111] When the user's task is to be calculated locally, the mobile device The local computing power is denoted as , the chip architecture factor of the mobile device is , each time slot of the mobile device Processable The cycle is recorded as , get the mobile device's time slot of The number of cycles is:

[0112] ;

[0113] For mobile devices , in each time slot Local processing of computing tasks Processing delay It can be expressed as:

[0114] ;

[0115] in, for Calculate the number of cycles for one bit of data.

[0116] In time slot In , the energy consumption of a mobile device to perform its tasks locally can be defined as:

[0117] ;

[0118] In case of edge offloading, from mobile devices Sending a computation task To the edge server The time it takes to uninstall It can be described as:

[0119] ;

[0120] Since the mobile device can allocate different transmission powers to different edge servers during task offloading, the total energy consumption of the mobile device transmission task is calculated by the time and power of the mobile device offloading the task to each edge server. The total energy consumption is recorded as :

[0121] ;

[0122] The edge server consumes its own computing resources to process the task according to the size of the task received. , Edge Server Complete the offload task processing delay for all mobile devices It can be expressed as:

[0123] ;

[0124] Since the edge server task processing result is smaller than the offloaded data, the download part is not modeled.

[0125] mobile device The task offloading delay depends on the maximum delay between offloading the task to each edge server and the edge server completing the calculation. In short, the mobile device Unloading its tasks The total delay at is given by:

[0126] ;

[0127] Assuming local computation and task offloading are parallel, the mobile device The total computational delay of It can be regarded as the maximum value of the two and can be expressed as:

[0128] ;

[0129] Since edge servers are located in fixed locations and have a stable power supply, the energy consumption generated when they handle offload tasks is not taken into account. The total energy consumption, namely the energy consumption of local computing and the energy consumption of offloaded tasks, can be expressed as:

[0130] ;

[0131] The task loss rate directly reflects the stability and efficiency of the system when facing different task loads. The possibility of task loss is considered during the unloading process and is set at a certain time slot. The tasks generated in need to be completed in the current time slot, and the unfinished ones will be discarded. These tasks are recorded as , which can be expressed as

[0132] ;

[0133] During the consensus process, the mobile device is set to allocate a part of computing resources for the consensus process, that is, the consensus process can be carried out synchronously with the tasks generated by the mobile device. By modeling the election phase and block consensus phase of RLR-RAFT, it can be concluded that during the election phase of the mobile device, the leader node in the mobile device needs to execute Voting requests, Election confirmation and At the same time, the leader node also needs to generate blocks and offload them to the edge server for block verification.

[0134] Therefore, the delay of the leader's transmission status confirmation signal between mobile devices is expressed as:

[0135] ;

[0136] Its energy consumption is expressed as:

[0137]

[0138] At the same time, get the delay of the leader generating blocks:

[0139] ;

[0140] The energy consumption of generating a block is:

[0141] ;

[0142] The leader offloads the block to the edge server The unloading delay is:

[0143] ;

[0144] Unloading energy consumption is:

[0145] ;

[0146] When blocks are offloaded to edge servers When , the edge server becomes the leader among edge servers and is responsible for the block verification process. The leader among edge servers needs to send the block to other edge servers to complete the block verification. The delay of this process can be expressed as:

[0147] ;

[0148] The energy consumption for data transmission is:

[0149] ;

[0150] During the election process, each follower in the mobile device needs to send a voting reply and a heartbeat reply to the leader who initiated the voting request. The transmission delay of the status confirmation signal of each follower in the mobile device is:

[0151] ;

[0152] The energy consumption for data transmission is

[0153] ;

[0154] During the block verification process, the followers in the edge server need to send a block confirmation reply to the leader who initiated the block verification, and the block confirmation signal transmission delay of each follower in the edge server is:

[0155] ;

[0156] The energy consumption of this process is:

[0157] ;

[0158] Therefore, the total delay of each mobile device in the consensus process is:

[0159] ;

[0160] The total energy consumption of each mobile device during the consensus process is:

[0161] ;

[0162] The total delay in the consensus process is thus:

[0163] ;

[0164] The total energy consumption during the consensus process is:

[0165] ;

[0166] The energy consumption and delay in the B-MEC environment are mainly concentrated in the local calculation and offloading process of user tasks and the delay and energy consumption in the block mining and offloading process of the consensus scheme. In order to jointly optimize the energy consumption and delay of the block mining task of the local mobile device and the local partial task calculation, the larger value of the two indicators is set as the local calculation delay and energy consumption of the system, that is,

[0167] ;

[0168] ;

[0169] At the same time, this strategy is still adopted during the unloading process, and the larger value of the indicator is selected as the unloading delay and energy consumption of the system:

[0170] ;

[0171] ;

[0172] So we can get the total delay and total energy consumption:

[0173] ;

[0174] ;

[0175] In order to minimize energy consumption, delay and task loss rate, the optimization problem can be expressed as:

[0176] ;

[0177] ;

[0178] ;

[0179] ;

[0180] in, Represents the non-negative weighted factors of these three indicators. Since the higher the delay, energy consumption and task loss rate, the greater the impact on the entire system, they are all set to negative values; Constraint Indicates that the sum of each user's local computing data and offloaded data does not exceed the generated amount; constraint Indicates that the mobile user The location remains within the service range of the edge server; constraints Indicates that the local computing power and transmission power of the mobile device do not exceed the maximum power.

[0181] According to the optimization problem It can be seen that its action space is complex and continuous, and as a dynamic and high-dimensional system state space, the use of traditional optimization methods will lead to challenges of high computational complexity, which seriously restricts the feasibility of the constructed model in the actual scenario of mobile edge computing B-MEC based on blockchain. In order to overcome this problem, in the complex environment of multi-agent collaborative operation, distributed multi-agent deep reinforcement learning MADRL is introduced to effectively deal with the complexity of multi-objective optimization problems, ensuring that the proposed model can be efficiently and flexibly applied to blockchain-based B-MEC deployment.

[0182] Markov Decision Process:

[0183] First, the objective function Transform it into a reward maximization problem; and formulate it as a multi-agent Markov process, consisting of the tuple Indicates that Represents the set of each mobile device, which learns the optimal strategy by observing the state in the environment and maximizes its decision reward. In addition, Represents a state set, represents the set of actions for each agent, Represents the set of rewards for all agents.

[0184] Next, set the items in each tuple:

[0185] (1) State space

[0186] The state space consists of two parts: each agent in the time slot Task status and location status , which is defined as a matrix .

[0187] (2) Action Space

[0188] Each intelligent agent makes actions at each time step by observing the system state, including offloading task decisions, transmission power, and local computing power selection. Each action is explained as follows:

[0189] Offloading decision: Each mobile device decides the number of tasks to be offloaded to the edge server based on the current task status. And the number of local computing tasks .

[0190] Transmit power: Each mobile device selects a transmit power value based on the current offloading strategy and observation information. , sending some tasks to edge servers .

[0191] Local computing power: Select the appropriate local computing power based on the current local computing task volume. distribute resources to perform local tasks.

[0192] Define the action space as a matrix ,in ;

[0193] At the same time, according to the mobile device The amount of tasks sent to each edge server , to determine the leader in the edge server and send the block to the leader. Here, the edge server with the least amount of tasks sent by mobile devices is elected as the leader, that is, .

[0194] (3) Reward Function

[0195] In a time slot In order to optimize the overall performance, reduce the energy consumption of mobile devices, reduce operation delays and avoid task loss, enhance the integrity and satisfaction of user experience, and maximize its cumulative reward value, each intelligent agent is set to perform a certain action in a certain state. The reward function within .

[0196] MADRL Algorithm

[0197] In reinforcement learning, the agent interacts with the environment to learn the optimal strategy, which aims to maximize the long-term cumulative reward. Compared with traditional reinforcement learning, deep reinforcement learning uses deep neural networks as nonlinear approximators, which can effectively extract features and make decisions in high-dimensional input space. In this context, the MADDPG algorithm is used to solve the multi-objective optimization problem in the B-MEC system. MADDPG is a multi-objective deep reinforcement learning algorithm for multi-agent collaborative decision-making problems. In order to cope with the complex interaction and collaboration challenges in multi-agent systems, the MADDPG algorithm solves this problem by introducing centralized training and distributed execution methods. Considering the existence of sensitive information such as user location privacy data in the B-MEC system, a trusted third-party agency is introduced, which acts as a secure data aggregator, allowing agents to upload their own experience samples and integrate them into a global experience sample. This method protects user privacy and ensures the consistency of the learning environment, and improves the convergence speed and quality of the learning algorithm. In each time period In the process, the agent downloads the latest policy network parameters from a trusted third party to guide its decision-making process. The mobile device can use the policy network to quantify the computational allocation between local execution tasks and offloaded tasks, and obtain the corresponding power value. The mobile devices also need to reach a consensus through a consensus mechanism and generate data blocks based on their respective calculated power values. Subsequently, these data blocks will be transmitted to the edge server, which will perform the block verification process to ensure the integrity of the data. The corresponding model framework is as follows: Figure 2 shown.

[0198] set up as the set of all proxy policies, and As the parameters of the corresponding strategy set, each agent updates its parameters To obtain the optimal strategy The MADDPG algorithm is based on Framework, each agent corresponds to 4 networks, one of which is a policy network , a value network , two target networks and Used for stable training. Runs in a continuous action space in a multi-agent environment, where Through the behavioral network in each time slot Generate deterministic actions in ,and Value Network Assessment The behavior of the agent during training In By calculating the objective function The gradient of is used to update the behavior network.

[0199] ;

[0200] in, Indicated by The state-action value function given by the network, represents the state set of all agents, yes The action strategy given by the network. In addition, The network minimizes the loss function Update the target network by:

[0201] ;

[0202] in, , is the discount factor for long-term rewards.

[0203] In order to smooth the update of the target network, and The target networks all adopt the soft update method, namely: .

[0204] in, is a soft update parameter. After a certain number of updates, The parameters of the policy network are sent to the user to perform action decisions. The update process of the proposed MADRL algorithm is shown in Algorithm 1. First, the network parameters of all agents are initialized, and then at each time step in each training phase , each agent chooses an action based on the current state , and then execute the agent's action , the observed reward value and the next state , and then store its experience samples in the experience replay buffer. At this point, the trusted third party obtains the experience samples from each agent and combines them into a global experience sample. The trusted third party starts centralized training and first randomly samples a size of from the global experience sample. Batch samples of , updated by minimizing the loss and policy gradient Network parameters and Network parameters. For each agent’s objective function, the target network parameters are updated using a soft update method to control the synchronization speed between the target network and the main network.

[0205] Algorithm 1. MADRL algorithm based on B-MEC system

[0206] 1: Input: location information of the mobile device and the size of the generated task

[0207] 2: Output: local computing strategy and offloading strategy

[0208] 3: :

[0209] 4: Initialize each agent of Network parameters ;

[0210] 5: Initialize each agent of Network parameters ;

[0211] 6: Initialize each agent Target network , ;

[0212] 7:

[0213] 8: for =1,2,...,T do

[0214] 9: , initialization state Select an action based on the Actor network

[0215] 10: Observed reward value and the next state

[0216] 11: Storage Experience Replay Pool

[0217] 12: for trusted third-party do

[0218] 13: Collect data samples uploaded by mobile devices to form a global experience pool

[0219] 14: Randomly draw from the global experience pool Samples Used to train the network

[0220] 15: Settings

[0221] 16: Update The network is passed through gradient descent:

[0222] ;

[0223] 18: end for

[0224] 19: Update each agent Target network parameters

[0225] 20:

[0226] 21: end for

[0227] 22: end for

[0228] 23: end for

[0229] Simulation parameter setting: The simulation environment is set to python3.9 and PyTorch1.8.1. All users are randomly distributed in an area of ​​1000mX1000m. In each time slot, the user will move continuously, and the displacement will not exceed 2 meters. Four edge servers are configured in the area to unload task data. Their coordinates are (300,300), (300,600), (600,300), (600,600). The number of users in the experiment is set to 10, and the task volume of the mobile device they carry is sampled from a normal distribution with a mean of 200, in KB. The weight factors of delay, energy consumption, and task loss rate in the reward function are 2, 2, and 1, respectively. Other simulation parameter configurations are shown in Table 1.

[0230] Table 1 Parameter setting table

[0231]

[0232] like Figure 3 As shown in the figure, the convergence and average return changes of different algorithms are compared. It is obvious that the proposed MADDPG reaches convergence in about 500 rounds. In contrast, the QMIX algorithm continues to fluctuate during the first 400 iterations and fails to converge effectively, which indicates that the algorithm fails to effectively learn a good strategy at this stage. In order to improve this situation, constraints on the action space are introduced through manual intervention after 400 rounds, which eventually converges to an acceptable range.

[0233] like Figure 4 As shown in the experimental results, the impact of the change in the number of users on the computational delay of user tasks is shown. As the number of users increases, the amount of tasks that users need to upload to the edge server also increases accordingly, which directly leads to an increase in the computational delay of tasks. The proposed algorithm can capture this dynamic change of intensified channel competition caused by the increase in the number of users. In comparison, this algorithm shows lower computational delay.

[0234] like Figure 5As shown in Figure 2, the energy consumption trend caused by user tasks under different numbers of users is shown. It can be seen that the energy consumption of all algorithms gradually increases with the increase in the number of users. This is mainly because the increase in the number of users aggravates the congestion of the uplink channel. Figure 5 It can be seen from the data that the energy consumption increment of the MADDPG algorithm is the smallest when dealing with the growth in the number of users.

[0235] like Figure 6 As shown in the figure, it shows that as the number of users increases, the overall delay in the consensus mechanism shows an upward trend. This trend is mainly composed of two parts: on the one hand, the inherent delay caused by local calculation of blocks, that is, the processing time required for each user node to generate blocks. On the other hand, the congestion of the uplink channel caused by the increase in the user base leads to significant delays in the transmission of blocks from user nodes to edge servers. In the comparative analysis, the MADDPG algorithm shows superior performance under the comprehensive consideration of these two types of delays, significantly reducing the delay in the overall calculation and transmission process.

[0236] like Figure 7 As shown in the figure, when the number of users increases, the energy consumption in the consensus also shows a significant growth trend. The reason is that latency is positively correlated with energy consumption. When comparing this performance indicator, the MADDPG algorithm also shows significant optimization results in terms of consensus energy consumption. The algorithm effectively reduces the energy consumption of local computing and network transmission by intelligently allocating computing resources and optimizing block upload strategies.

[0237] like Figure 8 As shown in the figure, the impact of changes in the number of users on the task processing loss rate is demonstrated. Since the arrival of tasks is random and unpredictable, under high task load conditions, task loss is a normal phenomenon for the system to face challenges during peak periods. In contrast, it can be seen that the task loss rate of the proposed MADDPG algorithm always remains at a relatively low and stable level.

[0238] like Fig. 9 As shown in the figure, the significant difference in delay performance between the proposed RLR-RAFT consensus mechanism and the RAFT consensus mechanism is illustrated through comparative analysis. As the number of system users continues to grow, the RAFT consensus algorithm frequently encounters election conflicts during the leader election process, resulting in a large consensus delay. The RLR-RAFT consensus algorithm introduces a reinforcement learning mechanism and uses the reinforcement learning reward value between nodes as the basis for the election of the leader node, thereby achieving a significant reduction in consensus delay.

[0239] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation modes, and they can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

Claims

1. A collaborative task offloading and block mining method for mobile edge computing, characterized in that: include: Step 1: Build a mobile edge computing B-MEC system, and based on the built B-MEC system, establish a data transmission network based on blockchain operations and task offloading; The B-MEC system is composed of multiple mobile users' mobile devices and multiple edge servers, and forms three types of data transmission paths, including: between mobile users' mobile devices, between edge servers, and between mobile users' mobile devices and edge servers; The set of edge servers is represented as: ; The mobile device collection is represented as: ; Step 2: Intelligently allocate transmission power based on deep reinforcement learning algorithm. When offloading tasks, offload tasks to edge servers according to different proportions, including: For mobile devices , in each time slot Local processing of computing tasks Processing delay It is expressed as: ； In the formula, for Calculate the number of cycles of one bit of data, For mobile devices In time slot Processable CPU cycles; Mobile device based Offload tasks to each edge server Time and power Calculate the total energy consumption of the mobile device transmission task, the total energy consumption is , ； In the formula, For mobile devices In time slot Offload to edge servers mission; Based on the computing power of the edge server , Edge Server Complete the offload task processing delay for all mobile devices for: ； In the formula, For mobile devices , In time slot Offload to edge servers mission; mobile device Unloading its tasks The total delay consumed is: ； Setting local computation and task offloading in parallel, mobile devices The total computational delay of for: ； In the formula, For mobile devices During the uninstall process and to the edge server The sum of execution time; mobile device The total energy consumption is the energy consumption of local computing and the energy consumption of offloading tasks, expressed as: ； In the formula, For mobile devices The total energy consumption, is the energy consumption of local computing, is the energy consumption of the offloading task; During the offloading process, based on task loss, set at any time slot The tasks generated in need to be completed in the current time slot, and the unfinished ones will be discarded. These tasks are recorded as , expressed as: ； In the formula, For mobile devices In time slot Processable CPU cycles, For mobile devices To the edge server The uplink transmission rate is mobile device In time slot Offload to edge servers mission; Step 3: Establish the RLR-RAFT consensus protocol based on deep learning reward value election, construct a multi-objective optimization function, and complete the collaborative optimization of task offloading and block mining based on the multi-agent deep deterministic policy gradient algorithm; Among them, the consensus process can be carried out synchronously with the tasks generated by the mobile device. By modeling the election phase and block consensus phase of RLR-RAFT, it is concluded that: in the election phase of the mobile device, the leader node in the mobile device needs to execute Voting requests, Election confirmation and The secondary heartbeat signal requires generating blocks and offloading them to the edge server for block verification; The total delay for each mobile device in the consensus process is: ； In the formula, The delay of the leader's transmission status confirmation signal between mobile devices, The delay in producing blocks for the leader, Offloading blocks to edge servers for leaders Uninstall delay, For mobile devices Probability of being a leader and a follower; The total energy consumption of each mobile device during the consensus process is: ； In the formula, The energy consumption of the leader's transmission status confirmation signal between mobile devices, The energy consumption of generating blocks for the leader, Offloading blocks to edge servers Unloading energy consumption; The total delay during the consensus process is: ； In the formula, The delay in sending the block to other edge servers to complete the block verification process, Block confirmation signal transmission delay for each follower in the edge server; The total energy consumption during the consensus process is: ； In the formula, is the energy consumption for block data transmission in the edge server, Energy consumption of the block confirmation reply process; Optimize total delay, total energy consumption and task loss rate. Set the objective function as: , transforming the objective function into a maximization problem, Set each agent in the time slot The reward function in is: ； In the formula, is the total delay, is the total energy consumption; , , They are the non-negative weighted factors of the three indicators: total delay, total energy consumption, and task loss rate; Based on the multi-agent deep deterministic policy gradient algorithm, the collaborative optimization of task offloading and block mining is completed, including: set up As the set of all proxy policies, As the parameters of the corresponding strategy set, each agent updates its parameters To obtain the optimal strategy ; based on Framework, each agent corresponds to 4 networks, including: policy network , value network , two target networks and Used for stable training, running in a continuous action space in a multi-agent environment; in, Through the behavioral network in each time slot Generate deterministic actions in ,and Value Network Assessment The behavior of the agent during training In By calculating the objective function The gradient of is used to update the behavior network; The agent is one of a set of mobile devices; After a certain number of updates, The parameters of the policy network are sent to the user to perform action decisions; The update process includes: Initialize the network parameters of all agents at each time slot in each training phase , each agent chooses an action based on the current state , execute the agent’s actions , the observed reward value and the next state , store its experience samples in the experience replay buffer, the trusted third party obtains the experience samples from each agent and combines them into global experience samples, and starts centralized training on the trusted third party; Randomly sample a size of Batch samples of , updated by minimizing the loss and policy gradient Network parameters and Network parameters,For the objective function of each agent, the target network parameters are updated by soft update,,controlling the synchronization speed between the target network and the main,network.

2. The collaborative task offloading and block mining method of mobile edge computing according to claim 1, characterized in that: Setting up your mobile device To the edge server The wireless channel gain is , ； In the formula, Distance to edge server Base channel gain at one meter, is the distance between the user and the edge server; mobile device To the edge server The uplink transmission rate is expressed as: ； In the formula, For mobile devices To the edge server The transmission power, For server bandwidth, is the background noise variance.

3. The collaborative task offloading and block mining method of mobile edge computing according to claim 2, characterized in that: Also includes: Setting up your mobile device In time slot Offload to edge servers The task is , then the mobile device The tasks processed locally are represented as: ； In the formula, For time slot Tasks generated by mobile devices, For mobile devices In time slot Offload to edge servers , mission; Setting up your mobile device The probability of becoming the leader in an election is: ； In the formula, , , The reward value for each user; Setting up your mobile device The transmission power between and and other mobile devices is , is the interference power of other mobile devices, and the wireless channel gain between users is , ； In the formula, For mobile devices and The distance between ; mobile device and The communication rate between is expressed as: ； In the formula, Bandwidth for mobile devices; Edge Server and The communication rate between is expressed as: ； In the formula, For server bandwidth, For edge servers The transmission power between the edge servers and other edge servers, is the interference power between edge servers, .

Images