A collaborative task offloading method based on blockchain in cloud-edge computing scenarios

Abstract

The present invention discloses a collaborative task offloading method based on blockchain in a cloud-edge computing scenario, the method comprising: obtaining the computing status of the computing tasks and resources that need to be offloaded through a task publishing smart contract; determining the optimal task offloading strategy according to the computing tasks and computing status and the task offloading contract; based on the optimal task offloading strategy, offloading the computing tasks that need to be offloaded to obtain the task processing result. The present invention enables user nodes to make decisions in a distributed manner under the premise of partial observability, ensuring the efficiency of task offloading and the privacy of user offloading strategies, while not relying on a third-party central control node, and constructing a blockchain system to achieve distributed optimal task offloading decisions in CED, as well as to achieve multi-party trust of CED resources and protect the security of their data.

Description

Technical Field

[0001] The present invention relates to the fields of computer technology such as blockchain, reinforcement learning, cloud-edge device systems, and in particular to a blockchain-based collaborative task offloading method in a cloud-edge computing scenario. Background Art

[0002] In recent years, various network applications with latency and computation-intensive characteristics, such as multimedia data processing, artificial intelligence, and the Internet of Things, have quickly exhausted computing and communication resources at the edge. With the rapid development of wireless communication technology and mobile devices, resource utilization in various mobile devices (MDs) has attracted more and more attention from researchers. This new trend has formed a new computing model, namely the cloud-edge device (CED) computing system. The cluster of MDs with elastic resource supply makes CED different from the existing centralized computing framework, while overcoming the problems of resource shortage, low backbone efficiency, and latency. In addition, since the cloud and edge computing servers provide stable resources, CED can also meet the QoS requirements of users.

[0003] Although CED is highly anticipated by academics and industry, there are still problems in bringing this new network computing paradigm into reality. CED is a network system where computing units are distributed in different locations, interconnected through the public Internet, and operated by different groups. This decentralized nature makes the performance of CED heavily dependent on task offloading schemes, that is, scheduling tasks between various nodes with different capabilities.

[0004] Currently, one solution to the CED task offloading problem is to formulate it as a non-convex optimization problem and prove its NP-completeness. Solving this problem requires heuristic algorithms, which are suboptimal and theoretically difficult to guarantee performance under the stochasticity of resources provided by MDs. Another emerging trend is to apply reinforcement learning (RL) tailored for optimal control of dynamic systems. RL-based methods adapt to system changes by formulating the task offloading problem as a Markov decision process (MDP) with the goal of maximizing long-term cumulative utility. By observing the interaction between decisions and the environment, the RL agent iteratively optimizes the policy to obtain the best long-term utility. Since the changes in the system state are unpredictable, ensuring the optimality of the output decision requires RL methods to explore as many changes in the system state as possible when training the model, which is time-consuming. In addition, considering the continuous expansion of the system scale, if relying on a centralized coordinator, processing all the information of the system will inevitably lead to privacy and security issues.

[0005] CED's resources come from multiple parties that may have conflicts of interest, which raises issues of trustworthiness and data security. Blockchain technology is a widely used technology to solve such problems. In the blockchain, each agreement on resource use / provision will be verified by all blockchain participants, a process that is easy to handle and cannot be tampered with. Currently, there are studies trying to use blockchain to build a real incentive mechanism when user devices participate in task offloading. There are also studies that apply blockchain to the data security issues of distributed learning systems. There are also studies that use blockchain to build a federated learning system. Therefore, if blockchain technology is used in combination with CED, it will be expected to solve the problems of multi-party trustworthiness and data security of CED's resources. However, the traditional blockchain consensus process will inevitably bring additional delays and costs to CED resource allocation. Summary of the invention

[0006] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0007] To this end, in view of the fact that the existing technology cannot make optimal task offloading decisions in CED in a distributed manner and that the multi-party trust and data security of CED resources are difficult to ensure, the present invention proposes a computation offloading scheme BC-CED based on blockchain multi-agent collaborative reinforcement learning. This scheme enables user nodes to make decisions in a distributed manner under the premise of partial observability, ensuring the efficiency of task offloading and the privacy of user offloading strategies, while not relying on third-party central control nodes. The participant with the best result is selected to output the block, including resource transaction information. In order to achieve this design goal, we dynamically model the entire system and describe the task offloading problem in the CED system as a partially observable Markov decision process (POMDP). Each blockchain participant uses the RL method to solve this problem. Blockchain participants further compete for block write rights by comparing the optimality of their output strategies. The present invention also proposes a real incentive mechanism to ensure the honesty of computing units in the BC-CED system.

[0008] To achieve the above objectives, the present invention proposes a blockchain-based collaborative task offloading method in a cloud-edge computing scenario, comprising:

[0009] Obtain the computing status of the computing tasks and resources that need to be offloaded through the task publishing smart contract;

[0010] Determine an optimal task offloading strategy according to the computing task and computing status and the task offloading contract;

[0011] Based on the optimal task offloading strategy, the computing task that needs to be offloaded is offloaded to obtain a task processing result.

[0012] The blockchain-based collaborative task offloading method in the cloud-edge computing scenario according to an embodiment of the present invention may also have the following additional technical features:

[0013] Furthermore, in one embodiment of the present invention, after obtaining the task processing result, the method further includes: outputting a reward to the resource provider according to the reward contract and the task processing result.

[0014] Furthermore, in one embodiment of the present invention, the method of determining the optimal task offloading strategy based on the computing task and the computing status and the task offloading contract includes: determining the task environment information based on the computing task and the computing status, and obtaining the corresponding strategy probability state based on the task environment information and the strategy network; obtaining the task offloading strategy based on the strategy probability state and the task offloading contract, and determining the optimal task offloading strategy based on the consensus protocol.

[0015] Furthermore, in one embodiment of the present invention, after obtaining the task offloading strategy according to the strategy probability state and the task offloading contract, it also includes: verifying whether the task offloading strategy meets the preset conditions based on the task offloading contract; if so, storing the task offloading strategy, and performing sorting voting according to the value function of the task offloading strategy, and obtaining the leader node according to the sorting voting results; wherein the leader node is used to generate blocks.

[0016] Furthermore, in one embodiment of the present invention, based on the optimal task offloading strategy, the computing task that needs to be offloaded by the task issuing unit is offloaded to obtain the task processing result, including: transmitting the computing task that needs to be offloaded to the resource provider based on the optimal task offloading strategy; and using local resources to settle the computing task to obtain the task execution result, and completing the task offloading based on the task execution result.

[0017] To achieve the above-mentioned purpose, the present invention proposes, on the other hand, a blockchain-based collaborative task offloading system in a cloud-edge computing scenario, comprising:

[0018] The task publishing module is used to obtain the computing status of the computing tasks and resources that need to be offloaded through the task publishing smart contract;

[0019] A strategy determination module, used to determine an optimal task offloading strategy according to the computing task and computing state and the task offloading contract;

[0020] The task offloading module is used to offload the computing tasks that need to be offloaded based on the optimal task offloading strategy to obtain the task processing result.

[0021] The collaborative task offloading method and system based on blockchain in the cloud-edge computing scenario of the present invention plays a core role through blockchain, realizing the functions of task offloading, resource use mediation and incentive. The method enables participants to reach a consensus by solving the task offloading problem by modifying the blockchain consensus process. Considering the offloading problem of the computing power and network performance of the candidate nodes, the method allows each participant to apply a reinforcement learning-based method to solve the problem, and to compete for the right to output the block by comparing the offloading strategy performance in the next time period and the best acceptance strategy as the offloading scheme. Finally, by establishing a real incentive mechanism, resource contribution is encouraged, and the computing unit is required to act honestly and output the correct task results, otherwise it will be punished. The present invention enables user nodes to make decisions in a distributed manner under the premise of partial observability, ensuring the efficiency of task offloading and the privacy of user offloading strategies, while not relying on third-party central control nodes, and building a blockchain system to achieve distributed optimal task offloading decisions in CED and realize multi-party trust of CED resources and protect the security of its data.

[0022] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0024] Figure 1 It is a flowchart of a collaborative task offloading method based on blockchain in a cloud-edge computing scenario according to an embodiment of the present invention;

[0025] Figure 2 This is a diagram of the blockchain-enabled BCD system architecture according to an embodiment of the present invention;

[0026] Figure 3 A schematic diagram of a computing queue and a link queue in a cloud-edge scenario according to an embodiment of the present invention;

[0027] Figure 4 is an overall flow chart of BC-BED according to an embodiment of the present invention;

[0028] Figure 5 A schematic diagram of an Actor-Critic network architecture according to an embodiment of the present invention;

[0029] Figure 6 This is a schematic diagram of the structure of a collaborative task offloading system based on blockchain in a cloud-edge computing scenario according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] The following describes a blockchain-based collaborative task offloading method and system in a cloud-edge computing scenario proposed according to an embodiment of the present invention with reference to the accompanying drawings.

[0033] Figure 1 It is a flowchart of a blockchain-based collaborative task offloading method in a cloud-edge computing scenario according to an embodiment of the present invention.

[0034] like Figure 1 As shown, the method includes but is not limited to the following steps:

[0035] S1, obtain the computing status of the computing tasks and resources that need to be offloaded through the task publishing smart contract;

[0036] S2, determine the optimal task offloading strategy based on the computing task, computing status and task offloading contract;

[0037] S3, based on the optimal task offloading strategy, offload the computing tasks that need to be offloaded to obtain the task processing results.

[0038] The following is a detailed description of the blockchain-based collaborative task offloading method in the cloud-edge computing scenario of an embodiment of the present invention in conjunction with the accompanying drawings.

[0039] It is understood that some terms involved in the present invention are briefly explained as follows:

[0040] The structure of BC-CED is as follows Figure 2As shown in the figure, it mainly consists of three layers. The application layer includes various network applications running on top of BC-CED. These applications act as clients of BC-CED, subscribe to their resource requests and offload tasks. The blockchain layer plays a core role in the functions of BC-CED, including decision-making, resource management, and incentives for task offloading. In addition, the modified consensus mechanism proof-of-optimum (OOP) is also implemented in this layer, which enables blockchains to compete for the right to output blocks by optimizing BC-CED's task offloading scheme. The resource layer keeps computing units contributing their resources to support BC-CED's task offloading.

[0041] Computing unit: (1) Data center or edge computing server, which has stable and powerful computing power, but is expensive in resource usage; (2) Mobile or portable device, which has limited and random resource supply, but is large in quantity and cheap in resource usage.

[0042] User: A network application or client offloads its tasks to BC-CED by specifying its resource needs and QoS requirements.

[0043] Transaction: When a task user issues a resource request in BC-CED, a transaction including resource and task QoS requirements will be generated, and the user agent will be responsible for the task. After the decision to offload the task is made, the information of the computing unit assigned to the task will also be recorded in the corresponding transaction. When the computing unit processes the task and returns the result to the task user, the reward for completing the task offloading will also be written into the transaction.

[0044] Blockchain: The transactions defined above are processed in the blockchain. Similar to the current blockchain, the output blocks are competed by achieving a pre-given purpose.

[0045] Optimizer: Selects from blockchain members to generate task offloading strategies through RL models. In addition, the optimizer compares the results of each person and selects the node with the best estimated performance as the leader for the next time period. The leader outputs a block and receives a block reward.

[0046] Compute Units (CUs): contribute their resources to process the offloaded tasks.

[0047] User Agent: This entity provides an interface for users to interact with the blockchain and CU. It acts as a transit station to forward information about tasks and CUs to the blockchain. Its main functions, including resource management, task offloading, and incentives, are deployed on the blockchain. In addition, this user agent can be a trusted component provided by a CED operator or distributedly operated by multiple entities (such as blockchain workers).

[0048] Rewards: Subscribers are required to pay for resource usage.

[0049] A computational offloading scheme for multi-agent collaborative reinforcement learning based on blockchain includes the following steps:

[0050] The solution mainly consists of two parts: high-performance task offloading design and blockchain system. The steps of high-performance task offloading are as follows:

[0051] Communication model: In this invention, the network topology of CED is modeled as an undirected graph G(v,e), where v and e are the sets of nodes and links in the cloud-edge system, respectively. Vu, Vc, Ve, Vm, and Vr represent users, cloud servers, edge servers, mobile devices, and network routers, respectively. Cloud servers, edge servers, and mobile devices have computing resources available for task offloading. The network link state is a virtual dynamic queue V l (t), the capacity of the queue indicates the maximum capacity of the link. The length of the real-time queue is the load of the current link. At the same time, the length of the link queue is stable in each time period. In order to ensure the smooth flow of the link, the bandwidth of the link should be greater than the arrival rate of the data.

[0052] Computation model: Node V k ,k=c,e,m has computing resources available for task offloading and can process random tasks generated by users. For each node u, the computing resources can be used u (t) indicates that. At the same time, F c (t)>>F e (t)>>F m (t) The computing capacity of each computing node is represented by the number of CPU clocks in each time interval. For cloud servers and edge servers, the computing capacity is constant. They will provide all their computing resources for computing task offloading at any time. For mobile devices (MD), their computing capacity is time-varying. Mobile devices can only use their idle resources at the current moment for computing offloading. At the same time, each computing node has a task queue Q u (t), the capacity of the queue is the computing capacity of the server, and the length of the queue is the current load of the server. For task i unloaded to computing node u at time t, its computing delay is P i,u =(Q u (t)+s i (t)) / F u (t).

[0053] It is understandable that the main purpose of task offloading is to minimize the overall delay of task execution and the overhead of resource usage. This problem is actually a POMDP problem. This POMDP problem can be represented by a tuple of the following parts:

[0054] State space: S is the state space in the CED scenario. This solution defines S = {Q, V, C, N}. Q is the value set of the backlog of tasks on each computing node. V is the value set of the virtual queue lengths of all links. C is the set of available bandwidths of all links. N is the value set of Vk (k = c, e, m).

[0055] Action space: Each agent needs to decide which computing node to offload the task to and the data rate of the task. For ease of description, this scheme assumes that each user generates a task at each t. The action (also called decision, strategy) of the agent corresponding to task i at t can be represented by a 2-tuple (w i (t),x i (t)) represents, where w i (t) is a vector representing the computing node to which task i is offloaded. i (t) represents the data rate transmitted by the task.

[0056] Instantaneous reward: r(w i (t),x i (t)) represents the direct reward given by the environment to the user agent after making a decision.

[0057] Observation space: Each user agent has only partial information about system dynamics. i , rather than having a system

[0058] The present invention uses the RL method to solve this problem using the Actor-Critic network, where the Critic network estimates the strategy based on the historical information and feedback rewards at each time step. The Actor network generates actions based on the parameters optimized by the values ​​calculated by the Critic network. The Actor-Critic network is trained:

[0059] Actor network: This solution defines the output strategy as π θ (w(t),x(t)|S t ), where θ is the parameter of the actor network, S t The network is in state t. The network iteratively updates the parameters θ through a gradient-based method.

[0060] Critic network: The critic network uses the TD(λ) method to optimize the advantage function.

[0061] According to the above design, AC optimizes the task offloading decision through subsequent iterations at each time t until the iteration standard is reached. The blockchain system part is the core part of the present invention, and the entire task offloading process of the present invention depends on the blockchain system.

[0062] Its important functions include maintaining the information of resource providers in BC-CED; verifying and recording task subscription information; managing resource allocation in BC-CED; and allocating rewards / penalties based on task completion. In order to complete the above functions, the present invention designs four smart contracts, namely, CU registration contract, task release contract, task unloading contract and reward contract, to enable blockchain. The specific components are as follows:

[0063] Resource Management: The computing resources of CUs in BC-CED can be geographically distributed and randomly provided. Therefore, how to manage resources and update the resource status of CUs in a timely manner is crucial to the performance of task offloading.

[0064] Task offloading: Based on the designed smart contract, this paper proposes a task offloading strategy to optimize long-term utility. We divide the task offloading of each time slot t in BC-CED into three stages: observation stage, decision stage, and offloading stage.

[0065] Incentive Mechanism: Since the present invention utilizes resources from multiple third parties, an incentive mechanism can be used to encourage participants to participate in the work honestly and actively. Under the incentive mechanism, rewards and punishments for task transfer participants are measured by their contributions. The incentive mechanism mainly includes two parts. (1) Rewards: rewards are set for CUs that provide BC-CED resources. (2) Punishments: penalties are imposed on CUs that fail to complete the results or provide false results.

[0066] In the above resource management phase, during time slot t, when CU j When a joining request is sent to the user agent, the CU registration contract is called to generate an ID and record resource information such as CPU frequency and memory size. This information is maintained by the blockchain and recorded in the blocks generated within t to ensure the credibility of resource supply. Once a CU joins BC-CED, it updates the available resources at every t to ensure that the task offloading scheme can always make decisions based on the latest CED resource status. Therefore, for any optimizer in BC-CED, the resource status can be obtained by accessing the resource information of all CUs in the block. When a CU needs to exit the CED system, an exit notification will be sent to the BC-CED user agent, and the CU registration contract will add a new information to the block, setting the CU's available resources to zero.

[0067] In the above task offloading phase, the optimizer in the blockchain observes the tasks in the arrival queue and the resource requirements and raw data volume of each task i. At the same time, the optimizer accesses the system status, including the task backlog Q u(t), the computing power of each CU Fu(t), and the end-to-end delay of each CU-user pair. In practice, it is very time-consuming to access all end-to-end delay pairs, so the present invention uses the sum of the delay from the task user to the nearest base station and the delay from the CU to the nearest base station to approximate the end-to-end delay.

[0068] In the decision-making stage of the above-mentioned task offloading, the present invention uses a customized proof of optimality (POO) consensus mechanism to complete it. The POO consensus incorporates task offloading optimization into the consensus mechanism, attempting to reuse the computing and communication resources owned by the blockchain while supporting high-performance task offloading. Its main steps include optimizer election, task offloading optimization, leader selection, and block establishment.

[0069] Optimizer Selection: Similar to traditional DPoS, at time slot t, the blockchain in POO selects a group of optimizers from the set based on account balances.

[0070] Task offloading optimization: Each optimizer uses the AC network mentioned above to get the output decision based on the input. Considering the large scale of the system and the strict constraints on the submission deadline of the offloading action, the optimizer with limited capabilities may not be able to process the state of the entire system horizontally, and can only use partial state information to optimize the offloading. Optimizers with strong computing power and communication capabilities can provide better task offloading solutions because they can explore as much environmental information as possible.

[0071] Leader selection: For each optimizer, as long as the optimal decision is made within a given period, the result will be broadcast to all optimizers, otherwise the result will be discarded. The optimizers that receive the broadcast will check whether the decision meets the optimization conditions. If it does, it will be added to the feasible list, and the optimizer with the highest reward ranking in the feasible list will be voted as the leader. The optimizer with the best performance within t will be selected as the leader.

[0072] Block creation: After the leader is generated, the leader constructs blocks containing the ones generated in t. Each block contains up to M transactions, and once the size limit is reached, a new block containing subsequent transactions will be generated. The new block will be broadcast to other optimizers, who will verify the transactions in the new block. In BC-CED, if more than 2 / 3 of the optimizers agree after verification, the new block is accepted. Once the new block is accepted, the rewards will be distributed to the optimizers. The leader p gets half of the reward, and the rest of the reward is evenly distributed to other optimizers.

[0073] In the above incentive mechanism: the self-reward transfer scheme of the present invention is as follows:

[0074] Freeze: Once a subscriber issues a "cancel delegation" request, the funds owed to the corresponding user agent will be recorded and frozen.

[0075] Reward: The subscriber confirms whether the task is successfully completed. If the task is completed, the reward will be automatically unlocked and transferred to the corresponding CU's account.

[0076] Punishment: In CED, CUs can be devices carried by humans, and CUs may lose trust in the process of resource provision, affecting system performance and sustainability. Therefore, it is necessary to punish CUs with dishonest behavior. The present invention introduces the concept of deposit. After the CU confirms the processing of the assigned task, the deposit is paid according to the resources it provides and its reliability. If the task is completed, the deposit will be waived and returned to the CU's wallet. Otherwise, the deposit will be confiscated to compensate for the loss of CED.

[0077] Further, Figure 3 The schematic diagram of the computing node calculation and network link transmission of the present invention. For any network link, its capacity is cl, that is, the sum of the transmission speeds of all tasks in the link in each time period shall not exceed cl. For computing node u, the execution of its computing tasks is an execution queue Qu(t). The node computing power is quantified by the CPU frequency as Pu(t). For computing nodes, especially mobile device nodes, the computing power Pu(t) available for task offloading is dynamic. The computational complexity of each task i is quantified by the number of CPU clocks it needs to consume as si. Suppose the set of tasks unloaded to node u in time period t is Au(t), then the state transition equation of the execution queue of the computing node is:

[0078]

[0079] Figure 4 This is a flowchart of the computational offloading solution for blockchain-based multi-agent collaborative reinforcement learning of the present invention. The specific process is as follows:

[0080] Resource request. A network application or client accesses the BC-CED system as a user, and proposes its resource needs and QoS requirements in the form of transactions, unloading its tasks to the BC-CED system. Afterwards, a corresponding user agent connects the generated transaction information to the blockchain system to wait for request confirmation and resource task allocation.

[0081] Strategy generation. After the resource request transaction is confirmed, voting is conducted based on the balance in the blockchain, and a set of optimizers to perform the task is selected based on the voting results. The optimizer in the blockchain will obtain the current observable environment information, and generate the corresponding strategy probability distribution through its own strategy network based on the partial environment information it observes. The architecture of the strategy network can be composed of multiple architectures. This embodiment adopts Figure 5 The multi-layer perceptron shown in the figure is combined with an LSTM network as the architecture of our policy network and evaluation network, which can effectively adapt to time series data.

[0082] Strategy optimization. Once the optimizer comes up with a task offloading strategy before the deadline, it broadcasts it to other optimizers. Other optimizers that receive the broadcast need to verify whether the strategy meets the constraints. If the strategy is feasible, it is included in the feasible list. The optimizers sort and vote according to the value function of the executed strategy and select the optimizer with the best performance as the leader.

[0083] Block update. After the leader is generated, the leader constructs a block containing the transactions generated in t. Each block contains at most M transactions, and once the size limit is reached, a new block containing subsequent transactions will be generated. The new block will be broadcast to other optimizers, who will verify the transactions in the new block. In BC-CED, if more than 2 / 3 of the optimizers agree after verification, the new block is accepted. Once the new block is accepted, the rewards will be distributed to the optimizers. The leader p gets half of the reward, and the rest of the reward is evenly distributed to other optimizers.

[0084] Task execution. After the optimal task offloading strategy is generated, the user agent passes the task to the task node. When the task arrives at the computing node, the expenses of the user agent of the user who issued the task will be recorded and frozen to prevent the user from not paying after the CU completes the task. At the same time, in order to prevent malicious CUs from deliberately providing incorrect task execution results to damage the stability and sustainability of the entire system, each CU will have a portion of its balance frozen as a deposit after receiving the task. Afterwards, the computing node uses its local resources to settle according to the assigned task and returns the task result to the user agent. The user agent then passes the received task result to the user to complete the entire task offloading process.

[0085] Rewards and penalties. After a user's task is completed, the user confirms whether the task is completed correctly. If the task is completed correctly, the corresponding CU's frozen deposit will be returned to the CU's account, and the user's frozen reward Ri(t) will be automatically unlocked and transferred to the corresponding CU's account. Moreover, the reward R(t)>C(t) for each CU that provides the correct task execution result, where C(t) is the cost of completing the task. Otherwise, the corresponding CU will not only not receive any reward, but its frozen deposit will also be confiscated by the system to compensate for the loss of BCD.

[0086] In summary, in the existing technical solutions, many studies on computing task offloading are mainly carried out for a single cloud computing or a single edge computing, considering static, homogeneous computing services, and lacking adaptability to dynamic heterogeneous cloud-edge systems. At the same time, the existing reinforcement learning-based computing offloading solutions have two characteristics: strategy and training centralization and search based on reward value. This makes the existing technical solutions have problems such as single point failure, easy to fall into local optimal rather than global optimal solution, and easy to be seized by some malicious participants for more resources. CED's resources come from multiple parties that may have conflicts of interest. How to protect the security of multiple parties' data during task offloading is also a problem that the current solutions have not solved.

[0087] The present invention proposes a computational offloading scheme for multi-agent collaborative reinforcement learning based on blockchain. The scheme enables user nodes to make decisions in a distributed manner under the premise of partial observability, ensuring the efficiency of task offloading and the privacy of user offloading strategies, while not relying on third-party central control nodes. The participant with the best result is selected to output the block, including resource transaction information. It incorporates the task offloading process into the consensus mechanism of the blockchain to improve resource utilization and uses the collective intelligence of blockchain participants to solve complex CED offloading problems. In addition, a real incentive mechanism is proposed to intelligently issue rewards or penalties to computing units to ensure the honesty of computing units in the BC-CED system. This scheme has the advantages of strong adaptability, high generalization, high offloading efficiency and decentralization.

[0088] In order to implement the above embodiment, Figure 6 As shown, this embodiment also provides a blockchain-based collaborative task offloading system 10 in a cloud-edge computing scenario. The system 10 includes: a task publishing module 100, a policy determination module 200 and a feature task offloading module 300.

[0089] The task publishing module 100 is used to obtain the computing status of the computing tasks and resources that need to be offloaded through the task publishing smart contract;

[0090] A strategy determination module 200, configured to determine an optimal task offloading strategy according to the computing task and computing state and the task offloading contract;

[0091] The task offloading module 300 is used to offload the computing task that needs to be offloaded based on the optimal task offloading strategy to obtain a task processing result.

[0092] Furthermore, after the above task offloading module 300, it also includes:

[0093] The reward module is used to output rewards to resource providers based on the reward contract and task processing results.

[0094] Furthermore, the above-mentioned strategy determination module 200 includes:

[0095] A strategy probability distribution subunit, used to determine task environment information according to the computing task and computing state, and obtain a corresponding strategy probability state according to the task environment information and the strategy network;

[0096] The optimal strategy determination subunit is used to obtain the task offloading strategy according to the strategy probability state and the task offloading contract, and determine the optimal task offloading strategy according to the consensus protocol.

[0097] Furthermore, after the above-mentioned strategy probability distribution subunit, it also includes: a block generation subunit, which is used to:

[0098] Verify whether the task offloading strategy meets the preset conditions based on the task offloading contract;

[0099] If it is satisfied, the task offloading strategy is stored, and a ranking vote is performed according to the value function of the task offloading strategy, and the leader node is obtained according to the ranking vote result; wherein, the leader node is used to generate blocks.

[0100] Furthermore, the task offloading module 300 is also used for:

[0101] Transmitting the computing task information that needs to be offloaded to the resource provider based on the optimal task offloading strategy; and

[0102] The computing tasks are settled using local resources to obtain the task execution results, and the computing tasks are unloaded based on the task execution results.

[0103] According to the blockchain-based collaborative task offloading system in the cloud-edge computing scenario of an embodiment of the present invention, user nodes can make decisions in a distributed manner under the premise of partial observability, thereby ensuring the efficiency of task offloading and the privacy of user offloading strategies, while not relying on third-party central control nodes. A blockchain system is built to achieve distributed optimal task offloading decisions in CED, realize multi-party trust of CED resources, and protect the security of its data.

[0104] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0105] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0106] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A collaborative task offloading method based on blockchain in a cloud-edge computing scenario, characterized in that: The following steps are involved: Obtain the computing status of the computing tasks and resources that need to be offloaded through the task publishing smart contract; Determine an optimal task offloading strategy according to the computing task and computing status and the task offloading contract; Based on the optimal task offloading strategy, the computing task that needs to be offloaded is offloaded to obtain a task processing result; Determining the optimal task offloading strategy according to the computing task, computing status and task offloading contract includes: Determine the task environment information according to the computing task and the computing state, and obtain the corresponding strategy probability state according to the task environment information and the strategy network, including: the optimizer in the blockchain obtains the current observable environment information, and generates the corresponding strategy probability distribution through its own strategy network according to the part of the environment information observed by itself, wherein the optimizer is selected from the blockchain members to generate the task offloading strategy through the RL model; Obtaining a task offloading strategy according to the strategy probability state and the task offloading contract, and determining an optimal task offloading strategy according to a consensus protocol, wherein the consensus protocol is an optimality proof consensus mechanism; After obtaining the task offloading strategy according to the strategy probability state and the task offloading contract, the method further includes: Verify whether the task offloading strategy meets the preset conditions based on the task offloading contract; If satisfied, the task offloading strategy is stored, and a ranking vote is performed according to the value function of the task offloading strategy, and a leader node is obtained according to the ranking vote result; wherein the leader node is used to generate a block; The step of offloading the computing tasks that need to be offloaded by the task issuing unit based on the optimal task offloading strategy to obtain the task processing result includes: transmitting the computing task to be offloaded to the resource provider based on the optimal task offloading strategy; and The computing tasks are settled using local resources to obtain task execution results, and task offloading is completed based on the task execution results.

2. The method according to claim 1, characterized in that After obtaining the task processing result, the method further includes: According to the reward contract and the task processing results, output the reward for the resource provider.

3. A blockchain-based collaborative task offloading system in a cloud-edge computing scenario, characterized in that: include: The task publishing module is used to obtain the computing status of the computing tasks and resources that need to be offloaded through the task publishing smart contract; A strategy determination module, used to determine an optimal task offloading strategy according to the computing task and computing state and the task offloading contract; A task offloading module, used for offloading the computing task to be offloaded to obtain a task processing result based on the optimal task offloading strategy; The strategy determination module includes: The strategy probability distribution subunit is used to determine the task environment information according to the computing task and the computing state, and obtain the corresponding strategy probability state according to the task environment information and the strategy network, which includes: the optimizer in the blockchain obtains the current observable environment information, and generates the corresponding strategy probability distribution through its own strategy network according to the part of the environment information observed by itself, wherein the optimizer is selected from the blockchain members to generate the task offloading strategy through the RL model; An optimal strategy determination subunit, used to obtain a task offloading strategy according to the strategy probability state and the task offloading contract, and determine an optimal task offloading strategy according to a consensus protocol; After the strategy probability distribution subunit, it also includes: a block generation subunit, which is used to: Verify whether the task offloading strategy meets the preset conditions based on the task offloading contract; If satisfied, the task offloading strategy is stored, and a ranking vote is performed according to the value function of the task offloading strategy, and a leader node is obtained according to the ranking vote result; wherein the leader node is used to generate a block; The task offloading module is further used for: Transmitting the computing task information that needs to be offloaded to the resource provider based on the optimal task offloading strategy; and The computing task is settled using local resources to obtain a task execution result, and the computing task is unloaded based on the task execution result.

4. The system according to claim 3, characterized in that After the task unloading module, it also includes: The reward module is used to output the reward to the resource provider according to the reward contract and the task processing result.

Images