A multi-operator edge resource scheduling method supporting mobile blockchain off-chain computing

Abstract

The application discloses a kind of multi-operator edge resource scheduling methods supporting mobile blockchain off-chain computing, to maximize the long-term expected utility of user, the application establishes Markov game model by considering block size and the task completion delay of blockchain producer, and under the condition that the model is solvable, the short-term optimization problem consistent with long-term problem is proposed. The offloading decision of each blockchain producer and block size can be learned independently by the designed neural network model, wherein the average field reinforcement learning is used to realize the edge network resource deployment scheme under incomplete information, the action reorganization and the construction of neural network are realized, and the utility of blockchain producer is optimized. Theoretical and experimental results show that, compared with other representative algorithms, the algorithm designed by the application performs better in terms of average utility and average reward of blockchain producer. The application provides a new method for deploying multi-operator edge computing resources in mobile blockchain off-chain computing.

Description

Technical Field

[0001] This invention relates to a multi-operator edge resource scheduling method that supports off-chain computing in mobile blockchain, and more particularly to a network edge resource scheduling method based on mean field reinforcement learning. Background Technology

[0002] Blockchain, due to its security and distributed management features, has been widely applied in various network environments, including smart grids and the Internet of Things (IoT). Proof-of-Work (PoW) is a commonly used consensus mechanism in blockchain, where blocks are generated through the PoW of blockchain producers. However, its high computational power requirements limit its application in mobile environments, especially in mobile blockchain applications. Fortunately, the development of edge computing and hashing technologies has greatly supported the application of PoW in mobile environments. Blockchain producers can offload the computational tasks related to the PoW problem to edge servers, i.e., design a reasonable off-chain computing scheme. However, most current off-chain computing schemes are based on edge resource scheduling by a single operator, where all edge servers in the network have the same configuration and are uniformly managed by a single operator. Blockchain producers only need to decide whether to offload the computational tasks related to the PoW problem. However, in reality, there are often multiple operators, and the edge servers managed by different operators have different configurations, such as different computing power and different prices for renting computing power resources. Therefore, how to optimize the configuration of edge server computing resources under multi-operator management to meet the off-chain computing needs of blockchain producers and improve the utility of blockchain producers requires further exploration by researchers. Summary of the Invention

[0003] This invention addresses some shortcomings of existing technologies by proposing a multi-operator edge resource scheduling method that supports off-chain computing in mobile blockchains. It utilizes mean-field reinforcement learning to implement an edge network resource deployment scheme under incomplete information conditions, while simultaneously optimizing the utility of blockchain producers. This provides a new method for multi-operator edge computing resource deployment that supports off-chain computing in mobile blockchains.

[0004] Therefore, the technical solution adopted by the present invention is a multi-operator edge resource scheduling method supporting off-chain computing of mobile blockchain, comprising the following steps:

[0005] (1) Construct a network model and a blockchain producer computational offloading model to determine the utility of blockchain producers;

[0006] (2) Construct a Markov game model and establish a reward maximization optimization problem based on the utility of blockchain producers;

[0007] (3) Based on mean field theory, action recombination and neural network model establishment are carried out;

[0008] (4) Under the local information state, construct the agent's online learning strategy based on step (3).

[0009] The above technical solution has the following advantages:

[0010] This invention proposes a resource scheduling method for off-chain computing of mobile blockchains with multiple operators. To maximize the long-term utility of users, this invention establishes a Markov game model by considering block size and task completion delay of blockchain producers, and proposes a short-term optimization problem consistent with the long-term problem under the condition that the model is solvable. The offloading decision and block size of each blockchain producer can be learned independently through a neural network model, in which mean-field reinforcement learning is used to realize the deployment scheme of edge network resources under incomplete information, realize action recombination and neural network construction, and simultaneously optimize the utility of blockchain producers. Theoretical and experimental results show that, compared with other representative algorithms, the algorithm designed in this invention performs better in terms of average utility and average revenue of blockchain producers. This invention provides a novel method for deploying multi-operator edge computing resources for off-chain computing of mobile blockchains. Attached Figure Description

[0011] Figure 1 For system model;

[0012] Figure 2 This is a schematic diagram of online algorithm training based on learning.

[0013] Figure 3 This is a performance comparison chart of the method of this invention and three other algorithms under different numbers of blockchain producers, showing the average utility and average revenue of blockchain producers.

[0014] Figure 4 This is a performance comparison chart of the method of the present invention with three other algorithms, showing the average blockchain producer utility and average revenue obtained under the management of a service provider for different numbers of edge servers.

[0015] Figure 5 This is a performance comparison chart of the method of this invention with three other algorithms under different resource requirements, showing the average utility and average revenue of blockchain producers.

[0016] Figure 6 This is a performance comparison chart of the method of the present invention and three other algorithms on different server computing units, showing the average utility and average revenue of blockchain producers.

[0017] Figure 7 This is a performance comparison chart of the method of this invention and three other algorithms under edge computing units (ECUs) of different values, showing the average utility and average revenue of blockchain producers.

[0018] Figure 8 This is a comparison chart of the convergence time of the method of the present invention under different numbers of blockchain producers;

[0019] Figure 9 This is a loss trend diagram of the strategy network of the present invention at different training stages. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below.

[0021] This invention provides a multi-carrier edge resource scheduling method that supports off-chain computing in mobile blockchains. The method includes:

[0022] Step 1: Construct a network model and a blockchain producer computation offloading model to determine the utility of blockchain producers.

[0023] This invention constructs a two-dimensional regional system model. Public blockchain applications initiated by the blockchain management terminal are implemented through a mobile edge computing network. When mobile users participate in the blockchain generation process, they offload their computing tasks to edge computing servers managed by different service providers. As users move, the service request data dynamically changes, such as... Figure 1 As shown, this system includes K service providers, M edge servers, and N blockchain producers. Blockchain producer n operates at a constant speed v throughout the system. n The system can move and offload Proof-of-Work (PoW) related computational tasks to edge computing servers, or process these tasks locally. The latency in completing a computational task comprises three parts: travel latency, waiting latency, and processing latency. Blockchain producer n enters edge server M. ki Travel delay within the communication range Represented as:

[0024]

[0025] in This represents the relationship between blockchain producer n and edge server M. ki The distance between them. R represents the distance between edge servers M. ki The communication radius.

[0026] If blockchain producer n has already made the offloading decision on edge server M ki Within the communication coverage area, its travel delay is 0. When blockchain producer n reaches its coverage area, it can directly upload its computing tasks to the edge server M. ki Edge server M kiUpon receiving a computation task, it is placed in a waiting queue and processed in a First-In-First-Out (FIFO) order. The typical waiting time for a computation task is... The M / G / 1-based queuing system is represented as follows:

[0027]

[0028] in, Represents edge server M ki The target task intensity is reached in phase t; Represents edge server M ki Average task processing latency; σ 2 This represents the variance of task processing delay.

[0029] Task processing delay of blockchain producer n It can be represented as:

[0030]

[0031] Among them, F t This represents the complexity of the Edge Computing Unit (ECU) required to solve the proof of work in phase t. Represents edge server M ki Maximum number of ECUs.

[0032] If the computation task is processed locally by blockchain producer n, there is no waiting delay; there is no local processing delay. Calculated by the following formula:

[0033]

[0034] in, Represents the maximum number of ECUs for blockchain producer n. The total latency for completing the computational task of blockchain producer n in phase t. It can be represented as:

[0035]

[0036] in, Indicates whether the computational task of blockchain producer n is processed locally; Indicates whether the task has been offloaded to the edge server M ki superior.

[0037] The utility of blockchain producer n in stage t It is the sum of the fixed part and the variable part, expressed as:

[0038]

[0039] in ε represents the fixed positive gain of the blockchain producer; ε represents the variable gain factor. This represents the block size of blockchain producer n in stage t, i.e., the number of transactions in the block. The cost of blockchain producer n. Represented as:

[0040]

[0041] in, Indicates whether the task is offloaded to the edge server M. ki The binary value; This indicates the price per unit of ECU usage per unit of time. When... At that time, blockchain producer n offloads its computing tasks to edge server M. ki Otherwise, the computational task is processed locally without additional overhead.

[0042] Because blockchain producers need to move within the communication coverage area of ​​the corresponding server to upload their tasks, the probability of successfully mining a block depends not only on the hash rate (which can be replaced by ECUs) but also on the travel latency. This invention considers the probability that blockchain producer n will successfully mine a block... It is inversely proportional to the task completion time, that is:

[0043]

[0044] in, This represents the total latency in completing the computational task of blockchain producer n in stage t. The utility of blockchain producer n in stage t. It can be approximated as:

[0045]

[0046] in, This represents the cost of blockchain producer n; This represents the probability of blockchain producer n being isolated in stage t; This represents the probability that blockchain producer n will successfully mine a block.

[0047] Step 2: Construct a Markov game model and establish a reward maximization optimization problem based on the utility of blockchain producers.

[0048] We formulate the computational offloading problem for blockchain producers as a Markov game model, aiming to maximize the long-term expected utility of blockchain producers by considering offloading decisions, server selection, and block size. In step 1), blockchain producer n can offload computational tasks related to the PoW problem to an edge server or process them locally. Blockchain producers do not have access to the global state; in a state of local information, they are perfectly competitive and need to independently adjust their offloading modules and block size to maximize their own utility.

[0049] In the constructed Markov game model, the strategy π = {π1, ..., π} is required. n ,…,π N This approach maximizes the long-term expected cumulative utility of all blockchain producers. Therefore, the optimization problem of this model can be formulated as:

[0050]

[0051]

[0052]

[0053]

[0054]

[0055]

[0056] in, This represents the cost of blockchain producer n; This represents the reward received by blockchain producer n in stage t; This represents the budget of blockchain producer n in stage t; This indicates whether the task is offloaded to the edge server M. ki The binary value of γ. t Let T represent the expected reward discount factor at stage t; E represents the expected reward; T represents the expected reward. b R represents the average time interval between the generation of two blocks; κ represents the effect of block size on block propagation delay; ′ Blockchain producer n moves to edge server M ki Required distance; D1 represents the distance that blockchain producer n moves out of edge server M. ki The required time and task are determined by the edge server M. ki The time difference required for processing.

[0057] The aforementioned problem of maximizing the profits of blockchain producers can be transformed into a Markov game problem, which can be represented by tuples.<S,O,A,R,P,N,γ> The meanings of each element are as follows:

[0058] State S: Represents the state information of the established Markov game model, denoted as... S1 represents the status of service providers, including the total number, resource value, and managed edge servers; S2 represents the status of edge servers, including location, computing power, coverage radius, task arrival rate, and average task processing latency; S3 represents the status of blockchain producers, including the speed, trajectory, budget, and computing power of blockchain producer actions; and S4 represents the status of the blockchain, including the complexity of the block generation problem and the expected block generation interval.

[0059] Observation state O: Blockchain producers in the system cannot observe system state S; they can only observe partial information, in which... It is the observed state of blockchain producer n, including the state S1 of the service provider, the state S2 of the edge server, and the state S4 of the blockchain.

[0060] Action A: The set of blockchain producer actions is represented as follows Where a t It is a joint action of all intelligent agents. It refers to the action of agent n at stage t.

[0061] The state transition probability P is represented as P: S×A×S→[0,1]. Based on probability... and action a t The system state changes from s t Jump to s t+1 .

[0062] The reward function R can be expressed as: Representing blockchain producer n in state s t Next action The instantaneous reward obtained afterward. Based on maximizing the utility of each blockchain producer in the long run, the definition is... in Let n be the current attribute of blockchain producer n.

[0063] Step 3: Based on mean field theory, perform action recombination and establish a neural network model.

[0064] In the system considered in this invention, blockchain producers decide whether to offload computational tasks related to the proof-of-work puzzle at each stage based on their budget and utility. If blockchain producer n decides to offload its tasks, the timing and location of the offloading should be determined. This process can be formed by a Markov game and represented by the long-run expected reward maximization problem shown in problem P1.

[0065] To solve problem P1, we need to calculate... and There are three values, but these three values ​​are interdependent and cannot be directly obtained through a neural network because a single neural network cannot reflect their interrelationships. The interdependence between the coupling parameters poses three challenges to the optimization problem described above. First, the offloading decision is dependent on the opponent's actions. Whether a blockchain producer can win in the proof-of-work stage depends not only on individual decisions but also on the opponent's strategies. Second, there is the balance between the blockchain producer's reward and block size; larger blocks can increase both the reward and the isolation probability. If the isolation probability increases, the blockchain producer may not be able to win in the proof-of-work stage. Third, based on the blockchain producer's local observations, it is difficult for blockchain producers to reach a Nash equilibrium. To address these challenges, this invention uses mean-field theory for action recombination, establishes a neural network model, trains and executes it in a distributed manner, and proposes a computational offloading algorithm based on online learning to solve the established optimization problem. Therefore, this invention first needs to... and The three values ​​are separated from the overall objective and form their local optimization problems in order to derive their optimal values.

[0066] To reorganize the actions of blockchain producers, this invention formulates a short-run optimization problem P2 and proves that it is consistent with the long-run optimization problem P1. Since both problems P1 and P2 depend on the strategies of other blockchain producers, this invention approximates the interactions between blockchain producers based on mean-field theory, where action reorganization can be performed using mean-field theory and a neural network model can be established.

[0067] This invention has defined a utility function for blockchain producer n, and based on this, the expected utility value of blockchain producer n during the mining phase t is... It can be represented as:

[0068]

[0069] in, This represents the cost of blockchain producer n; This represents the probability of blockchain producer n being isolated in stage t; This represents the probability that blockchain producer n will successfully mine a block.

[0070] In stage t, maximize expected utility This will allow blockchain producer n to gain more utility. To maximize... The probability of successfully mining a block should be maximized. and rewards And minimize the isolation probability and cost Based on this condition, we can obtain the following short-run optimization problem P2:

[0071]

[0072] in, This represents a small, fixed positive value to ensure the denominator is not zero. By solving the short-run optimization problem P2, we can obtain the short-run optimization... and The optimal value.

[0073] The optimization problem of blockchain producer n depends on the actions of other blockchain producers. Therefore, an approximation of the interactions between blockchain producers is used to predict the actions of other blockchain producers, where the value function of blockchain producer n is determined by the expected effect v. n (s) definition, namely:

[0074]

[0075] Wherein, π(s) t ,a t ) is in state s t The possibility of all blockchain producers taking action. This represents the reward received by blockchain producer n, which can be abbreviated as... Using the above formula, the optimization problem P2 can be expressed in another form:

[0076]

[0077]

[0078] in, This indicates that blockchain producer n is in state s. t Next action Boundary values; π n Represents the strategy of blockchain producer n; This represents the expectation under the n-strategy of blockchain producer; This represents the expectation of blockchain producer n under the optimal strategy. This represents the transition probability.

[0079] Step 4: Under the local information state, construct the agent's online learning strategy based on step 3).

[0080] In the context of local information, to enable agents to achieve better online learning results, this invention first proposes a local optimization problem based on global optimality, then reorganizes the actions of blockchain producers based on mean field theory, approximates the interaction between blockchain producers, and uses a distributed network for corresponding policy training, thereby enabling blockchain producers to reach Nash equilibrium.

[0081] In a locally observed state, the blockchain producer's local information cannot obtain the global state s. t Therefore, use Replace s t Approximation based on mean-field theory And defined as The problems that need to be solved are as follows:

[0082]

[0083] Wherein, the function of blockchain producer n in stage t It can be achieved through average action and actions Approximation, i.e.

[0084] The algorithm pseudocode flow for training the blockchain producer strategy model is shown in Table 1.

[0085] Table 1. Pseudocode for the training process of the blockchain producer strategy model.

[0086]

[0087] The pseudocode flow of the designed online algorithm LOBE is shown in Table 2.

[0088] Table 2. LOBE Algorithm Pseudocode

[0089]

[0090]

[0091] See Figure 2 This invention utilizes a multilayer perceptron for distributed online learning, where the four fully connected layers are a policy network, an offloading and block size adjustment module, and a value network. Multiple agents interact with the environment, acquiring partial observable information from the system and performing online training based on the library generated by the agents.

[0092] Figure 3The performance of the LOBE algorithm proposed in this invention was compared with three other algorithms for average blockchain producer utility and average revenue under different numbers of blockchain producers. Experimental results show that the proposed algorithm outperforms the comparison algorithms for different numbers of blockchain producers.

[0093] Figure 4 The performance of the LOBE algorithm proposed in this invention was compared with three other algorithms under different numbers of edge servers managed by a service provider, focusing on the average blockchain producer utility and average revenue. Experimental results show that the proposed algorithm outperforms the comparison algorithms under different numbers of edge servers.

[0094] Figure 5 The performance of the LOBE algorithm proposed in this invention, compared with three other algorithms, was analyzed under different resource requirements to assess the average utility and average revenue of blockchain producers. Experimental results show that the proposed algorithm outperforms the comparison algorithms under different resource requirements.

[0095] Figure 6 The performance of the LOBE algorithm proposed in this invention, along with three other algorithms, was compared under different server computing units in terms of average blockchain producer utility and average revenue. Experimental results show that the proposed algorithm outperforms the comparison algorithms under different server computing units.

[0096] Figure 7 The performance of the LOBE algorithm proposed in this invention, along with three other algorithms, was compared to the average blockchain producer utility and average revenue under different value edge computing units (ECUs). Experimental results show that online scheduling is beneficial when local information is available, and compared to the compared algorithms, this invention achieves higher system blockchain producer utility and average revenue.

[0097] Figure 8 This represents the convergence time of the LOBE algorithm proposed in this invention under different numbers of blockchain producers. As more blockchain producers and more offloading tasks are generated, the convergence time increases with the number of blockchain producers, requiring more iterations to reach the Nash equilibrium point.

[0098] Figure 9 This diagram shows the loss trend of the policy network in this invention at different training stages. At the beginning of the training stage, the loss of the policy network fluctuates significantly. As the number of stages increases, the amplitude of the loss trend gradually decreases. After 400 stages, the loss fluctuates around 0, indicating that the learning algorithm has converged. Experimental results show that the LOBE algorithm proposed in this invention can better learn and train the model based on mean-field theory, and gradually converges to reach the Nash equilibrium point.

[0099] The above description describes specific embodiments of the present invention and the technical principles employed. Any changes made in accordance with the concept of the present invention that do not exceed the spirit of the specification and drawings should still fall within the protection scope of the present invention.

Claims

1. A multi-carrier edge resource scheduling method supporting off-chain computing in mobile blockchain, characterized in that, Includes the following steps: (1) Construct a network model and a blockchain producer computation offloading model to determine the utility of blockchain producers; The delays in completing computation tasks in the blockchain producer computation offloading model include travel, waiting, and processing delays. Blockchain producer n enters the edge server Travel delay within the communication range Represented as: in Indicates blockchain producers With edge servers The distance between them Represents edge server The communication radius; Indicates blockchain producers Speed ​​within the system; Calculate the waiting time of the task The M / G / 1-based queuing system is represented as follows: in, Represents edge server exist The intensity of the task reaching the stage; Represents edge server Average task processing latency; The variance representing the task processing delay; Blockchain producer task processing delay Represented as: in, This represents the edge computing unit complexity required to solve the proof-of-work problem in phase t. Represents edge server Maximum number of edge computing units; If the computational task is processed locally by blockchain producer A, there is no waiting delay or local processing delay. Calculated by the following formula: in, Indicates blockchain producers Maximum number of edge computing units; Total latency to complete the phase t blockchain producer's computation task Represented as: in, Indicates whether the computational tasks of blockchain producer A are processed locally; Indicates whether the task has been offloaded to the edge server. superior; Blockchain producers exist Stage utility Represented as: in, Indicates blockchain producers The cost; This indicates that blockchain producer 𝑛 is in The probability of isolation in a stage; Indicates blockchain producers The probability of successfully mining a block. This represents a fixed positive gain for blockchain producers. Indicates a variable gain factor; Indicates blockchain producers exist The block size of the stage; (2) Construct a Markov game model and establish a reward maximization optimization problem based on the utility of blockchain producers; the reward maximization optimization problem is: in, Indicates blockchain producers The cost; Indicates blockchain producers exist Compensation earned during the phase; Indicates blockchain producers exist Budget for the phase; Indicates whether the task is offloaded to the edge server. The binary value; Indicates in The expected reward discount factor for the stage; Indicates the expectation of a reward; This represents the average time interval between the two blocks. This indicates the impact of block size on block propagation latency; Blockchain producer Mobile Edge Server Required distance; Indicates blockchain producers Mobile Edge Server The required time and tasks are handled by the edge server. The time difference required for processing; (3) Based on mean-field theory, action recombination and neural network model establishment are carried out; , and The three values ​​are separated from the overall objective and their local optimization problems are formed to obtain their optimal values; the short-run optimization problem P2 is constructed as follows: in, Let a fixed positive value be used to ensure that the denominator is not zero. By solving the short-run optimization problem P2, the short-run optimization problem can be obtained. , and The optimal value; (4) Under the local information state, construct the agent's online learning strategy based on step (3); In a locally observed state, the blockchain producer's local information cannot obtain the global state. Therefore, use replace Approximation based on mean-field theory And defined as The problems that need to be solved are as follows: Wherein, the function of blockchain producer n in stage t By average movement and actions Approximation, i.e.

2. The multi-operator edge resource scheduling method supporting off-chain computing of mobile blockchain according to claim 1, characterized in that: The network model includes individual service providers An edge server and A blockchain producer, a blockchain producer The system always maintains a constant speed It can move and offload computational tasks related to proof-of-work puzzles to edge servers or process these tasks locally.

3. The multi-operator edge resource scheduling method supporting off-chain computing of mobile blockchain according to claim 1, characterized in that: The optimization problem P2 can be expressed in another form, P3: in, This indicates that blockchain producer n is in state. Next action Boundary values.

Images