Distributed edge cloud resource renting decision-making method for Internet of Things block chain

Abstract

The invention provides a distributed edge cloud resource renting decision-making method for an internet of things block chain. The method comprises the following steps: S1, acquiring resource prices, residual resource quantity, block outlet speed and maximum tolerant time delay of an edge service provider (ESP) and a cloud service provider (CSP) in a network environment, and residual energy, residual budget and maximum computing resource of local equipment; s2, constructing an optimization objective function; S3, modeling the optimization objective function into a Markov decision process, and then solving the Markov decision process by adopting a strategy-based reinforcement learning method so as to obtain a resource renting strategy. According to the method, efficient distributed task scheduling can be realized in a dynamic Internet of Things environment. And in combination with the block chain technology, not only are the security and transparency of data transmission improved, but also the computing efficiency and stability of the system are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things blockchain, and in particular to a distributed edge cloud resource leasing decision method for Internet of Things blockchain. Background Art

[0002] In the traditional Internet of Things (IoT) blockchain resource leasing field, the centralized leasing method has shown shortcomings in many application scenarios, such as high data transmission overhead, low system reliability and poor scalability. In contrast, the distributed leasing method has shown obvious advantages in many aspects, including smooth expansion, performance, reliability, availability, cost and disaster recovery. In particular, the distributed leasing method has shown good performance in the scenario of processing large-scale data and high concurrent access.

[0003] At present, distributed rental decision-making mainly adopts multi-agent reinforcement learning methods. However, these methods usually require a lot of time and experience to learn the optimal strategy. In the IoT environment, the arrival of tasks and changes in the environment are frequent, which requires the agent to adapt to new situations quickly. However, traditional multi-agent reinforcement learning methods may take a long time to converge to a better strategy, which is not efficient enough in scenarios with high real-time requirements.

[0004] In addition, the interaction between agents may lead to instability in the learning process, thus affecting convergence. Different agents may fall into local optimal solutions during the learning process and fail to find the global optimal resource leasing strategy. This situation limits the optimization of system performance and thus affects the overall effect of distributed leasing. Summary of the invention

[0005] The present invention aims to at least solve the technical problems existing in the prior art, and in particular innovatively proposes a distributed edge cloud resource leasing decision method for the Internet of Things blockchain.

[0006] The present invention proposes a distributed edge cloud resource leasing decision method for the Internet of Things blockchain, comprising:

[0007] S1, obtains the resource pricing of the edge service provider (ESP) and cloud service provider (CSP) in the network environment, as well as the remaining resource volume, block speed, maximum tolerable delay, and remaining energy, remaining budget, and maximum computing resources of the local device;

[0008] S2, construct optimization objective function:

[0009] The block time of the IoT blockchain system is defined as t b , when the average block time of the device is t mean Greater than t bWhen the system reduces the difficulty of participating in the competition for the right to bookkeeping, the difficulty of participating in the competition for the right to bookkeeping will be increased, that is:

[0010]

[0011] Among them, h new Indicates the difficulty of generating a new round of blocks;

[0012] Let μ be the number of CPU revolutions required for a hash calculation. Then, the probability of device n creating a new block per unit time is:

[0013]

[0014] Among them, f n (t) represents the total computing power of device n when it participates in the accounting right competition for the tth time;

[0015] D=2 h , h represents the difficulty of finding the random number nonce; the larger h is, the more difficult it is to find the random number nonce.

[0016] For any device n, the probability of obtaining a reward by creating a block before other devices is expressed as:

[0017]

[0018] Among them, λ i (t) represents the probability of device i creating a new block;

[0019] λ n (t) represents the probability of device n creating a new block;

[0020] f i (t) represents the total computing power of device i when it participates in the accounting right competition for the tth time;

[0021] The reward obtained by creating a block for the tth time is represented by R(t), and the expected reward of device n participating in the competition for the bookkeeping right for the tth time is:

[0022]

[0023] Among them, R(t) represents the block reward for participating in the bookkeeping right competition for the tth time;

[0024] In summary, the expected benefit of device n participating in the tth accounting right competition is expressed as:

[0025] U n (t) = R n (t)-C n (t)(27)

[0026] Among them, Rn (t) represents the expected block reward of device n;

[0027] C n (t) represents the payment for equipment n;

[0028] The optimization objective is expressed as:

[0029]

[0030] Where T represents the maximum number of iterations;

[0031] Represents a collection of IoT devices;

[0032] S3, the optimization objective function is modeled as a Markov decision process, and then a policy-based reinforcement learning method is used to solve the Markov decision process to obtain the resource rental strategy;

[0033] The policy-based reinforcement learning method needs to continuously optimize the policy network through the swarm intelligence optimization algorithm (SIOA) until the optimal network parameters are found, decode the location information of the optimal individual in the population, and obtain the optimal parameters of the policy network of all IoT devices; the specific steps are as follows:

[0034] S3-1, the objective function of each policy network learning is expressed as:

[0035]

[0036] in, Indicates expected return;

[0037] s0 represents the initial state;

[0038] represents the value function;

[0039] S3-2, using swarm intelligence based optimization algorithm (SIOA) to optimize policy network parameters, thereby maximizing the policy network objective function;

[0040] The method of the present invention replaces the gradient parameter update method in traditional multi-agent reinforcement learning and uses SIOA to update the policy network parameters, thus overcoming the instability of traditional multi-agent reinforcement learning during training and the shortcomings of being easily trapped in local optimality.

[0041] Furthermore, the optimization objective function is constructed based on the rental model, the calculation model, the delay model, the energy consumption model, the payment model and the reward model, that is, the optimization objective function needs to meet the prerequisites of C1 to C5:

[0042] Assume that the maximum computing capability of device n is That is, the CPU clock frequency. The local computing power used by device n when participating in the accounting right competition for the tth time is recorded as Then we have:

[0043]

[0044] In addition, due to the limited computing resources of edge servers, the use To represent the maximum computing power that ESPs m can provide, where ESPsm represents the mth ESP, represents the set of ESPs; the size of the computing resources rented by m to n during the tth competition for bookkeeping rights is recorded as Let ESPs allocate computing resources in proportion to the size of the resources rented by the user, that is:

[0045]

[0046] Among them, π n (t) represents the ratio of task resource rental of device n when participating in the accounting right competition for the tth time;

[0047] π i (t) represents the ratio of task resource rental of device i when participating in the accounting right competition for the tth time;

[0048] Indicates the selected service provider, which is ESP and / or CSP. ESP is an edge service provider and CSP is a cloud service provider.

[0049] It means that the service provider selected by device n for leasing is ESP when participating in the accounting right competition for the tth time;

[0050] Indicates which service provider is selected for resource leasing. Indicates that an edge service provider ESP is selected for leasing, m = 1 to M; M represents the total number of edge service providers;

[0051] I {x} is the indicator function. When x is true, I {x} =1, otherwise I {x} =0;

[0052] As for cloud servers, their computing resources are usually sufficient, so each IoT can use all the computing resources of the rented cloud servers; the computing resources that the cloud server can provide are recorded as Then the computing resources rented by device n from CSP when participating in the competition for bookkeeping rights for the tth time are expressed as:

[0053]

[0054] in, Indicates that a cloud service provider CSP has been selected for leasing;

[0055] In summary, the computing power of device n when participating in the accounting right competition for the tth time is expressed as:

[0056]

[0057] Finally, the minimum computing power required to participate in the bookkeeping right competition is defined as f min , then:

[0058] C2:f n (t)≥f min (5)

[0059] In a blockchain based on the PoW consensus mechanism, if the generation delay of a block is too large and exceeds the threshold set in the network consensus rules, other nodes may consider the block to be outdated and choose not to accept it. Therefore, when IoT devices choose to rent resources from edge-cloud servers, they also need to consider the latency factor. The subtasks that IoT devices choose to calculate locally only have computing latency, which can be expressed as:

[0060]

[0061] in, It represents the computation delay of the subtask that device n chooses to compute locally when participating in the competition for bookkeeping rights for the tth time;

[0062] π n (t) represents the computing resource rental ratio;

[0063] ρ(t) represents the number of CPU cycles required to complete the task;

[0064] Indicates the local computing power used by device n when participating in the accounting right competition for the tth time;

[0065] In addition to the computational delay, there is also a communication delay when renting the resources of ESPs to perform subtask calculations. When participating in the accounting right competition for the tth time, the computational delay of device n renting the resources of m to perform subtask calculations is expressed as:

[0066]

[0067] in, It represents the computational delay of device n renting the resources of the mth ESP to perform subtask computation when participating in the competition for bookkeeping rights for the tth time;

[0068] α is the delay discount parameter caused by too many devices renting m computing resources;

[0069] n m (t) represents the number of devices that choose to rent m resources when participating in the accounting right competition for the tth time;

[0070] The amount of computing resources that ESP m rents to device n during the tth participation in the bookkeeping rights competition;

[0071] The communication delay of renting m's resources for subtask calculation is expressed as:

[0072]

[0073] in, It represents the communication delay of device n renting the resources of the mth ESP to perform subtask calculation when participating in the competition for bookkeeping rights for the tth time;

[0074] τ(t) represents the size of the computing task participating in the bookkeeping right competition for the tth time;

[0075] R m is the link transmission rate from IoT device to ESP m;

[0076] Therefore, the total delay of device n renting the resources of m to perform subtask calculation is expressed as:

[0077]

[0078] in, It represents the total delay of device n choosing to rent the resources of ESP m to perform subtask calculation when participating in the competition for bookkeeping rights for the tth time;

[0079] Similarly, renting cloud server resources for subtask computing also has computing delay and communication delay; the delay of device n renting cloud server resources for subtask computing is expressed as:

[0080]

[0081] in, It represents the total delay of device n choosing to rent CSP resources to perform subtask calculation when participating in the accounting right competition for the tth time;

[0082] It represents the communication delay from IoT device to cloud server, and its calculation formula is:

[0083]

[0084] Among them, R c Indicates the link transmission rate from the IoT system to the cloud server;

[0085] It indicates the computational delay of the IoT device choosing to rent cloud server resources for subtask computation. The calculation formula is:

[0086]

[0087] In summary, when participating in the competition for the bookkeeping right for the tth time, the total delay of device n in participating in the competition for the bookkeeping right is:

[0088]

[0089] In addition, using D max To express the maximum delay that can be accepted in the competition for bookkeeping rights, we have:

[0090] C3:D n (t)≤D max (14)

[0091] The task that the device chooses to calculate locally will consume more energy due to a large number of hash calculations. The energy consumption generated by device n when participating in the competition for accounting rights for the tth time is expressed as:

[0092]

[0093] Among them, k n is the energy consumption coefficient;

[0094] Indicates the size of local computing power;

[0095] π n (t) represents the proportion of rented resources;

[0096] ρ(t) represents the number of CPU cycles required to complete the task;

[0097] Since the energy of IoT devices is limited, n (t) represents the remaining energy of device n, then:

[0098] C4:E n (t)≤e n (t) (16)

[0099] When IoT devices rent computing resources from edge-cloud servers, they also need to pay a fee to the service provider. From the perspective of ESP, ESPs will give users certain price discounts in order to attract more devices to use their services to increase potential revenue and compensate for the decline in service quality caused by the increase in the number of service devices. Therefore, the fee that the competing device n that participates in the accounting right for the tth time needs to pay to ESP m is expressed as:

[0100]

[0101] Among them, c m It represents the charge of ESP m for unit computing amount;

[0102] β is the discount rate;

[0103] Indicates the number of devices that choose to rent the resources of m for subtask calculation;

[0104] π n (t) represents the proportion of rented resources;

[0105] ρ(t) represents the number of CPU cycles required to complete the task;

[0106] Indicates whether the resources of the mth ESP are rented for the calculation of the subtask;

[0107] The computing resource fee paid to CSP is expressed as:

[0108]

[0109] Among them, c c It represents the charge of CSP for unit computing amount;

[0110] Indicates whether to rent CSP resources to perform subtask calculations;

[0111] In summary, the fee that device n needs to pay for participating in the accounting right competition for the tth time is:

[0112]

[0113] In addition, each IoT device has a certain budget for purchasing computing resources. n (t) represents the remaining budget of device n when it participates in the accounting right competition for the tth time, then:

[0114] C5: C n (t)≤b n (t)(20).

[0115] Furthermore, the optimization of the policy network parameters using a swarm intelligence-based optimization algorithm (SIOA) includes: firstly initializing the position information of each individual in the population (i.e., the policy network parameters of multiple agents) and calculating their fitness information;

[0116] Then, the position is updated through the position update formula. After the number of iterations, the historical optimal position is decoded to extract the parameters of all its IoT device policy networks;

[0117] The position update formula includes the following three methods:

[0118] Method 1: Individual i calculates its own position X i After further investigation, the update formula is expressed as:

[0119]

[0120] in, represents the position of individual i at the ep iteration;

[0121] ep indicates the current iteration number;

[0122] exp() represents the natural exponential function;

[0123] iter max Indicates the maximum number of iterations;

[0124] Both r1 and r2 are random numbers with values ​​of (0,1);

[0125] L represents a All values ​​of a matrix follow a standard normal distribution.

[0126] Method 2: Individual i updates its position by using the best individual position in the current population. The update formula at this time is expressed as:

[0127]

[0128] in, represents the position of the individual with the best fitness;

[0129] represents the individual position of individual i;

[0130] W is a random number that follows a standard normal distribution;

[0131] UB and LB represent the upper and lower limits of individual position values, respectively;

[0132] Method 3: In order to improve the exploration degree of individuals and avoid falling into the local optimal solution, individual i also searches for the optimal position by randomly changing its position. We express its random update formula as:

[0133]

[0134] Among them, X r represents a random position;

[0135] A means one The matrix has values ​​of 1 or -1.

[0136] Furthermore, when the individual fitness size of an individual is in the top Q% of the population, the individual is a high fitness individual and has a probability of more than 50% to use formula (40) to update the position;

[0137] When the individual fitness size of an individual is not in the top Q% of the population, the individual is a low fitness individual, and there is a probability of more than 50% that the position is updated using formula (41);

[0138] In addition, each individual has a K% probability of updating its position using formula (42).

[0139] Furthermore, the value of Q is 30, and the value of K is 5.

[0140] Furthermore, the fitness information is obtained through a fitness function, and the formula is as follows:

[0141]

[0142] in, In the strategy π X Expected return under

[0143] π X Represents the joint policy of all IoT devices;

[0144] T represents the maximum number of iterations;

[0145] Represents a collection of IoT devices;

[0146] r n (t) represents the reward function of device n.

[0147] The greater the fitness of an individual, the better the solution represented by the individual, that is, the better the network parameters, and thus the greater the cumulative reward.

[0148] Furthermore, after executing steps S1 to S3, the trained policy network is deployed on the corresponding IoT device. Since each IoT device has an independent policy network, each IoT device can independently make resource leasing decisions based on the environmental status.

[0149] The multi-agent reinforcement learning algorithm adopted in the present invention adopts a centralized training distributed execution solution paradigm. During training, some global information that cannot be seen by a single agent is used to achieve better training results, while this information is not used during execution. Each agent takes actions directly according to its own strategy to achieve the effect of distributed execution.

[0150] In summary, due to the adoption of the above technical solutions, the distributed resource leasing method based on swarm intelligence multi-agent reinforcement learning proposed in the present invention can realize efficient distributed task scheduling in a dynamic Internet of Things environment. Combined with blockchain technology, it not only improves the security and transparency of data transmission, but also significantly improves the computing efficiency and stability of the system. By optimizing resource scheduling and task allocation, the method of the present invention can achieve rapid convergence and high-quality strategy optimization in a dynamic Internet of Things environment. It effectively reduces the energy consumption of the system and ensures the good convergence and adaptability of the system, thereby maintaining excellent performance in various complex network scenarios. The specific advantages are:

[0151] (1) The present invention designs a swarm intelligence-enabled multi-agent reinforcement learning algorithm SIMARL. The algorithm uses SIOA's powerful exploration and optimization capabilities to replace the traditional gradient descent method in DRL, overcoming the instability of DRL during training and the shortcomings of being easily trapped in local optimality. The innovation of SIMARL is that it can not only enhance the efficiency of collaboration between agents in a multi-agent system, but also significantly improve the adaptability of the optimization process to complex environmental changes.

[0152] (2) This invention proposes a distributed dynamic resource leasing scheme for PoW computing tasks in IoT devices. This scheme intelligently determines the allocation of computing tasks between the cloud and edge servers based on real-time environmental changes and dynamic resource allocation, optimizes resource utilization and significantly improves overall system benefits.

[0153] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0154] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0155] Figure 1 This is the IoT blockchain system architecture of the present invention.

[0156] Figure 2 It is a schematic diagram of the framework of the PoW distributed dynamic resource leasing solution based on SIMARL of the present invention.

[0157] Figure 3 It is a schematic diagram of the convergence analysis of SIMARL under different numbers of IoT devices in the present invention.

[0158] Figure 4 It is a convergence analysis diagram of SIMARL under different numbers of ESPs of the present invention.

[0159] Figure 5 3 is a schematic diagram of the scheme comparison when the number of IoT devices of the present invention is 20.

[0160] Figure 6 3 is a schematic diagram for comparing solutions when the number of IoT devices of the present invention is 40.

[0161] Figure 7 3 is a schematic diagram of the method comparison when the number of IoT devices of the present invention is 20.

[0162] Figure 8 3 is a schematic diagram of the method comparison when the number of IoT devices of the present invention is 40.

[0163] Fig. 9 This is a schematic diagram of the solution quality comparison when the number of IoT devices of the present invention is 20.

[0164] Fig.10 This is a schematic diagram of the solution quality comparison when the number of IoT devices of the present invention is 40.

[0165] Fig.11 It is a schematic diagram of robustness comparison under different numbers of IoT devices of the present invention.

[0166] Fig.12 It is a schematic diagram of robustness comparison under different numbers of ESPs of the present invention. DETAILED DESCRIPTION

[0167] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0168] 1 System framework and modeling

[0169] The present invention relates to a typical IoT blockchain system, which includes multiple edge service providers (Edge Service Provider, ESP) and a cloud service provider (Cloud Service Provider, CSP). The specific architecture is as follows Figure 1 As shown. The system consists of a large number of heterogeneous IoT devices, which can be connected to the nearest base station (BS) through a wireless network. Multiple edge servers are deployed near the BS, and IoT devices can also connect to the cloud server through the BS. In this system, IoT devices can offload PoW computing tasks to the edge or cloud by renting computing resources from edge servers or cloud servers, so as to enable the cloud to participate in the competition for accounting rights. We use represents a collection of N IoT devices, represents a set of M ESPs, Denotes CSP. A tuple (τ(t), ρ(t)) is used to represent the size of the computing task (bit) of the tth competition for the right to keep accounts and the number of CPU cycles (cycles) required to complete the task. In addition, we consider that the computing tasks participating in the competition for the right to keep accounts can be partially offloaded. IoT devices participating in the competition for the right to keep accounts can choose to calculate all tasks locally, or offload part of their tasks to edge servers or cloud servers.

[0170] We define a two-tuple To represent IoT devices (terminals) The rental decision when participating in the competition for bookkeeping rights for the tth time, where π n (t) represents the resource rental ratio, Indicates the choice of rental service provider. n The value range of (t) is [0, 1], The possible values ​​of are {0, 1, 2, ..., M}, where a value of 0 indicates renting the resources of the cloud server, and values ​​from 1 to M indicate M ESPs, respectively.

[0171] Since participating in the competition for bookkeeping rights requires a large amount of computing resources, the terminals in the IoT need to rent certain computing resources from edge servers or cloud servers to improve their own computing power in order to participate in the competition for bookkeeping rights. Since most of the terminals in the IoT are heterogeneous, their computing power also varies. Suppose the maximum computing power of terminal n is That is, the CPU clock frequency (CPU cycles / s), and the local computing power used by terminal n when participating in the competition for bookkeeping rights for the tth time is recorded as Then we have:

[0172]

[0173] In addition, due to the limited computing resources of edge servers, we use To represent ESPs The maximum computing power that can be provided. The size of the computing resources rented by m to n during the tth competition for bookkeeping rights is recorded as Since the computing resources of ESPs are limited, in order to meet the needs of users as much as possible, we assume that ESPs will allocate computing resources in proportion to the size of the resources rented by users, that is:

[0174]

[0175] Among them, I {x} is the indicator function. When x is true, I{x} =1, otherwise I {x} = 0. As for cloud servers, their computing resources are usually sufficient, so we believe that each IoT can use all the computing resources of the rented cloud servers. We denote the computing resources that the cloud server can provide as Then the computing resources rented by terminal n from CSP when participating in the competition for bookkeeping rights for the tth time can be expressed as

[0176]

[0177] In summary, the computing power that terminal n possesses when participating in the competition for bookkeeping rights for the tth time can be expressed as:

[0178]

[0179] Finally, we define the minimum computing power required to participate in the competition for bookkeeping rights as f min , then:

[0180] C2:f n (t)≥f min (5)

[0181] In a blockchain based on the PoW consensus mechanism, if the generation delay of a block is too large and exceeds the threshold set in the network consensus rules, other nodes may consider the block to be outdated and choose not to accept it. Therefore, when IoT devices choose to rent resources from edge-cloud servers, they also need to consider the latency factor. The subtasks that IoT devices choose to calculate locally only have computing latency, which can be expressed as:

[0182]

[0183] In addition to the computational delay, there is also a communication delay when renting the resources of ESPs to perform subtask calculations. The computational delay of terminal n renting the resources of m to perform subtask calculations when participating in the competition for bookkeeping rights for the tth time can be expressed as:

[0184]

[0185] Among them, α is the delay discount parameter caused by too many terminals renting m computing resources, n m (t) represents the number of terminals that choose to rent m resources to perform subtask calculations when participating in the competition for bookkeeping rights for the tth time. The communication delay of renting m resources to perform subtask calculations can be expressed as:

[0186]

[0187] Among them, R mis the link transmission rate from IoT device to ESP m. Therefore, the total delay of terminal n renting m’s resources to perform subtask calculation can be expressed as:

[0188]

[0189] Similarly, renting the computing resources of the cloud server for subtask computing also has computing delay and communication delay. The delay of terminal n renting the resources of the cloud server for subtask computing can be expressed as:

[0190]

[0191] in, It represents the communication delay from IoT device to cloud server, and its calculation formula is:

[0192]

[0193] Among them, R c Indicates the link transmission rate from the IoT system to the cloud server. It represents the computational delay of the IoT device choosing to rent cloud server resources for subtask computation. The calculation formula can be expressed as:

[0194]

[0195] In summary, when participating in the competition for the bookkeeping right for the tth time, the total delay of terminal n in participating in the competition for the bookkeeping right is:

[0196]

[0197] In addition, we use D max To express the maximum delay that can be accepted in the competition for bookkeeping rights, we have:

[0198] C3:D n (t)≤D max (14)

[0199] The task that the terminal chooses to calculate locally will consume more energy due to a large number of hash calculations. The energy consumption generated by terminal n when participating in the competition for bookkeeping rights for the tth time can be expressed as:

[0200]

[0201] Among them, k n The energy consumption coefficient is different for different types of terminals.

[0202] Indicates the size of local computing power;

[0203] π n(t) represents the proportion of resource rental;

[0204] ρ(t) represents the number of CPU cycles required to complete the task;

[0205] We consider that the energy of IoT devices is limited. n (t) represents the residual energy of terminal n, then:

[0206] C4:E n (t)≤e n (t) (16)

[0207] When IoT devices rent computing resources from edge-cloud servers, they also need to pay a fee to the service provider. From the perspective of ESP, ESPs will give users certain price discounts in order to attract more terminals to use their services to increase potential revenue and compensate for the decline in service quality caused by the increase in the number of service terminals. Therefore, the fee that terminal n, which participates in the accounting right competition for the tth time, needs to pay to ESP m can be expressed as:

[0208]

[0209] Where β is the discount rate;

[0210] represents the number of terminals that choose to rent the resources of m for subtask calculation;

[0211] π n (t) represents the proportion of resource leasing;

[0212] ρ(t) represents the number of CPUT cycles required to complete the task;

[0213] Indicates whether the mth ESP is selected for leasing;

[0214] c m It represents the charge of ESP m for unit computing amount;

[0215] The computing resource fee paid to CSP can be expressed as:

[0216]

[0217] Among them, c c Indicates the CSP charge per unit of computing power. Indicates whether a CSP is selected for leasing. In summary, the fee that terminal n needs to pay for participating in the competition for the accounting right for the tth time is:

[0218]

[0219] In addition, we consider that each IoT device has a certain budget for purchasing computing resources. n (t) represents the remaining budget of terminal n when it participates in the competition for bookkeeping rights for the tth time, then:

[0220] C5: C n (t)≤b n (t) (20)

[0221] In addition to the above restrictions and required fees, each IoT terminal may receive a block reward to generate income by producing a block first when participating in the competition for bookkeeping rights. In order to generate a new block, IoT terminals need to compete to complete a mathematical problem based on a hash encryption algorithm. The winner has the right to record transactions on the blockchain and receive rewards. The hash function H outputs an L-bit hash value based on the Merkle tree root data tx of all transaction data in the block, the hash value prev.hash of the parent block, and the random number nonce. L represents the L-bit hash value. The IoT terminal needs to find a random number nonce that satisfies:

[0222] H(tx,prev.hash,nonce)≤1 L-h (twenty one)

[0223] Where h represents the difficulty of finding the random number nonce. The larger h is, the more difficult it is to find the random number nonce. In this paper, we consider defining the block time of the IoT blockchain system as t b , when the average block time of the terminal is t mean Greater than t b When the system reduces the difficulty of participating in the competition for the right to bookkeeping, the difficulty of participating in the competition for the right to bookkeeping will be increased, that is:

[0224]

[0225] Among them, h new Indicates the difficulty of generating a new round of blocks; D = 2 h .

[0226] Assuming that the number of CPU revolutions required for a hash calculation is μ, the probability of terminal n creating a new block per unit time is:

[0227]

[0228] Among them, f n (t) represents the total computing power of terminal n when it participates in the accounting right competition for the tth time;

[0229] For any terminal n, the probability of obtaining benefits by creating a block before other terminals can be expressed as:

[0230]

[0231] Among them, λ i (t) represents the probability of terminal i creating a new block;

[0232] λ n (t) represents the probability of terminal n creating a new block;

[0233] f i (t) represents the computing power of device i when it participates in the accounting right competition for the tth time;

[0234] We denote the reward obtained by creating a block for the tth time by R(t). Then the expected reward of terminal n participating in the competition for bookkeeping rights for the tth time is:

[0235]

[0236] Among them, R(t) represents the block reward for participating in the bookkeeping right competition for the tth time;

[0237] In summary, the expected benefit of terminal n participating in the tth competition for bookkeeping rights can be expressed as:

[0238] U n (t) = R n (t)-C n (t) (26)

[0239] Among them, C n (t) represents the fee that terminal n needs to pay when participating in the accounting right competition for the tth time

[0240] This paper mainly considers that within a period of competition for accounting rights, each IoT terminal is subject to the constraints of latency, energy consumption, budget, etc. and considers the leasing decisions of other IoT terminals, and maximizes the cumulative benefits of the entire IoT system by reasonably leasing the computing resources of the edge cloud. Therefore, the optimization goal of this paper can be expressed as:

[0241]

[0242] stC1-C

[0243] 2 Distributed Resource Leasing Method Based on SIMARL

[0244] The present invention uses a swarm intelligence optimization algorithm-enabled multi-agent reinforcement learning algorithm (SIMARL) to solve the distributed resource leasing problem in the IoT blockchain system. By using SIOA's powerful exploration and optimization capabilities to replace the traditional gradient descent method in DRL, SIMARL overcomes the instability of DRL during training and the shortcomings of being easily trapped in local optimality. It can not only enhance the collaboration efficiency between agents in the multi-agent system, but also significantly improve the adaptability of the optimization process to complex environmental changes. Figure 2 The relevant details of the distributed resource leasing solution framework based on SIMARL proposed in the present invention are presented in detail. (1) In the training stage (centralized training): first obtain the resource pricing of ESP and CSP in the network environment, as well as the remaining resource volume, block speed, maximum tolerable delay and other parameters, initialize the position information of each individual in the population (i.e., the policy network parameters of multiple intelligent agents), and calculate its fitness information, and update its own position through three position update formulas. After the number of iterations is completed, the historical optimal position is decoded and the parameters of the policy network of all IoT devices are extracted. (2) Execution stage (distributed execution): The trained policy network is deployed on the corresponding IoT device. Since each IoT device has an independent policy network, each IoT device can make leasing decisions independently based on the environmental status.

[0245] In order to effectively use SIMARL to solve the above complex optimization problem, we model it as a Markov decision process (MDP) and use a five-tuple It indicates that, Represents a collection of intelligent agents. In the scenario considered in this article, each IoT device represents an intelligent agent, so represents the state set of all agents, represents the joint action set of all agents, represents the state transition probability, Represents the reward function. The state space, action space and reward function of each agent are defined as follows:

[0246] (1) State space: The system state when participating in the bookkeeping right competition for the tth time is expressed as It consists of the states of all IoT devices, namely s n (t) represents the state of device n when it participates in the competition for bookkeeping rights for the tth time. We define

[0247] s n (t) = [h(t), b n (t), e n (t)] (28)

[0248] That is, the state of each intelligent agent is composed of the current difficulty of the competition for the right to participate in accounting, the remaining budget, and the remaining energy.

[0249] (2) Action space: The joint action when participating in the bookkeeping right competition for the tth time is expressed as in

[0250]

[0251] It represents the leasing decision of device n when it participates in the accounting right competition for the tth time. It should be noted that the decision made by the IoT device needs to satisfy the constraints C1-C5.

[0252] (3) Reward function: The reward function for the tth participation in the bookkeeping rights competition is expressed as It consists of the rewards of all IoT devices, namely in

[0253] r n (t) = U n (t)(30)

[0254] There are two main learning paradigms for reinforcement learning: policy-based methods and value-based methods. SIMARL is essentially a policy-based reinforcement learning (RL) method. Unlike the value-based method, the policy-based method directly and explicitly learns a target policy π during the learning process. θ , where θ represents the weight and bias parameters of the policy network. Policy-based reinforcement learning methods are more applicable in our scenario than value-based methods. The main reasons include the following: First, the policy-based method directly processes the continuous action space, avoiding the complexity of action discretization. Second, it makes the policy search more direct by optimizing the policy itself, reducing the impact of the value function estimation error. In addition, the policy-based method balances exploration and utilization through random sampling, which is more suitable for dealing with complex dynamic environments. In high-dimensional state space, the policy-based method has better scalability and convergence. Therefore, in the IoT blockchain resource leasing problem, the policy-based method can provide a better solution.

[0255] The objective function of policy network learning can be expressed as:

[0256]

[0257] in, Indicates expected return;

[0258] s0 represents the initial state;

[0259] Represents the value function.

[0260] In traditional policy-based RL methods, the gradient ascent method is basically used to maximize this objective function to obtain the optimal strategy. The gradient of the objective function can be obtained:

[0261]

[0262] in, In the strategy π θ Expected return under

[0263] π θ (a|s) represents the strategy under the network parameter θ;

[0264] After the gradient is obtained by formula (32), all parameters are then updated in the direction of the gradient increase to maximize the objective function of the policy network.

[0265] Represents the action value function. Traditional policy-based RL methods can estimate it in many ways. For example, in the REINFORCE algorithm, the Monte Carlo method is used to estimate it. Therefore, the policy gradient in the REINFORCE algorithm is:

[0266]

[0267] Among them, γ is the discount factor and r is the reward. In the RL method based on the Actor-Critic structure, the Critic value network is used to fit a value function to guide the strategy learning, and the Actor network is the policy network, and its policy gradient can be expressed as:

[0268]

[0269] Then the gradient ascent method is also used to update the Actor strategy network parameters. The gradient of its value function can be expressed as:

[0270]

[0271] The Critic value network also uses the gradient descent method to update the network parameters.

[0272] However, there are many challenges involved in using the gradient update method to optimize network parameters. First, the choice of learning rate is crucial, and its size directly affects the quality of experimental results. A larger learning rate may lead to unstable learning and difficulty in convergence, while a smaller learning rate may be trapped in a local optimal solution and unable to escape. Secondly, the design of the network structure also has an important impact on the results. An overly complex structure may cause gradient vanishing or gradient explosion, while an overly simple structure cannot effectively learn complex function mappings. Finally, even if researchers have determined the optimal learning rate, network structure and other parameters through a large number of experiments, when solving different problems, it is usually necessary to readjust the parameters, which will bring a large time cost.

[0273] In SIMARL, each agent (IoT device) contains only one policy network, and SIOAs is used to replace the traditional gradient update method to optimize the policy network parameters. The advantage of this method is that there is no need to adjust the learning rate and network structure through large-scale experiments, which saves a lot of time and resources. The reason for adopting such a simple network structure is that SIMARL needs to achieve fast convergence and efficient calculation in dynamic resource rental scenarios. Simple network structures are more suitable for combining with swarm intelligence to cope with dynamic changes in high-dimensional state space. In addition, some updated network structures usually introduce higher algorithm complexity and computational overhead, which may not meet the design goals of SIMARL. SIOAs is an optimization algorithm based on group cooperation, which represents a set of solutions by the position of each individual in the search space. The quality of individual positions is judged by an indicator called fitness (usually an objective function). Each individual can use the position information and fitness of other individuals to adjust its own position to find better fitness, that is, to find a better solution. In SIMARL, we represent the positions of all individuals in the population as a dimensional space, where represents the population size, represents the position space dimension, that is, the dimension of the optimization problem. The position information of each individual represents the weight and bias information in the policy network, so The size of depends on the number of policy network parameters. Since there are N agents in the scenario we consider, there are N policy networks in SIMARL, which all include an input layer, a hidden layer, and an output layer. To represent the total number of policy network parameters for all IoT devices, They represent the number of weight parameters and the number of bias parameters in the strategy network of agent i respectively. Then the optimization problem dimension Let θ be the parameter of the policy network of device n, which is expressed as follows:

[0274]

[0275] Let X i Represents an individual , then

[0276]

[0277] Of course, we need to decode the individual's position information into N policy networks and replace it with neural network parameters. The decoding pseudocode is shown in Algorithm 1.

[0278]

[0279] We define f(X) as the fitness function to evaluate the quality of individual positions. In this paper, we directly use the optimization goal of the policy network (the optimization goal is formulated for the Markov decision process, and the actual optimization goal is the same as before, because the reward function is the optimization goal of this paper) as the fitness function (fitness can also be regarded as the optimization goal of SIOA), that is:

[0280]

[0281] Among them, π X Represents the joint strategy of all IoT devices. More generally, we can express the fitness function as follows:

[0282]

[0283] The greater the fitness of an individual, the better the solution represented by the individual, that is, the better the network parameters, and thus the greater the cumulative reward. The pseudo code for calculating the fitness function is shown in Algorithm 2.

[0284]

[0285] Each individual in the population can reach a better fitness position by updating its own position. In this paper, each individual can update its position in three ways to seek the optimal solution. First, individual i can update its position X i After further investigation, the update formula can be expressed as:

[0286]

[0287] in, represents the position of individual i at the ep iteration;

[0288] ep indicates the current iteration number;

[0289] exp() represents the natural exponential function;

[0290] iter maxIndicates the maximum number of iterations;

[0291] Both r1 and r2 are random numbers with values ​​of (0,1);

[0292] L represents a All values ​​of a matrix obey the standard normal distribution.

[0293] Secondly, individual i can update its own position by using the best individual position in the current population. The update formula at this time can be expressed as:

[0294]

[0295] in, represents the position of the individual with the best fitness;

[0296] W is a random number that follows a standard normal distribution;

[0297] UB and LB represent the upper and lower limits of individual position values, respectively;

[0298] Finally, in order to improve the exploration degree of individuals and avoid falling into the local optimal solution, individual i can also find the optimal position by randomly changing its position. We express its random update formula as:

[0299]

[0300] Among them, X r represents a random position; A represents a The matrix has values ​​of 1 or -1.

[0301] In addition, we also define that individuals with higher fitness (individual fitness size is in the top 30%) are more inclined to use formula (40) to update their positions, and individuals with lower fitness (individual fitness size is not in the top 30%) use formula (41) to update their positions, and each individual has a 5% probability of randomly updating its position. In each iteration, the historical optimal individual position will be saved. When the current optimal position after the next iteration is better than the historical optimal position, the historical optimal position will be replaced by the current optimal position, otherwise the historical optimal position remains unchanged. When the iteration is completed, the individual in the historical optimal position is decoded to obtain the optimal parameters of the IoT device strategy network to obtain more cumulative rewards. In summary, the pseudo code of the distributed resource leasing algorithm based on SIMARL is shown in Algorithm 3.

[0302]

[0303]

[0304] 3 Simulation Experiment and Result Analysis

[0305] In this simulation environment, the number of basic IoT devices is set to 20, the number of edge servers is set to 10, the system block time is set to 30s, the maximum acceptable delay is set to 2 times the block time, and the maximum computing power of the local terminal is set to The value is randomly generated between 10 and 80 Mcycles / s, and the maximum computing power of the edge node The value of is randomly generated between 4 and 6 Gcycles / s. The minimum values ​​of the task size τ(t) and the number of CPU cycles ρ(t) required to complete the task are 100 Mb and 2×10 9 cycles, the minimum computing power to participate in the bookkeeping right competition is 50Mcycles / s, the delay discount α and discount rate β are both 0.04, and the CSP charges c for unit computing power. c The value is 5×10 -10 units / cycles. ESPs set the price of computing resources at 7×10 -10 to 10×10 -10 Units / cycles are randomly generated, and the block reward R(t) for participating in the bookkeeping right competition is 30 units.

[0306] In order to verify the convergence performance of SIMARL, we evaluated the convergence of SIMARL under different numbers of IoT devices and different numbers of ESPs. The experimental results are as follows: Figure 3 and Figure 4 As shown in the figure, as the problem scale increases and the scenario changes, SIMARL can still reach convergence in a relatively small number of iterations. This is because SIMARL uses SIOAs to optimize the policy network, and the SIOA used shows good results in optimizing neural network parameters. Therefore, SIMARL has good convergence performance when solving various optimization problems.

[0307] Subsequently, to verify the effectiveness of the resource leasing scheme based on the SIMARL method, this paper compares the average cumulative benefits of IoT devices under the proposed resource leasing scheme and the full leasing computing scheme when the number of IoT devices is 20 and 40. The results are as follows: Figure 5 and Figure 6As shown in the figure, we can see that under the full resource leasing scheme, as the number of block generation increases, the average revenue of IoT devices gradually decreases, especially when the request scale of IoT devices increases. The revenue of IoT devices under the full resource leasing scheme reaches a plateau after generating 60 blocks, while the resource leasing scheme based on SIMARL can always maintain a high revenue. This is because the resource leasing scheme based on SIMARL can make full use of local energy consumption and budget, and allocate them reasonably to improve the cumulative revenue of IoT devices.

[0308] Then we make a detailed comparison between SIMARL and the classic MARL algorithms MADDPG, MASAC, MATD3, HATRPO and H-MADDPG in recent years. First, we evaluate the convergence of SIMARL and other MARL algorithms in solving the optimization problem in this paper. Figure 7 and Figure 8 As shown in the figure, it can be seen that the SIMARL method proposed in this paper is superior to other methods in terms of convergence speed and solution quality. After increasing the scale of the optimization problem, some classic algorithms are difficult to converge, while SIMARL can still converge at a faster speed and has better solution quality than other methods. This is because other methods use gradient descent for optimization, which is easy to fall into the local optimal solution, and the setting of hyperparameters will lead to unstable convergence. SIMARL uses the SIOAs method to optimize the policy network and has a good ability to jump out of the local optimal solution.

[0309] Then, we compared the solution quality of SIMARL-based resource leasing scheme with other methods under different problem scales. Fig. 9 and Fig.10 As shown in the figure, no matter the size of the problem, the solution quality of the resource leasing scheme based on SIMARL is always better than other methods.

[0310] Finally, to verify the robustness of the SIMARL method, we compared the cumulative benefits of IoT devices with SIMARL and other methods under different numbers of IoT devices and different numbers of ESPs. Fig.11 and Fig.12As shown in the figure. As can be seen from the figure, with the increase in the number of IoT devices, the average cumulative benefit of IoT devices is gradually decreasing, and with the increase in the number of ESPs, the average cumulative benefit of IoT devices is gradually increasing. This is because the more IoT devices there are, the greater the resource competition between the devices will be. As the number of ESPs increases, the amount of computing resources will increase and the resource competition will decrease. In addition, it can be clearly seen from the figure that the solution quality of the resource leasing scheme based on SIMARL is always better than other MARL methods, and it has good robustness in solving the optimization problem proposed in this paper. This is because SIOAs is not sensitive to the parameters of the optimization problem. Even if the problem scale becomes larger or the environmental parameters change, it can still maintain good solution quality and has good robustness.

[0311] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A distributed edge cloud resource leasing decision method for IoT blockchain, characterized in that: include: S1, obtains the resource pricing of edge service providers and cloud service providers in the network environment, as well as the remaining resource volume, block speed, maximum tolerable delay, and remaining energy, remaining budget, and maximum computing resources of local devices; S2, construct the optimization objective function: The block time of the IoT blockchain system is defined as t b , when the average block time of the device is t mean Greater than t b When the system reduces the difficulty of participating in the competition for the right to bookkeeping, the difficulty of participating in the competition for the right to bookkeeping will be increased, that is: Among them, h new Indicates the difficulty of generating a new round of blocks; Let μ be the number of CPU revolutions required for a hash calculation. Then, the probability of device n creating a new block per unit time is: Among them, f n (t) represents the total computing power of device n when it participates in the accounting right competition for the tth time; D=2 h , h represents the difficulty of finding random numbers; For any device n, the probability of obtaining revenue by creating a block before other devices is expressed as: Among them, λ i (t) represents the probability of device i creating a new block; λ n (t) represents the probability of device n creating a new block; f i (t) represents the total computing power of device i when it participates in the accounting right competition for the tth time; The reward obtained by creating a block for the tth time is represented by R(t), and the expected reward of device n participating in the competition for the bookkeeping right for the tth time is: Among them, R(t) represents the block reward for participating in the bookkeeping right competition for the tth time; In summary, the expected benefit of device n participating in the tth accounting right competition is expressed as: U n (t)=R n (t)-C n (t)(27) Among them, R n (t) represents the expected block reward of device n; C n (t) represents the payment for equipment n; The optimization objective is expressed as: Where T represents the maximum number of iterations; N represents the set of IoT devices; S3, the optimization objective function is modeled as a Markov decision process, and then a policy-based reinforcement learning method is used to solve the Markov decision process to obtain the resource rental strategy; The policy-based reinforcement learning method needs to continuously optimize the policy network through the swarm intelligence optimization algorithm until the optimal network parameters are found and the optimal parameters of the policy network of all IoT devices are obtained; the specific steps are as follows: S3-1, the objective function of each policy network learning is expressed as: in, Indicates expected return; s0 represents the initial state; represents the value function; S3-2, uses a swarm intelligence-based optimization algorithm to optimize the policy network parameters, thereby maximizing the policy network objective function.

2. According to a distributed edge cloud resource leasing decision-making method for the Internet of Things blockchain according to claim 1, it is characterized in that: The optimization objective function is constructed based on the rental model, calculation model, delay model, energy consumption model, payment model and reward model, that is, the optimization objective function must meet the prerequisites of C1 to C5: Assume that the maximum computing capability of device n is That is, the CPU clock frequency. The local computing power used by device n when participating in the accounting right competition for the tth time is recorded as Then we have: In addition, due to the limited computing resources of edge servers, the use To represent the maximum computing power that ESPs m can provide, where ESPsm represents the mth ESP, represents the set of ESPs; the size of the computing resources rented by m to n during the tth competition for bookkeeping rights is recorded as Let ESPs allocate computing resources in proportion to the size of the resources rented by the user, that is: Among them, π n (t) represents the resource rental ratio of device n when participating in the accounting right competition for the tth time; π i (t) represents the resource rental ratio of device i when participating in the accounting right competition for the tth time; Indicates the selected service provider, the service provider is ESP and / or CSP, ESP is the edge service provider, and CSP is the cloud service provider; It means that the service provider selected by device n for leasing is ESP when participating in the accounting right competition for the tth time; Indicates which service provider is selected for resource leasing. Indicates that an edge service provider ESP is selected for leasing, m = 1 to M; M represents the total number of edge service providers; I {x} is the indicator function. When x is true, I {x} =1, otherwise I {x} =0; As for cloud servers, their computing resources are usually sufficient, so each IoT can use all the computing resources of the rented cloud servers; the computing resources that the cloud server can provide are recorded as Then the computing resources rented by device n from CSP when participating in the competition for bookkeeping rights for the tth time are expressed as: in, Indicates that a cloud service provider CSP has been selected for leasing; In summary, the computing power of device n when participating in the accounting right competition for the tth time is expressed as: Finally, the minimum computing power required to participate in the bookkeeping right competition is defined as f min , then: C2:f n (t)≥f min (5) The subtask that the IoT device chooses to calculate locally only has a computational delay, which can be expressed as: in, It represents the computation delay of the subtask that device n chooses to compute locally when participating in the competition for bookkeeping rights for the tth time; π n (t) represents the resource rental ratio; ρ(t) represents the number of CPU cycles required to complete the task; Indicates the local computing power used by device n when participating in the accounting right competition for the tth time; In addition to the computational delay, there is also a communication delay when renting ESPs computing resources to perform subtasks. When participating in the accounting right competition for the tth time, the computational delay of device n renting m's resources to perform subtask calculations is expressed as: in, It represents the computation delay of device n renting the mth ESP resource to perform subtask computation when participating in the competition for bookkeeping rights for the tth time; α is the delay discount parameter caused by too many devices renting m computing resources; n m (t) represents the number of devices that choose to rent m resources when participating in the accounting right competition for the tth time; The amount of computing resources that ESPm leases to device n during the tth participation in the bookkeeping rights competition; The communication delay of choosing to rent m resources for subtask calculation is expressed as: in, It represents the communication delay of device n renting the mth ESP resource to perform subtask calculation when participating in the competition for bookkeeping rights for the tth time; τ(t) represents the size of the computing task participating in the bookkeeping right competition for the tth time; R m is the link transmission rate from IoT device to ESPm; Therefore, the total delay of device n renting m resources to perform subtask calculation is expressed as: in, It represents the total delay of device n choosing to rent the resources of ESPm to perform subtask calculation when participating in the competition for bookkeeping rights for the tth time; Similarly, the subtasks that use cloud server resources for computing also have computing delays and communication delays. The delay of device n using cloud server resources for computing subtasks is expressed as: in, It represents the total delay of device n choosing to rent CSP resources to perform subtask calculation when participating in the accounting right competition for the tth time; It represents the communication delay from IoT device to cloud server, and its calculation formula is: Among them, R c Indicates the link transmission rate from the IoT system to the cloud server; It indicates the computing delay of IoT devices choosing to rent cloud server resources for subtask computing. The calculation formula is: In summary, when participating in the competition for the bookkeeping right for the tth time, the total delay of device n in participating in the competition for the bookkeeping right is: In addition, using D max To express the maximum delay that can be accepted in the competition for bookkeeping rights, we have: C3:D n (t)≤D max (14) The task that the device chooses to calculate locally will consume more energy due to a large number of hash calculations. The energy consumption generated by device n when participating in the competition for accounting rights for the tth time is expressed as: Among them, k n is the energy consumption coefficient; Indicates the size of local computing power; π n (t) indicates the size of the selected resource rental ratio; ρ(t) represents the number of CPU cycles required to complete the task; Since the energy of IoT devices is limited, n (t) represents the remaining energy of device n, then: C4:E n (t)≤e n (t) (16) The fee that the competing device n that participates in the bookkeeping right for the tth time needs to pay to ESPm is expressed as: Among them, c m It indicates the charge of ESPm per unit of computing volume; β is the discount rate; Indicates the number of devices selected to rent the resources of m for subtask calculation; π n (t) represents the proportion of rented resources; ρ(t) represents the number of CPU cycles required to complete the task; Indicates whether the resources of the mth ESP are rented for subtask calculation; The computing resource fee paid to CSP is expressed as: Among them, c c It represents the charge of CSP for unit computing amount; Indicates whether to rent CSP resources to perform subtask calculations; In summary, the fee that device n needs to pay for participating in the accounting right competition for the tth time is: In addition, each IoT device has a certain budget for purchasing computing resources. n (t) represents the remaining budget of device n when it participates in the accounting right competition for the tth time, then: C5:C n (t)≤b n (t)(20)。 3. According to a distributed edge cloud resource leasing decision-making method for the Internet of Things blockchain according to claim 1, it is characterized in that: The optimization algorithm based on swarm intelligence is used to optimize the strategic network parameters, including: firstly, initializing the position information of each individual in the population and calculating its fitness information; Then, the position is updated through the position update formula. After the number of iterations, the historical optimal position is decoded to extract the parameters of all its IoT device policy networks; The position update formula includes the following three methods: Method 1: Individual i calculates its position X i After further investigation, the update formula is expressed as: in, represents the position of individual i at the ep iteration; ep indicates the current iteration number; exp() represents the natural exponential function; iter max Indicates the maximum number of iterations; Both r1 and r2 are random numbers with values ​​of (0,1); L represents a A matrix whose values ​​all obey the standard normal distribution; Method 2: Individual i updates its own position by using the best individual position in the current population. The update formula at this time is expressed as: in, represents the position of the individual with the best fitness; represents the individual position of individual i; W is a random number that follows a standard normal distribution; UB and LB represent the upper and lower limits of individual position values, respectively; Method three: In order to improve the exploration degree of individuals and avoid falling into the local optimal solution, individual i also searches for the optimal position by randomly changing its position. The random update formula is expressed as: Among them, X r represents a random position; A means one The matrix has values ​​of 1 or -1.

4. According to a distributed edge cloud resource leasing decision method for the Internet of Things blockchain according to claim 3, it is characterized by: When the individual fitness size of an individual is in the top Q% of the population, the individual is a high fitness individual and has a probability of more than 50% to use formula (40) to update the position; When the individual fitness size of an individual is not in the top Q% of the population, the individual is a low fitness individual, and there is a probability of more than 50% that the position is updated using formula (41); In addition, each individual has a K% probability of updating its position using formula (42).

5. According to a distributed edge cloud resource leasing decision method for IoT blockchain according to claim 4, it is characterized in that: The Q value is 30, and the K value is 5.

6. According to a distributed edge cloud resource leasing decision method for IoT blockchain according to claim 3, it is characterized in that: The fitness information is obtained through the fitness function, and the formula is as follows: in, In the strategy π X Expected return under π X Represents the joint policy of all IoT devices; T represents the maximum number of iterations; Represents a collection of IoT devices; r n (t) represents the reward function of device n.

7. According to a distributed edge cloud resource leasing decision method for IoT blockchain according to claim 1, it is characterized in that: After executing steps S1 to S3, the trained policy network is deployed on the corresponding IoT device. Since each IoT device has an independent policy network, each IoT device can independently make resource leasing decisions based on the environmental status.

Images