A distributed edge cloud resource leasing decision-making method for IoT blockchain

Abstract

The present invention proposes a distributed edge-cloud resource leasing decision-making method for the Internet of Things blockchain, including: S1, obtaining the resource pricing of the edge service provider (ESP) and the cloud service provider (CSP) in the network environment, as well as the remaining resource amount, block speed, maximum tolerable delay and the remaining energy, remaining budget, and maximum computing resources of the local device; S2, constructing an optimization objective function; S3, modeling the optimization objective function as a Markov decision process, and then using a policy-based reinforcement learning method to solve the Markov decision process to obtain a resource leasing strategy. The method of the present invention can achieve efficient distributed task scheduling in a dynamic Internet of Things environment. Combined with blockchain technology, it not only improves the security and transparency of data transmission, but also significantly improves the computing efficiency and stability of the system.

Description

Technical Field

[0001] The present invention relates to the field of Internet of Things blockchain technology, and in particular to a distributed edge cloud resource leasing decision-making method for Internet of Things blockchain. Background Art

[0002] In the traditional Internet of Things (IoT) blockchain resource leasing sector, centralized leasing methods have demonstrated shortcomings in many application scenarios, such as high data transmission overhead, low system reliability, and poor scalability. In contrast, distributed leasing methods offer significant advantages in multiple areas, including smooth scalability, performance, reliability, availability, cost, and disaster recovery. Distributed leasing methods demonstrate particularly strong performance in scenarios involving large-scale data processing and high concurrent access.

[0003] Currently, distributed rental decision-making primarily relies on multi-agent reinforcement learning methods. However, these methods typically require significant time and experience to learn optimal policies. In IoT environments, tasks arrive and the environment changes frequently, requiring agents to adapt quickly to new situations. However, traditional multi-agent reinforcement learning methods can take a long time to converge on a good policy, making them inefficient in scenarios with high real-time requirements.

[0004] Furthermore, interactions between agents can lead to instability in the learning process, hindering convergence. Different agents may become trapped in local optima during learning, failing to find the globally optimal resource leasing strategy. This situation limits system performance optimization and, in turn, impacts the overall effectiveness of distributed leasing. Summary of the Invention

[0005] The present invention aims to at least solve the technical problems existing in the prior art, and in particular innovatively proposes a distributed edge-cloud resource leasing decision-making method for the Internet of Things blockchain.

[0006] The present invention proposes a distributed edge-cloud resource leasing decision-making method for the Internet of Things blockchain, comprising:

[0007] S1, obtains the resource pricing, remaining resource volume, block generation speed, maximum tolerable latency, and remaining energy, remaining budget, and maximum computing resources of the local device of the edge service provider (ESP) and cloud service provider (CSP) in the network environment;

[0008] S2, construct the optimization objective function:

[0009] The block time of the IoT blockchain system is defined as t b , when the average block time of the device is t mean Greater than t bWhen , the system will reduce the difficulty of participating in the competition for the right to record books, and vice versa increase the difficulty of participating in the competition for the right to record books, that is:

[0010]

[0011] Among them, h new Indicates the difficulty of generating blocks in a new round;

[0012] Let μ be the number of CPU revolutions required for a hash calculation. Then, the probability of device n creating a new block per unit time is:

[0013]

[0014] Among them, f n (t) represents the total computing power of device n when it participates in the accounting right competition for the tth time;

[0015] D=2 h , h represents the difficulty of finding the random number nonce; the larger h is, the more difficult it is to find the random number nonce.

[0016] For any device n, the probability of gaining benefits by creating a block before other devices is expressed as:

[0017]

[0018] Among them, λ i (t) represents the probability that device i creates a new block;

[0019] λ n (t) represents the probability that device n creates a new block;

[0020] f i (t) represents the total computing power of device i when it participates in the accounting right competition for the tth time;

[0021] The reward obtained from creating a block for the tth time is represented by R(t), and the expected reward of device n participating in the competition for the bookkeeping right for the tth time is:

[0022]

[0023] Where R(t) represents the block reward for participating in the bookkeeping rights competition for the tth time;

[0024] In summary, the expected benefit of device n participating in the tth accounting right competition is expressed as:

[0025] U n (t) = R n (t)-C n (t)(27)

[0026] Among them, Rn (t) represents the expected block reward of device n;

[0027] C n (t) represents the payment for equipment n;

[0028] The optimization objective is expressed as:

[0029]

[0030] Where T represents the maximum number of iterations;

[0031] Represents a collection of IoT devices;

[0032] S3, the optimization objective function is modeled as a Markov decision process, and then a policy-based reinforcement learning method is used to solve the Markov decision process to obtain the resource rental strategy;

[0033] The policy-based reinforcement learning method requires continuously optimizing the policy network through the swarm intelligence optimization algorithm (SIOA) until the optimal network parameters are found. The position information of the best individuals in the population is decoded to obtain the optimal parameters of the policy network for all IoT devices. The specific steps are as follows:

[0034] S3-1, the objective function of each policy network learning is expressed as:

[0035]

[0036] in, Expresses expected return;

[0037] s0 represents the initial state;

[0038] represents the value function;

[0039] S3-2, uses a swarm intelligence-based optimization algorithm (SIOA) to optimize the policy network parameters to maximize the policy network objective function;

[0040] The method of the present invention replaces the gradient parameter update method in traditional multi-agent reinforcement learning and uses SIOA to update the policy network parameters, overcoming the shortcomings of traditional multi-agent reinforcement learning instability during training and the susceptibility to falling into local optimality.

[0041] Furthermore, the optimization objective function is constructed based on the rental model, calculation model, delay model, energy consumption model, payment model and reward model, that is, the optimization objective function must meet the prerequisites C1 to C5:

[0042] Assume that the maximum computing power of device n is That is, the CPU clock frequency. The local computing power used by device n when participating in the accounting right competition for the tth time is recorded as Then we have:

[0043]

[0044] In addition, due to the limited computing resources of edge servers, using To represent the maximum computing power that ESPs m can provide, where ESPsm represents the mth ESP, Denotes the set of ESPs; the size of the computing resources rented by m to n during the tth competition for bookkeeping rights is recorded as ESPs will allocate computing resources proportionally based on the size of the user's leased resources, that is:

[0045]

[0046] Among them, π n (t) represents the task resource rental ratio of device n when participating in the accounting right competition for the tth time;

[0047] π i (t) represents the ratio of task resource rental of device i when participating in the accounting right competition for the tth time;

[0048] Indicates the selected service provider, which can be ESP and / or CSP. ESP is an edge service provider, and CSP is a cloud service provider.

[0049] Indicates that when participating in the accounting right competition for the tth time, the service provider selected by device n for leasing is ESP;

[0050] Indicates which service provider is selected for resource leasing. Indicates that an edge service provider ESP is selected for leasing, m = 1 to M; M represents the total number of edge service providers;

[0051] I {x} is the indicator function. When x is true, I {x} =1, otherwise I {x} =0;

[0052] As for cloud servers, their computing resources are usually sufficient, so each IoT can use all the computing resources of the rented cloud servers; the computing resources that the cloud servers can provide are recorded as The computing resources rented by device n from CSP when it participates in the competition for accounting rights for the tth time are expressed as:

[0053]

[0054] in, Indicates that a cloud service provider CSP has been selected for leasing;

[0055] In summary, the computing power of device n when participating in the bookkeeping right competition for the tth time is expressed as:

[0056]

[0057] Finally, the minimum computing power required to participate in the bookkeeping right competition is defined as f min , then:

[0058] C2:f n (t)≥f min (5)

[0059] In a blockchain based on the PoW consensus mechanism, if the delay in generating a block is too long, exceeding the threshold set in the network consensus rules, other nodes may consider the block outdated and choose not to accept it. Therefore, IoT devices need to consider latency when choosing to rent resources from edge-cloud servers. The subtasks that IoT devices choose to compute locally only have computational latency, which can be expressed as:

[0060]

[0061] in, represents the computational latency of the subtask that device n chooses to compute locally when participating in the competition for bookkeeping rights for the tth time;

[0062] π n (t) represents the computing resource rental ratio;

[0063] ρ(t) represents the number of CPU cycles required to complete the task;

[0064] represents the local computing power used by device n when participating in the accounting right competition for the tth time;

[0065] In addition to computational delay, there is also communication delay when renting ESPs' resources to perform subtask computations. The computational delay of device n renting m's resources to perform subtask computations during the tth participation in the bookkeeping right competition is expressed as:

[0066]

[0067] in, It represents the computational delay of device n renting the resources of the mth ESP to perform subtask computation when participating in the accounting right competition for the tth time;

[0068] α is the delay discount parameter caused by too many devices renting m computing resources;

[0069] n m (t) represents the number of devices that choose to rent m resources when participating in the accounting right competition for the tth time;

[0070] The amount of computing resources that ESP m leases to device n during the tth competition for bookkeeping rights;

[0071] The communication delay of renting m's resources for subtask calculation is expressed as:

[0072]

[0073] in, represents the communication delay of device n renting the resources of the mth ESP to perform subtask calculation when participating in the accounting right competition for the tth time;

[0074] τ(t) represents the size of the computing task participating in the bookkeeping right competition for the tth time;

[0075] R m is the link transmission rate from IoT device to ESP m;

[0076] Therefore, the total delay of device n renting the resources of m to perform subtask calculation is expressed as:

[0077]

[0078] in, represents the total delay of device n choosing to rent ESP m's resources to perform subtask calculations when participating in the accounting right competition for the tth time;

[0079] Similarly, renting cloud server resources for subtask computing also incurs computational delay and communication delay. The latency for device n to rent cloud server resources for subtask computing is expressed as:

[0080]

[0081] in, represents the total delay of device n choosing to rent CSP resources to perform subtask calculations when participating in the accounting right competition for the tth time;

[0082] The communication delay from the IoT device to the cloud server is calculated as follows:

[0083]

[0084] Among them, R c Indicates the link transmission rate from the IoT system to the cloud server;

[0085] The computational latency of the IoT device when it chooses to rent cloud server resources for subtask computation is expressed as:

[0086]

[0087] In summary, when participating in the competition for bookkeeping rights for the tth time, the total delay of device n in participating in the competition for bookkeeping rights is:

[0088]

[0089] In addition, using D max To express the maximum delay that can be accepted in the competition for bookkeeping rights, we have:

[0090] C3:D n (t)≤D max (14)

[0091] The task that the device chooses to perform calculations locally will consume more energy due to a large number of hash calculations. The energy consumption generated by device n when participating in the competition for accounting rights for the tth time is expressed as:

[0092]

[0093] Among them, k n is the energy consumption coefficient;

[0094] Indicates the size of local computing power;

[0095] π n (t) represents the proportion of rented resources;

[0096] ρ(t) represents the number of CPU cycles required to complete the task;

[0097] Since the energy of IoT devices is limited, n (t) represents the remaining energy of device n, then:

[0098] C4:E n (t)≤e n (t) (16)

[0099] When IoT devices rent computing resources from edge-cloud servers, they also pay a fee to the service provider. From an ESP's perspective, ESPs offer discounts to attract more devices to use their services, increase potential revenue, and compensate for the decline in service quality caused by the increase in the number of devices served. Therefore, the fee that device n, competing for accounting rights for the tth time, needs to pay to ESP m is expressed as:

[0100]

[0101] Among them, c m It represents the charge of ESP m for unit computing amount;

[0102] β is the discount rate;

[0103] Indicates the number of devices that choose to rent m's resources for subtask calculation;

[0104] π n (t) represents the proportion of rented resources;

[0105] ρ(t) represents the number of CPU cycles required to complete the task;

[0106] Indicates whether the resources of the mth ESP are rented for the calculation of the subtask;

[0107] The computing resource fee paid to CSP is expressed as:

[0108]

[0109] Among them, c c Indicates the CSP's charge per unit of computing power;

[0110] Indicates whether to rent CSP resources for subtask calculation;

[0111] In summary, the fee that device n needs to pay for participating in the bookkeeping right competition for the tth time is:

[0112]

[0113] In addition, each IoT device has a certain budget for purchasing computing resources. n (t) represents the remaining budget of device n when it participates in the accounting right competition for the tth time, then:

[0114] C5:C n (t)≤b n (t)(20).

[0115] Furthermore, the optimization of the policy network parameters using a swarm intelligence-based optimization algorithm (SIOA) includes: first initializing the position information of each individual in the population (i.e., the policy network parameters of multiple agents) and calculating their fitness information;

[0116] Then, the position update formula is used to update the position. After the number of iterations is completed, the historical optimal position is decoded and the parameters of all its IoT device strategy networks are extracted;

[0117] The position update formula includes the following three methods:

[0118] Method 1: Individual i calculates his position X i After further investigation, the update formula is expressed as:

[0119]

[0120] in, represents the position of individual i at ep iterations;

[0121] ep represents the current iteration number;

[0122] exp() represents the natural exponential function;

[0123] iter max Indicates the maximum number of iterations;

[0124] Both r1 and r2 are random numbers with values ​​of (0,1);

[0125] L represents a All values ​​of the matrix obey the standard normal distribution.

[0126] In the second method, individual i updates its own position by using the optimal individual position in the current population. The update formula at this time is expressed as:

[0127]

[0128] in, represents the position of the individual with the best fitness;

[0129] represents the individual position of individual i;

[0130] W is a random number that follows a standard normal distribution;

[0131] UB and LB represent the upper and lower limits of individual position values, respectively;

[0132] Method three: In order to improve the individual's exploration degree and avoid falling into the local optimal solution, individual i also searches for the optimal position by randomly changing its position. We express its random update formula as follows:

[0133]

[0134] Among them, X r represents a random position;

[0135] A means one The matrix has a value of 1 or -1.

[0136] Furthermore, when the individual fitness size of an individual is in the top Q% of the population, the individual is considered to have a high fitness and has a probability of more than 50% to use formula (40) to update the position;

[0137] When the individual fitness size of an individual is not in the top Q% of the population, the individual is considered to have low fitness and has a probability of more than 50% to use formula (41) to update its position;

[0138] In addition, each individual has a K% probability of updating its position using formula (42).

[0139] Furthermore, the value of Q is 30, and the value of K is 5.

[0140] Furthermore, the fitness information is obtained through a fitness function, and the formula is as follows:

[0141]

[0142] in, In the strategy π X Expected return under

[0143] π X Represents the joint policy of all IoT devices;

[0144] T represents the maximum number of iterations;

[0145] Represents a collection of IoT devices;

[0146] r n (t) represents the reward function of device n.

[0147] The greater the fitness of an individual, the better the solution represented by the individual, that is, the better the network parameters, and thus the greater the cumulative reward.

[0148] Furthermore, after executing steps S1 to S3, the trained policy network is deployed on the corresponding IoT device. Since each IoT device has an independent policy network, each IoT device can independently make resource leasing decisions based on the environmental status.

[0149] The multi-agent reinforcement learning algorithm adopted in this invention adopts a centralized training distributed execution solution paradigm. During training, some global information that cannot be seen by a single agent is used to achieve better training results, but this information is not used during execution. Each agent takes actions directly according to its own strategy to achieve the effect of distributed execution.

[0150] In summary, due to the adoption of the above technical solutions, the distributed resource leasing method based on swarm intelligence multi-agent reinforcement learning proposed in the present invention can achieve efficient distributed task scheduling in a dynamic Internet of Things environment. Combined with blockchain technology, it not only improves the security and transparency of data transmission, but also significantly improves the computing efficiency and stability of the system. By optimizing resource scheduling and task allocation, the method of the present invention can achieve rapid convergence and high-quality strategy optimization in a dynamic Internet of Things environment. It effectively reduces system energy consumption and ensures good convergence and adaptability of the system, thereby maintaining excellent performance in various complex network scenarios. The specific advantages are:

[0151] (1) This paper designs a swarm intelligence-enabled multi-agent reinforcement learning algorithm, SIMARL. This algorithm utilizes SIOA's powerful exploration and optimization capabilities to replace the traditional gradient descent method in DRL, overcoming DRL's instability during training and its tendency to fall into local optimality. The innovation of SIMARL lies in that it can not only enhance the efficiency of collaboration between agents in a multi-agent system, but also significantly improve the adaptability of the optimization process to complex environmental changes.

[0152] (2) This paper proposes a distributed dynamic resource leasing scheme for PoW computing tasks in IoT devices. This scheme intelligently determines the allocation of computing tasks between cloud and edge servers based on real-time environmental changes and dynamic resource allocation, optimizing resource utilization and significantly improving overall system benefits.

[0153] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0154] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which:

[0155] Figure 1 This is the IoT blockchain system architecture of the present invention.

[0156] Figure 2 This is a schematic diagram of the framework of the PoW distributed dynamic resource leasing solution based on SIMARL of the present invention.

[0157] Figure 3 It is a convergence analysis diagram of SIMARL under different numbers of IoT devices in the present invention.

[0158] Figure 4 It is a convergence analysis diagram of SIMARL under different numbers of ESPs in the present invention.

[0159] Figure 5 2 is a schematic diagram for comparing solutions when the number of IoT devices in the present invention is 20.

[0160] Figure 6 3 is a schematic diagram of the comparison of solutions when the number of IoT devices of the present invention is 40.

[0161] Figure 7 3 is a schematic diagram for comparing the methods of the present invention when the number of IoT devices is 20.

[0162] Figure 8 3 is a schematic diagram for comparing the methods of the present invention when the number of IoT devices is 40.

[0163] Figure 9 This is a schematic diagram of the solution quality comparison when the number of IoT devices in the present invention is 20.

[0164] Figure 10 This is a schematic diagram of the solution quality comparison when the number of IoT devices in the present invention is 40.

[0165] Figure 11 This is a schematic diagram of the robustness comparison under different numbers of IoT devices in the present invention.

[0166] Figure 12 This is a schematic diagram comparing the robustness of the present invention under different numbers of ESPs. DETAILED DESCRIPTION

[0167] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0168] 1 System framework and modeling

[0169] The present invention relates to a typical IoT blockchain system, which includes multiple edge service providers (ESPs) and a cloud service provider (CSP). The specific architecture is as follows: Figure 1 As shown. The system consists of a large number of heterogeneous IoT devices, which can be connected to the nearest base station (BS) through a wireless network. Multiple edge servers are deployed near the BS, and IoT devices can also connect to the cloud server through the BS. In this system, IoT devices can offload PoW computing tasks to the edge or cloud by renting computing resources from edge servers or cloud servers, so as to enable the cloud to participate in the competition for accounting rights. We use Represents a collection of N IoT devices, represents a set of M ESPs, Denote the CSP. A two-tuple (τ(t), ρ(t)) represents the size (in bits) of the computational task for the tth accounting right competition and the number of CPU cycles required to complete the task. Furthermore, we consider that the computational tasks participating in the accounting right competition can be partially offloaded. IoT devices participating in the accounting right competition can choose to perform all computations locally or offload part of their tasks to edge servers or cloud servers.

[0170] We define a two-tuple To represent IoT devices (terminals) The rental decision when participating in the competition for bookkeeping rights for the tth time, where π n (t) represents the resource rental ratio, Indicates the service provider selected for leasing. n The value range of (t) is [0, 1], The possible values ​​of are {0, 1, 2, ..., M}, where a value of 0 indicates renting cloud server resources, and values ​​from 1 to M indicate M ESPs, respectively.

[0171] Since participating in the competition for bookkeeping rights requires a large amount of computing resources, the terminals in the IoT need to rent certain computing resources from edge servers or cloud servers to improve their own computing power in order to participate in the competition for bookkeeping rights. Since most of the terminals in the IoT are heterogeneous, their computing power also varies. Let the maximum computing power of terminal n be That is, the CPU clock frequency (CPU cycles / s), and the local computing power used by terminal n when participating in the competition for bookkeeping rights for the tth time is recorded as Then we have:

[0172]

[0173] In addition, due to the limited computing resources of edge servers, we use To represent ESPs The maximum computing power that can be provided. The size of the computing resources rented by m to n during the tth competition for bookkeeping rights is recorded as Since ESPs have limited computing resources, in order to meet user needs as much as possible, we assume that ESPs will allocate computing resources proportionally based on the size of the user's leased resources, that is:

[0174]

[0175] Among them, I {x} is the indicator function. When x is true, I{x} =1, otherwise I {x} = 0. As for cloud servers, their computing resources are usually sufficient, so we assume that each IoT can use all the computing resources of the rented cloud servers. We denote the computing resources that the cloud server can provide as Then the computing resources rented by terminal n from CSP when participating in the competition for accounting rights for the tth time can be expressed as

[0176]

[0177] In summary, the computing power of terminal n when participating in the competition for bookkeeping rights for the tth time can be expressed as:

[0178]

[0179] Finally, we define the minimum computing power required to participate in the competition for bookkeeping rights as f min , then:

[0180] C2:f n (t)≥f min (5)

[0181] In a blockchain based on the PoW consensus mechanism, if the delay in generating a block is too long, exceeding the threshold set in the network consensus rules, other nodes may consider the block outdated and choose not to accept it. Therefore, IoT devices need to consider latency when choosing to rent resources from edge-cloud servers. The subtasks that IoT devices choose to compute locally only have computational latency, which can be expressed as:

[0182]

[0183] In addition to computational delay, renting ESPs' resources for subtask computation also incurs communication delay. The computational delay of terminal n renting m's resources for subtask computation during the tth competition for bookkeeping rights can be expressed as:

[0184]

[0185] Among them, α is the delay discount parameter caused by too many terminals renting m computing resources, n m (t) represents the number of terminals that choose to rent m resources to perform subtask calculations when participating in the competition for bookkeeping rights for the tth time. The communication delay of renting m resources to perform subtask calculations can be expressed as:

[0186]

[0187] Among them, R mis the link transmission rate from the IoT device to ESP m. Therefore, the total delay of terminal n renting m’s resources to perform subtask calculation can be expressed as:

[0188]

[0189] Similarly, renting the computing resources of the cloud server for subtask calculations also has computing delays and communication delays. The delay of terminal n renting the resources of the cloud server for subtask calculations can be expressed as:

[0190]

[0191] in, The communication delay from the IoT device to the cloud server is calculated as follows:

[0192]

[0193] Among them, R c Indicates the link transmission rate from the IoT system to the cloud server. The computational latency of the IoT device when it chooses to rent cloud server resources for subtask computation can be expressed as:

[0194]

[0195] In summary, when participating in the competition for bookkeeping rights for the tth time, the total delay of terminal n in participating in the competition for bookkeeping rights is:

[0196]

[0197] In addition, we use D max To express the maximum delay that can be accepted in the competition for bookkeeping rights, we have:

[0198] C3:D n (t)≤D max (14)

[0199] The task that the terminal chooses to perform calculations locally will consume more energy due to a large number of hash calculations. The energy consumption generated by terminal n when participating in the competition for accounting rights for the tth time can be expressed as:

[0200]

[0201] Among them, k n The energy consumption coefficient is different for different types of terminals.

[0202] Indicates the size of local computing power;

[0203] π n(t) represents the proportion of resource rental;

[0204] ρ(t) represents the number of CPU cycles required to complete the task;

[0205] We consider that the energy of IoT devices is limited. n (t) represents the remaining energy of terminal n, then:

[0206] C4:E n (t)≤e n (t) (16)

[0207] When IoT devices rent computing resources from edge-cloud servers, they also pay a fee to the service provider. From an ESP's perspective, ESPs offer discounts to attract more terminals to use their services, increase potential revenue, and compensate for the decline in service quality caused by the increase in the number of terminals served. Therefore, the fee that terminal n, competing for accounting rights for the tth time, needs to pay to ESP m can be expressed as:

[0208]

[0209] Where β is the discount rate;

[0210] Indicates the number of terminals that choose to rent the resources of m for subtask calculation;

[0211] π n (t) represents the proportion of resource rental;

[0212] ρ(t) represents the number of CPUT cycles required to complete the task;

[0213] Indicates whether the mth ESP is selected for leasing;

[0214] c m It represents the charge of ESP m for unit computing amount;

[0215] The computing resource fee paid to CSP can be expressed as:

[0216]

[0217] Among them, c c Indicates the CSP's charge per unit of computing power. Indicates whether a CSP is selected for leasing. In summary, the fee that terminal n needs to pay for participating in the tth competition for accounting rights is:

[0218]

[0219] In addition, we consider that each IoT device has a certain budget for purchasing computing resources. n (t) represents the remaining budget of terminal n when it participates in the competition for bookkeeping rights for the tth time, then:

[0220] C5:C n (t)≤b n (t) (20)

[0221] In addition to the above restrictions and fees, each IoT terminal competing for the right to record data has the potential to earn a block reward by being the first to produce a block. To generate a new block, IoT terminals must compete to complete a mathematical puzzle based on a hash encryption algorithm. The winner has the right to record transactions on the blockchain and receive a reward. The hash function H outputs an L-bit hash value based on the Merkle tree root data tx of all transaction data in the block, the hash value prev.hash of the parent block, and the random number nonce. L represents the L-bit hash value. The IoT terminal needs to find a random number nonce that satisfies the following conditions:

[0222] H(tx,prev.hash,nonce)≤1 L-h (twenty one)

[0223] Where h represents the difficulty of finding the random number nonce. The larger h is, the more difficult it is to find the random number nonce. In this paper, we consider the block time of the IoT blockchain system to be defined as t b , when the average block time of the terminal is t mean Greater than t b When , the system will reduce the difficulty of participating in the competition for the right to record books, and vice versa increase the difficulty of participating in the competition for the right to record books, that is:

[0224]

[0225] Among them, h new Indicates the difficulty of generating a new round of blocks; D = 2 h .

[0226] Assuming that the number of CPU revolutions required for a hash calculation is μ, the probability of terminal n creating a new block per unit time is:

[0227]

[0228] Among them, f n (t) represents the total computing power of terminal n when it participates in the accounting right competition for the tth time;

[0229] For any terminal n, the probability of gaining benefits by creating a block before other terminals can be expressed as:

[0230]

[0231] Among them, λ i (t) represents the probability of terminal i creating a new block;

[0232] λ n (t) represents the probability of terminal n creating a new block;

[0233] f i (t) represents the computing power of device i when it participates in the accounting right competition for the tth time;

[0234] We denote the reward obtained from creating a block for the tth time by R(t). Then the expected reward of terminal n participating in the competition for bookkeeping rights for the tth time is:

[0235]

[0236] Where R(t) represents the block reward for participating in the bookkeeping rights competition for the tth time;

[0237] In summary, the expected benefit of terminal n participating in the t-th competition for bookkeeping rights can be expressed as:

[0238] U n (t) = R n (t)-C n (t) (26)

[0239] Among them, C n (t) represents the fee that terminal n needs to pay when participating in the accounting right competition for the tth time

[0240] This paper mainly considers the optimization of each IoT terminal within a period of competition for accounting rights, under the constraints of latency, energy consumption, budget, etc., and taking into account the leasing decisions of other IoT terminals, to maximize the cumulative benefits of the entire IoT system by rationally leasing computing resources on the edge cloud. Therefore, the optimization goal of this paper can be expressed as:

[0241]

[0242] stC1-C

[0243] 2 Distributed Resource Leasing Method Based on SIMARL

[0244] This paper uses a swarm intelligence-enabled multi-agent reinforcement learning algorithm (SIMARL) to solve the distributed resource leasing problem in IoT blockchain systems. By replacing the traditional gradient descent method in DRL with SIOA's powerful exploration and optimization capabilities, SIMARL overcomes DRL's instability and tendency to fall into local optimality during training. This not only enhances the collaboration efficiency between agents in a multi-agent system, but also significantly improves the optimization process's adaptability to complex environmental changes. Figure 2 The relevant details of the distributed resource leasing solution framework based on SIMARL proposed in this invention are presented in detail. (1) In the training phase (centralized training): first obtain the resource pricing of ESP and CSP in the network environment as well as the remaining resource amount, block speed, maximum tolerance delay and other parameters, initialize the position information of each individual in the population (i.e., the policy network parameters of multiple intelligent agents), and calculate its fitness information. Update its own position through three position update formulas. After the number of iterations is completed, decode the historical optimal position and extract the parameters of the policy network of all its IoT devices. (2) Execution phase (distributed execution): Place the trained policy network on the corresponding IoT device. Since each IoT device has an independent policy network, each IoT device can make a rental decision independently based on the environmental status.

[0245] In order to effectively use SIMARL to solve the above complex optimization problem, we model it as a Markov decision process (MDP) and use a five-tuple Indicates that, Represents a collection of intelligent agents. In the scenario considered in this article, each IoT device represents an intelligent agent, so represents the state set of all agents, represents the joint action set of all agents, represents the state transition probability, Represents the reward function. The state space, action space, and reward function of each agent are defined as follows:

[0246] (1) State space: The system state when participating in the bookkeeping right competition for the tth time is expressed as It consists of the states of all IoT devices, namely s n (t) represents the state of device n when it participates in the competition for bookkeeping rights for the tth time. We define

[0247] s n (t)=[h(t),b n (t), e n (t)] (28)

[0248] That is, the state of each intelligent agent is composed of the current difficulty of the competition for the right to participate in accounting, the remaining budget, and the remaining energy.

[0249] (2) Action space: The joint action when participating in the bookkeeping right competition for the tth time is expressed as in

[0250]

[0251] It represents the leasing decision of device n when it participates in the accounting right competition for the tth time. It should be noted that the decision made by the IoT device needs to satisfy the constraints C1-C5.

[0252] (3) Reward function: The reward function for the tth participation in the bookkeeping right competition is expressed as It consists of the rewards of all IoT devices, namely in

[0253] r n (t) = U n (t)(30)

[0254] There are two main learning paradigms for reinforcement learning: policy-based methods and value-based methods. SIMARL is essentially a policy-based reinforcement learning (RL) method. Unlike the value-based method, the policy-based method directly and explicitly learns a target policy π during the learning process. θ , where θ represents the weights and bias parameters of the policy network. Policy-based reinforcement learning methods are more suitable for our scenario than value-based methods for the following reasons: First, policy-based methods directly process the continuous action space, avoiding the complexity of discretizing actions. Second, by optimizing the policy itself, they make policy search more direct and reduce the impact of errors in value function estimation. Furthermore, policy-based methods balance exploration and exploitation through random sampling, making them more suitable for complex dynamic environments. In high-dimensional state spaces, policy-based methods have better scalability and convergence. Therefore, policy-based methods can provide a more optimal solution to the IoT blockchain resource leasing problem.

[0255] The objective function of policy network learning can be expressed as:

[0256]

[0257] in, Expresses expected return;

[0258] s0 represents the initial state;

[0259] Represents the value function.

[0260] In traditional policy-based RL methods, the gradient ascent method is basically used to maximize this objective function to obtain the optimal policy. The gradient of this objective function can be obtained:

[0261]

[0262] in, In the strategy π θ Expected return under

[0263] π θ (a|s) represents the strategy under the network parameter θ;

[0264] After the gradient is calculated using formula (32), all parameters are then updated in the direction of the gradient increase to maximize the objective function of the policy network.

[0265] represents the action-value function. Traditional policy-based RL methods can estimate it in many ways. For example, in the REINFORCE algorithm, the Monte Carlo method is used to estimate it. Therefore, the policy gradient in the REINFORCE algorithm is:

[0266]

[0267] Where γ is the discount factor and r is the reward. In the RL method based on the Actor-Critic structure, the Critic value network is used to fit a value function to guide policy learning, and the Actor network is the policy network. Its policy gradient can be expressed as:

[0268]

[0269] Then, the gradient ascent method is also used to update the Actor strategy network parameters. The gradient of its value function can be expressed as:

[0270]

[0271] The Critic value network also uses the gradient descent method to update the network parameters.

[0272] However, there are several challenges involved in using gradient update methods to optimize network parameters. First, the choice of learning rate is crucial, and its size directly affects the quality of experimental results. A larger learning rate may lead to unstable learning and difficulty in convergence; while a smaller learning rate may lead to being trapped in a local optimal solution and unable to escape. Secondly, the design of the network structure also has a significant impact on the results. An overly complex structure may cause gradient vanishing or gradient explosion, while an overly simple structure cannot effectively learn complex function mappings. Finally, even if researchers have determined the optimal learning rate, network structure and other parameters through a large number of experiments, it is usually necessary to readjust the parameters when solving different problems, which will incur a large time cost.

[0273] In SIMARL, each agent (IoT device) contains only one policy network, and SIOAs is used instead of the traditional gradient update method to optimize the policy network parameters. The advantage of this method is that there is no need to adjust the learning rate and network structure through large-scale experiments, which saves a lot of time and resources. The reason for adopting such a simple network structure is that SIMARL needs to achieve fast convergence and efficient calculation in dynamic resource leasing scenarios. Simple network structures are more suitable for combining with swarm intelligence to cope with dynamic changes in high-dimensional state spaces. In addition, some updated network structures usually introduce higher algorithm complexity and computational overhead, which may not meet the design goals of SIMARL. SIOAs is an optimization algorithm based on group cooperation, which represents a set of solutions by the position of each individual in the search space. The quality of an individual's position is judged by an indicator called fitness (usually an objective function). Each individual can use the position information and fitness of other individuals to adjust its own position in order to find better fitness, that is, to find a better solution. In SIMARL, we represent the positions of all individuals in the population as a dimensional space, where represents the population size, Represents the position space dimension, that is, the dimension of the optimization problem. The position information of each individual represents the weight and bias information in the policy network, so The size of depends on the number of policy network parameters. Since there are N agents in the scenario we consider, there are N policy networks in SIMARL, which all include an input layer, a hidden layer and an output layer. To represent the total number of parameters of all IoT device policy networks, where Respectively represent the number of weight parameters and bias parameters in the strategy network of agent i. Then the optimization problem dimension is Let θ be the parameter of the policy network of device n, which is expressed as follows:

[0274]

[0275] Let X i Represents an individual 's location information, then

[0276]

[0277] Of course, we need to decode the individual's position information into N policy networks and replace it with neural network parameters. The decoding pseudocode is shown in Algorithm 1.

[0278]

[0279] We define f(X) as the fitness function to evaluate the quality of individual positions. In this paper, we directly use the optimization goal of the policy network (the optimization goal is formulated for the Markov decision process, and the actual optimization goal is the same as before, because the reward function is the optimization goal of this paper) as the fitness function (fitness can also be regarded as the optimization goal of SIOA), that is:

[0280]

[0281] Among them, π X represents the joint strategy of all IoT devices. More generally, we can express the fitness function as follows:

[0282]

[0283] The greater the fitness of an individual, the better the solution it represents, that is, the better the network parameters, and thus the greater the cumulative reward. The pseudo code for calculating the fitness function is shown in Algorithm 2.

[0284]

[0285] Each individual in the population can reach a better fitness position by updating its own position. In this paper, each individual can update its position in three ways to seek the optimal solution. First, individual i can update its position X i After further investigation, the update formula can be expressed as:

[0286]

[0287] in, represents the position of individual i at ep iterations;

[0288] ep represents the current iteration number;

[0289] exp() represents the natural exponential function;

[0290] iter maxIndicates the maximum number of iterations;

[0291] Both r1 and r2 are random numbers with values ​​of (0,1);

[0292] L represents a All values ​​of the matrix obey the standard normal distribution.

[0293] Secondly, individual i can update its own position by using the optimal individual position in the current population. The update formula at this time can be expressed as:

[0294]

[0295] in, represents the position of the individual with the best fitness;

[0296] W is a random number that follows a standard normal distribution;

[0297] UB and LB represent the upper and lower limits of individual position values, respectively;

[0298] Finally, in order to improve the individual's exploration degree and avoid falling into the local optimal solution, individual i can also find the optimal position by randomly changing its position. We express its random update formula as:

[0299]

[0300] Among them, X r Indicates a random position; A indicates a The matrix has a value of 1 or -1.

[0301] In addition, we also define that individuals with higher fitness (individual fitness size in the top 30%) are more inclined to use formula (40) to update their positions, and individuals with lower fitness (individual fitness size not in the top 30%) use formula (41) to update their positions, and each individual has a 5% probability of randomly updating its position. During each iteration, the historical optimal individual position will be saved. When the current optimal position after the next iteration is better than the historical optimal position, the historical optimal position will be replaced by the current optimal position. Otherwise, the historical optimal position remains unchanged. After the iteration is completed, the individual with the historical optimal position is decoded to obtain the optimal parameters of the IoT device strategy network to obtain more cumulative rewards. In summary, the pseudo code of the distributed resource leasing algorithm based on SIMARL is shown in Algorithm 3.

[0302]

[0303]

[0304] 3 Simulation Experiment and Result Analysis

[0305] In this simulation environment, the number of basic IoT devices is set to 20, the number of edge servers is set to 10, the system block time is set to 30s, the maximum acceptable delay is set to 2 times the block time, and the maximum computing power of the local terminal is set to The value of is randomly generated between 10 and 80Mcycles / s, and the maximum computing power of the edge node The value of is randomly generated between 4 and 6 Gcycles / s. The minimum values ​​of the task size τ(t) and the number of CPU cycles ρ(t) required to complete the task are 100Mb and 2×10 9 cycles, the minimum computing power to participate in the bookkeeping right competition is 50Mcycles / s, the delay discount α and discount rate β are both 0.04, and the CSP charges c for unit computing power. c The value is 5×10 -10 units / cycles, ESPs price computing resources at 7×10 -10 to 10×10 -10 Units / cycles are randomly generated, and the block reward R(t) for participating in the bookkeeping right competition is 30 units.

[0306] In order to verify the convergence performance of SIMARL, we evaluated the convergence of SIMARL under different numbers of IoT devices and different numbers of ESPs. The experimental results are as follows: Figure 3 and Figure 4 As shown in the figure, as the problem size increases and the scenario changes, SIMARL can still reach convergence in a relatively small number of iterations. This is because SIMARL uses SIOAs to optimize the policy network, and the SIOA used shows good results in optimizing neural network parameters. Therefore, SIMARL has good convergence performance when solving various optimization problems.

[0307] Subsequently, to verify the effectiveness of the resource leasing scheme based on the SIMARL method, this paper compares the average cumulative benefits of IoT devices under the proposed resource leasing scheme with the full leasing computing scheme when the number of IoT devices is 20 and 40. The results are as follows: Figure 5 and Figure 6As shown in the figure, under the full resource leasing scheme, as the number of block generation increases, the average revenue of IoT devices gradually decreases, especially when the scale of IoT device requests increases. The revenue of IoT devices under the full resource leasing scheme reaches a plateau after generating 60 blocks, while the resource leasing scheme based on SIMARL can always maintain a high revenue. This is because the resource leasing scheme based on SIMARL can fully utilize local energy consumption and budget and allocate them reasonably to improve the cumulative revenue of IoT devices.

[0308] Then we conduct a detailed comparison of SIMARL with the more classic MARL algorithms in recent years, MADDPG, MASAC, MATD3, HATRPO and H-MADDPG. First, we evaluate the convergence of SIMARL and other MARL algorithms in solving the optimization problem in this paper. The results are as follows: Figure 7 and Figure 8 As shown in the figure, the SIMARL method proposed in this paper outperforms other methods in terms of convergence speed and solution quality. When the scale of the optimization problem is increased, some classic algorithms become difficult to converge, while SIMARL can still converge at a faster speed and achieve better solution quality than other methods. This is because other methods use gradient descent for optimization, which easily fall into local optimal solutions, and the setting of hyperparameters can lead to unstable convergence. SIMARL uses the SIOAs method to optimize the policy network, which has a good ability to escape from local optimal solutions.

[0309] Then, we compared the solution quality of SIMARL-based resource leasing scheme with other methods under different problem scales. Figure 9 and Figure 10 As shown in the figure, no matter the size of the problem, the solution quality of the resource leasing scheme based on SIMARL is always better than other methods.

[0310] Finally, to verify the robustness of the SIMARL method, we compared the cumulative benefits of SIMARL and other methods under different numbers of IoT devices and different numbers of ESPs. The results are as follows: Figure 11 and Figure 12As shown in the figure, as the number of IoT devices increases, the average cumulative benefit of IoT devices gradually decreases, while as the number of ESPs increases, the average cumulative benefit of IoT devices gradually increases. This is because the more IoT devices there are, the greater the resource competition between them. As the number of ESPs increases, the more computing resources there are, and the resource competition decreases. In addition, it can be clearly seen from the figure that the solution quality of the resource leasing scheme based on SIMARL is always better than other MARL methods, and it has good robustness in solving the optimization problem proposed in this paper. This is because SIOAs is not sensitive to the parameters of the optimization problem. Even if the problem scale increases or the environmental parameters change, it can still maintain good solution quality and has good robustness.

[0311] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

Claims

1. A distributed edge cloud resource leasing decision-making method for IoT blockchain, characterized by: include: S1, obtains the resource pricing, remaining resources, block generation speed, maximum tolerable latency, and remaining energy, remaining budget, and maximum computing resources of the edge service provider (ESP) and cloud service provider (CSP) in the network environment; S2, construct the optimization objective function: The block time of the IoT blockchain system is defined as , when the average block time of the device Greater than When the IoT blockchain system reduces the difficulty of competing for the right to participate in bookkeeping, it will increase the difficulty of competing for the right to participate in bookkeeping, that is: in, Indicates the difficulty of generating blocks in a new round; Let the number of CPU revolutions required for a hash calculation be , then per unit time, the device The probability of creating a new block is: in, Representation device In the Total computing power when participating in the bookkeeping rights competition; , Indicates the difficulty of finding a random number; For any device , the probability of gaining benefits by creating blocks before other devices is expressed as: in, Representation device The probability of creating a new block; Representation device The probability of creating a new block; Representation device In the Total computing power when participating in the bookkeeping rights competition; The first The reward for creating a block is used Indicates that the device No. The expected reward for participating in the competition for bookkeeping rights is: in, Indicates the The block reward for participating in the bookkeeping rights competition; In summary, equipment Participate in The expected return of participating in the bookkeeping right competition is expressed as: in, Representation device Expected block reward of ; Representation device the expenses paid; The optimization objective is expressed as: in, Indicates the maximum number of iterations; Represents a collection of local devices; S3, the optimization objective function is modeled as a Markov decision process, and then a policy-based reinforcement learning method is used to solve the Markov decision process to obtain the resource rental strategy; The policy-based reinforcement learning method requires continuously optimizing the policy network through a swarm intelligence optimization algorithm until the optimal network parameters are found, thereby obtaining the optimal parameters of the policy network for all local devices. The specific steps are as follows: S3-1, the objective function of each policy network learning is expressed as: in, Expresses expected return; Indicates the initial state; represents the value function; S3-2, uses a swarm intelligence-based optimization algorithm to optimize the policy network parameters, thereby maximizing the policy network objective function.

2. A distributed edge cloud resource leasing decision-making method for IoT blockchain according to claim 1, characterized in that: The optimization objective function is constructed based on the rental model, calculation model, delay model, energy consumption model, payment model and reward model, that is, the optimization objective function must satisfy Prerequisites: Equipment The maximum computing power is , that is, the CPU clock frequency, the first Equipment when participating in the bookkeeping rights competition The local computing power used is denoted as , then: In addition, due to the limited computing resources of edge service providers, using To represent ESPs The maximum computing power that can be provided, including ESPs Indicates the ESP, , Represents the set of ESPs; Participating in the bookkeeping rights competition Lease to The computing resource size is denoted as , so that ESPs will allocate computing resources proportionally according to the size of the user's leased resources, that is: in, Indicates the calculation of Equipment when participating in the bookkeeping rights competition The proportion of resource rental; Indicates the calculation of Equipment when participating in the bookkeeping rights competition The proportion of resource rental; Indicates the selected service provider, which can be ESP and / or CSP. ESP is an edge service provider, and CSP is a cloud service provider. Indicates in Equipment when participating in the bookkeeping rights competition Choose ESP as the service provider for leasing; Which service provider is used for resource leasing? Indicates that the edge service provider ESP is selected for leasing. ; represents the total number of edge service providers; is the indicator function, when When true, ,otherwise ; As for cloud service providers, their computing resources are usually sufficient, so each IoT can use all the computing resources rented by the cloud service provider; the computing resources that the cloud service provider can provide are recorded as , then the device The computing resources rented from CSP in the competition for bookkeeping rights in the tth round are expressed as: in, Select a cloud service provider (CSP) for leasing; In summary, equipment In the The computing power possessed when participating in the bookkeeping rights competition is expressed as: Finally, the minimum computing power required to participate in the bookkeeping rights competition is defined as , then: The subtasks that the local device chooses to compute locally only have computational latency, which can be expressed as: in, Indicates in Equipment participating in the competition for bookkeeping rights Select the computational latency of the subtask to be computed locally; Indicates the resource rental ratio; Indicates the number of CPU cycles required to complete the task; Indicates the Equipment when participating in the bookkeeping rights competition The local computing power used; In addition to the computational delay, renting ESPs computing resources to perform computational subtasks also incurs communication delay. Equipment when participating in the bookkeeping rights competition Rental The computational delay of subtask computation using resources is expressed as: in, Indicates in Equipment participating in the competition for bookkeeping rights Rent the The computational latency of each ESP resource performing subtask computation; Due to lease Delay discount parameter caused by too many devices with computing resources; Indicates the Choose to rent when participating in the bookkeeping rights competition The number of devices for the resource; No. ESP participated in the bookkeeping rights competition Equipment rental The size of computing resources; Choose to rent The communication delay of the resource for subtask calculation is expressed as: in, Indicates in Equipment participating in the competition for bookkeeping rights Rent the The communication delay of each ESP resource for subtask calculation; Indicates the The size of the computing task participating in the bookkeeping rights competition; For local devices to ESP Link transmission rate; Therefore, the device Rental The total latency of a resource performing subtask calculations is expressed as: in, Indicates in Equipment participating in the competition for bookkeeping rights Choose to rent an ESP The total delay of subtask calculation by resources; Similarly, subtasks that use cloud service provider resources for computing also have computing and communication delays; The latency of renting cloud service provider resources for subtask computing is expressed as: in, Indicates in Equipment participating in the competition for bookkeeping rights The total latency of choosing to rent CSP resources for subtask computation; represents the communication delay from the local device to the cloud service provider, and its calculation formula is: in, Indicates the link transmission rate from the IoT blockchain system to the cloud service provider; It represents the computational latency of the local device when it chooses to rent cloud service provider resources to perform subtask computation. Its calculation formula is: In summary, When participating in the competition for bookkeeping rights for the first time, the equipment The total delay of competing for the right to participate in bookkeeping is: In addition, use To express the maximum delay that can be accepted in the competition for bookkeeping rights, we have: The task that the device chooses to calculate locally will consume more energy due to a large number of hash calculations. In the The energy consumption generated when participating in the competition for bookkeeping rights is expressed as: in, is the energy consumption coefficient; Indicates the size of local computing power; Indicates the size of the selected resource rental ratio; Indicates the number of CPU cycles required to complete the task; Since the energy of local devices is limited, Representation device The remaining energy is: No. Equipment that participates in the competition for bookkeeping rights Need to pay to ESP The cost is expressed as: in, Indicates ESP Charges per unit of computational capacity; is the discount rate; Indicates that you will rent The number of devices that use resources to perform subtask calculations; Indicates the proportion of rented resources; Indicates the number of CPU cycles required to complete the task; Indicates whether the first The resources of each ESP are used to calculate the subtasks; The computing resource fee paid to CSP is expressed as: in, Indicates the CSP's charge per unit of computing power; Indicates whether to rent CSP resources for subtask calculation; In summary, equipment Participate in The fees required to participate in the bookkeeping rights competition are: In addition, each local device has a certain budget for purchasing computing resources. Representation device In the The remaining budget for the second time participating in the bookkeeping right competition is: 。 3. A distributed edge cloud resource leasing decision-making method for IoT blockchain according to claim 1, characterized in that: The optimization algorithm based on swarm intelligence is used to optimize the strategic network parameters, including: firstly initializing the position information of each individual in the population and calculating its fitness information; Then, the position update formula is used to update the position. After the number of iterations is completed, the historical optimal position is decoded and the parameters of all local device policy networks are extracted; The position update formula includes the following three methods: Method 1: Individual By positioning oneself After further investigation, the update formula is expressed as: in, Represents an individual exist The position at the iteration; Indicates the current iteration number; () represents the natural exponential function; Indicates the maximum number of iterations; and They are all random numbers with values ​​of (0,1); Indicates a A matrix whose values ​​all obey the standard normal distribution; Method 2: Individual By using the optimal individual position in the current population to update its own position, the update formula at this time is expressed as: in, represents the position of the individual with the best fitness; Represents an individual individual position; is a random number that follows a standard normal distribution; and Respectively represent the upper and lower limits of individual position values; Method 3: In order to improve the exploration degree of individuals and avoid falling into local optimal solutions, individuals It also searches for the optimal position by randomly changing its position. Its random update formula is expressed as: in, represents a random position; Indicates a The matrix has a value of 1 or -1.

4. The distributed edge cloud resource leasing decision-making method for the Internet of Things blockchain according to claim 3 is characterized by: When the individual fitness size of an individual is in the top Q% of the population, the individual is considered to have a high fitness and has a probability of more than 50% to update its position using formula (40); When the individual fitness of an individual is not in the top Q% of the population, the individual is considered to have low fitness and has a probability of more than 50% to update its position using formula (41); In addition, each individual has a K% probability of updating its position using formula (42).

5. A distributed edge cloud resource leasing decision-making method for IoT blockchain according to claim 4, characterized in that: The value of Q is 30, and the value of K is 5.

6. A distributed edge cloud resource leasing decision-making method for IoT blockchain according to claim 3, characterized in that: The fitness information is obtained through the fitness function, and the formula is as follows: in, Indicates that in the strategy Expected return under Represents the joint policy of all local devices; Indicates the maximum number of iterations; Represents a collection of local devices; Representation device The reward function of .

7. A distributed edge cloud resource leasing decision-making method for IoT blockchain according to claim 1, characterized in that: After executing steps S1 to S3, the trained policy network is deployed on the corresponding local device. Since each local device has an independent policy network, each local device can independently make resource leasing decisions based on the environmental status.

Images