Internet of Things block chain fragmentation method and device

Abstract

The invention provides an Internet of Things block chain fragmentation method and device, and the method comprises the steps: selecting a node through a preset mechanism, and carrying out the supervision and execution of transaction control and block chain fragmentation based on the calculation capability and credit and reputation screening policy committee of each node. A deep reinforcement learning DRL behavior network is utilized to execute selection of fragment actions based on environment states, two state spaces and corresponding action spaces are established for transaction transactions and block chain fragments, and a dual deep reinforcement learning model is constructed. Meanwhile, a first target network is introduced to stabilize a first target value predicted by a first evaluation network, a second target network is introduced to stabilize a second target value predicted by a second evaluation network, and transaction transactions and blockchain fragmentation are synchronously regulated and controlled to adapt to dynamic changes of a network environment so as to realize efficient network fragmentation.

Description

Technical Field

[0001] The present invention relates to the field of blockchain technology, and in particular to a method and device for sharding an Internet of Things blockchain. Background Art

[0002] With the rapid development of the Internet of Things, the number of devices worldwide continues to increase. It is estimated that by 2030, the number of IoT devices connected to the Internet will reach tens of billions. These devices generate and exchange massive amounts of data, which has promoted the development of applications such as smart homes, smart cities, self-driving cars, and industrial automation. However, the expansion of the Internet of Things also faces huge challenges, especially in terms of data security, privacy protection, storage capacity, bandwidth, and computing power. The traditional centralized architecture is increasingly unable to cope with such a large scale of devices and data, and it is difficult to meet the requirements of decentralization, security, and scalability.

[0003] In this context, blockchain technology is considered a potential solution to the challenges of the Internet of Things due to its decentralization, security and immutability. Through blockchain, IoT devices can safely transmit and trade data in a decentralized network, avoiding dependence on centralized servers, thereby improving the reliability and anti-attack capabilities of the system. However, blockchain itself also faces scalability issues, especially when processing large-scale IoT data, traditional blockchain faces storage and computing bottlenecks.

[0004] In order to solve these problems, innovative solutions continue to emerge in the blockchain field. Blockchain sharding technology is one of the key breakthroughs. Sharding technology divides the blockchain network into multiple independent sub-networks, called shards, each of which is responsible for handling different transactions, thereby improving the transaction concurrency capacity of the blockchain. Dynamic sharding further improves this technology. It can automatically adjust the number and size of shards according to factors such as traffic and load in the network to ensure the best use of network resources.

[0005] Traditional sharding technology cannot adjust and expand the blockchain according to network conditions when facing large-scale networks. It has limitations in increasing the number of transactions processed per second, resulting in performance degradation. Another difficulty in traditional sharding methods is the processing of cross-shard transactions. Frequent cross-shard transactions will increase latency and complexity, affecting the efficiency of the blockchain system. The use of certain algorithms such as community partitioning algorithms during sharding can reduce the occurrence of cross-shard transactions, reduce the interaction cost between shards, and improve the overall performance of the system. However, such sharding is likely to cause the number of corrupt nodes in some community shards to exceed the normal range, thereby affecting the security of the entire blockchain. Therefore, optimizing cross-shard transactions while ensuring security is a major problem.

[0006] Therefore, there is an urgent need for a dynamic sharding technology for the IoT blockchain to overcome the challenges posed to the traditional blockchain architecture by the surge in the number of IoT devices and the explosive growth in data exchange. Summary of the invention

[0007] In view of this, an embodiment of the present invention provides an Internet of Things blockchain sharding method and device to eliminate or improve one or more defects existing in the prior art, and to solve the problem that the existing Internet of Things blockchain sharding technology cannot adapt to changes in network conditions to perform sharding to achieve reasonable resource utilization, resulting in slow processing speed and high network latency.

[0008] One aspect of the present invention provides an Internet of Things blockchain sharding method, which is performed by an intelligent agent of an Internet of Things blockchain network; when executing a cross-shard transaction, the Internet of Things blockchain network introduces a relay block to relay the cross-shard transaction from a source shard to a target shard, and completes subsequent operations of the transaction in the target shard; in each epoch, the method includes the following steps:

[0009] A preset mechanism is used to select nodes to join the IoT blockchain network. Based on the computing resources of each node and the score in the creditor scoring table, multiple nodes are selected to form a strategy committee.

[0010] The strategy committee, as an agent, constructs the transaction number of the transaction pool, the number of completed transactions and the total number of transaction transactions as the first state space, and constructs the epoch length, block size, number of shards and block interval as the first action space; constructs the shard state of each node in the IoT blockchain network as the second state space, and constructs the shard adjustment of each node as the second action space;

[0011] The policy committee acts as an agent, and the first actor network based on deep reinforcement learning takes the parameters of the first state space as input and outputs the prediction of the first action in the first action space; the first evaluation network is used to predict the first target value of the transaction throughput reward according to the first action, and the first target value introduces the prediction of the transaction throughput reward by the first target network as a stable reference; the structure of the first target network is consistent with the first actor network and the first evaluation network;

[0012] The policy committee acts as an agent, and the second actor network based on deep reinforcement learning takes the parameters of the second state space as input and outputs the prediction of the second action in the second action space; the second evaluation network is used to predict the second target value of the cross-shard transaction optimization reward according to the second action, and the second target value introduces the prediction of the cross-shard transaction optimization reward by the second target network as a stable reference; the structure of the second target network is consistent with the second actor network and the second evaluation network;

[0013] performing parameter updates on the first actor network and the first evaluation network according to the first target value, and performing parameter updates on the second actor network and the second evaluation network according to the second target value; and performing soft updates on the first target network and the second target network;

[0014] The policy committee performs the first action, and when the number of cross-shard transactions in the IoT blockchain network is higher than a set number of total transaction transactions, performs the second action.

[0015] In some embodiments, based on the computing resources of each node and the score in the trustee score table, multiple nodes are selected to form a policy committee, including:

[0016] Calculate the historical behavior reputation, the calculation formula is:

[0017]

[0018] in, represents the historical behavior reputation of node i, represents the number of transactions processed by node i in time period t, Represents the total number of transactions of all nodes in the blockchain, represents the online time of node i, T c Indicates the current system time. Indicates the time when node i is registered to the blockchain; α1 and α2 are weight coefficients, and α1+α2=1;

[0019] Calculate the computing power reputation, the calculation formula is:

[0020]

[0021] in, represents the computing power reputation of node i, Indicates the CPU performance value of the node. Represents the network delay performance value of the node, Represents the network delay performance value of the node, Indicates the storage speed performance value of the node; The total CPU performance value of all nodes, Represents the total network delay performance value of all nodes, Represents the total network bandwidth performance value of all nodes, Represents the total storage speed performance value of all nodes. β1, β2, β3, β4 are weight coefficients, and β1+β2+β3+β4=1.

[0022] The calculation formula of the comprehensive reputation of a node is:

[0023]

[0024] in, The comprehensive reputation of node i, ω1 and ω2 are weight coefficients, and ω1+ω2=1.

[0025] In some embodiments, the preset mechanism includes: a proof-of-work mechanism, a proof-of-stake mechanism, or a reputation mechanism; the first actor network, the first evaluation network, the second actor network, and the second evaluation network all adopt a deep neural network structure.

[0026] In some embodiments, the calculation formula for the transaction throughput reward is:

[0027]

[0028] Among them, R tps represents the transaction throughput reward, K represents the number of shards, T represents the epoch length, Γ represents the block interval, B represents the block size, and R r Indicates the average number of redundant transactions.

[0029] In some embodiments, the calculation formula for the cross-shard transaction optimization reward is:

[0030]

[0031] Among them, R cts represents the cross-shard transaction optimization reward, α is Indicates the number of transactions in the shard, α cs represents the number of cross-shard transactions, A represents the total number of transactions, and U represents the load imbalance; w1, w2, and w3 are parameters.

[0032] In some embodiments, the first target value introduces the prediction of the transaction throughput reward of the first target network as a stable reference, and the calculation formula is:

[0033] y 1i =r 1i +γ1Q1′(s i+1 ,μ′(s i+1 |θ1μ′)|θ1Q′;

[0034] Among them, y 1i represents the first target value, r 1i represents the current value of the transaction throughput reward, γ1 is the first discount factor, Q1′(s i+1 ,μ′(s i+1 |θ1μ ′ )|θ1 Q′ represents the predicted value of the transaction throughput reward of the first target network; s i+1represents the state of the i+1th epoch, θ1 μ′ represents the parameters of the first target actor network in the first target network, θ1 Q′ Representing parameters of a first target evaluation network in the first target network;

[0035] The second target value introduces the second target network's prediction of the cross-shard transaction optimization reward as a stable reference, and the calculation formula is:

[0036] y 2i =r 2i +γ2Q2′(s i+1 ,μ′(s i+1 |θ2 μ′ )|θ2 Q′ ;

[0037] Among them, y 2i represents the second target value, r 2i represents the current value of the cross-shard transaction optimization reward, γ2 is the second discount factor, Q2′(s i+1 ,μ′(s i+1 |θ2 μ′ )|θ2 Q′ represents the predicted value of the cross-shard transaction optimization reward of the second target network; s i+1 represents the state of the i+1th epoch, θ2 μ′ represents the parameters of the second target actor network in the second target network, θ2 Q′ Represents parameters of the second target evaluation network in the second target network.

[0038] In some embodiments, updating parameters of the first actor network and the first evaluation network according to the first target value includes:

[0039] The parameters of the first evaluation network are updated based on the first target value, and the expression is:

[0040]

[0041] Among them, θ1 Q represents the weight parameter of the first evaluation network, y 1i represents the first target value, η represents the learning rate, represents the weight gradient of the first evaluation network, Q(s i ,a i |θ1 Q ) represents the first target value currently estimated by the first evaluation network; s 1i represents the state of the first state space at the i-th epoch, a 1i represents the action of the first action space at the i-th epoch;

[0042] The parameters of the first actor network are updated, and the expression is:

[0043]

[0044] Among them, θ1 μ represents the weight parameter of the first actor network, η represents the learning rate, represents the weight gradient of the first actor network, Q(s1,μ(s1|θ1 μ )|θ1 Q ) represents the first target value currently estimated by the first evaluation network, μ(s1|θ1 μ ) represents the policy function of the first actor network;

[0045] Updating parameters of the second actor network and the second evaluation network according to the second target value includes:

[0046] The parameters of the second evaluation network are updated based on the second target value, and the expression is:

[0047]

[0048] Among them, θ2 Q represents the weight parameter of the second evaluation network, y 2i represents the second target value, η represents the learning rate, represents the weight gradient of the second evaluation network, Q(s 2i ,a 2i |θ2 Q ) represents the second target value currently estimated by the second evaluation network; s 2i represents the state of the second state space at the i-th epoch, a 2i represents the action of the second action space at the i-th epoch;

[0049] The parameters of the second actor network are updated, and the expression is:

[0050]

[0051] Among them, θ2 μ represents the weight parameter of the second actor network, η represents the learning rate, represents the weight gradient of the second actor network, represents the second target value currently estimated by the second evaluation network, μ(s1|θ1 μ ) represents the policy function of the second actor network.

[0052] In some embodiments, performing a soft update on the first target network and the second target network includes:

[0053] The weight update expression of the first target actor network is:

[0054] θ1 μ′ ←τθ1 μ +(1-τ)θ1 μ′ ;

[0055] Among them, θ1 μ′ represents the weight parameter of the first target actor network, τ represents the soft update coefficient, θ1 μ represents the current weight parameter of the first actor network;

[0056] The weight update expression of the first target evaluation network is:

[0057] θ1 Q′ ←τθ1 Q +(1-τ)θ1 Q′ ;

[0058] Among them, θ1 Q′ represents the weight parameter of the first target evaluation network, τ represents the soft update coefficient, θ1 Q represents the current weight parameter of the first evaluation network;

[0059] The weight update expression of the second target actor network is:

[0060] θ2 μ′ ←τθ2 μ +(1-τ)θ2 μ′ ;

[0061] Among them, θ2 μ′ represents the weight parameter of the second target actor network, τ represents the soft update coefficient, θ2 μ represents the current weight parameter of the second actor network;

[0062] The weight update expression of the second target evaluation network is:

[0063] θ2 Q′ ←τθ2 Q +(1-τ)θ2 Q′ ;

[0064] Among them, θ2 Q′ represents the weight parameter of the second target evaluation network, τ represents the soft update coefficient, θ2 Q Represents the current weight parameters of the second evaluation network.

[0065] On the other hand, the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon, which implements the steps of the above method when the computer program / instruction is executed by a processor.

[0066] On the other hand, the present invention also provides a computer program product, comprising a computer program / instruction, characterized in that the computer program / instruction implements the steps of the above method when executed by a processor.

[0067] The beneficial effects of the present invention are at least:

[0068] The IoT blockchain sharding method and device of the present invention selects nodes to be included in the IoT blockchain network through a preset mechanism, and then selects a strategy committee based on the computing power and credit reputation of each node to supervise and execute transaction control and blockchain sharding. The deep reinforcement learning DRL actor network is used to select sharding actions based on the environmental state, and two state spaces and their corresponding action spaces are established for transaction transactions and blockchain sharding respectively, and a dual deep reinforcement learning model is constructed. At the same time, the first target network is introduced to stabilize the first target value predicted by the first evaluation network, and the second target network is introduced to stabilize the second target value predicted by the second estimation network. The transaction transactions and blockchain sharding are synchronously regulated to adapt to the dynamic changes of the network environment and realize efficient network sharding.

[0069] Furthermore, when screening the policy committee, a trustee scoring table is introduced to maintain a global account score to prevent the influence of malicious nodes.

[0070] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0071] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings:

[0073] Figure 1 The present invention is a flowchart of a method for sharding an Internet of Things blockchain according to an embodiment of the present invention.

[0074] Figure 2This is a schematic diagram of the deep reinforcement neural network structure in the Internet of Things blockchain network sharding solution described in one embodiment of the present invention. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0076] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0077] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0078] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0079] When traditional blockchain is applied in the Internet of Things, it faces scalability problems, especially when processing large-scale Internet of Things data, traditional blockchain faces storage and computing bottlenecks. Dynamic sharding is a method to improve blockchain performance. The core purpose of the present invention is to overcome the challenges brought to the traditional blockchain architecture by the surge in the number of Internet of Things devices and the explosive growth in data exchange, and to achieve efficient expansion of the blockchain network through innovative dynamic sharding technology driven by reputation and deep reinforcement learning (DRL). This technology focuses on improving transaction processing speed, reducing network latency, optimizing storage solutions, and achieving reasonable allocation and utilization of resources under the premise of ensuring data security and privacy protection. In addition, the present invention also aims to reduce the energy consumption and computing cost of the network by dynamically adjusting the number and scale of shards to adapt to changing network conditions, and integrating a lightweight consensus mechanism, thereby providing a flexible and efficient Internet of Things blockchain network sharding solution for Internet of Things devices.

[0080] Specifically, one aspect of the present invention provides an Internet of Things blockchain sharding method, which is performed by an intelligent agent of an Internet of Things blockchain network; when the Internet of Things blockchain network performs a cross-shard transaction, a relay block is introduced to relay the cross-shard transaction from a source shard to a target shard, and subsequent operations of the transaction are completed in the target shard; in each epoch, the method includes the following steps S101 to S104:

[0081] Step S101: A preset mechanism is used to select nodes to join the IoT blockchain network, and multiple nodes are selected to form a strategy committee based on the computing resources of each node and the score in the creditor scoring table.

[0082] Step S102: The strategy committee acts as an agent to construct the transaction number of the transaction pool, the number of completed transactions and the total number of transaction transactions as the first state space, and the epoch length, block size, number of shards and block interval as the first action space; the shard state of each node in the IoT blockchain network is constructed as the second state space, and the shard adjustment of each node is constructed as the second action space.

[0083] Step S103: The strategy committee acts as an agent, and the first actor network based on deep reinforcement learning takes the parameters of the first state space as input and outputs the prediction of the first action in the first action space; the first evaluation network is used to predict the first target value of the transaction throughput reward according to the first action, and the first target value introduces the prediction of the transaction throughput reward by the first target network as a stable reference; the structure of the first target network is consistent with the first actor network and the first evaluation network.

[0084] Step S104: The policy committee acts as an agent, and the second actor network based on deep reinforcement learning takes the parameters of the second state space as input and outputs the prediction of the second action in the second action space; the second evaluation network is used to predict the second target value of the cross-shard transaction optimization reward according to the second action, and the second target value introduces the prediction of the cross-shard transaction optimization reward by the second target network as a stable reference; the structure of the second target network is consistent with the second actor network and the second evaluation network.

[0085] Step S105: updating parameters of the first actor network and the first evaluation network according to the first target value, updating parameters of the second actor network and the second evaluation network according to the second target value; and soft updating the first target network and the second target network.

[0086] Step S106: The policy committee member performs the first action, and when the number of cross-shard transactions in the IoT blockchain network is higher than the set number of total transaction transactions, the second action is performed.

[0087] In step S101, the nodes constituting the IoT blockchain are selected first, and the preset mechanisms adopted include: proof of work mechanism (POW), proof of stake mechanism (Pos) or reputation mechanism.

[0088] In the PoW mechanism, nodes need to solve a complex mathematical problem, called the PoW problem, to prove that they have the right to add new blocks to the blockchain. The PoW problem is usually a problem that is complex to calculate but relatively simple to verify. Nodes need to try a lot of calculations to find a solution that meets the conditions, and it is relatively easy to verify whether the solution is correct.

[0089] In Proof of Stake, nodes are selected based on the amount of cryptocurrency they hold in the network (i.e., "stake"), rather than by solving complex math problems (as in PoW). Nodes that hold more tokens have a higher probability of being selected as validators and responsible for creating new blocks.

[0090] The reputation mechanism is based on the historical behavior and contribution of the node. The system selects the validation node of the epoch through a reputation or credit scoring mechanism. The higher the reputation of the node, the greater the chance of being selected to participate in the validation.

[0091] After completing the construction of the IoT blockchain network, the present invention also introduces a lender role in the blockchain to maintain the global trustworthiness score of each node based on historical data. Specifically, each node will save the user's account, which is equivalent to the node corresponding to a part of the users, and then the user account has a lender score. The user's lender score is aggregated to obtain the node's lender score. After the node has a lender score, the global information sharing synchronization is the global trust, which can be used as the historical behavior reputation.

[0092] At the same time, the reputation of each node is comprehensively evaluated based on the computing resources of the node, and members of the strategy committee are selected based on this.

[0093] In some embodiments, based on the computing resources of each node and the score in the trustee score table, multiple nodes are selected to form a policy committee, including steps S1011 to S1013:

[0094] Step S1011: Calculate historical behavior reputation, the calculation formula is:

[0095]

[0096] in, represents the historical behavior reputation of node i, represents the number of transactions processed by node i in time period t, Represents the total number of transactions of all nodes in the blockchain, represents the online time of node i, T c Indicates the current system time. Represents the time when node i is registered to the blockchain; α1 and α2 are weight coefficients, and α1+α2=1.

[0097] Step S1012: Calculate computing power reputation, the calculation formula is:

[0098]

[0099] in, represents the computing power reputation of node i, Indicates the CPU performance value of the node. Represents the network delay performance value of the node, Represents the network delay performance value of the node, Indicates the storage speed performance value of the node; The total CPU performance value of all nodes, Represents the total network delay performance value of all nodes, Represents the total network bandwidth performance value of all nodes, Represents the total storage speed performance value of all nodes. β1, β2, β3, β4 are weight coefficients, and β1+β2+β3+β4=1.

[0100] Step S1013: The calculation formula of the node's comprehensive reputation is:

[0101]

[0102] in, The comprehensive reputation of node i, ω1 and ω2 are weight coefficients, and ω1+ω2=1.

[0103] During the specific implementation process, nodes whose comprehensive reputation is higher than a set value can be included in the policy committee, or a set proportion of nodes with high comprehensive reputation rankings can be included in the policy committee.

[0104] In steps S102 to S105, a dual deep reinforcement learning DRL model is introduced. On the one hand, the first DRL constructs the number of transactions in the transaction pool, the number of completed transactions, and the total number of transaction transactions as the first state space, and constructs the epoch length, block size, number of shards, and block interval as the first action space, and adopts transaction throughput rewards; the first DRL includes a first actor network and a first evaluation network. The first actor network predicts actions according to state execution, and the first evaluation network is used to predict transaction throughput rewards to obtain a first target value. The first target value is introduced into the first target network to stabilize the prediction of transaction throughput rewards. The structure of the first target network is consistent with the first actor network and the first evaluation network, and it adopts the form of soft update, and its prediction changes less with the environment.

[0105] On the other hand, the second DRL constructs the shard state of each node in the IoT blockchain network as the second state space, and constructs the shard adjustment of each node as the second action space, using cross-shard transaction optimization rewards. The second DRL contains a second actor network and a second evaluation network. The second actor network predicts the action according to the state execution. The second evaluation network is used to predict the cross-shard transaction optimization reward to obtain the second target value. The second target value introduces the second target network to stabilize the prediction of the cross-shard transaction optimization reward. The structure of the second target network is consistent with the second actor network and the second evaluation network, and it adopts the form of soft update, and its prediction changes less with the environment.

[0106] The first actor network, the first evaluation network, the second actor network and the second evaluation network all adopt a deep neural network structure. The structure of the first target network is consistent with the first actor network and the first evaluation network. The structure of the second target network is consistent with the second actor network and the second evaluation network.

[0107] In some embodiments, the transaction throughput reward is calculated as:

[0108]

[0109] Among them, R tps represents the transaction throughput reward, K represents the number of shards, T represents the epoch length, Γ represents the block interval, B represents the block size, and R r Indicates the average number of redundant transactions.

[0110] In some embodiments, the calculation formula for cross-shard transaction optimization rewards is:

[0111]

[0112] Among them, R cts represents the cross-shard transaction optimization reward, α is Indicates the number of transactions in the shard, α cs represents the number of cross-shard transactions, A represents the total number of transactions, and U represents the load imbalance; w1, w2, and w3 are parameters.

[0113] In some embodiments, the first target value introduces the prediction of the transaction throughput reward of the first target network as a stable reference, and the calculation formula is:

[0114] y 1i =r 1i +γ1Q1′(s i+1 ,μ′(s i+1 |θ1 μ′ )|θ1 Q′ ;

[0115] Among them, y1i represents the first target value, r 1i represents the current value of the transaction throughput reward, γ1 is the first discount factor, Q1′(s i+1 ,μ′(s i+1 |θ1 μ′ )|θ1 Q′ represents the predicted value of the transaction throughput reward of the first target network; s i+1 represents the state of the i+1th epoch, θ1 μ′ represents the parameters of the first target actor network in the first target network, θ1 Q′ Representing parameters of a first target evaluation network in the first target network;

[0116] The second target value introduces the second target network's prediction of the cross-shard transaction optimization reward as a stable reference, and the calculation formula is:

[0117] y 2i =r 2i +γ2Q2′(s i+1 ,μ′(s i+1 |θ2 μ′ )|θ2 Q′ ;

[0118] Among them, y 2i represents the second target value, r 2i represents the current value of the cross-shard transaction optimization reward, γ2 is the second discount factor, Q2′(s i+1 ,μ′(s i+1 |θ2 μ′ )|θ2 Q′ represents the predicted value of the cross-shard transaction optimization reward of the second target network; s i+1 represents the state of the i+1th epoch, θ2 μ′ represents the parameters of the second target actor network in the second target network, θ2 Q′ Represents parameters of the second target evaluation network in the second target network.

[0119] In some embodiments, updating parameters of the first actor network and the first evaluation network according to the first target value includes steps S201 to S202:

[0120] Step S201: Update the parameters of the first evaluation network based on the first target value, and the expression is:

[0121]

[0122] Among them, θ1 Q represents the weight parameter of the first evaluation network, y 1irepresents the first target value, η represents the learning rate, represents the weight gradient of the first evaluation network, Q(s i ,a i |θ1 Q ) represents the first target value estimated by the current first evaluation network; s 1i represents the state of the first state space at the i-th epoch, a 1i Represents the action of the first action space at the i-th epoch.

[0123] Step S202: Update the parameters of the first actor network, the expression is:

[0124]

[0125] Among them, θ1 μ represents the weight parameter of the first actor network, η represents the learning rate, represents the weight gradient of the first actor network, Q(s1,μ(s1|θ1 μ )|θ1 Q ) represents the first target value currently estimated by the first evaluation network, μ(s1|θ1 μ ) represents the policy function of the first actor network;

[0126] The second actor network and the second evaluation network are updated with parameters according to the second target value, including steps S301 to S302:

[0127] Step S301: Update the parameters of the second evaluation network based on the second target value, and the expression is:

[0128]

[0129] Among them, θ2 Q represents the weight parameter of the second evaluation network, y 2i represents the second target value, η represents the learning rate, represents the weight gradient of the second evaluation network, Q(s 2i ,a 2i |θ2 Q ) represents the second target value estimated by the current second evaluation network; s 2i represents the state of the second state space at the i-th epoch, a 2i Represents the action of the second action space at the i-th epoch.

[0130] Step S302: Update the parameters of the second actor network, the expression is:

[0131]

[0132] Among them, θ2μ represents the weight parameter of the second actor network, η represents the learning rate, represents the weight gradient of the second actor network, represents the second target value estimated by the current second evaluation network, μ(s1|θ1 μ ) represents the policy function of the second actor network.

[0133] In some embodiments, performing a soft update on the first target network and the second target network includes steps S401 to S404:

[0134] Step S401: The weight update expression of the first target actor network is:

[0135] θ1 μ′ ←τθ1 μ +(1-τ)θ1 μ′ ;

[0136] Among them, θ1 μ′ represents the weight parameter of the first target actor network, τ represents the soft update coefficient, θ1 μ Represents the current weight parameters of the first actor network.

[0137] Step S402: The weight update expression of the first target evaluation network is:

[0138] θ1 Q′ ←τθ1 Q +(1-τ)θ1 Q′ ;

[0139] Among them, θ1 Q′ represents the weight parameter of the first target evaluation network, τ represents the soft update coefficient, θ1 Q Represents the current weight parameters of the first evaluation network.

[0140] Step S403: The weight update expression of the second target actor network is:

[0141] θ2 μ′ ←τθ2 μ +(1-τ)θ2 μ′ ;

[0142] Among them, θ2 μ′ represents the weight parameter of the second target actor network, τ represents the soft update coefficient, and θ2 μ Represents the current weight parameters of the second actor network.

[0143] Step S404: The weight update expression of the second target evaluation network is:

[0144] θ2 Q′ ←τθ2Q +(1-τ)θ2 Q′ ;

[0145] Among them, θ2 Q′ represents the weight parameter of the second target evaluation network, τ represents the soft update coefficient, θ2 Q Represents the current weight parameters of the second evaluation network.

[0146] In step S106, at the end of each epoch, the strategy committee initiates resharding when the number of cross-shard transactions exceeds the set ratio. For example, if the cross-shard ratio of transaction transactions reaches 50%, resharding is performed, that is, the second action is performed. The tolerance for cross-sharding varies in different environments, and this ratio can be set as needed. Resharding is equivalent to restarting the entire model.

[0147] On the other hand, the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon, which implements the steps of the above method when the computer program / instruction is executed by a processor.

[0148] On the other hand, the present invention also provides a computer program product, comprising a computer program / instruction, characterized in that the computer program / instruction implements the steps of the above method when executed by a processor.

[0149] The present invention is described below in conjunction with a specific embodiment:

[0150] This embodiment provides an IoT blockchain network sharding solution, and the system model is as follows:

[0151] This embodiment proposes the architecture of the Credit and Dual DRL Verification (CDDV) model, such as Figure 2 As shown, at the beginning of each epoch, new nodes that complete the PoW problem within the specified time can join the blockchain network. In each shard, any node can propose a node as the leader. The epoch strategy is introduced to reset the policy committee (SC) of the shard, which is composed of members democratically elected by network users. Inspired by Elastico, SC ensures decentralized and reliable supervision. This design prevents single points of failure and minimizes risks from centralized agents. SC acts as a decentralized and trusted coordinator that oversees the user list of the entire network and assigns nodes to shards.

[0152] The most important work for SC is to judge and give a new sharding strategy at the end of each epoch and start a new epoch, redistributing nodes between shards according to changes in the environment. This adaptability ensures that the system remains scalable and secure, and adapts to higher transaction volumes without compromising security. The DRL model is used to assist the blockchain reconfiguration process. The agent consists of a committee elected by all peers in the blockchain network. The agent obtains the state from the node allocation, obtains rewards by virtually resharding, deciding the optimal allocation strategy, and executing new node allocation actions. The permissioned blockchain system restricts network access to specific nodes. In order for the training process to be considered credible, it is important that the agent is reliable and has transparent records when training the DRL model. In addition, it is assumed that each shard involves at least a certain number of members, and the claim that the number of dishonest nodes in each shard does not exceed a certain number of the total number of nodes is based on the Byzantine Fault Tolerance (BFT) principle. In the event of a node failure or a dishonest node attack, the system continues to operate with the remaining nodes. Each node maintains a lender ledger for this node, where lender scores are distributed to other network nodes. It aggregates the lender scores of each user in the node to derive the global trust metric of each node, thereby protecting trust and node distribution. Specifically, each node will save the user's account, which is equivalent to the node corresponding to a part of the users, and then the user account will have a lender score. Aggregating the user's lender score is the node's lender score. After the node has a lender score, global information sharing and synchronization is the global trust.

[0153] In order to find nodes with good reputation and abundant computing resources and form a sharding strategy committee SC (Strategy Committee), a temporary directory committee is elected in the same way as elastico when the blockchain is first started. The directory committee calculates and stores the reputation of each node, and randomly selects 10 nodes from the set of high-reputation candidate nodes as the sharding strategy committee SC. After the sharding strategy committee SC is selected, the SC will gradually replace the function of the temporary directory committee, and the SC will synchronize and store the reputation of each node and handle the update of the SC. By comprehensively evaluating the reputation of the node, the present invention designs a node reputation mechanism composed of historical behavior reputation and computing power reputation to accurately reflect the performance and historical behavior of the node. Use historical behavior reputation Represents the credibility of the node's historical behavior in the most recent t period of time, defined as:

[0154]

[0155] in, represents the historical behavior reputation of node i, represents the number of transactions processed by node i in time period t, Represents the total number of transactions of all nodes in the blockchain, represents the online time of node i, T c Indicates the current system time. Represents the time when node i is registered to the blockchain; α1 and α2 are weight coefficients, and α1+α2=1.

[0156] The computing power reputation reflects the computing power performance of the node hardware, which is defined as

[0157]

[0158] in, represents the computing power reputation of node i, Indicates the CPU performance value of the node. Represents the network delay performance value of the node, Represents the network delay performance value of the node, Indicates the storage speed performance value of the node; The total CPU performance value of all nodes, Represents the total network delay performance value of all nodes, Represents the total network bandwidth performance value of all nodes, Represents the total storage speed performance value of all nodes. β1, β2, β3, β4 are weight coefficients, and β1+β2+β3+β4=1.

[0159] The comprehensive reputation of a node is defined as

[0160]

[0161] in, The comprehensive reputation of node i, ω1 and ω2 are weight coefficients, and ω1+ω2=1.

[0162] In the deep reinforcement learning model DRL, the SC (Strategy Committee) is an agent, which consists of several nodes. These nodes are selected from each shard based on computing resources and lender records, and execute the PBFT protocol to reach a consensus, representing the collective action of the blockchain. The agent performs a learning process and determines the shard allocation based on the real-time network status.

[0163] The environment is considered as a black box that executes the agent's actions and obtains the state. This embodiment takes the shard reconstruction process in the blockchain shard network as the environment. Consider that the agent obtains a state s∈S according to the node allocation of the current epoch t, ​​where s represents the distribution of all nodes on different shards in the current episode, and the matrix A accurately identifies the specific nodes that exist in a specific shard.

[0164] s 1,t=(q1,q2,n);

[0165] s 2,t =([M x,1 ,…,M x,i ,…,M x,N ]),x∈D,i∈N;

[0166] Where q1 is the number of transactions in the transaction pool, q2 is the number of completed transactions, and n is the total number of transactions. x,1 is the i-th shard to which the current x-th node belongs, D is the number of nodes, and N is the number of shards.

[0167] Constructing the action space includes epoch length T, block size B, number of shards K, block interval Γ and node shard matrix A x,i , which plays an important role in solving the problem of dynamic sharding system.

[0168] When considering that the arrival of nodes follows a certain distribution, the epoch length will determine the number of system nodes in the next epoch. In addition, the block interval and block size can change the state of the transaction pool by affecting the rate of processing transactions. Therefore, it needs to be adjusted to adapt to the dynamic environment. Considering the calculation in the system, for the entire action space a1 and a2 at time t, we define them as:

[0169] a 1,t =(T,B,K,Γ);

[0170] a 2,t =([A x,1 ,…,A x,i ,…,A x,N ]),x∈D,i∈N;

[0171] A x,i The xth node is assigned to the i-th shard, D is the number of nodes, and N is the number of shards.

[0172] Setting rewards,The optimization goal of this embodiment is to obtain a sharding strategy that can balance performance and security in each period.Since scalability can be easily quantified by transaction throughput, transaction throughput rewards are used as the reward function, and cross-shard transaction optimization rewards are introduced.

[0173] The calculation formula for transaction throughput rewards is:

[0174]

[0175] Among them, R tps represents the transaction throughput reward, K represents the number of shards, T represents the epoch length, Γ represents the block interval, B represents the block size, and R r Indicates the average number of redundant transactions.

[0176] The calculation formula for cross-shard transaction optimization rewards is:

[0177]

[0178] Among them, R cts represents the cross-shard transaction optimization reward, α is Indicates the number of transactions in the shard, α cs represents the number of cross-shard transactions, A represents the total number of transactions, and U represents the load imbalance; w1, w2, and w3 are parameters.

[0179] The dynamic sharding framework of blockchain based on CDDV model includes: blockchain network status s t , lender ledger records (global trust of nodes, ratings), experience replay buffer R, replay buffer size τ, actor network initial weight θ μ , the initial weight of the evaluator network θ Q , learning rate η and discount factor γ. The output includes: shard reconfiguration strategy shard number K, epoch length T, block interval Γ, block size B and node allocation strategy ([A x,1 ,…,A x,i ,…,A x,N ]).

[0180] Specifically, a dual DRL is constructed, where DRL 1 corresponds to the state space s 1,t , action space a 1,t , using transaction throughput rewards; DRL 2 corresponds to the state space s 2,t , action space a 2,t , using cross-shard transactions to optimize rewards.

[0181] The specific steps are as follows:

[0182] S1. Initialize the network and data structure: Initialize the actor network μ(s|θ μ ) and the evaluator network Q(s,a|θ Q ).

[0183] S2. Initialize the target network μ′(s|θ μ′ ) and Q′(s,a|θ Q′ ).

[0184] S3. Initialize the experience replay buffer R and the global lender scoring ledger.

[0185] S4. Initialize the control variables T, B, K, Γ of the shard reconfiguration strategy.

[0186] S5. Node selection and allocation at the beginning of each epoch: The PoW mechanism is used for node selection. The selected new nodes can join the blockchain network and their contribution value and trustworthiness are evaluated based on the lender score table. The allocation of nodes is supervised by a democratically elected Strategy Committee (SC).

[0187] S6. Using the actor network μ(s|θ μ ) According to the current blockchain status s t Select shard reconfiguration action a t , including epoch length T, block size B, number of shards K and block interval Γ.

[0188] S7. Execute actions and feedback rewards: Execute shard reconfiguration actions, adjust node allocation and shard size, and receive rewards based on the execution results. t .

[0189] S8.Store state transfer information: transfer the state (s t ,a t ,r t ,s t+1 ) is stored in the experience replay buffer R, and the state transitions (s i ,a i ,r i ,s i+1 ) is used for model training.

[0190] S9. Use the evaluation network to calculate the target value: y i =r i +γQ′(s i+1 ,μ′(s i+1 |θ μ′ )|θ Q′ , update the evaluation network θ Q Weight: Update the actor network θ μ The weights are set to maximize the value of the evaluation network:

[0191] S10. Soft update of the target network: Update the weight θ of the target network by soft update μ′ and θ Q′ :θ μ′ ←τθ μ +(1-τ)θ μ′ ,θ Q′ ←τθ Q +(1-τ)θ Q′ .

[0192] S11. Sharding reconfiguration and cross-sharding transaction optimization: At the end of each epoch, the Strategy Committee (SC) decides whether to reshard based on the results of the DRL model. If there are many cross-sharding transactions, resharding and node reconfiguration are triggered, and malicious or low-reputation nodes are identified with the help of the lender scoring mechanism.

[0193] The execution process of the DRL-based dynamic sharding framework in the algorithm. First, initialize the network state, lender ledger, and sharding reconfiguration strategy. In each epoch, the algorithm selects new nodes through the PoW mechanism, and the strategy committee supervises the allocation of nodes. The proxy maintains the sharding strategy μ(s|θ μ ) and the value function Q(s,a|θ Q ), where θ μ and θ Q are the parameters of the actor network and the critic network. The agent uses the target network modified for the actor-critic (initialized as a copy of the actor-critic network) to calculate the target value of the network update. In order to ensure that the training samples are independent and identically distributed, a finite-sized replay buffer is used for sampling. At each time step t, the agent selects and executes a shard action A based on the current blockchain state s, and then applies noise N for exploration. Afterwards, the blockchain environment will give the agent a reward, which is measured by system security and throughput, and enter the next state s t The agent will convert (s t ,a t ,r t ,s t+1 ) is stored in R, and then a constant number of previous transformations are batched from the replay buffer R to compute the loss function L C and policy gradient The network parameters θ used to update the critic μ and the actor network parameters θ Q Finally, the algorithm uses a soft target update to slowly change the target network: θ Q′ ←τθ Q +(1-τ)θ Q′ and θ μ′ ←τθ μ +(1-τ)θ μ′ .

[0194] Furthermore, when cross-shard transactions are required, the system introduces relay blocks to relay cross-shard transactions from the source shard to the target shard, and completes the subsequent operations of the transaction in the target shard. In the traditional two-phase submission process, the transaction is divided into multiple sub-transactions. After the last sub-transaction is completed, the transaction confirmation submission process is carried out. Since the source shard needs to wait for the confirmation submission message of the target shard after opening the transaction, the confirmation process prolongs the cross-shard transaction time submitted in the second phase, which makes the cross-shard transaction time much longer than the intra-shard transaction time.

[0195] After introducing the final atomic cross-sharding, the confirmation time of cross-shard transactions can be reduced, thereby improving the efficiency of cross-shard transactions. This embodiment divides cross-shard transactions into two operations, a withdrawal operation and a collection operation. The withdrawal operation is completed in the source shard, and a relay block is generated. After the relay block is generated, it will be forwarded to the target shard, and the target shard will verify the relay block to complete the collection operation. After the relay block is forwarded to the target shard, it will ensure that it will eventually execute the deposit operation, that is, the final atomicity. Payment transactions involving withdrawal and deposit operations should be atomic, and the final atomicity ensures the correctness of the sharding system.

[0196] This embodiment introduces a lender role in the blockchain. The lender role maintains a global account score and withdrawal record book, evaluates each account, and when the account meets the requirements, the account performs an early withdrawal operation when conducting cross-shard transactions. The withdrawal amount is temporarily deducted and recorded from the lender, without waiting for the cross-shard shard confirmation. When the lender receives the transaction confirmation submission from the target shard, the lender will deduct the amount of the account's early withdrawal operation. The account can quickly change its account balance when conducting cross-shard transactions, without waiting for transaction confirmation, greatly reducing the confirmation time of cross-shard transactions.

[0197] Lender will record the information of each account that conducts cross-shard transactions. After the account initiates early withdrawal, the lender will update the score of the account. The score includes the user's enthusiasm and security. The higher the score, the higher the amount of early withdrawal initiated by the account. Accounts that do not meet the score will not be able to enjoy the early withdrawal service. After the introduction of the lender role, accounts that perform a large number of cross-shard transactions will not reduce efficiency due to the long time of cross-shard transactions, and malicious accounts will be quickly identified due to low scores.

[0198] The simulation results show that the CCDV blockchain sharding model proposed in the present invention has a significant improvement in the number of blockchain transactions (TPS) compared to random sharding, community-based sharding and reputation-based sharding models, and reduces cross-sharding transactions while ensuring a low proportion of corrupt nodes in the sharding. The dynamic sharding based on reputation and dual DRL verification proposed in the present invention can adjust the appropriate blockchain sharding strategy in real time according to changes in the blockchain environment and improve the efficiency of cross-sharding through improved relay transaction processing. Therefore, the dynamic sharding model proposed in the present invention can ensure the security of the blockchain system while reducing the number of cross-sharding transactions, thereby achieving a higher number of transactions processed per second by the system.

[0199] This embodiment realizes adaptive reconfiguration of blockchain shards by introducing a dual deep reinforcement learning model, and dynamically adjusts the sharding strategy according to network load and transaction demand. The proxy node monitors and learns the current state of the system in real time through the DRL algorithm, including the node load, number of transactions, resource usage of the shard, etc. The DRL model continuously adjusts the node allocation and sharding structure in the blockchain through virtual resharding and policy evaluation to adapt to the dynamic changes of the network environment. The lender role is introduced to maintain the global account score and record the ledger. In cross-shard transactions, the lender uses the early withdrawal mechanism to allow nodes with high credit scores to quickly complete cross-shard transactions without waiting for confirmation from the target shard, thereby significantly reducing the transaction confirmation time and improving the system efficiency. The simulation experiment results show that the scheme proposed in the present invention can achieve a high number of transactions processed per second while ensuring high security.

[0200] Corresponding to the above method, the present invention also provides an apparatus / system, which includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, the processor is used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the apparatus / system implements the steps of the method described above.

[0201] The embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned edge computing server deployment method are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0202] In summary, the IoT blockchain sharding method and device of the present invention selects nodes to be included in the IoT blockchain network through a preset mechanism, and then selects a strategy committee based on the computing power and credit reputation of each node to supervise and execute transaction control and blockchain sharding. The deep reinforcement learning DRL actor network is used to select sharding actions based on the environmental state, and two state spaces and their corresponding action spaces are established for transaction transactions and blockchain sharding, respectively, to construct a dual deep reinforcement learning model, and at the same time, the first target network is introduced to stabilize the first target value predicted by the first evaluation network, and the second target network is introduced to stabilize the second target value predicted by the second estimation network, and the transaction transactions and blockchain sharding are synchronously regulated to adapt to the dynamic changes of the network environment and realize efficient network sharding.

[0203] Furthermore, when screening the policy committee, a trustee scoring table is introduced to maintain a global account score to prevent the influence of malicious nodes.

[0204] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0205] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0206] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0207] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for sharding an Internet of Things blockchain, characterized in that: The method is performed by an intelligent agent of an IoT blockchain network; when executing a cross-shard transaction, the IoT blockchain network introduces a relay block to relay the cross-shard transaction from a source shard to a target shard, and completes subsequent operations of the transaction in the target shard; in each epoch, the method includes the following steps: A preset mechanism is used to select nodes to join the IoT blockchain network. Based on the computing resources of each node and the score in the creditor scoring table, multiple nodes are selected to form a strategy committee. The strategy committee, as an agent, constructs the transaction number of the transaction pool, the number of completed transactions and the total number of transaction transactions as the first state space, and constructs the epoch length, block size, number of shards and block interval as the first action space; constructs the shard state of each node in the IoT blockchain network as the second state space, and constructs the shard adjustment of each node as the second action space; The policy committee acts as an agent, and the first actor network based on deep reinforcement learning takes the parameters of the first state space as input and outputs the prediction of the first action in the first action space; the first evaluation network is used to predict the first target value of the transaction throughput reward according to the first action, and the first target value introduces the prediction of the transaction throughput reward by the first target network as a stable reference; the structure of the first target network is consistent with the first actor network and the first evaluation network; The policy committee acts as an agent, and the second actor network based on deep reinforcement learning takes the parameters of the second state space as input and outputs the prediction of the second action in the second action space; the second evaluation network is used to predict the second target value of the cross-shard transaction optimization reward according to the second action, and the second target value introduces the prediction of the cross-shard transaction optimization reward by the second target network as a stable reference; the structure of the second target network is consistent with the second actor network and the second evaluation network; performing parameter updates on the first actor network and the first evaluation network according to the first target value, and performing parameter updates on the second actor network and the second evaluation network according to the second target value; and performing soft updates on the first target network and the second target network; The policy committee performs the first action, and when the number of cross-shard transactions in the IoT blockchain network is higher than a set number of total transaction transactions, performs the second action.

2. The IoT blockchain sharding method according to claim 1, characterized in that: Based on the computing resources of each node and the score in the creditor scoring table, multiple nodes are selected to form a strategy committee, including: Calculate the historical behavior reputation, the calculation formula is: in, represents the historical behavior reputation of node i, represents the number of transactions processed by node i in time period t, Represents the total number of transactions of all nodes in the blockchain, represents the online time of node i, T c Indicates the current system time. Indicates the time when node i is registered to the blockchain; α1 and α2 are weight coefficients, and α1+α2=1; Calculate the computing power reputation, the calculation formula is: in, represents the computing power reputation of node i, Indicates the CPU performance value of the node. Represents the network delay performance value of the node, Represents the network delay performance value of the node, Indicates the storage speed performance value of the node; The total CPU performance value of all nodes, Represents the total network delay performance value of all nodes, Represents the total network bandwidth performance value of all nodes, Represents the total storage speed performance value of all nodes. β1, β2, β3, β4 are weight coefficients, and β1+β2+β3+β4=1. The calculation formula of the comprehensive reputation of a node is: in, The comprehensive reputation of node i, ω1 and ω2 are weight coefficients, and ω1+ω2=1.

3. The IoT blockchain sharding method according to claim 1, characterized in that: The preset mechanism includes: proof of work mechanism, proof of equity mechanism or reputation mechanism; the first actor network, the first evaluation network, the second actor network and the second evaluation network all adopt a deep neural network structure.

4. The IoT blockchain sharding method according to claim 1, characterized in that: The calculation formula for the transaction throughput reward is: Among them, R tps represents the transaction throughput reward, K represents the number of shards, T represents the epoch length, Γ represents the block interval, B represents the block size, and R r Indicates the average number of redundant transactions.

5. The IoT blockchain sharding method according to claim 4 is characterized in that: The calculation formula for the cross-shard transaction optimization reward is: Among them, R cts represents the cross-shard transaction optimization reward, α is Indicates the number of transactions in the shard, α cs represents the number of cross-shard transactions, A represents the total number of transactions, and U represents the load imbalance; w1, w2, and w3 are parameters.

6. The IoT blockchain sharding method according to claim 1, characterized in that: The first target value introduces the prediction of the transaction throughput reward of the first target network as a stable reference, and the calculation formula is: y 1i =r 1i +γ1Q1′(s i+1 ,μ′(s i+1 |θ1 μ′ )|θ1 Q′ ; Among them, y 1i represents the first target value, r 1i represents the current value of the transaction throughput reward, γ1 is the first discount factor, Q1′(s i+1 ,μ′(s i+1 |θ1 μ′ )|θ1 Q′ represents the predicted value of the transaction throughput reward of the first target network; s i+1 represents the state of the i+1th epoch, θ1 μ′ represents the parameters of the first target actor network in the first target network, θ1 Q′ Representing parameters of a first target evaluation network in the first target network; The second target value introduces the second target network's prediction of the cross-shard transaction optimization reward as a stable reference, and the calculation formula is: y 2i =r 2i +γ2Q2′(s i+1 ,μ′(s i+1 |θ2 μ′ )|θ2 Q′ ; Among them, y 2i represents the second target value, r 2i represents the current value of the cross-shard transaction optimization reward, γ2 is the second discount factor, Q2′(s i+1 ,μ′(s i+1 |θ2 μ′ )|θ2 Q′ represents the predicted value of the cross-shard transaction optimization reward of the second target network; s i+1 represents the state of the i+1th epoch, θ2 μ′ represents the parameters of the second target actor network in the second target network, θ2 Q′ Represents parameters of the second target evaluation network in the second target network.

7. The IoT blockchain sharding method according to claim 6, characterized in that: Updating parameters of the first actor network and the first evaluation network according to the first target value includes: The parameters of the first evaluation network are updated based on the first target value, and the expression is: Among them, θ1 Q represents the weight parameter of the first evaluation network, y 1i represents the first target value, η represents the learning rate, represents the weight gradient of the first evaluation network, Q(s i ,a i |θ1 Q ) represents the first target value currently estimated by the first evaluation network; s 1i represents the state of the first state space at the i-th epoch, a 1i represents the action of the first action space at the i-th epoch; The parameters of the first actor network are updated, and the expression is: Among them, θ1 μ represents the weight parameter of the first actor network, η represents the learning rate, represents the weight gradient of the first actor network, Q(s1,μ(s1|θ1 μ )|θ1 Q ) represents the first target value currently estimated by the first evaluation network, μ(s1|θ1 μ ) represents the policy function of the first actor network; Updating parameters of the second actor network and the second evaluation network according to the second target value includes: The parameters of the second evaluation network are updated based on the second target value, and the expression is: Among them, θ2 Q represents the weight parameter of the second evaluation network, y 2i represents the second target value, η represents the learning rate, represents the weight gradient of the second evaluation network, Q(s 2i ,a 2i |θ2 Q ) represents the second target value currently estimated by the second evaluation network; s 2i represents the state of the second state space at the i-th epoch, a 2i represents the action of the second action space at the i-th epoch; The parameters of the second actor network are updated, and the expression is: Among them, θ2 μ represents the weight parameter of the second actor network, η represents the learning rate, represents the weight gradient of the second actor network, represents the second target value currently estimated by the second evaluation network, μ(s1|θ1 μ ) represents the policy function of the second actor network.

8. The IoT blockchain sharding method according to claim 7, characterized in that: Performing a soft update on the first target network and the second target network includes: The weight update expression of the first target actor network is: θ1 μ′ ←tθ1 μ +(1-τ)θ1 μ′ ; Among them, θ1 μ′ represents the weight parameter of the first target actor network, τ represents the soft update coefficient, θ1 μ represents the current weight parameter of the first actor network; The weight update expression of the first target evaluation network is: θ1 Q′ ←tθ1 Q +(1-τ)θ1 Q′ ; Among them, θ1 Q′ represents the weight parameter of the first target evaluation network, τ represents the soft update coefficient, θ1 Q represents the current weight parameter of the first evaluation network; The weight update expression of the second target actor network is: θ2 μ′ ←tθ2 μ +(1-τ)θ2 μ′ ; Among them, θ2 μ′ represents the weight parameter of the second target actor network, τ represents the soft update coefficient, θ2 μ represents the current weight parameter of the second actor network; The weight update expression of the second target evaluation network is: θ2 Q′ ←tθ2 Q +(1-τ)θ2 Q′ ; Among them, θ2 Q′ represents the weight parameter of the second target evaluation network, τ represents the soft update coefficient, θ2 Q Represents the current weight parameters of the second evaluation network.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method as claimed in any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images