Block chain fragment dynamic optimization method based on two-stage reinforcement learning

Abstract

The invention discloses a block chain fragment dynamic optimization method based on two-stage reinforcement learning. Firstly, security and performance contribution values of nodes are quantified to calculate security and performance values of fragments, then a two-stage migration decision model is constructed, to-be-migrated nodes are screened through a DQN algorithm, and then a proper target fragment is selected by adopting an A2C algorithm according to fragment load difference and security deviation constraints. According to the method, the node migration in the blockchain fragments is disassembled into a'screening-distribution 'two-stage reinforcement learning task, so that the local optimization defect of a traditional single-stage strategy is effectively overcome, the node load balance between the fragments is remarkably improved, and the transaction processing efficiency of the system is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of blockchain sharding optimization, and specifically relates to a blockchain sharding dynamic optimization method based on two-stage reinforcement learning, which is suitable for load balancing and communication overhead optimization in high-concurrency scenarios. Background Art

[0002] Blockchain technology, with its decentralized, secure, and reliable nature, provides an ideal infrastructure for IoT applications. However, with the surge in the number of IoT devices, traditional blockchain architectures are facing bottlenecks such as insufficient processing power and high transaction latency. To overcome these bottlenecks, blockchain sharding has emerged. This technology divides the network into multiple shards to process transactions in parallel, significantly improving the system's overall throughput and scalability. It effectively meets the IoT's rigid demands for large-scale, high-frequency data processing and interaction, providing solid technical support for its continued development.

[0003] However, existing blockchain sharding solutions still face challenges in terms of shard security, performance, and load balancing. On the one hand, differences in the behavior and reputation of different nodes can make some shards vulnerable to attacks or centralization, compromising system security. On the other hand, frequent transactions and interactions can lead to overload or uneven resource utilization on some shards, impacting performance. Furthermore, shard node migration often lacks intelligent and dynamic mechanisms, making it difficult to adapt to real-time changes in network status, limiting system stability and security.

[0004] Therefore, there is an urgent need for an innovative solution that can dynamically quantify the multi-dimensional capabilities of nodes, balance security and performance objectives, and achieve shard load balancing and system stability through efficient migration decisions. To address these issues, this paper proposes a dynamic optimization method for blockchain sharding based on two-stage reinforcement learning. Through a two-stage migration decision-making method, node migration is broken down into a two-stage "screening-allocation" task, aiming to optimize the balance between security and performance of blockchain sharding and achieve deep self-adaptation of blockchain networks. Summary of the Invention

[0005] According to the first aspect of the invention, a two-stage reinforcement learning-based dynamic optimization method for blockchain sharding is provided. This method aims to improve the performance and security of blockchain systems and address issues such as irrational node allocation and uneven shard load in existing sharding technologies. By quantifying the security and performance contributions of nodes and shards, the invention enables more rational node sharding allocation and migration strategies, thereby improving the overall efficiency and security of blockchain networks.

[0006] The core idea of ​​this invention is to combine the security and performance of nodes, optimize their reasonable division in blockchain shards by dynamically quantifying the capabilities of nodes in shards, and thus improve system performance and stability.

[0007] First, before assigning nodes to shards, the security contribution and performance contribution of the nodes are quantified. The security contribution is calculated based on the node's historical voting behavior, dividing the node into block leaders and followers. By analyzing the node's voting in successful and failed blocks, a binary variable is defined to identify the block submission status, thereby calculating the node's security contribution. Specifically, if a node votes in favor of a successfully submitted block, its security contribution is higher; conversely, if it votes against a failed block, its security contribution is also higher.

[0008] Secondly, the security value of a shard is calculated based on the sum of the node's security contribution values ​​and the node's interaction distribution entropy. By constructing an interaction graph between nodes and defining the normalized weighted degree of each node, Shannon entropy is used to quantify the degree of decentralization of nodes within the shard. This way, the security value of a shard not only reflects the node's security contribution but also takes into account the degree of decentralization, effectively reducing the risk of single points of failure. Regarding the performance contribution value, this invention quantifies the performance contribution of a node by the total number of successful transactions submitted per second within an epoch. The performance contribution values ​​of the nodes are similarly weighted and averaged to ultimately calculate the performance value of the shard. This process ensures that the allocation of shard resources takes into account both node security and performance.

[0009] Next, for node migration selection, the present invention uses the DQN algorithm to screen the set of nodes to be migrated. This algorithm prioritizes nodes whose security contribution or performance contribution values ​​significantly deviate from the shard mean to optimize the migration strategy. This approach effectively identifies nodes with poor security or performance performance, allowing for appropriate migration.

[0010] Finally, the A2C algorithm selects the target shard for the node migration. The policy network actors generate the probability of assigning the target shard based on shard load variance and safety deviation constraints. By incorporating metrics such as the shard's real-time load and communication latency, the actor network achieves load balancing while ensuring security. The final reward function encompasses security, performance variance reduction, and overall cost, ensuring improved overall system performance and security after the migration is complete.

[0011] In summary, this paper proposes a novel sharding node optimization method by comprehensively considering the security and performance contributions of nodes. By rationally allocating sharding nodes, the security and performance of the blockchain system can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The present invention will be further described below with reference to the accompanying drawings and embodiments;

[0013] Figure 1 It is a system flow chart of the present invention;

[0014] Figure 2 It is a system architecture diagram of the present invention;

[0015] Figure 3 It is a schematic diagram of screening migration nodes;

[0016] Figure 4 It is a schematic diagram of node redistribution;

[0017] Figure 5 This is a schematic diagram of the redistribution results. DETAILED DESCRIPTION

[0018] The present invention is further described and illustrated in detail below with reference to specific embodiments, but it should not be understood that the scope of the present invention is limited to the following embodiments.

[0019] Reference Figure 1 As shown, the specific steps of the present invention are:

[0020] Step 1: Quantify the security contribution and performance contribution of the node in the shard, and further calculate the security and performance values ​​of the shard based on these two quantitative results. Specifically, first quantify the security contribution based on the node's historical voting behavior, divide the node into the role weights of block leaders and followers, and combine the voting results with the block submission status to quantify the individual security contribution. If the node votes in favor of the successfully submitted block, its contribution value is higher; if the node votes against the block that failed to submit, its contribution value is higher. Define μ j ∈{0,1} binary variable is used to identify whether the block is submitted successfully, e ij ∈{0,1} indicates whether the node voted against or in favor. The total number of blocks submitted successfully and failed is N, and the total number of nodes is m k , where k represents the index of the shard. The set of block votes that the node participates in as a leader is N L , the block voting set N that participates as a follower F , where N = N L ∪N F . Define the security contribution of a node as It reflects the security of node i in epoch t and is calculated as follows:

[0021]

[0022] Where w is the voting weight of the block leader. After the end of epoch t, ​​the security contribution of node i is s(n i) Use the current safety contribution value for weighted averaging.

[0023] Secondly, the security value of the shard is calculated based on the sum of the security contribution values ​​of the nodes and the interaction distribution entropy of the nodes. On the one hand, if the security contribution value of the nodes in the shard is higher, the security value of the shard is higher. On the other hand, if the degree of decentralization within the shard is higher, the security value of the shard is higher. Define the interaction graph between nodes as G(V,E), where each vertex in V represents a node, and each edge in E represents the frequency value of the interaction between two nodes. Define p d (n i ) is the normalized weighted degree value of the node. Based on this, Shannon entropy is introduced to quantify the degree of decentralization of nodes in the shard, and the shard sd k The safety value s(sd k ) is calculated as follows,

[0024]

[0025] Among them, α and β are the weight coefficients of security and decentralization respectively. It not only encourages the accumulation of security contribution value within the shard, but also punishes the risk of excessive concentration of node interactions through Shannon entropy, thus balancing the goals of security and decentralization. For example, when the node interactions within the shard are evenly distributed, the entropy value is high, and the security value of the shard will be improved. The arithmetic mean of the security contribution value of all nodes in the shard is,

[0026]

[0027] Then, the performance contribution is quantified based on the effective throughput of the node processing transactions, which is specifically calculated as the total number of successful transactions submitted per second by the node in the epoch. i,j represents the number of transactions contributed by node i, E r represents the number of nodes that voted in favor. The performance contribution of node i in epoch t is calculated as follows:

[0028]

[0029] Similarly, the performance contribution of node i is weighted averaged using the current performance contribution value. Finally, the performance value p(sd k ),

[0030]

[0031] The average performance contribution of all nodes in the shard is μ p (sd k ),

[0032]

[0033] Step 2: Migration node selection. In this stage, the DQN algorithm is used to select the set of nodes to be migrated from all nodes, and the state space is defined as is the shard security value matrix state matrix, is the sharding performance value matrix, Contribute vector to node security, Contribution vector for performance, is the set of nodes to be migrated; K is the number of shards and N is the total number of nodes. is the shard load difference index, which reflects the imbalance of resource pressure between shards. k is the real-time load of shard k, is the average load of the system shards,

[0034]

[0035] where Q k is the number of transactions in the current shard transaction pool that have been counted in shard k but have not yet been packaged into the block.

[0036] is the average communication delay between shard k and other shards, d kj is the communication delay between shard k and shard j, ω, z, and η are weight coefficients, satisfying ω+z+η=1.

[0037] The action space is defined as a(t) = {kt, zt}, where kt represents the migration node selection and zt represents the target shard selection. Specifically, kt represents the selection of nodes that deviate from the shard performance or security and enters the set of nodes to be migrated. When a new node joins, it will also directly enter the set of nodes to be migrated. The binary decision vector set A = {A1, A2, A3, ..., A N}, A i ∈{0, 1} N Indicates whether to migrate the corresponding node, A i =1 means that node i enters the set of nodes to be migrated. zt means selecting the target shard for the node to be migrated, and the output is the target shard index vector corresponding to the node to be migrated.

[0038] The action is selected through the ε-greedy strategy, and nodes whose security or performance deviates from the threshold are randomly selected with probability ε. Nodes whose migration action Q value is higher than the non-migration action Q value are selected with probability 1-ε to be added to the set to be migrated. Right now,

[0039] d s (n i )>Threshold s or d p (ni )>Threshold p

[0040] where d s (n i )=|Δ s (n i )-μ s (sd k )|、d p (n i )=|Δ p (n i )-μ p (sd k )| are the absolute differences between the node security and performance contribution values ​​and the shard mean. s =α·σ s +∈·δ s and Threshold p =β·σ p +∈·δ p are the dynamic thresholds of safety and performance, σ s , σ p is the variance of the security and performance contribution values ​​within the shard, α and β are adjustable parameters that can be dynamically adjusted according to the variance fluctuation, δ s and δ p To explore the disturbance term, it decays with the training rounds.

[0041] During the training process, the experience replay mechanism is used to store the state transition tuple (s t ,a t ,r t ,s t+1 ), and calculate the target Q value through batch sampling,

[0042]

[0043] where r t is the immediate reward, r=λ1(s(sd k ) t+1 -s(sd k ) t )+λ2(p(sd k ) t+1 -p(sd k ) t ), λ1, λ2 are weight coefficients, γ is the discount factor, and the online network Q θ By minimizing the mean square error loss function Update parameters, target network The soft update mechanism is used to synchronize parameters, i.e. θ - ←τθ+(1-τ)θ -, where τ is the soft update coefficient to stabilize the Q-value estimation and improve training convergence.

[0044] This design uses a dual-stream network structure to extract security matrix and performance matrix features respectively, and generates Q values ​​after fusion through a fully connected layer. It not only retains the multi-dimensional information of shard security and performance, but also achieves efficient node screening through the ε-greedy strategy and dynamic threshold, ultimately optimizing the long-term benefits of migration decisions.

[0045] Step 3: Target shard selection and allocation. In this stage, the A2C algorithm is used to select appropriate shards and allocate them to the nodes to be migrated. load And the set of nodes to be migrated output in the first stage As input, the policy network actor is subject to the constraints, target shard load And safety deviation Generate assignment probabilities, where ΔS k =|s(sd k )-μ S |,μ S is the system safety mean, The preset security deviation tolerance threshold is adaptively adjusted based on the historical security variance to ensure the consistency of the two-stage goals. The policy network Actor outputs the target shard allocation probability distribution π z (z t |s t [k t ]), the value network Critic evaluates the state value V θ (s t ). The advantage function is defined as the difference between the actual cumulative reward and the critic's prediction value,

[0046]

[0047] The loss function of the policy network Actor is,

[0048]

[0049] The loss function of the value network Critic is,

[0050]

[0051] The final reward function is calculated after the migration is completed, covering security, performance variance reduction and comprehensive cost.

[0052]

[0053] Where λ1 is the safety variance reduction weight, λ2 is the performance variance reduction weight, λ3 is the migration cost penalty weight, σ s , σp are the variances of security and performance contributions within the shard before migration, σ' s ,σ' p are the variances of security and performance contributions within the shard after migration, C select The computational cost for node selection is calculated as follows:

[0054]

[0055] O(s i ) The time complexity of calculating the security contribution value of node i is linearly related to the number of historical voting records, that is, O(s i )=k1·|N L ∪N F |. O(p i ) is the time complexity of calculating the performance contribution value of node i, which is linearly related to the transaction throughput, that is, O(p i )=k2·TX i , k1 and k2 are constants. A2C (A) is the time complexity of a single iteration of the A2C algorithm. decision The delay of real-time decision making is proportional to the number of nodes to be migrated.

[0056] C migrate The migration cost includes bandwidth, latency and other overheads and is calculated as follows:

[0057]

[0058] B data The amount of data that needs to be transferred for a single node migration, d avg is the average network distance between the source shard and the target shard, L delay (k, j) is the one-way delay between shard k and shard j, and t is the time it takes for a single node state to be synchronized to the target shard.

[0059] Step 4: After each round of migration decision is completed, the system implements dynamic control of training through the following process. First, calculate the global composite reward function R total The absolute difference ΔR between the current value and the previous value of Or the current training round has reached the maximum training round, that is, TR ≥ TR max , the training is terminated; otherwise, the system recalculates the shard security value s(sd k ) and performance value p(sd k ) and load difference index σ load , recalculate the node security and performance contribution values ​​and update the shard status matrix, then return to step 1 to trigger a new round of two-stage migration decisions, forming a closed-loop optimization process until the convergence conditions are met or the maximum round limit is reached.

Claims

1. A blockchain sharding dynamic optimization method based on two-stage reinforcement learning, characterized in that: The following steps are involved: Step 1: For all nodes in the shard, the security contribution value is quantified based on historical voting behavior and role weight, and the performance contribution value is quantified based on the effective transaction throughput. The shard security value is calculated based on the sum of the security contribution values ​​and the interaction distribution entropy index, and the shard performance value is calculated using the sum of the performance contribution values. Step 2: Build a DQN network, using the shard security matrix, shard performance matrix, and node contribution vector as network inputs. The migration value Q value of each node is output. Nodes whose security contribution value or performance contribution value significantly deviates from the average level of their shard are identified to form a set to be migrated. The experience replay mechanism and the target network are used to stabilize the training process. The loss function of the Q network is defined as the mean squared error between the predicted Q value and the target Q value. Step 3: Build an A2C policy network actor. Based on the generated set of nodes to be migrated, the policy network actor generates a target shard index allocation action. Under the constraints of safety deviation and load balancing, the target shard is allocated to the nodes to be migrated. The critic network evaluates the long-term benefits of the allocation strategy and selects the target shard that is suitable for the selected node set. Step 4: After each round of training, if the change in the global compound reward function is lower than the preset threshold, or the preset maximum number of training rounds has been reached, the algorithm training process is exited. Otherwise, return to step 1 to form a closed-loop control mechanism for dynamic optimization.

2. The blockchain sharding dynamic optimization method based on two-stage reinforcement learning according to claim 1 is characterized in that: The specific calculation of the shard security value is as follows: First, the security contribution value is quantified based on the node's historical voting behavior, and the nodes are divided into the role weights of block leaders and followers. The voting results and block submission status are combined to quantify the individual security contribution, and μ is defined. j ∈{0,1} binary variable is used to identify whether the block is submitted successfully, e ij ∈{0,1} indicates whether the node voted against or in favor. The total number of blocks submitted successfully and failed is N, and the total number of nodes is m k , where k represents the index of the shard; The set of block votes that the node participates in as a leader is N L , the block voting set N that participates as a follower F , where N = N L ∪N F , defining the security contribution of a node as reflects the security of node i in epoch t and is calculated as follows, Where w is the voting weight of the block leader, after epoch t ends, the security contribution of node i is s(n i ) Using the current safety contribution value for weighted averaging; Secondly, the security value of the shard is calculated based on the sum of the security contribution value of the node and the interaction distribution entropy of other nodes. The interaction graph between nodes is defined as G(V,E), where each vertex in V represents a node, and each edge in E represents the frequency value of the interaction between two nodes. Define p d (n i ) is the normalized weighted degree value of the node, Shannon entropy is introduced to quantify the degree of decentralization of nodes in the shard, and the shard sd k The safety value s(sd k ) is calculated as follows, Among them, α and β are the weight coefficients of security and decentralization, respectively. They not only encourage the accumulation of security contribution value within the shard, but also punish the risk of excessive concentration of node interactions through Shannon entropy, thereby balancing the goals of security and decentralization.

3. The blockchain sharding dynamic optimization method based on two-stage reinforcement learning according to claim 1 is characterized in that: Assign target shards to the nodes to be migrated under the constraints of safety deviation and load balancing. The specific constraints are the target shard load. And the safety deviation ΔS k ≤θ s , L k is the real-time load of shard k, where Q k is the number of transactions in the current shard transaction pool that have been counted in shard k but have not yet been packaged into the block. is the average communication delay between shard k and other shards, d kj is the communication delay between shard k and shard j, ω, z, η are weight coefficients, satisfying ω+z+η=1, is the average load of the system slice; ΔS k =|s(sd k )-μ S |,s(sd k ) is the shard security value, μ S is the system security mean, θ s The preset safety deviation tolerance threshold is adaptively adjusted based on historical safety variance to ensure the consistency of the two-stage goals.

Images