Block chain hierarchical consensus optimization method for Internet of Vehicles

Abstract

A block chain hierarchical consensus optimization method for the Internet of Vehicles comprises the following operation steps: (1) constructing a block chain consensus framework in a fragmentation mode, a head vehicle of a fragment representing the fragment communicates with an RSU of the fragment and other fragments; all vehicles in one fragment form a formation; (2) the block chain consensus framework executes two layers of consensus processes, namely local consensus and global consensus; the local consensus process refers to a local consensus achievement process performed among all vehicles in the same fragment; the global consensus process refers to a global consensus achievement process carried out among the RSUs of the fragments; according to the method, the formation vehicles only need to be subjected to consensus in the fragments, the communication complexity and resource consumption are greatly reduced, and the efficiency of the whole consensus process is improved.

Description

Technical Field

[0001] The present invention relates to a blockchain layered consensus optimization method for an Internet of Vehicles, and belongs to the technical field of Internet of Vehicles, and in particular to the technical field of blockchain layered consensus optimization for an Internet of Vehicles. Background Art

[0002] With the development of Internet of Vehicles (IoV) related technologies and the continuous progress of vehicle intelligence, more and more IoV applications and services have emerged, requiring each IoV node to generate and exchange a large amount of data. However, the high mobility, heterogeneity and low latency requirements of IoV make traditional cloud-based and centralized data management not applicable in many scenarios, and it is difficult to ensure data security. Blockchain, as a distributed, decentralized, tamper-proof and interoperable data management method, has broad application prospects in IoV.

[0003] However, there are also some problems with the deployment and application of blockchain in the Internet of Vehicles. One of the most important problems is the inefficiency caused by the blockchain consensus process. The most widely used consensus algorithm, Practical Byzantine Fault Tolerance (PBFT), forms a consensus through multiple rounds of information exchange and signature verification between all nodes. When faced with a large network scale and complex communication Internet of Vehicles environment, it will cause a large consensus delay, consume a lot of resources, and reduce the efficiency of the entire system. This is unacceptable in Internet of Vehicles applications that require low latency and fast response.

[0004] How to optimize the blockchain consensus process and improve the efficiency of the blockchain consensus process is a technical problem that urgently needs to be solved in the field of Internet of Vehicles consensus optimization technology. Summary of the invention

[0005] In view of this, the purpose of the present invention is to invent a blockchain hierarchical consensus optimization method for the vehicle networking vehicle formation scenario, aiming to reduce the complexity of the vehicle networking blockchain consensus process and improve the consensus optimization efficiency of the vehicle networking blockchain.

[0006] In order to achieve the above object, the present invention proposes a blockchain hierarchical consensus optimization method for Internet of Vehicles, which includes the following steps:

[0007] (1) A sharded blockchain consensus framework is constructed. Under this framework, vehicles and roadside units (RSUs) that are close to each other are assigned to the same shard. Each shard contains a roadside unit (RSU) and multiple vehicles. In each shard, one vehicle is selected as the lead vehicle from all vehicles in the shard. That is, there are two types of vehicles in the shard: ordinary vehicles and lead vehicles. The lead vehicle of a shard communicates with the RSU of the shard and other shards on behalf of the shard. All vehicles in a shard form a formation. There are two types of blockchain nodes in the blockchain consensus framework, namely, vehicles and roadside units (RSUs).

[0008] (2) The blockchain consensus framework implements a two-layer consensus process, namely local consensus and global consensus; the local consensus process refers to the local consensus process carried out among all vehicles in the same shard; the local consensus process does not require long-distance communication with other shards, which greatly reduces the communication complexity of the consensus process; the local consensus process can be carried out in parallel without interfering with each other, thereby improving system efficiency; each shard has its own local blockchain, and the platoon vehicles use the information in this shard more frequently, so that the platoon vehicles in this shard can quickly obtain local blockchain information; the global consensus process refers to the global consensus process carried out among the RSUs of each shard; the global consensus process ensures the consistency of information in all shards; the complete blockchain is stored on the RSU, and the complete blockchain information on each RSU is consistent, recording the local blockchain information in each shard, and when a vehicle in a shard needs the complete blockchain information, it can be obtained from the RSU.

[0009] In step (2), the blockchain consensus framework executes the local consensus process within the shard through a consensus optimization model; the consensus optimization model is constructed based on the generative diffusion model GDM and deep reinforcement learning DRL;

[0010] In the consensus optimization model, the reverse diffusion process of GDM introduces the deep reinforcement learning SAC algorithm, and replaces the policy network in the SAC algorithm with the reverse diffusion process to generate the strategy of the intelligent agent; the policy network is used in traditional SAC to generate the probability distribution of actions, and after the introduction of GDM, the probability distribution of actions is obtained by denoising the Gaussian noise;

[0011] The consensus optimization model is deployed on RSU;

[0012] The consensus optimization goal of the consensus optimization model of the blockchain consensus framework is to maximize the throughput of the blockchain system while minimizing the consensus delay; the state space of the consensus optimization model takes into account the physical parameters that affect the blockchain throughput in the formation environment, including the available computing resources of the vehicle and the communication rate between the vehicle and the RSU; the consensus optimization model needs to select the optimal action in the current state according to the state space parameters, including adjusting the head vehicle, block size and block interval; the consensus optimization model can learn the optimal consensus adjustment strategy by online learning in the process of continuously collecting environmental status and executing adjustment actions to obtain rewards.

[0013] The specific content of the consensus optimization model state modeling is:

[0014] (11) Select state parameters: (a) The available computing resources of the vehicle, including chips, GPUs and other on-board computing units; the available computing resources of the vehicle may fluctuate due to the vehicle performing other tasks, resulting in fluctuations in the computing resources allocated to the consensus task; (b) The communication rate between the vehicle and the RSU; the vehicle exchanges information through the RSU during the consensus process, so the communication rate with the RSU affects the transmission delay in the consensus, and the communication rate will also fluctuate with a certain probability;

[0015] (12) The consensus optimization model obtains the above state parameters from the Internet of Vehicles environment to form a state vector S = [n, (c1, ..., cn), (r1, ..., rn)], where n is the number of platoon vehicles in the shard, ci is the available computing resources of vehicle i, and ri is the communication rate between vehicle i and the RSU.

[0016] The specific content of the consensus optimization model action space modeling is: the action space includes:

[0017] (a) Lead vehicle within a shard: The lead vehicle is responsible for generating block information, and the available computing resources of the vehicle affect the rate at which the vehicle generates information, which in turn affects the blockchain throughput. Therefore, the consensus optimization model needs to select the vehicle with more available computing resources as the lead vehicle. At the same time, it is necessary to appropriately predict the changes in vehicle computing resources and select a suitable lead vehicle.

[0018] (b) Block size: This is the block size generated by the lead vehicle. A larger block size can contain more information, but will result in higher transmission delays. Therefore, the consensus optimization model needs to balance the setting of block capacity.

[0019] (c) Block interval: the time interval between two consecutive blocks generated by the leading vehicle. In theory, a smaller time interval can enable vehicles to generate blocks faster and improve throughput. However, too fast a block generation speed may prevent consensus from being completed in time, resulting in information loss or chain forks, which puts additional pressure on the system. Therefore, a consensus optimization model is also needed to make a trade-off.

[0020] In summary, the action vector of the consensus optimization model is A=[H, SB, IB], where H is the head vehicle, SB is the block size, and IB is the block interval.

[0021] The specific content of the consensus optimization model generation action is: after the consensus optimization model obtains the state vector, it takes it together with Gaussian noise as the input of the reverse diffusion model; the purpose of the diffusion model is to learn the strategy π(a|s), that is, under the state s∈{S}, obtain the optimal consensus adjustment plan a∈{A}; after T steps of denoising process, the current consensus adjustment plan is obtained; the denoising process is the same as the reverse diffusion process of the diffusion model, and the above denoising process is implemented by a neural network, where each step of the denoising process can be expressed as q(a i |a i+1 ,s), i∈{0,...,T-1}, where q is the action probability distribution predicted by the denoising model, a i is the intermediate action generated in the Ti-th step during the iterative denoising process, a 0 That is, the action executed by the model, that is, the intermediate action and state vector obtained by the previous denoising step are used as the input of the denoising model, the mean of the action distribution predicted by the model is obtained, the next intermediate action is obtained by reparameterization, and the final executed action is obtained after T steps of denoising in the above manner; this iterative step of denoising in the intermediate action can fine-tune the consensus optimization model and improve the exploration ability of the consensus optimization model.

[0022] The specific content of the consensus optimization model strategy update is: in order to train the inverse denoising process of the consensus optimization model so that it can learn the optimal strategy π(a|s), the SAC strategy update method is used to guide the training of the consensus optimization model; that is, two critic networks and two target-critic networks are set, the critic network evaluates the action value generated by the diffusion model, and avoids overestimation by taking the minimum value of the Q value predicted by the two critic networks, and the target-critic network is used to predict the target value; after each interaction between the agent and the environment, the experience (s, a, s', r) generated by the interaction is stored in the experience replay pool, where s is the current state, a is the action performed by the agent, s' is the new state after the action is performed, and r is the reward returned by the environment; each strategy update randomly takes N experience samples {(s i , a i , s′ i , r i )}, i∈{1,...,N}, where s i , a i , S′ i , r iRespectively represent the environment state in the i-th experience sample, the action performed by the model, the new state obtained after the action is performed, and the reward after the action is performed. The policy network loss is calculated according to the following formula, and the policy network is updated by gradient descent;

[0023]

[0024] In the above formula, Q(s i , a i ) is the value of the action in the i-th experience sample predicted by the critic network, logπ θ (a i |s i ) is the entropy of the strategy generated by the strategy network, α is the reward coefficient of entropy, which determines the importance of entropy, and θ is the strategy network parameter;

[0025] The updating method of the critic network is consistent with the traditional SAC algorithm. The updating goal is to minimize the gap between its predicted value and the target value. The target-critic network is updated by soft-update. After training, the critic network can accurately predict the action value and guide the denoising model to learn the optimal strategy.

[0026] The ordinary vehicles will collect road condition information through their own sensors during driving and provide necessary information for the formation; ordinary vehicles also have the need to communicate with vehicles in this shard and vehicles in other shards. When communicating with vehicles in this shard, it can be done directly through vehicle-to-vehicle communication (V2V), while communication with other shards needs to be done through the lead vehicle and RSU; at the same time, ordinary vehicles will also participate in the consensus process within the shard as blockchain nodes;

[0027] The head vehicle in the shard is responsible for communicating with the RSU on behalf of the shard to reduce communication pressure, so it needs to have a vehicle with more computing resources. When information is generated in the shard, the head vehicle will be responsible for forming a block and executing the local consensus process in the shard. At the same time, the head vehicle is also responsible for transmitting information to the RSU, reaching a global consensus with other shards, and finally completing the block chain.

[0028] Vehicles within a shard must communicate with other shards through RSU; RSU also acts as a blockchain node and executes the global consensus process with other RSUs, that is, after local consensus within the shard, global consensus needs to be achieved among all shards; since RSU is generally operated by trusted government agencies or operators and has more storage and computing resources, it also acts as a trusted edge storage node to store the complete blockchain.

[0029] The beneficial effects of the present invention are as follows: the vehicles in each platoon only need to reach consensus within the shard, and do not need to reach consensus among all nodes, which greatly reduces the complexity of communication and resource consumption; the vehicles only need to store local blockchain information, and do not need to store the complete blockchain, which reduces the consumption of storage resources; the consensus of each shard can be carried out in parallel without interfering with each other, which improves the efficiency of the entire consensus process. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flow chart of a blockchain hierarchical consensus optimization method for Internet of Vehicles proposed in the present invention. DETAILED DESCRIPTION

[0031] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings.

[0032] See also Figure 1 , introduces a blockchain hierarchical consensus optimization method for Internet of Vehicles proposed by the present invention, the method comprising the following steps:

[0033] (1) A sharded blockchain consensus framework is constructed. Under this framework, vehicles and roadside units (RSUs) that are close to each other are assigned to the same shard. Each shard contains a roadside unit (RSU) and multiple vehicles. In each shard, one vehicle is selected as the lead vehicle from all the vehicles in the shard. That is, there are two types of vehicles in the shard: ordinary vehicles and lead vehicles. The lead vehicle of a shard communicates with the RSU of the shard and other shards on behalf of the shard. All vehicles in a shard form a formation. There are two types of blockchain nodes in the blockchain consensus framework, namely, vehicles and roadside units (RSUs).

[0034] (2) The blockchain consensus framework implements a two-layer consensus process, namely local consensus and global consensus; the local consensus process refers to the local consensus process carried out among all vehicles in the same shard; the local consensus process does not require long-distance communication with other shards, which greatly reduces the communication complexity of the consensus process; the local consensus process can be carried out in parallel without interfering with each other, thereby improving system efficiency; each shard has its own local blockchain, and the platoon vehicles use the information in this shard more frequently, so that the platoon vehicles in this shard can quickly obtain local blockchain information; the global consensus process refers to the global consensus process carried out among the RSUs of each shard; the global consensus process ensures the consistency of information in all shards; the complete blockchain is stored on the RSU, and the complete blockchain information on each RSU is consistent, recording the local blockchain information in each shard, and when a vehicle in a shard needs the complete blockchain information, it can be obtained from the RSU.

[0035] In step (2), the blockchain consensus framework executes the local consensus process within the shard through a consensus optimization model; the consensus optimization model is constructed based on a generative diffusion model (GDM) and deep reinforcement learning (DRL).

[0036] In the consensus optimization model, the reverse diffusion process of GDM introduces the deep reinforcement learning SAC (Soft Actor Critic) algorithm, and replaces the policy network in the SAC algorithm with the reverse diffusion process to generate the strategy of the intelligent agent; in traditional SAC, the policy network is used to generate the probability distribution of actions, and after the introduction of GDM, the probability distribution of actions is obtained by denoising the Gaussian noise;

[0037] The consensus optimization model is deployed on RSU;

[0038] The consensus optimization goal of the consensus optimization model of the blockchain consensus framework is to maximize the throughput of the blockchain system while minimizing the consensus delay; the state space of the consensus optimization model takes into account the physical parameters that affect the blockchain throughput in the formation environment, including the available computing resources of the vehicle and the communication rate between the vehicle and the RSU; the consensus optimization model needs to select the optimal action in the current state according to the state space parameters, including adjusting the head vehicle, block size and block interval; the consensus optimization model can learn the optimal consensus adjustment strategy by online learning in the process of continuously collecting environmental status and executing adjustment actions to obtain rewards.

[0039] The specific content of the consensus optimization model state modeling is:

[0040] (11) Select state parameters: (a) The available computing resources of the vehicle, including chips, GPUs and other on-board computing units; the available computing resources of the vehicle may fluctuate due to the vehicle performing other tasks, resulting in fluctuations in the computing resources allocated to the consensus task; (b) The communication rate between the vehicle and the RSU; the vehicle exchanges information through the RSU during the consensus process, so the communication rate with the RSU affects the transmission delay in the consensus, and the communication rate will also fluctuate with a certain probability;

[0041] (12) The consensus optimization model obtains the above state parameters from the Internet of Vehicles environment to form a state vector S = [n, (c1, ..., cn), (r1, ..., rn)], where n is the number of platoon vehicles in the shard, ci is the available computing resources of vehicle i, and ri is the communication rate between vehicle i and the RSU. In the embodiment, the value range of n is 5-15.

[0042] The specific content of the consensus optimization model action space modeling is: the action space includes:

[0043] (d) Lead vehicle within a shard: The lead vehicle is responsible for generating block information, and the available computing resources of the vehicle affect the rate at which the vehicle generates information, which in turn affects the blockchain throughput. Therefore, the consensus optimization model needs to select the vehicle with more available computing resources as the lead vehicle. At the same time, it is necessary to appropriately predict the changes in vehicle computing resources and select a suitable lead vehicle.

[0044] (e) Block size: This is the block size generated by the lead vehicle. A larger block size can contain more information, but will result in higher transmission delays. Therefore, the consensus optimization model needs to balance the setting of block size.

[0045] (f) Block interval: the time interval between two consecutive blocks generated by the leading vehicle. In theory, a smaller time interval can enable vehicles to generate blocks faster and improve throughput. However, too fast a block generation speed may prevent consensus from being completed in time, resulting in information loss or chain forks, which puts additional pressure on the system. Therefore, a consensus optimization model is also needed to make a trade-off.

[0046] In summary, the action vector of the consensus optimization model is A=[H, SB, IB], where H is the head vehicle, SB is the block size, and IB is the block interval. In the embodiment, the value range of the block size SB is an integer between 200-500, and the value range of the block interval IB is an integer between 5-20;

[0047] The specific content of the consensus optimization model generation action is: after the consensus optimization model obtains the state vector, it takes it together with Gaussian noise as the input of the reverse diffusion model; the purpose of the diffusion model is to learn the strategy π(a|s), that is, under the state s∈{S}, obtain the optimal consensus adjustment plan a∈{A}; after T steps of denoising process, the current consensus adjustment plan is obtained; the denoising process is the same as the reverse diffusion process of the diffusion model, and the above denoising process is implemented by a neural network, where each step of the denoising process can be expressed as q(a i |a i+1 ,s), i∈{0,...,T-1}, where q is the action probability distribution predicted by the denoising model, a i is the intermediate action generated in the Ti-th step during the iterative denoising process, a 0 That is, the action executed by the model, that is, the intermediate action and state vector obtained by the previous denoising step are used as the input of the denoising model, the mean of the action distribution predicted by the model is obtained, the next intermediate action is obtained by reparameterization, and the final executed action is obtained after T steps of denoising in the above manner; this iterative step of denoising in the intermediate action can fine-tune the consensus optimization model and improve the exploration ability of the consensus optimization model.

[0048] In the embodiment, T is generally set to 5;

[0049] The specific content of the consensus optimization model strategy update is: in order to train the inverse denoising process of the consensus optimization model so that it can learn the optimal strategy π(a|s), the SAC strategy update method is used to guide the training of the consensus optimization model; that is, two critic networks and two target-critic networks are set, the critic network evaluates the action value generated by the diffusion model, and avoids overestimation by taking the minimum value of the Q value predicted by the two critic networks, and the target-critic network is used to predict the target value; after each interaction between the agent and the environment, the experience (s, a, s', r) generated by the interaction is stored in the experience replay pool, where s is the current state, a is the action performed by the agent, s' is the new state after the action is performed, and r is the reward returned by the environment; each strategy update randomly takes N experience samples {(s i , a i , s′ i , r i )}, i∈{1,...,N}, where s i , a i , S i ',r i Respectively represent the environment state in the i-th experience sample, the action performed by the model, the new state obtained after performing the action, and the reward after performing the action; in the embodiment, N is generally taken as 128;

[0050] Calculate the policy network loss according to the following formula and update the policy network by gradient descent;

[0051]

[0052] In the above formula, Q(s i , a i ) is the value of the action in the i-th experience sample predicted by the critic network, logπ θ (a i |s i ) is the entropy of the strategy generated by the strategy network, α is the reward coefficient of entropy, in the embodiment, α is generally taken as 0.2; determines the importance of entropy, θ is the strategy network parameter;

[0053] The updating method of the critic network is consistent with the traditional SAC algorithm. The updating goal is to minimize the gap between its predicted value and the target value. The target-critic network is updated by soft-update. After training, the critic network can accurately predict the action value and guide the denoising model to learn the optimal strategy.

[0054] The ordinary vehicles will collect road condition information through their own sensors during driving and provide necessary information for the formation; ordinary vehicles also have the need to communicate with vehicles in this shard and vehicles in other shards. When communicating with vehicles in this shard, they can directly communicate through vehicle-to-vehicle communication V2V (Vehicle to Vehicle), while communicating with other shards requires communication through the lead vehicle and RSU; at the same time, ordinary vehicles will also participate in the consensus process within the shard as blockchain nodes;

[0055] The head vehicle in the shard is responsible for communicating with the RSU on behalf of the shard to reduce communication pressure, so it needs to have a vehicle with more computing resources. When information is generated in the shard, the head vehicle will be responsible for forming a block and executing the local consensus process in the shard. At the same time, the head vehicle is also responsible for transmitting information to the RSU, reaching a global consensus with other shards, and finally completing the block chain.

[0056] Vehicles within a shard must communicate with other shards through RSU; RSU also acts as a blockchain node and executes the global consensus process with other RSUs, that is, after local consensus within the shard, global consensus needs to be achieved among all shards; since RSU is generally operated by trusted government agencies or operators and has more storage and computing resources, it also acts as a trusted edge storage node to store the complete blockchain.

[0057] The inventors have conducted a large number of experiments on the method proposed in the present invention.

[0058] 15 vehicle nodes, 3 RSUs and 3 shards were set up in the experimental environment. The performance of the method proposed in the present invention was verified during the operation of the Internet of Vehicles, and consensus optimization based on the SAC and DQN (Deep Q-Network) algorithms was set up for comparison in the same environment. The results of 100 rounds of training with 1000 steps per round show that after combining with diffusion, the SAC algorithm is significantly better than the pure SAC algorithm in terms of convergence, and after convergence, the environmental reward of diffusion-sac is still slightly higher than that of the SAC algorithm, achieving better optimization effect. It can be seen that the introduction of the denoising process effectively improves the algorithm's ability to adapt to complex environments, while enhancing the algorithm's exploration ability to avoid falling into local optimal strategies. However, the DQN algorithm performs poorly in complex environments.

[0059] At the same time, the throughput performance of blockchain sharding method is compared under different ranges and amplitudes of environmental state parameters, and the throughput of blockchain is compared under three conditions: 3, 2 and 1 (no sharding). The experimental results show that under various environmental dynamics, the throughput of blockchain after adopting sharding method is higher than that of blockchain system without sharding, which reflects the effective improvement of system performance by blockchain sharding method.

[0060] The above experimental results prove that the method proposed in the present invention is feasible and effective.

Claims

1. A blockchain hierarchical consensus optimization method for Internet of Vehicles, characterized by: The method comprises the following steps: (1) A sharded blockchain consensus framework is constructed. Under this framework, vehicles and roadside units (RSUs) that are close to each other are assigned to the same shard. Each shard contains a roadside unit (RSU) and multiple vehicles. In each shard, one vehicle is selected as the lead vehicle from all vehicles in the shard. That is, there are two types of vehicles in the shard: ordinary vehicles and lead vehicles. The lead vehicle of a shard communicates with the RSU of the shard and other shards on behalf of the shard. All vehicles in a shard form a formation. There are two types of blockchain nodes in the blockchain consensus framework, namely, vehicles and roadside units (RSUs). (2) The blockchain consensus framework implements a two-layer consensus process, namely local consensus and global consensus; the local consensus process refers to the local consensus process carried out among all vehicles in the same shard; the local consensus process does not require long-distance communication with other shards, which greatly reduces the communication complexity of the consensus process; the local consensus process can be carried out in parallel without interfering with each other, thereby improving system efficiency; each shard has its own local blockchain, and the platoon vehicles use the information in this shard more frequently, so that the platoon vehicles in this shard can quickly obtain local blockchain information; the global consensus process refers to the global consensus process carried out among the RSUs of each shard; the global consensus process ensures the consistency of information in all shards; the complete blockchain is stored on the RSU, and the complete blockchain information on each RSU is consistent, recording the local blockchain information in each shard, and when a vehicle in a shard needs the complete blockchain information, it can be obtained from the RSU.

2. According to claim 1, a blockchain layered consensus optimization method for Internet of Vehicles is characterized by: In step (2), the blockchain consensus framework executes the local consensus process within the shard through a consensus optimization model; the consensus optimization model is constructed based on the generative diffusion model GDM and deep reinforcement learning DRL; In the consensus optimization model, the reverse diffusion process of GDM introduces the deep reinforcement learning SAC algorithm, and replaces the policy network in the SAC algorithm with the reverse diffusion process to generate the strategy of the intelligent agent; the policy network is used in traditional SAC to generate the probability distribution of actions, and after the introduction of GDM, the probability distribution of actions is obtained by denoising the Gaussian noise; The consensus optimization model is deployed on RSU.

3. According to claim 2, a blockchain layered consensus optimization method for Internet of Vehicles is characterized in that: The consensus optimization goal of the consensus optimization model of the blockchain consensus framework is to maximize the throughput of the blockchain system while minimizing the consensus delay; the state space of the consensus optimization model takes into account the physical parameters that affect the blockchain throughput in the formation environment, including the available computing resources of the vehicle and the communication rate between the vehicle and the RSU; the consensus optimization model needs to select the optimal action in the current state according to the state space parameters, including adjusting the head vehicle, block size and block interval; the consensus optimization model can learn the optimal consensus adjustment strategy by online learning in the process of continuously collecting environmental status and executing adjustment actions to obtain rewards.

4. A blockchain hierarchical consensus optimization method for Internet of Vehicles according to claim 2 or 3, characterized in that: The specific content of the consensus optimization model state modeling is: (11) Select state parameters: (a) The available computing resources of the vehicle, including chips, GPUs and other on-board computing units; the available computing resources of the vehicle may fluctuate due to the vehicle performing other tasks, resulting in fluctuations in the computing resources allocated to the consensus task; (b) The communication rate between the vehicle and the RSU; the vehicle exchanges information through the RSU during the consensus process, so the communication rate with the RSU affects the transmission delay in the consensus, and the communication rate will also fluctuate with a certain probability; (12) The consensus optimization model obtains the above state parameters from the Internet of Vehicles environment to form a state vector S = [n, (c1, ..., cn), (r1, ..., rn)], where n is the number of platoon vehicles in the shard, ci is the available computing resources of vehicle i, and ri is the communication rate between vehicle i and RSU; The specific content of the consensus optimization model action space modeling is: the action space includes: (a) Lead vehicle within a shard: The lead vehicle is responsible for generating block information, and the available computing resources of the vehicle affect the rate at which the vehicle generates information, which in turn affects the blockchain throughput. Therefore, the consensus optimization model needs to select the vehicle with more available computing resources as the lead vehicle. At the same time, it is necessary to appropriately predict the changes in vehicle computing resources and select a suitable lead vehicle. (b) Block size: This is the block size generated by the lead vehicle. A larger block size can contain more information, but will result in higher transmission delays. Therefore, the consensus optimization model needs to balance the setting of block capacity. (c) Block interval: the time interval between two consecutive blocks generated by the leading vehicle. In theory, a smaller time interval can enable vehicles to generate blocks faster and improve throughput. However, too fast a block generation speed may prevent consensus from being completed in time, resulting in information loss or chain forks, which puts additional pressure on the system. Therefore, a consensus optimization model is also needed to make a trade-off. In summary, the action vector of the consensus optimization model is A=[H, SB, IB], where H is the head vehicle, SB is the block size, and IB is the block interval; The specific content of the consensus optimization model generation action is: after the consensus optimization model obtains the state vector, it takes it together with Gaussian noise as the input of the reverse diffusion model; the purpose of the diffusion model is to learn the strategy π(a|s), that is, under the state s∈{S}, obtain the optimal consensus adjustment plan a∈{A}; after T steps of denoising process, the current consensus adjustment plan is obtained; the denoising process is the same as the reverse diffusion process of the diffusion model, and the above denoising process is implemented by a neural network, where each step of the denoising process can be expressed as q(a i |a i+1 ,s), i∈{0,...,T-1}, where q is the action probability distribution predicted by the denoising model, a i is the intermediate action generated in the Ti-th step during the iterative denoising process, a 0 That is, the action executed by the model, that is, the intermediate action and state vector obtained by the previous denoising step are used as the input of the denoising model, the mean of the action distribution predicted by the model is obtained, the next intermediate action is obtained by reparameterization, and the final action is obtained after T steps of denoising in the above manner; this iterative step of denoising in the intermediate action can fine-tune the consensus optimization model and improve the exploration ability of the consensus optimization model; The specific content of the consensus optimization model strategy update is: in order to train the inverse denoising process of the consensus optimization model so that it can learn the optimal strategy π(a|s), the SAC strategy update method is used to guide the training of the consensus optimization model; that is, two critic networks and two target-critic networks are set, the critic network evaluates the action value generated by the diffusion model, and avoids overestimation by taking the minimum value of the Q value predicted by the two critic networks, and the target-critic network is used to predict the target value; after each interaction between the agent and the environment, the experience (s, a, s', r) generated by the interaction is stored in the experience replay pool, where s is the current state, a is the action performed by the agent, s' is the new state after the action is performed, and r is the reward returned by the environment; each strategy update randomly takes N experience samples {(s i , a i , s′ i , r i )}, i∈{1,...,N}, where s i , a i , S′ i , r i Respectively represent the environment state in the i-th experience sample, the action performed by the model, the new state obtained after the action is performed, and the reward after the action is performed. The policy network loss is calculated according to the following formula, and the policy network is updated by gradient descent; In the above formula, Q(s i , a i ) is the value of the action in the i-th experience sample predicted by the critic network, logπ θ (a i |s i ) is the entropy of the strategy generated by the strategy network, α is the reward coefficient of entropy, which determines the importance of entropy, and θ is the strategy network parameter; The updating method of the critic network is consistent with the traditional SAC algorithm. The updating goal is to minimize the gap between its predicted value and the target value. The target-critic network is updated by soft-update. After training, the critic network can accurately predict the action value and guide the denoising model to learn the optimal strategy.

5. The method for optimizing the blockchain layered consensus for Internet of Vehicles according to claim 1 is characterized in that: The ordinary vehicles will collect road condition information through their own sensors during driving and provide necessary information for the formation; ordinary vehicles also have the need to communicate with vehicles in this shard and vehicles in other shards. When communicating with vehicles in this shard, it can be done directly through vehicle-to-vehicle communication (V2V), while communication with other shards needs to be done through the lead vehicle and RSU; at the same time, ordinary vehicles will also participate in the consensus process within the shard as blockchain nodes; The head vehicle in the shard is responsible for communicating with the RSU on behalf of the shard to reduce communication pressure, so it needs to have a vehicle with more computing resources. When information is generated in the shard, the head vehicle will be responsible for forming a block and executing the local consensus process in the shard. At the same time, the head vehicle is also responsible for transmitting information to the RSU, reaching a global consensus with other shards, and finally completing the block chain. Vehicles within a shard must communicate with other shards through RSU; RSU also acts as a blockchain node and executes the global consensus process with other RSUs, that is, after local consensus within the shard, global consensus needs to be achieved among all shards; since RSU is generally operated by trusted government agencies or operators and has more storage and computing resources, it also acts as a trusted edge storage node to store the complete blockchain.

Images