A reinforcement learning-based super-heuristic blockchain storage optimization method

Abstract

The application discloses a kind of hyper-heuristic blockchain storage optimization methods based on reinforcement learning, it is related to artificial intelligence, blockchain technology field, including vehicle save the cropped blockchain, periodically send its basic information to roadside unit, when the storage space of vehicle is insufficient, transaction clipping request is proposed to roadside unit;After receiving the transaction clipping request of vehicle, roadside unit considers the utility brought by clipping transaction under various factors comprehensively, optimizes clipping transaction strategy by hyper-heuristic algorithm based on reinforcement learning, decides clipping transaction scheme and sends to vehicle;Vehicle clips local saved transaction according to clipping transaction scheme.Therefore, by the above-mentioned one kind of hyper-heuristic blockchain storage optimization methods based on reinforcement learning, irrelevant or obsolete historical data is clipped locally by lightweight node scheme, so as to realize the storage optimization of blockchain, more suitable for Internet of Vehicles environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and blockchain technology, and particularly relates to a super-heuristic blockchain storage optimization method based on reinforcement learning. BACKGROUND

[0002] The scalability of the blockchain has become the biggest bottleneck of the implementation of the blockchain application, and the storage scalability problem has become a key problem restricting the landing of the blockchain application. In order to ensure the security of the blockchain, all nodes need to synchronize the complete blockchain data. With the running of the blockchain, the amount of data on the chain continues to grow, which causes great pressure on devices with limited storage resources such as vehicles. Therefore, in the vehicle networking environment with limited storage resources, the blockchain application is difficult to implement.

[0003] In order to solve the problem that the blockchain cannot be used due to insufficient storage resources of the node, at present, many researches have focused on the storage scalability of the blockchain. These researches mainly include two categories, off-chain storage and on-chain storage. Among them, the off-chain storage is to reduce the storage amount of the node by saving the actual data of the blockchain to other locations off the chain, and only saving the index address corresponding to the off-chain on the chain; the on-chain storage is that the blockchain data is still saved on the blockchain, and each node only stores part of the block data, and the storage space is reduced through the collaborative storage between nodes or the light node.

[0004] At present, based on the off-chain storage and collaborative storage method, the node needs to maintain continuous communication with the block storage node. However, the high dynamics of the vehicle networking environment makes it impossible to guarantee that the vehicle and the block storage node are always connected. In order to solve this problem, the present application introduces a light node scheme, which reduces the storage space by pruning the historical transactions of the blockchain. In the vehicle networking scene, the node only pays attention to the data of interest, and in the rapidly changing traffic scene, the historical data that is out of date has limited reference value for real-time decision-making. This light node scheme is more suitable for the vehicle networking environment, and the storage optimization can be realized by pruning irrelevant or outdated historical data locally. SUMMARY

[0005] The purpose of the present application is to provide a super-heuristic blockchain storage optimization method based on reinforcement learning, which can realize storage optimization through a light node scheme and is suitable for the vehicle networking environment; at the same time, a super-heuristic transaction pruning strategy based on reinforcement learning is adopted to maximize the system utility.

[0006] In order to achieve the above purpose, the present application provides a super-heuristic blockchain storage optimization method based on reinforcement learning, comprising the following steps:

[0007] S1, the vehicle stores the pruned blockchain, periodically sends its basic information to the roadside unit, and when the storage space of the vehicle is insufficient, requests the roadside unit for transaction pruning;

[0008] S2, after the roadside unit receives the transaction pruning request of the vehicle, the stored transaction information and node information are obtained, and a super-heuristic algorithm based on reinforcement learning is used to optimize the pruning transaction strategy, and the pruning transaction scheme is determined and sent to the vehicle;

[0009] In the process of optimizing the transaction pruning strategy, the system comprehensive utility is maximized as the optimization target, the system comprehensive utility includes the comprehensive utility of all nodes, and the comprehensive utility of each node considers the utility brought by the number of transaction copies, transaction generation time and transaction size factors when the node is pruned;

[0010] S3, the vehicle prunes the locally stored transactions according to the pruning transaction scheme.

[0011] In a possible implementation, in step S2, the node information includes N nodes stored in the vehicle, does not include the node itself, and the set is represented as The transaction set of each node node i i=1,2,…,N is represented as M i is the number of transactions of the stored node node i .

[0012] In a possible implementation, in step S2, the utility i generated in the number of transaction copies after the transaction Tr i,j of the node node is pruned is:

[0013]

[0014] wherein, represents the number of stored transactions Tr i,j on other vehicles, μ Cop controls the change rate of the utility ;

[0015] The utility generated in the transaction generation time after the transaction Tr i,j of the node node i is pruned is:

[0016]

[0017] wherein, represents the generation time of the transaction Tr i,j , and timestampCur denotes the current system time when the transaction is pruned, μ Tim control utility rate of change of the control utility

[0018] node node i transaction Tr i,j the utility brought by the transaction size after being pruned is:

[0019]

[0020] wherein, denotes the size of the transaction Tr i,j , μ Net control utility rate of change of the control utility denotes the probability of the vehicle VNode Pru meeting the node node i again.

[0021] In a possible implementation, in step S2, when the roadside unit decides to prune the transaction scheme, a super-heuristic algorithm based on reinforcement learning is used to determine the pruning strategy of each node, and the system comprehensive utility is maximized as the optimization objective to obtain the optimal pruning transaction strategy.

[0022] In a possible implementation, the expression of the system comprehensive utility Total U is as follows:

[0023]

[0024] wherein,

[0025] denotes the utility brought by the transaction in the node node i , and the expression is:

[0026]

[0027] denotes whether to prune the transaction Tr i in the node node i,j , and the expression is:

[0028]

[0029] denotes the comprehensive utility brought by the transaction Tr i,j , and the expression is:

[0030]

[0031] Wherein, a, b, g are weights corresponding to effects respectively.

[0032] In a possible implementation, the expression of the optimization target is:

[0033]

[0034] Wherein, is a mixed integer nonlinear programming problem of the optimization target; constraint C1 indicates that the processing of transactions includes two states, which are clipping or saving respectively; constraint C2 ensures that the clipping scheme meets the clipping demand of the node; constraints C3-C5 ensure the value range of the utility, i.e., the utility is negative.

[0035] In a possible implementation, in the reinforcement learning-based hyper-heuristic algorithm, the number of clipped transactions [Num1, Num2, …, Num N ] in each node is taken as a clipping transaction strategy, and initial solutions are generated for the low-level heuristic operators, and the evaluation function is used to evaluate the system comprehensive utility brought by each solution, and solutions that do not meet the clipping demand are punished.

[0036] The initial solutions include historical received solutions and randomly generated solutions, and a received solution set acceptSet i is designed for each low-level heuristic operator, i.e., i=1, 2, …, n, used to record received solutions obtained by using the low-level heuristic operator, wherein n is the number of low-level heuristic operators.

[0037] In a possible implementation, when the initial solutions are generated for the low-level heuristic operators, first, according to the proportion P his of historical received solutions in the initial solutions, M·P i initial solutions are selected from the received solution set acceptSet his , wherein the number M of initial solutions is the population size of the low-level heuristic operator, at this time, two cases are divided:

[0038] When the number of solutions in the set acceptSet i is less than M·P his , all received solutions in acceptSet i are selected, and the remaining solutions are generated in a random manner; otherwise, a roulette strategy is used to select M·P i solutions from the set acceptSet his .

[0039] In a possible implementation, the expression of the evaluation function is:

[0040]

[0041] wherein, represents a penalty factor for solutions that do not meet the clipping requirements, Size Pru represents the total size of the clipping transactions corresponding to the current solution, and the formula is:

[0042]

[0043] In a possible implementation, the reinforcement learning-based hyper-heuristic algorithm includes representing the optimization problem in a Markov decision process model, as follows:

[0044] The state space contains all system states at time slots t = 1, 2, 3, …, and the state S t is defined as:

[0045] S t = {s 1 ,s 2};

[0046] wherein, s 1 represents that the current optimal clipping scheme is improved after the action is performed, and s 2 represents that the current optimal clipping scheme is not improved after the action is performed;

[0047] At time slot t, the action A t adopted from the state S t+1 to the state S t is defined as:

[0048] A t = {a 1 ,a 2 ,a 3 ,a 4};

[0049] wherein, a 1 represents that the genetic algorithm is selected for optimization, a 2 represents that the differential optimization algorithm is selected for optimization, a 3 represents that the particle swarm algorithm is selected for optimization, and a 4 represents that the ant colony algorithm is selected for optimization;

[0050] The reward R t obtained from the current state S t performing the action A t+1 to the state S t+1 is defined as:

[0051]

[0052] When R t +1> 0, it means that there is an improvement to the current optimal pruning scheme after the action is performed, and the system jumps from the current state to state {s 1}, that is, S t+1 ={s 1} ; otherwise, it means that there is no improvement to the current optimal pruning scheme after the action is performed, and the system jumps from the current state to state {s 2}, that is, S t+1 ={s 2}.

[0053] Therefore, the application adopts the above-mentioned super-heuristic blockchain storage optimization method based on reinforcement learning, which has the following technical effects:

[0054] (1) The application adopts a lightweight node scheme, which can reduce the storage space by pruning irrelevant or outdated historical data in the blockchain historical transactions, thereby realizing the storage optimization of the blockchain.

[0055] (2) The application provides a super-heuristic algorithm optimization pruning transaction strategy based on reinforcement learning, the core idea of which is to design multiple different types of low-level heuristic operators in the low layer to realize diversified global search, and to dynamically select low-level heuristic operators in the high layer by using reinforcement learning technology, thereby realizing the universality of the algorithm.

[0056] The technical solutions of the application will be further described in detail below with the aid of the drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 is a pruning transaction flowchart in an embodiment of the super-heuristic blockchain storage optimization method based on reinforcement learning;

[0058] Figure 2 is a super-heuristic algorithm architecture based on reinforcement learning in an embodiment of the super-heuristic blockchain storage optimization method based on reinforcement learning. DETAILED DESCRIPTION

[0059] The application can be explained in more detail through the following examples, and the purpose of disclosing the application is to protect all changes and improvements within the scope of the application, and the application is not limited to the following examples.

[0060] Example 1

[0061] The application provides a super-heuristic blockchain storage optimization method based on reinforcement learning, which includes proposing a lightweight node scheme to realize storage optimization, as follows:

[0062] Please refer to Figure 1, the vehicle saves the pruned blockchain. The vehicle periodically sends its basic information to the roadside unit during its movement, including the node identification of the vehicle, the location, the moving direction, and the locally saved transaction information of other vehicles. In order to reduce the amount of data sent, the sent transaction storage information only includes the transactions that have changed compared with the last sending, and the vehicle only sends the hash value of the transaction.

[0063] The roadside unit saves the entire blockchain and records the basic information of all vehicles. At the same time, as an edge node, the roadside unit has relatively strong computing resources and is responsible for making pruning strategies for the pruning request proposed by the vehicle.

[0064] When the storage space of the vehicle VNode Pru is insufficient to store more transactions, a transaction pruning request needs to be initiated to the roadside unit, and the format of the transaction pruning request is shown in Table 1. represents the account address of the vehicle, respectively represent the moving direction, the location and the moving speed of the vehicle VNode Pru , represents the size of the pruning space applied by the vehicle.

[0065] Table 1: Format of transaction pruning request

[0066]

[0067] After receiving the transaction pruning request of the vehicle VNode Pru , the roadside unit first obtains the transaction information of the other N nodes (excluding the vehicle node itself) stored in the vehicle VNode Pru from the transaction storage information, and the set of the transaction information of the other N nodes is In the N nodes, the transaction set of each stored node node i (i = 1, 2,..., N) is M i is the number of transactions of the stored node node i .

[0068] Define the variable to represent whether to prune the transaction Tr i in the node node i,j . The specific assignment method of is as follows:

[0069]

[0070] After obtaining the vehicle VNode PruAfter the transactions stored in the middle are determined, the specific pruning transaction scheme is determined, that is, the final In the pruning process, the total size of the pruned transactions needs to be greater than the VNode Pru The space size of the application The roadside unit sends the determined pruning scheme to the node, and the node prunes the locally stored transactions according to the pruning scheme.

[0071] Because the utility brought by each factor is different when the transaction is pruned, it will affect the overall utility of the blockchain system. In order to reduce the impact of pruning transactions in the Internet of Vehicles, the embodiment comprehensively considers the three factors of transaction copy number, transaction generation time and transaction size when executing pruning transactions, as follows:

[0072] Assume that the transaction Tr i (i = 1, 2, …, N) in the node node i,j (j = 1, 2, …, M i ) is pruned, the utility brought in the transaction copy number is The utility brought in the transaction generation time is The utility brought in the transaction size is

[0073] A. Transaction copy number: for each transaction in the system, the more the transaction copy number, the higher the system security. Pruning transactions will inevitably reduce the number of transaction copies in the blockchain node, and thus affect the system security. For the overall utility of the system, it produces negative utility. After the transaction Tr i of the node node i,j is pruned, the utility brought in the transaction copy number is:

[0074]

[0075] Wherein, represents the storage number of the transaction Tr i,j on other vehicles, μ Cop controls the rate of change of utility .

[0076] B. Transaction Generation Time: In the Internet of Vehicles (IoV), vehicles typically pay more attention to transactions recently generated by other nodes. For any given transaction, the later its generation time, the higher the probability of it being acquired by other nodes. Therefore, the likelihood of a transaction being acquired is closely related to its timeliness, which necessitates considering the transaction generation time during the pruning process. Transaction pruning can lead to node failures in reading transactions, thus generating negative utility for the overall system. The later the transaction's generation time, the greater the probability of node failures due to transaction pruning, resulting in greater negative utility. Node i Transaction Tr i,j The utility gained in terms of transaction time after being cropped for:

[0077]

[0078] in, Indicates transaction Tr i,j The time of its generation, timestamp Cur μ represents the current system time at the time of the pruning transaction. Tim Control Utility The rate of change.

[0079] C. Transaction Size: Vehicles in the Internet of Vehicles (IoV) are always in motion. When two nodes meet again, they need to obtain transaction data from the other's account chain. If the required transaction has been pruned, resynchronization is necessary. Resynchronizing transactions through other vehicles or roadside units consumes additional network resources, thus pruning transactions has a negative effect. The resources required for resynchronizing transactions mainly depend on the probability of vehicles meeting again and the transaction size, as well as the node's... i Transaction Tr i,j The utility of being cropped in terms of transaction size for:

[0080]

[0081] in, Indicates transaction Tr i,j Size, μ Net Control Utility rate of change, Indicates vehicle VNode Pru With node i The probability of meeting again.

[0082] The overall utility of transaction trimming is determined by the different utilities it brings under each factor when the transaction is trimmed. Defined as:

[0083]

[0084] where, α, β, γ are the weights of each utility respectively.

[0085] Pruning node node i Utility brought by transaction in (i = 1, 2, …, N) The formula is:

[0086]

[0087] When the roadside unit decides the specific pruning transaction scheme, the utility brought by all nodes needs to be considered comprehensively, that is, the system comprehensive utility Total U , and the calculation formula is:

[0088]

[0089] In order to better realize the pruning of transactions and reduce the impact of pruning transactions on the blockchain system, while meeting the pruning needs of nodes, the system comprehensive utility Total U needs to be maximized. Therefore, the process of determining the final pruning scheme can be defined as a mixed integer nonlinear programming (MINLP) problem with the optimization goal of maximizing the system comprehensive utility Total U Specifically, it can be expressed as:

[0090]

[0091] Constraint C1 indicates that the processing of transactions includes two states, pruning or saving. Constraint C2 ensures that the final pruning scheme meets the pruning needs of nodes. Constraints C3-C5 ensure the value range of utility, and utility is negative.

[0092] Embodiment two

[0093] The super-heuristic blockchain storage optimization method based on reinforcement learning provided by the application further includes a super-heuristic pruning transaction strategy based on reinforcement learning. Please refer to Figure 2 The super-heuristic algorithm based on reinforcement learning adopted in this embodiment includes two parts of high layer and low layer.

[0094] In the low layer, a specific optimization problem is described, and a plurality of low-level heuristics (LLH) operators of different types are selected to realize diversified global search, which is specifically as follows:

[0095] A. Problem representation:

[0096] Assuming that the node​i The number of transactions to be cut is k, i.e., Num. i =k. Variable The possible values ​​are as follows:

[0097]

[0098] Set [Num1,Num2,…,Num] N This is a pruning trading strategy that needs to solve the problem of using a reinforcement learning-based hyperheuristic algorithm to obtain the total system utility. U The biggest cut-off trading strategy.

[0099] B. Initial solution:

[0100] To find the optimal pruning strategy more efficiently, the initial solution needs to reflect the diversity and quality of solutions. Therefore, the initial solution consists of two parts: one part comes from historical received solutions, and the other part is generated randomly. Assume that the proportion of solutions from historical reception to the initial solution is P. his The initial number of solutions, M, is equal to the population size of the low-level heuristic operators. To facilitate the acquisition of historically received solutions, a set of accepted solutions, acceptSet, is designed for each low-level heuristic operator. i (i = 1, 2, ..., n) is used to record the received solutions obtained by solving using low-level heuristic operators, where n is the number of low-level heuristic operators.

[0101] When generating an initial solution for a low-level heuristic operator, the first step is to start from the acceptSet of solutions. i Select M·P his An initial solution. If the set acceptSet... i Number of solutions Less than M·P his Then select acceptSet i All the received solutions, the rest A solution is generated randomly; otherwise, a roulette wheel strategy is used to select a solution from the acceptSet. i Select M·P his One solution.

[0102] C. Evaluation function:

[0103] For each solution [Num'1,Num'2,…,Num' N The variable can be determined by the number of pruned transactions at each node. The optimization objective is to maximize the total pruning utility; therefore, the evaluation function primarily assesses the overall system utility Total' brought by each solution. UIn evaluating each solution, Total' U is calculated, and a penalty is given to the solution that does not meet the cutting demand. The specific formula of the evaluation function is:

[0104]

[0105] wherein, is the penalty factor of the solution that does not meet the cutting demand, Size Pru represents the total size of the cutting transaction corresponding to the current solution, and the specific calculation formula is:

[0106]

[0107] D. Low-level heuristic operator:

[0108] Each operator in the LLH algorithm library represents a search heuristic strategy in the neighborhood space. Different operators have different search efficiencies in different problems. Meta-heuristic algorithms are inspired by natural phenomena and use random probability, survival of the fittest and other principles for optimization, resulting in strong timeliness and high search efficiency. Therefore, the operators in the LLH algorithm library in the embodiment use meta-heuristic algorithms, and four different types of meta-heuristic algorithms are selected: genetic algorithm (Genetic Algorithm, GA), differential evolution algorithm (Differential Evolution, DE), particle swarm optimization algorithm (Particle Swarm Optimization, PSO), and ant colony optimization algorithm (Ant Colony Optimization, ACO).

[0109] In the high layer, the low-level heuristic operator is dynamically selected through reinforcement learning technology to achieve the universality of the algorithm, and the solution obtained by the low-level heuristic operator is determined to be accepted as the current solution through the acceptance strategy, as follows:

[0110] A. Selection strategy:

[0111] The selection strategy is realized through reinforcement learning technology. The agent selects a low-level heuristic operator as an action by interacting with the environment according to the current observation state obtained. Then, the action value function is updated according to the reward obtained by executing the action, which provides a basis for the selection of the low-level heuristic operator. Through multiple interactions between the agent and the environment, the selection strategy of the low-level heuristic operator in this environment can be obtained.

[0112] The important mathematical model of reinforcement learning is Markov Decision Process (MDP). When solving optimization problems based on reinforcement learning, the problem is represented using the MDP model. The MDP model can use state space Action space and reward space The three elements are represented.

[0113] a. State Space: The state space needs to contain all system states in time slot t = (1, 2, 3, ...). After selecting low-level heuristic operators for optimization, a better solution than the current optimal pruning scheme may be found, or it may not. Therefore, the state space... It needs to reflect whether the executed action improves the current optimal pruning scheme. Specifically, this should be reflected in the state of time slot t. Defined as:

[0114] S t ={s 1 ,s 2};

[0115] Among them, s 1 This indicates that the action improves the current optimal clipping solution. 2 This indicates that the action performed did not improve the current optimal trimming solution.

[0116] b. Action Space: For each state S t The agent performs actions by interacting with its environment. The state can change by selecting different low-level heuristic operators. Therefore, in time slot t, from state S... t to state S t+1 Possible actions Defined as:

[0117] A t ={a 1 ,a 2 ,a 3 ,a 4};

[0118] Where a 1 This indicates that a genetic algorithm is chosen for optimization. 2 This indicates that the differential optimization algorithm is selected for optimization, a 3 This indicates that the particle swarm optimization algorithm is chosen for optimization, a 4 This indicates that the ant colony algorithm has been selected for optimization.

[0119] c. Reward Space: The MDP model starts from the current state S t Perform action A t Jump to state S t+1 When, the reward R is obtained t+1 The optimization objective of the pruning scheme is to maximize the overall system utility, therefore, the reward R will be... t+1 Defined as action A t The resulting overall system utility The overall system utility of the current optimal trimming scheme The size of the improvement. From the current state S t Perform action A t Jump to state S t+1 Rewards received Defined as:

[0120]

[0121] If R t+1 >0 indicates that the action improves the current optimal trimming solution, and the system transitions from the current state to the state {s}. 1}, that is, S t+1 ={s 1 Otherwise, it means that performing the action does not improve the current optimal pruning solution, and the system jumps from the current state to state {s}. 2}, that is, S t+1 ={s 2}

[0122] Q-learning is a classic and widely used value-based algorithm in reinforcement learning. It uses a Q-table to record the action value of each action in each state. This algorithm is mainly suitable for discrete state and action spaces. Since the states and actions in this optimization problem are discrete and their number is small, Q-learning is chosen for updating action values ​​in this embodiment.

[0123] When updating action values ​​using the Q-learning algorithm, an improved strategy is required. The specific update method is as follows:

[0124]

[0125] Where α represents the learning rate and γ represents the discount factor. q(S) represents the estimated return. t A t ) represents the actual value.

[0126] B. Mobile reception strategy:

[0127] The state space includes states that improve upon the optimal solution and states that do not. When receiving solutions, it is necessary to receive both improved and unimproved solutions—that is, all solutions. Therefore, the mobile receiving strategy adopts the total receiving strategy from the deterministic receiving strategy.

[0128] Based on the above, this embodiment provides a pruning trading strategy optimization process based on a Q-learning hyperheuristic algorithm, specifically including 6 steps:

[0129] (1) Initialization: This process mainly initializes the low-level heuristic operator hyper-parameters, Q-learning hyper-parameters (a, g, e), Q-table, initial state and the maximum system overall utility

[0130] (2) Action selection: In order to prevent the obtained cutting scheme from being a local optimal scheme, the agent should explore more action space as much as possible. Therefore, in each time slot, the action A is selected by using the e-greedy strategy t If the random number is less than e, a random action A is selected from the action space t Otherwise, the action with the maximum action value is selected according to the Q table Then, the initial solution (i.e., the initial population) of the LLH operator is initialized according to the selected action A t .

[0131] (3) Action execution: In this process, the action A is executed t , and the cutting utility obtained by the corresponding optimization operator is observed Then, the reward R t+1 and the next state S t+1 are calculated according to and .

[0132] (4) Action value update: After each sampling, first, the estimated value U of the return is calculated according to the Q-learning mechanism, and then q(S t , A t ) is updated to reduce [U-q(S t , A t )] 2 .

[0133] (5) Update the received solution set: The optimal cutting strategy and cutting utility obtained by executing the action A t corresponding to the optimization operator are added to the corresponding received solution set acceptSet i (i = 1, 2, 3, 4).

[0134] (6) Update the maximum system overall utility If the current maximum system overall utility is less than the optimal cutting utility observed by executing the action A t , it means that the obtained cutting trading strategy can make the system overall utility greater. At this time, the maximum system overall utility is updated, and the current maximum system overall utility is replaced by

[0135] By the above-mentioned super heuristic algorithm based on Q learning, a feasible solution that maximizes the system comprehensive utility can be obtained. In the solution, the number of pruning transactions of each node is the optimal pruning transaction strategy.

[0136] Therefore, the application adopts the above-mentioned super heuristic blockchain storage optimization method based on reinforcement learning, can realize storage optimization by using the pruning transaction strategy of the super heuristic algorithm based on Q learning through the lightweight node scheme to prune irrelevant or obsolete historical data.

[0137] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, but not to limit them. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can still be modified or replaced by equivalents, and these modifications or equivalent replacements should not make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present application.

Claims

1. A reinforcement learning based hyper-heuristic blockchain storage optimization method, characterized in that, The method comprises the following steps: S1, the vehicle saves the pruned blockchain, periodically sends its basic information to the roadside unit, and proposes a transaction pruning request to the roadside unit when the storage space of the vehicle is insufficient; S2, after the roadside unit receives the transaction pruning request of the vehicle, the stored transaction information and node information are obtained, and a pruning transaction strategy is optimized through a super-heuristic algorithm based on reinforcement learning, a pruning transaction scheme is determined and sent to the vehicle; The super-heuristic algorithm based on reinforcement learning includes two parts of high layer and low layer, in the low layer, a specific optimization problem is described, and a plurality of low-level heuristics (LLH) operators of different types are selected to realize diversified global search, in the high layer, the low-level heuristic operators are dynamically selected through reinforcement learning technology to realize the universality of the algorithm, and the solution obtained by the low-level heuristic operator is determined whether to be accepted as the current solution through the acceptance strategy; In the process of optimizing the transaction pruning strategy, the system comprehensive utility is maximized as the optimization target, the system comprehensive utility includes the comprehensive utility of all nodes, and the comprehensive utility of each node includes the utility brought by the number of transaction copies, the transaction generation time and the transaction size factor when the node is pruned; S3, the vehicle prunes the locally saved transactions according to the pruning transaction scheme.

2. The method of claim 1, wherein, In step S2, the node information includes data stored in the vehicle. The set of nodes, excluding vehicle nodes themselves, is represented as follows: Each node The transaction set is represented as , It is a storage node The number of transactions. 3.The method of claim 1, wherein, In step S2, the node trims the transaction , after which the resulting utility in terms of the number of transaction replicas is : ； wherein, representing a transaction a stored quantity on other vehicles, control utility a rate of change; Nodes of transactions The utility generated in terms of transaction origination time after clipping is: ； wherein, represents the transaction time of generation, represents the current system time at the time of clipping the transaction, controls the rate of change of the utility . Nodes of transactions The utility in terms of transaction size after clipping is: ； wherein, representing a transaction size, controlling the rate of change of utility , a change rate of utility representing a probability of a vehicle meeting a node again.

4. The method of claim 1, wherein, In step S2, when the roadside unit determines the pruning transaction scheme, the pruning strategy of each node is determined through the super-heuristic algorithm based on reinforcement learning, and the system comprehensive utility is maximized as the optimization target to obtain the optimal pruning transaction strategy.

5. The method of claim 2, wherein, Systemic utility the expression of the system, as follows: ； Wherein, represents the utility from a transaction in the cutting node The expression for the utility from a transaction in the cutting node is: ， ， ； represents whether to trim the node transaction in the node , expressed as: ； represents the cut transaction The overall utility resulting from the cut transaction is expressed as: ； wherein, , , are the weights corresponding to the effects, respectively.

6. The method of claim 5, wherein, The expression of the optimization target is: ， ； wherein, is a mixed integer nonlinear programming problem with the optimization objective; constraints indicates that the processing of the transaction includes two states, respectively, clipping or saving; constraints ensure that the clipping scheme meets the clipping needs of the node; constraints - ensure the value range of the utility, that is, the utility , , is negative.

7. The method of claim 1, wherein, In the reinforcement learning based hyper-heuristic algorithm, the low level generates a number of candidate solutions by pruning the transactions in each node As a pruning strategy, the low level generates a number of candidate solutions, and the initial solutions are evaluated by the evaluation function. The solutions that do not meet the pruning requirement are penalized. The initial solution includes historical received solutions and is randomly generated, and a received solution set is designed for each low-level heuristic operator , , and a received solution set is designed for each low-level heuristic operator, and a received solution set is designed for each low-level heuristic operator is the number of low-level heuristic operators.

8. The reinforcement learning based hyper-heuristic blockchain storage optimization method of claim 7, wherein, When generating initial solutions for low-level heuristic operators, first, according to the proportion of historical received solutions in initial solutions , select initial solutions from the received solution set , where the number of initial solutions is the population size of the low-level heuristic operator, at this time, there are two cases: When the set The number of solutions Is less than Then all the received solutions are selected The rest of the Solutions are generated by random Otherwise, select one solution from the set using a roulette wheel strategy solution.​ 9. The method of claim 7, wherein, The expression of the evaluation function is: ； In the formula, represents a penalty factor for the solution that does not meet the clipping requirement, represents the total size of the clipping transaction corresponding to the current solution, and the formula is: 。 10. The method of claim 1, wherein, The super-heuristic algorithm based on reinforcement learning includes that the optimization problem is represented by using a Markov decision process model, as follows: In the time slot of the state is defined as: ； wherein represents that there is an improvement to the currently optimal cropping scheme after performing the action, represents that there is no improvement to the currently optimal cropping scheme after performing the action; In the time slot From state To state The action taken Is defined as: ； wherein denotes that a selection genetic algorithm is optimized, denotes that a selection differential optimization algorithm is optimized, denotes that a selection particle swarm algorithm is optimized, denotes that a selection ant colony algorithm is optimized; from the current state performing an action jumping to a state the resulting reward is defined as: ； rewards defined as performing action resulting system aggregate utility system aggregate utility of the current optimal pruning scheme improvement size, when , indicates that performing action results in an improvement over the current optimal pruning scheme, and the system jumps from the current state to state , i.e. ; Otherwise, it means that there is no improvement on the current optimal pruning scheme after performing the action, and the system jumps from the current state to state , i.e. .

Images