A blockchain-based spectrum allocation method for drone-assisted Internet of Vehicles

By introducing blockchain technology and deep reinforcement learning into the Internet of Vehicles system, a decentralized UAV auxiliary network is solved, and the problems of low spectrum allocation efficiency and security threats in the Internet of Vehicles are achieved, and efficient and reliable spectrum resource management is achieved. Reduces maintenance costs and delays.

CN114173419BActive Publication Date: 2024-11-19GUANGDONG UNIV OF TECH +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111334196.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-11
Publication Date
2024-11-19
Estimated Expiration
2041-11-11

AI Technical Summary

Technical Problem

There are problems in the Internet of Vehicles system with networking systems such as network congestion, high maintenance costs, low spectrum allocation efficiency, complex interference management and serious security threats. The centralized spectrum management model is difficult to meet the needs of massive radio equipment.

Method used

Adopt the blockchain-based drone-assisted vehicle spectrum allocation method, and build a UAV-assisted vehicle network, use deep reinforcement learning and Ripple consensus mechanism to optimize spectrum and power allocation, and establish a decentralized resource management model. Improve the utilization and reliability of spectrum resources.

Benefits of technology

It improves the resource utilization and reliability of the Internet of Vehicles system, reduces the system maintenance cost and delay, enhances the efficiency and security of spectrum allocation, and avoids network congestion and interference management complexity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114173419B_ABST
    Figure CN114173419B_ABST
Patent Text Reader

Abstract

The present invention discloses a spectrum allocation method for UAV-assisted Internet of Vehicles based on blockchain, constructs a UAV-assisted Internet of Vehicles blockchain resource management model, configures a unique node list (UNL) for each UAV node; the vehicle submits a spectrum request to the connected UAV master node, the master node sets an optimization problem and uses a deep reinforcement learning method to solve the communication spectrum and transmission power between the vehicle and the UAV; the master node broadcasts the request transaction to the UAV node, the UAV node uses the Ripple consensus mechanism to execute the smart contract and consensus protocol on the request transaction, and the UAV node votes on the transaction to confirm whether the request transaction is passed; the request transaction that has not passed is discarded, and all vehicle information, transaction records and timestamps are packaged into a new block for the request transaction that has passed, and attached to the blockchain, broadcast to all UAV nodes, thereby confirming the spectrum and the transmission power obtained by the vehicle requesting the spectrum. The present invention manages wireless resources with a distributed structure, reduces congestion, and improves resource utilization and reliability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a spectrum allocation method, in particular to a blockchain-based drone-assisted Internet of Vehicles spectrum allocation method, which belongs to the technical field of Internet of Vehicles and intelligent driving. Background Art

[0002] The Internet of Vehicles is the product of the integration of the Internet and the Internet of Things, providing convenient and diverse services for intelligent transportation. At present, the two major camps of Internet of Vehicles technology are the General Short Range Communication (DSRC) led by the United States and the Long Term Evolution of Vehicle-to-Vehicle Communication (LTE-V) system promoted by domestic enterprises. With the rapid development of industrial Internet of Things technology in vehicle networks, data exchange between vehicles, vehicles and pedestrians, and vehicles and infrastructure units is becoming more and more frequent. The vehicle network faces huge challenges with high reliability and real-time requirements. Neither DSRC nor LTE-V communication can meet the needs of the Internet of Vehicles at this stage. In addition, the current Internet of Vehicles system is mainly a heterogeneous network based on enodeB and roadside facility units (RSU). RSU, as a vehicle wireless access point, uploads information such as vehicles and roads to the Internet and publishes it. This collaborative communication model between vehicles and infrastructure requires a large number of RSUs to support, which increases construction costs and energy consumption, and also causes serious interference between different radio devices.

[0003] In recent years, UAVs have attracted widespread attention due to their high mobility and easy deployment in wireless communication systems. The typical feature of UAVs is that they can achieve long-distance line-of-sight connections and provide wireless connections to users outside the base station coverage area at any time. They can be used to offload business flows and reduce interference with ground communication systems. Applying UAVs to intelligent transportation systems as aerial base stations to assist ground communication systems in the Internet of Vehicles can improve the capacity of the Internet of Vehicles system and the quality of vehicle-mounted applications.

[0004] In the Internet of Vehicles, spectrum management becomes more complicated because a large number of vehicles are distributed in a limited area. Current research on the Internet of Vehicles mainly relies on a centralized model. In a pyramid-like organizational structure, decision-making power is basically controlled by the highest-level central management center. Due to too many levels of management, a large amount of communication and coordination costs will be consumed within the organization, which makes the management of vehicle network systems lack flexibility. Specifically, the centralized management model has the following disadvantages:

[0005] 1) Severe network congestion. When the vehicle is crowded or a large number of devices are connected to the vehicle network, the centralized network structure is likely to cause network congestion in the center, and the central cloud server can easily reach a bottleneck or even be paralyzed, or the entire network can be destroyed due to a single point of failure. On the other hand, in remote areas where base station equipment cannot fully radiate, the communication system cannot fully exert its performance.

[0006] 2) High maintenance costs and low spectrum allocation efficiency. With the rapid development of technologies such as the Internet of Things, the Internet of Vehicles, and artificial intelligence, the number of radio devices will grow exponentially, and the data processing pressure of centralized spectrum management systems will increase sharply. In order to meet the spectrum requirements of a large number of different user devices, centralized spectrum management systems need to continuously upgrade equipment to ensure the performance of their users. This approach will incur huge maintenance costs. In addition, the Internet of Vehicles has extremely high requirements for end-to-end latency, which needs to be maintained within 1ms. This requires a centralized spectrum management system to improve its spectrum allocation efficiency. However, the surge in the number of radios makes it increasingly difficult to ensure spectrum allocation efficiency. Therefore, a more economical and efficient spectrum allocation method is needed.

[0007] 3) Increasingly complex interference management. Currently, wireless communications are widely used in various communication networks such as indoor access, short-distance communications, satellite communications, mobile communications, and sensor networks. With the surge in radio equipment and the increasing variety of business types, wireless bandwidth is unable to cope with the surge in mobile data traffic demand. To this end, operators generally increase the spectrum reuse rate by adding infrastructure (such as base stations, access points, etc.). This heterogeneous and layered network structure will lead to serious interference between different radio devices. Interference management in scenarios where massive radio devices coexist and change dynamically will become increasingly complex. The centralized spectrum management system will generate huge system overhead, and users' performance requirements will become increasingly difficult to guarantee.

[0008] 4) Increasingly serious security threats. In the future, B5G or 6G networks will accommodate tens of billions of radio devices to achieve ubiquitous intelligence. At the same time, the number of untrusted nodes (such as malicious nodes) will also increase. Illegal radio stations such as fake base stations, black broadcasts, eavesdropping devices, and radio cheating devices will seriously disrupt public radio order, pose a huge security risk to user privacy and data security, and also create obstacles for more accurate interference management. At the same time, malicious nodes' attacks on centralized databases and central agencies' malicious tampering and leakage of data will pose a great threat to the security of user data.

[0009] In recent years, the emergence of blockchain has provided a new direction for improving the security of Internet of Vehicles communications. Blockchain is a new decentralized infrastructure and distributed computing paradigm, which has the advantages of decentralization, immutability, traceability and openness compared to the centralized model. Blockchain technology can achieve peer-to-peer transactions, coordination and collaboration in distributed systems through means such as data encryption, timestamps and distributed consensus, thereby solving the problems of high cost, low efficiency and insecure data storage in the centralized model. Using blockchain to manage wireless spectrum resources can effectively improve the allocation efficiency of spectrum resources, the reliability and security of the system, and reduce the maintenance cost of the system. Summary of the invention

[0010] The technical problem to be solved by the present invention is to provide a blockchain-based drone-assisted Internet of Vehicles spectrum allocation method to improve the resource utilization and reliability of the Internet of Vehicles.

[0011] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0012] A blockchain-based drone-assisted vehicle network spectrum allocation method, characterized by comprising the following steps: Figure 1 As shown:

[0013] Step 1: Build a UAV-assisted IoV network, obtain the number of UAVs, the number of vehicles, the number of sub-channels, the interruption probability and the delay information in the IoV network, and build a blockchain resource management model based on the obtained information. Configure a trusted node list for each UAV node, called the unique node list (UNL);

[0014] Step 2: The UAV node connected to the vehicle is the master node. The vehicle submits a spectrum request to the master node. The master node sets the optimization problem and uses deep reinforcement learning to solve it, and obtains the communication spectrum and transmission power between the vehicle and the UAV.

[0015] Step 3: The master node broadcasts the request transaction to the UAV nodes on the unique node list UNL. After receiving the request, the UAV node uses the Ripple consensus mechanism to execute the smart contract and consensus protocol for the request transaction. The UAV nodes in the unique node list UNL vote on the transaction to confirm whether the request transaction is passed;

[0016] Step 4: Discard the failed request transactions. For the passed request transactions, pack all vehicle information, transaction records and timestamps into a new block, attach it to the blockchain, and broadcast it to all UAV nodes.

[0017] Furthermore, the UAV-assisted Internet of Vehicles network is composed of multiple UAVs and vehicles, the number of UAVs is U, the number of vehicles is I, each UAV flies in a set area and acts as an aerial base station to assist in communication between vehicle source nodes and target nodes. The UAV-assisted communication link provides vehicles with high reliability and low latency communication, the system spectrum is shared among all UAVs, and the number of system channels is N.

[0018] Furthermore, the blockchain resource management model is specifically as follows:

[0019] For the downlink transmission system of the UAV to vehicle information link, it is assumed that the information transmission is periodic data packet transmission; considering that when the UAV is communicating with the vehicle, other UAVs will interfere with this communication link, and the rate R between UAV u and the vehicle in channel n is u,n It can be expressed as

[0020]

[0021] Among them, p u,n represents the transmission power of UAV u in channel n, is the channel gain of UAV u on channel n to the connected vehicle, is the channel gain of UAV u' on channel n to other UAV-connected vehicles, and Obtained by the statistical average of the channel gains of all vehicles, σ 2 is the Gaussian white noise power.

[0022] When the signal-to-noise ratio of the communication link between the UAV and the vehicle is lower than γ0, the communication transmission from the UAV to the vehicle will be interrupted; in any time period, the probability that the data packet is not successfully sent is given by the interruption probability P out,u It means that the probability of signal-to-noise ratio being less than γ0; the interruption probability expression is

[0023]

[0024] Among them, a u',n represents the allocation variable of UAV u' in channel n.

[0025] Assumption Y u represents the average transmission time of the data packet of UAV u. When the data packet fails to be successfully transmitted in the current time slot, it is retransmitted. If the number of retransmissions does not reach N, it will continue to be retransmitted in the next time slot. If the number of retransmissions has reached N, it will be discarded. Assuming that the length of each time slot is T, Y u The probability function is expressed as

[0026]

[0027] Using D u Represents the average sending delay, expressed as

[0028]

[0029] Where E{·} represents the mean of the random variable.

[0030] Furthermore, the step 2 is specifically as follows:

[0031] After the vehicle is connected to the Internet of Vehicles, it requests the UAV for the spectrum to communicate with it. After receiving the spectrum request from the vehicle, the UAV sets an optimization problem and then solves the optimization problem to obtain the requested channel and power.

[0032] With the goal of optimizing energy efficiency, the optimization problem is set as:

[0033]

[0034]

[0035]

[0036]

[0037]

[0038]

[0039]

[0040] where a u,n represents the allocation variable of UAV u in channel n, the constraint C1 indicates that the communication rate of the vehicle connected to UAV u is higher than the minimum requirement, C2 and C3 represent the maximum limits of the interruption probability and the average delay, respectively, C4 represents the channel allocation variable of UAV, C5 represents the number of channels requested each time is 1, and C6 represents the maximum transmission power limit of UAV u;

[0041] In order to solve the above optimization problem, the deep Q learning (DQL) method is used to obtain the optimization variable values; the UAV acts as a learning agent to perceive the state s(t) at time t and outputs a strategy π, which determines the action a(t) to be taken and then executes this action; in order to allow the learning agent to accumulate experience and better choose actions, the environment will give the learning agent a reward r(t) as feedback; at this time, it enters the next state s(t+1), the learning agent continues to perceive s(t+1), and makes a new strategy π, and so on, eventually making the environment state converge and the agent obtains the maximum reward.

[0042] Furthermore, the steps of solving the optimization problem using the Deep Q-Learning method by the UAV node are as follows:

[0043] 1) Define the joint optimization problem as a Markov decision process, defining the state space, action space and reward function;

[0044] The state S(t) contains the channel F(t) requested by the UAV at the current time t and the power P(t) transmitted by the UAV; therefore, the state space is expressed as

[0045]

[0046] F(t) and P(t) are expressed as:

[0047]

[0048]

[0049] f n (t) indicates that channel n is requested for the current UAV at time t, p l (t) represents the size of the UAV transmission power requested at time t;

[0050] According to the current state S(t), the UAV takes action A(t) to change the current environment; the action space is represented by the following formula

[0051]

[0052] According to the state space and action space, the energy efficiency of UAV is modeled as a reward function, expressed as

[0053]

[0054] 2) Define Q(s,a,θ) as the action value function, which represents the expected total reward of action a in state s. The expression is

[0055] Q(s,a,θ)=E[R t |s t =s,a t =a,θ]

[0056] Where θ is the weight and bias set in the evaluation network;

[0057] Define the experience replay set D, {s t ,a t ,r t ,s t+1} represents the set of state changes and corresponding actions and rewards;

[0058] To simplify the description, we use Q network to represent the neural network in the DQL method, and use Q value to represent the value of the function Q(s, a, θ). We randomly initialize the network parameters θ, initialize the Q values ​​of all states and actions based on θ, clear the experience replay set D, and initialize the state s. t ;

[0059] 3) Use the state s in the function Q(s,a,θ) t As input, get the Q value output corresponding to all actions of the Q network, and select the action corresponding to the maximum Q value When Q(s,a,θ) takes the maximum value, it is recorded as

[0060]

[0061] 4) In state s t Next, perform action a t , get reward r t , get the new state s t+1 , and {st ,a t ,r t ,s t+1}Store in set D;

[0062] 5) Randomly select m samples from set D {{s j ,a j ,r j ,s j+1}}, j = 1, 2, ..., m, calculate the current target Q value y j :

[0063]

[0064] It means that the UAV obtains the maximum long-term reward and terminates the learning;

[0065] 6) Define the loss function L(θ) to represent the difference between the output Q value of the target network and the evaluation network, as shown in the formula

[0066]

[0067] Among them, θ - are the weights and biases set in the target network, and θ are the weights and biases set in the evaluation network;

[0068] The parameters θ at time t and time t+1 are represented by θ(t) and θ(t+1) respectively; the gradient of the neural network parameters is calculated according to L(θ), and then the parameters θ(t) are updated, and then the neural network module is updated using the parameters θ(t+1), which is expressed as follows:

[0069]

[0070] 7) Set the state s t Updated to t+1 ;

[0071] Repeat steps 2 to 7 until y j It is in the convergence state, that is, the requested channel and power values ​​are obtained.

[0072] Furthermore, the process of reaching a consensus among nodes after the master node requests the channel and power is as follows:

[0073] 1) The master node request channel and power resources are defined as request transactions, and the master node broadcasts the request transactions to the UAV nodes in the UNL;

[0074] 2) The nodes in the UNL are compared with their local candidate set. The local candidate set is the spectrum resource information set that the UAV retains in its local area. The resource information is updated in real time, and only idle spectrum is in this candidate set. If the requested channel is idle in the local candidate set, the transaction will get a vote; if the requested spectrum is not in the local candidate set, the node determines whether to allow the requesting node to use this spectrum based on whether it will cause harmful interference to itself. The expression is:

[0075]

[0076] If the above formula is satisfied, the requesting node is allowed to use, that is, it gets a vote, otherwise it will not get a vote;

[0077] 3) The nodes in UNL broadcast the voting results back to the master node through the network, and the master node counts the votes of the transaction;

[0078] 4) When the requested transaction obtains more than 50% of the votes, the transaction is passed and the voting stops; if all nodes of UNL have completed voting on the transaction or more than 50% of the node votes are not obtained within a certain time limit, the transaction is discarded and the master node resubmits other spectrum and power requests.

[0079] Furthermore, the step 4 is specifically as follows:

[0080] The transaction information, timestamp and public key of the accounting node that pass the vote are packaged into a new block;

[0081] The block body records specific data, and voting transaction information is stored in newly generated records, including vehicle information, allocated channels, and power. The block data structure is a Merkle tree, and data is recorded in leaf nodes. The values ​​of non-leaf nodes are hash values ​​of all leaf node data. Every two adjacent leaves are hashed together to become the content of the upper node. The hash value of the top-level Merkle root node is continuously calculated to quickly summarize and verify the existence and integrity of block data.

[0082] The block is broadcast to all nodes in the network, and each node adds the received new block to the blockchain; each node updates the local candidate set and removes the newly used channel from the local candidate set.

[0083] Compared with the prior art, the present invention has the following advantages and effects:

[0084] 1. The present invention improves the rate of the Internet of Vehicles by constructing a UAV-assisted Internet of Vehicles system and deriving the communication transmission rate, interruption probability and automatic retransmission delay;

[0085] 2. The present invention adopts the Ripple consensus mechanism for spectrum allocation and power control. The data is backed up at each node, and the UAV nodes jointly maintain the blockchain. If a malicious node wants to tamper with the spectrum data stored in the block, it can only be achieved by combining at least 51% of the nodes to modify the data at the same time, which improves the reliability of the system;

[0086] 3. The present invention calculates the maximum rate and efficiency of spectrum resource allocation by solving optimization problems and deep reinforcement learning, thereby reducing the system latency. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 It is a flow chart of the master node of the present invention requesting a channel and power and then reaching a consensus among nodes.

[0088] Figure 2 This is a network structure diagram of a blockchain-based drone-assisted Internet of Vehicles of the present invention.

[0089] Figure 3 It is a learning agent and environment interaction diagram of a blockchain-based drone-assisted vehicle network spectrum allocation method of the present invention.

[0090] Figure 4 It is a block structure schematic diagram of the present invention. DETAILED DESCRIPTION

[0091] In order to elaborate on the technical scheme adopted by the present invention to achieve the predetermined technical purpose, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only partial embodiments of the present invention, rather than all embodiments, and the technical means or technical features in the embodiments of the present invention can be replaced without paying creative work. The present invention will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0092] The present invention provides a blockchain-based drone-assisted vehicle networking spectrum allocation method, which is characterized by comprising the following steps: Figure 1 As shown:

[0093] Step 1: Build a UAV-assisted IoV network, obtain the number of UAVs, the number of vehicles, the number of sub-channels, the interruption probability and the delay information in the IoV network, and build a blockchain resource management model through the obtained information. Configure a trusted node list for each UAV node, called the unique node list (UNL).

[0094] like Figure 2As shown in the figure, the UAV-assisted Internet of Vehicles network consists of multiple UAVs and vehicles, the number of UAVs is U, the number of vehicles is I, each UAV flies in a set area and acts as an aerial base station to assist the communication between the vehicle source node and the target node. The UAV-assisted communication link provides high reliability and low latency communication for the vehicle. The system spectrum is shared among all UAVs, and the number of system channels is N.

[0095] The specific blockchain resource management model is:

[0096] For the downlink transmission system of the UAV to vehicle information link, it is assumed that the information transmission is periodic data packet transmission; considering that when the UAV is communicating with the vehicle, other UAVs will interfere with this communication link, and the rate R between UAV u and the vehicle in channel n is u,n It can be expressed as

[0097]

[0098] Among them, p u,n represents the transmission power of UAV u in channel n, is the channel gain of UAV u on channel n to the connected vehicle, is the channel gain of UAV u' on channel n to other UAV-connected vehicles, and Obtained by the statistical average of the channel gains of all vehicles, σ 2 is the Gaussian white noise power.

[0099] When the signal-to-noise ratio of the communication link between the UAV and the vehicle is lower than γ0, the communication transmission from the UAV to the vehicle will be interrupted; in any time period, the probability that the data packet is not successfully sent is given by the interruption probability P out,u It means that the probability of signal-to-noise ratio being less than γ0; the interruption probability expression is

[0100]

[0101] Among them, a u',n represents the allocation variable of UAV u' in channel n.

[0102] Assumption Y u represents the average transmission time of the data packet of UAV u. When the data packet fails to be successfully transmitted in the current time slot, it is retransmitted. If the number of retransmissions does not reach N, it will continue to be retransmitted in the next time slot. If the number of retransmissions has reached N, it will be discarded. Assuming that the length of each time slot is T, Y u The probability function is expressed as

[0103]

[0104] Using Du Represents the average sending delay, expressed as

[0105]

[0106] Where E{·} represents the mean of the random variable.

[0107] Based on the advantages of network characteristics and distributed management, the present invention uses blockchain technology to manage wireless resources. After the vehicle accesses the network, it applies to the UAV for spectrum communication with the UAV. The blockchain system formed between the UAVs determines the channel allocated to the UAV that requests the spectrum. A list of trusted nodes is configured for each UAV node, called the unique node list (UNL). The UAV nodes in the list can vote on transactions.

[0108] Step 2: The UAV node to which the vehicle is connected is the master node. The vehicle submits a spectrum request to the master node. The master node sets the optimization problem and uses deep reinforcement learning method to solve it to obtain the communication spectrum and transmission power between the vehicle and the UAV.

[0109] After the vehicle is connected to the Internet of Vehicles, it requests the UAV for the spectrum to communicate with it. After receiving the spectrum request from the vehicle, the UAV sets an optimization problem and then solves the optimization problem to obtain the requested channel and power.

[0110] With the goal of optimizing energy efficiency, the optimization problem is set as:

[0111]

[0112]

[0113]

[0114]

[0115]

[0116]

[0117]

[0118] where a u,n represents the allocation variable of UAV u in channel n, the constraint C1 indicates that the communication rate of the vehicle connected to UAV u is higher than the minimum requirement, C2 and C3 represent the maximum limits of the interruption probability and the average delay, respectively, C4 represents the channel allocation variable of UAV, C5 represents the number of channels requested each time is 1, and C6 represents the maximum transmission power limit of UAV u;

[0119] In order to solve the above optimization problem, the Deep Q-Learning (DQL) method is used to obtain the optimization variable value. The DQL method contains a deep neural network model to evaluate the action value and set the target value, called the evaluation network and the target network, and then make dynamic network updates and decisions. In the DQL method, the agent interacts with the environment and learns from experience, and finally obtains and executes an intelligent strategy based on the value function. The DQL method not only provides the best strategy to achieve the proposed optimization goal, but also obtains the maximum long-term reward.

[0120] like Figure 3 As shown, in the present invention, UAV acts as a learning agent to perceive state s(t) at time t and outputs strategy π, which determines the action a(t) to be taken and then executes this action; in order to allow the learning agent to accumulate experience and thus better select actions, the environment will reward the learning agent r(t) as feedback; at this time, it enters the next state s(t+1), the learning agent continues to perceive s(t+1), and makes a new strategy π, and so on, until the environmental state converges and the agent obtains the maximum reward.

[0121] The steps for the UAV node to solve the optimization problem using the Deep Q-Learning method are as follows:

[0122] 1) Define the joint optimization problem as a Markov decision process, defining the state space, action space and reward function;

[0123] The state S(t) contains the channel F(t) requested by the UAV at the current time t and the power P(t) transmitted by the UAV; therefore, the state space is expressed as

[0124]

[0125] F(t) and P(t) are expressed as:

[0126]

[0127]

[0128] f n (t) indicates that channel n is requested for the current UAV at time t, p l (t) represents the size of the UAV transmission power requested at time t;

[0129] According to the current state S(t), the UAV takes action A(t) to change the current environment; the action space is represented by the following formula

[0130]

[0131] According to the state space and action space, the energy efficiency of UAV is modeled as a reward function, expressed as

[0132]

[0133] 2) Define Q(s,a,θ) as the action value function, which represents the expected total reward of action a in state s. The expression is

[0134] Q(s,a,θ)=E[R t |s t =s,a t =a,θ]

[0135] Where θ is the weight and bias set in the evaluation network;

[0136] Define the experience replay set D, {s t ,a t ,r t ,s t+1} represents the set of state changes and corresponding actions and rewards;

[0137] To simplify the description, we use Q network to represent the neural network in the DQL method, and use Q value to represent the value of the function Q(s, a, θ). We randomly initialize the network parameters θ, initialize the Q values ​​of all states and actions based on θ, clear the experience replay set D, and initialize the state s. t ;

[0138] 3) Use the state s in the function Q(s,a,θ) t As input, get the Q value output corresponding to all actions of the Q network, and select the action corresponding to the maximum Q value When Q(s,a,θ) takes the maximum value, it is recorded as

[0139]

[0140] 4) In state s t Next, perform action a t , get reward r t , get the new state s t+1 , and {s t ,a t ,r t ,s t+1}Store in set D;

[0141] 5) Randomly select m samples from set D {{s j ,a j ,r j ,s j+1}}, j = 1, 2, ..., m, calculate the current target Q value y j :

[0142]

[0143] It means that the UAV obtains the maximum long-term reward and terminates the learning;

[0144] 6) Define the loss function L(θ) to represent the difference between the output Q value of the target network and the evaluation network, as shown in the formula

[0145]

[0146] Among them, θ - are the weights and biases set in the target network, and θ are the weights and biases set in the evaluation network;

[0147] The parameters θ at time t and time t+1 are represented by θ(t) and θ(t+1) respectively; the gradient of the neural network parameters is calculated according to L(θ), and then the parameters θ(t) are updated, and then the neural network module is updated using the parameters θ(t+1), which is expressed as follows:

[0148]

[0149] 7) Set the state s t Updated to t+1 ;

[0150] Repeat steps 2 to 7 until y j It is in the convergence state, that is, the requested channel and power values ​​are obtained.

[0151] Step 3: The master node broadcasts the request transaction to the UAV nodes on the unique node list UNL. After receiving the request, the UAV node uses the Ripple consensus mechanism to execute the smart contract and consensus protocol for the request transaction. The UAV nodes in the unique node list UNL vote on the transaction to confirm whether the request transaction is passed.

[0152] like Figure 1 As shown in the figure, the process of reaching consensus among nodes after the master node requests the channel and power is as follows:

[0153] 1) The master node request channel and power resources are defined as request transactions, and the master node broadcasts the request transactions to the UAV nodes in the UNL;

[0154] 2) The nodes in the UNL are compared with their local candidate set. The local candidate set is the spectrum resource information set that the UAV retains in its local area. The resource information is updated in real time, and only idle spectrum is in this candidate set. If the requested channel is idle in the local candidate set, the transaction will get a vote; if the requested spectrum is not in the local candidate set, the node determines whether to allow the requesting node to use this spectrum based on whether it will cause harmful interference to itself. The expression is:

[0155]

[0156] If the above formula is satisfied, the requesting node is allowed to use, that is, it gets a vote, otherwise it will not get a vote;

[0157] 3) The nodes in UNL broadcast the voting results back to the master node through the network, and the master node counts the votes of the transaction;

[0158] 4) When the requested transaction obtains more than 50% of the votes, the transaction is passed and the voting stops; if all nodes of UNL have completed voting on the transaction or more than 50% of the node votes are not obtained within a certain time limit, the transaction is discarded and the master node resubmits other spectrum and power requests.

[0159] Step 4: Discard the failed request transactions. For the passed request transactions, pack all vehicle information, transaction records and timestamps into a new block, attach it to the blockchain, and broadcast it to all UAV nodes.

[0160] The transaction information, timestamp and public key of the accounting node that pass the vote are packaged into a new block. The block structure is as follows: Figure 4 shown.

[0161] The block body records specific data, and voting transaction information is stored in newly generated records, including vehicle information, allocated channels, and power. The block data structure is a Merkle tree, and data is recorded in leaf nodes. The values ​​of non-leaf nodes are hash values ​​of all leaf node data. Every two adjacent leaves are hashed together to become the content of the upper node. The hash value of the top-level Merkle root node is continuously calculated to quickly summarize and verify the existence and integrity of block data.

[0162] The block is broadcast to all nodes in the network, and each node adds the received new block to the blockchain; each node updates the local candidate set and removes the newly used channel from the local candidate set.

[0163] The blockchain-based drone-assisted vehicle network spectrum allocation method of the present invention manages wireless resources in a distributed structure, which can effectively reduce the possibility of congestion, thereby improving resource utilization and reliability. Blockchain is a shared database with distributed characteristics, which integrates distributed structure, encryption technology, consensus mechanism and smart contract technology. Compared with the centralized model, the decentralization of blockchain can improve reliability and security and effectively reduce costs.

[0164] Compared with traditional Internet of Vehicles resource management technology, this technology has the characteristics of decentralization, immutability, collective maintenance, openness and transparency, making resource management more reliable and stable, avoiding the risk of network congestion or even collapse caused by a large number of vehicles accessing at the same time. This enhances the robustness of the network and optimizes the communication system to maximize the number of devices that can be supported in the system.

[0165] The present invention constructs a UAV-assisted vehicle network, where multiple UAVs exist in an area, each of which covers an area within a certain range. The transmission rate, interruption probability and automatic retransmission delay of the UAV channel are derived. Blockchain-based vehicle network resource management is proposed to avoid network congestion and improve network stability. UAV uses the Ripple consensus mechanism for spectrum allocation and power control. In order to maximize the rate of new additions to the system, an optimization problem is proposed. Deep reinforcement learning is used to solve the resource allocation problem. A discrete method is used in learning to obtain the final target value.

[0166] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.

Claims

1. A blockchain-based drone-assisted vehicle networking spectrum allocation method, characterized in that: The following steps are involved: Step 1: Build a UAV-assisted IoV network, obtain the number of UAVs, the number of vehicles, the number of sub-channels, the interruption probability and the delay information in the IoV network, and build a blockchain resource management model based on the obtained information. Configure a trusted node list for each UAV node, called the unique node list UNL; Step 2: The UAV node connected to the vehicle is the master node. The vehicle submits a spectrum request to the master node. The master node sets the optimization problem and uses deep reinforcement learning to solve it, and obtains the communication spectrum and transmission power between the vehicle and the UAV. Step 3: The master node broadcasts the request transaction to the UAV nodes on the unique node list UNL. After receiving the request, the UAV node uses the Ripple consensus mechanism to execute the smart contract and consensus protocol for the request transaction. The UAV nodes in the unique node list UNL vote on the transaction to confirm whether the request transaction is passed; Step 4: Discard the failed request transactions. For the passed request transactions, pack all vehicle information, transaction records and timestamps into a new block, attach it to the blockchain, and broadcast it to all UAV nodes. The UAV-assisted Internet of Vehicles network is composed of multiple UAVs and vehicles, the number of UAVs is U, the number of vehicles is I, each UAV flies in a set area, and acts as an aerial base station to assist the communication between the vehicle source node and the target node. The UAV-assisted communication link provides high reliability and low latency communication for the vehicle, the system spectrum is shared among all UAVs, and the number of system channels is N; The blockchain resource management model is specifically as follows: For the downlink transmission system of the UAV to vehicle information link, it is assumed that the information transmission is periodic data packet transmission; considering that when the UAV is communicating with the vehicle, other UAVs will interfere with this communication link, and the rate R between UAV u and the vehicle in channel n is u,n It can be expressed as Among them, p u,n represents the transmission power of UAV u in channel n, is the channel gain of UAV u on channel n to the connected vehicle, is the channel gain of UAV u' on channel n to other UAV-connected vehicles, and Obtained by the statistical average of the channel gains of all vehicles, σ 2 is the Gaussian white noise power; When the signal-to-noise ratio of the communication link between the UAV and the vehicle is lower than γ0, the communication transmission from the UAV to the vehicle will be interrupted; in any time period, the probability that the data packet is not successfully sent is given by the interruption probability P out,u It means that the probability of signal-to-noise ratio being less than γ0; the interruption probability expression is Among them, a u',n represents the allocation variable of UAV u' in channel n, Assumption Y u represents the average transmission time of the data packet of UAV u. When the data packet fails to be successfully transmitted in the current time slot, it is retransmitted. If the number of retransmissions does not reach N, it will continue to be retransmitted in the next time slot. If the number of retransmissions has reached N, it will be discarded. Assuming that the length of each time slot is T, Y u The probability function is expressed as Using D u Represents the average sending delay, expressed as Where E{·} represents the mean of the random variable; The step 2 is specifically as follows: After the vehicle is connected to the Internet of Vehicles, it requests the UAV for the spectrum to communicate with it. After receiving the spectrum request from the vehicle, the UAV sets an optimization problem and then solves the optimization problem to obtain the requested channel and power. With the goal of optimizing energy efficiency, the optimization problem is set as: s.t.C1: C2: C3: C4: C5: C6: where a u,n represents the allocation variable of UAV u in channel n, the constraint C1 indicates that the communication rate of the vehicle connected to UAV u is higher than the minimum requirement, C2 and C3 represent the maximum limits of the interruption probability and the average delay, respectively, C4 represents the channel allocation variable of UAV, C5 represents the number of channels requested each time is 1, and C6 represents the maximum transmission power limit of UAV u; In order to solve the above optimization problem, the Deep Q-Learning (DQL) method is used to obtain the optimization variable value; UAV, as a learning agent, perceives the state s(t) at time t and outputs a strategy π, which determines the action a(t) to be taken and then executes this action; in order to allow the learning agent to accumulate experience and better choose actions, the environment will give the learning agent a reward r(t) as feedback; at this time, it enters the next state s(t+1), the learning agent continues to perceive s(t+1), and makes a new strategy π, and so on, eventually making the environment state converge and the agent obtains the maximum reward; The steps of using the Deep Q-Learning method to solve the optimization problem of the UAV node are as follows: 1) Define the joint optimization problem as a Markov decision process, defining the state space, action space and reward function; The state S(t) contains the channel F(t) requested by the UAV at the current time t and the power P(t) transmitted by the UAV; therefore, the state space is expressed as F(t) and P(t) are expressed as: f n (t) indicates that channel n is requested for the current UAV at time t, p l (t) represents the size of the UAV transmission power requested at time t; According to the current state S(t), the UAV takes action A(t) to change the current environment; the action space is represented by the following formula According to the state space and action space, the energy efficiency of UAV is modeled as a reward function, expressed as 2) Define Q(s,a,θ) as the action value function, which represents the expected total reward of action a in state s. The expression is Q(s,a,θ)=E[R t |s t =s,a t [a,θ] Where θ is the weight and bias set in the evaluation network; Define the experience replay set D, {s t ,a t ,r t ,s t+1 } represents the set of state changes and corresponding actions and rewards; To simplify the description, we use Q network to represent the neural network in the DQL method, and use Q value to represent the value of the function Q(s, a, θ). We randomly initialize the network parameters θ, initialize the Q values ​​of all states and actions based on θ, clear the experience replay set D, and initialize the state s. t ; 3) Use the state s in the function Q(s,a,θ) t As input, get the Q value output corresponding to all actions of the Q network, and select the action corresponding to the maximum Q value When Q(s,a,θ) takes the maximum value, it is recorded as 4) In state s t Next, perform action a t , get reward r t , get the new state s t+1 , and {s t ,a t ,r t ,s t+1 }Store in set D; 5) Randomly select m samples from set D {{s j ,a j ,r j ,s j+1 }}, j = 1, 2, ..., m, calculate the current target Q value y j : It means that the UAV obtains the maximum long-term reward and terminates the learning; 6) Define the loss function L(θ) to represent the difference between the output Q value of the target network and the evaluation network, as shown in the formula Among them, θ - are the weights and biases set in the target network, and θ are the weights and biases set in the evaluation network; The parameters θ at time t and time t+1 are represented by θ(t) and θ(t+1) respectively; the gradient of the neural network parameters is calculated according to L(θ), and then the parameters θ(t) are updated, and then the neural network module is updated using the parameters θ(t+1), which is expressed as follows: 7) Set the state s t Updated to t+1 ; Repeat steps 2) to 7) until y j It is in the convergence state, that is, the requested channel and power values ​​are obtained.

2. According to claim 1, a blockchain-based drone-assisted vehicle network spectrum allocation method is characterized by: The process of reaching consensus among nodes after the master node requests the channel and power is as follows: 1) The master node request channel and power resources are defined as request transactions, and the master node broadcasts the request transactions to the UAV nodes in the UNL; 2) The nodes in the UNL compare with their local candidate set. The local candidate set is the spectrum resource information set kept by the UAV in its local area. The resource information is updated in real time. Only idle spectrum is in this candidate set. If the requested channel is idle in the local candidate set, the transaction will get a vote. If the requested spectrum is not in the local candidate set, the node determines whether to allow the use of the spectrum based on whether the requesting node's use of the spectrum will cause harmful interference to itself, based on the expression: If the above formula is satisfied, the requesting node is allowed to use, that is, it gets a vote, otherwise it will not get a vote; 3) The nodes in UNL broadcast the voting results back to the master node through the network, and the master node counts the votes of the transaction; 4) When the request transaction obtains more than 50% of the votes, the transaction is passed and the voting stops; If all nodes of UNL have completed voting on the transaction or more than 50% of the nodes vote within a certain time limit, the transaction is discarded and the master node resubmits other spectrum and power requests.

3. The blockchain-based drone-assisted vehicle networking spectrum allocation method according to claim 1 is characterized by: The step 4 is specifically as follows: The transaction information, timestamp and public key of the accounting node that pass the vote are packaged into a new block; The block body records specific data, and voting transaction information is stored in newly generated records, including vehicle information, allocated channels, and power. The block data structure is a Merkle tree, and data is recorded in leaf nodes. The values ​​of non-leaf nodes are hash values ​​of all leaf node data. Every two adjacent leaves are hashed together to become the content of the upper node. The hash value of the top-level Merkle root node is continuously calculated to quickly summarize and verify the existence and integrity of block data. The block is broadcast to all nodes in the network, and each node adds the received new block to the blockchain; each node updates the local candidate set and removes the newly used channel from the local candidate set.